Process Mining Trends

Process Cubes
Slicing, Dicing, Rolling Up and Drilling

Down Event Data for Process Mining
prof.dr.ir. Wil van der Aalst
www.processmining.org
PAGE 1
PAGE 2
Positioning Process Mining

process model analysis
(simulation, verification, optimization, gaming, etc.)
performanceoriented
questions,
problems and
solutions
process
mining
complianceoriented
questions,
problems and
solutions
data-oriented analysis
(data mining, machine learning, business intelligence)
PAGE 3
Example Process Discovery

(Vestia, Dutch housing agency, 208 cases, 5987 events)
PAGE 4

(ASML, test process lithography systems, 154966 events)
PAGE 5

(AMC, 627 gynecological oncology patients, 24331 events)
PAGE 6
Models are like the glasses required to see

and understand event data!
PAGE 7
PAGE 8
Alignments are essential!
conformance checking to diagnose deviations

squeezing reality into the model to do model-based
analysis
PAGE 9
process
model
event log
synchronous
move
move on
model only
move on log
only
PAGE 10
Example: BPI Challenge 2012

(Dutch financial institute, doi:10.4121/uuid:3926db30-f712-4394-aebc-75976070e91f)
Loops of W_Completeren aanvraag and

W_Nabellen offertes are often performed
Many moves on log of

O_CANCELLED,
O_CREATED,
O_SELECTED,
O_SENT occurred
with the same
frequency value (i.e.
60) before parallel
branch
Work of Arya Adriansyah (Replay project)
O_DECLINED and W_Wijzigen

contractgegevens are often skipped

W_Afhandelen
leads ( > 2200 times)
occurred in the end of
traces
PAGE 11
O_DECLINED and W_Wijzigen

Loops of W_Completeren
aanvraag and
Synchronous moves
of
contractgegevens
are often skipped
Move
on
log
of
Completeren
aanvraag
CompleterenW_Nabellen
aanvraagoffertes are often performed
The average waiting time for the input place of

W_Nabellen offertes+START is very long (2.83 days)
compares to the average waiting time of other places

O_CANCELLED,
O_CREATED,
O_SELECTED,
Move onoccurred
log of O_CANCELLED and A_CANCELLED
O_SENT
W_Afhandelen
with the same
Moves
on
model
towards
end
of
traces
leads
( > 2200 times)
frequency value (i.e.
occurred in the end of
60) before parallel
traces
branch
O_ACCEPTED has average sojourn time of 27.07 minutes,

while A_REGISTERED, A_ACTIVATED, and
A_APPROVED have average sojourn time of 29.56 minutes
Activity W_Wijzigen contractgegevens is the

bottleneck, but it occured rarely (onlyPAGE
4 times)
12
event logs
pre
mortem
current
Process Mining Spectrum
data
navigation
organizations
cartography
auditing
documents
recommend
machines
business
processes
enhance
promote
enhance
cartography
discover
compare
check
detect
recommend
models
discover
post
mortem
auditing
predict
explore
navigation
historic
data
promote
current
data
compare
pre
mortem
check
event logs
detect
provenance
de jure models
diagnose
predict
explore
information system(s)
diagnose
world
people
historic
data
post
mortem
de facto models
models
de jure models
control-flow
data/rules
resources/
organization
de facto models
control-flow
control-flow
control-flow
data/rules
data/rules
resources/
organization
resources/
data/rules
resources/
PAGE 13
ProM (2004 now)
See www.processmining.org
PAGE 14
Overview of ProM releases

ProM 1.1 ( 29 plug-ins:
7 analysis, 3
conversion, 9 export, 4
import, 0 filter, 6
mining)
ProM 4.1 (149 plug-ins)

4/16/2007
ProM 6.0 (170 plug-ins

distributed over 50
packages)
1/1/2005
ProM 2.0
7/1/2005
9/14/2007
9/27/2010
ProM 3.0
2/6/2006
6/17/2008

distributed over 57
packages)
9/5/2011
ProM 3.1 (91 plug-ins:

25 analysis, 10
conversion, 22 export,
10 import, 9 log filter,
15 mining)
ProM 1.0
9/1/2004

10/16/2008
5/18/2006
ProM 2.1
9/1/2005
11/23/2006
ProM 6.2 (standard,

468 plug-ins distributed
over 68 packages)

90 analysis, 44
22 import, 32 filter, 37
mining)
8/16/2012
ProM 6.2 (extended,

over 99 packages)
9/3/2009
5/27/2013
2004
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2013
Today
PAGE 15
ProM 1.1
7 analysis, 3
import, 0 filter, 6
mining)
SNA

4/16/2007

distributed over 50
packages)
1/1/2005
ProM 2.0
7/1/2005
9/14/2007
MFM
9/27/2010
ProM 3.0
2/6/2006
6/17/2008

distributed over 57
packages)
9/5/2011

25 analysis, 10
15 mining)
ProM 1.0
9/1/2004

10/16/2008
5/18/2006
ProM 2.1
9/1/2005
11/23/2006
ProM 6.2 (standard,

over 68 packages)

90 analysis, 44
mining)
8/16/2012
ProM 6.2 (extended,

over 99 packages)
AlphaM
9/3/2009
5/27/2013
2004
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2013
Today
PAGE 16
ProM 4.0
conformance checking

7 analysis, 3
import, 0 filter, 6
mining)

4/16/2007

distributed over 50
packages)
1/1/2005
ProM 2.0
7/1/2005
9/14/2007
9/27/2010
ProM 3.0
2/6/2006
6/17/2008

distributed over 57
packages)
9/5/2011

25 analysis, 10
15 mining)
ProM 1.0
9/1/2004
performance analysis
10/16/2008
5/18/2006
ProM 2.1
9/1/2005
11/23/2006
ProM 6.2 (standard,

over 68 packages)

90 analysis, 44
mining)
8/16/2012
ProM 6.2 (extended,

over 99 packages)
9/3/2009
5/27/2013
2004
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2013
Today
PAGE 17
ProM 5.2
7 analysis, 3
import, 0 filter, 6
mining)

4/16/2007

distributed over 50
packages)
1/1/2005
ProM 2.0
7/1/2005
9/14/2007
9/27/2010
ProM 3.0
2/6/2006
6/17/2008

distributed over 57
packages)
9/5/2011

25 analysis, 10
15 mining)
ProM 1.0
9/1/2004

10/16/2008
5/18/2006
ProM 2.1
9/1/2005
11/23/2006
ProM 6.2 (standard,

over 68 packages)
ProM 5.2 (274 plugins: 90 analysis, 44

mining)
8/16/2012
ProM 6.2 (extended,

over 99 packages)
9/3/2009
5/27/2013
2004
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2013
Today
PAGE 18
ProM 6.0: A new start

7 analysis, 3
import, 0 filter, 6
mining)

4/16/2007

distributed over 50
packages)
1/1/2005
ProM 2.0
7/1/2005
9/14/2007
9/27/2010
ProM 3.0
2/6/2006
6/17/2008

distributed over 57
packages)
9/5/2011

25 analysis, 10
15 mining)
ProM 1.0
9/1/2004

10/16/2008
5/18/2006
ProM 2.1
9/1/2005
11/23/2006
ProM 6.2 (standard,

over 68 packages)

90 analysis, 44
mining)
8/16/2012
ProM 6.2 (extended,

over 99 packages)
9/3/2009
5/27/2013
2004
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2013
Today
PAGE 19
ProM Today
7 analysis, 3
import, 0 filter, 6
mining)

4/16/2007

distributed over 50
packages)
1/1/2005
ProM 2.0
7/1/2005
9/14/2007
9/27/2010
ProM 3.0
2/6/2006
6/17/2008

distributed over 57
packages)
9/5/2011

25 analysis, 10
15 mining)
ProM 1.0
9/1/2004

10/16/2008
5/18/2006
ProM 2.1
9/1/2005
11/23/2006
ProM 6.2 (standard,

over 68 packages)

90 analysis, 44
mining)
8/16/2012
ProM 6.2 (extended, 676

plug-ins distributed over
99 packages)
9/3/2009
5/27/2013
2004
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2013
Today
PAGE 20
ProM 6.3
ProM
New features in ProM 6.3
The process mining framework ProM provides a

versatile and extendible environment for process
mining. It provides plug-ins to extract different types
of models from event logs, e.g., the construction of a
process and organizational models. Moreover, it
supports the conversion and analysis of models.
Using conformance checking techniques models can
also be compared with reality and existing models
can be enhanced with additional information, e.g.,
indicating bottlenecks in a process.
Petri nets with data
Improved features in ProM 6.3
A new inductive miner for structured processes
Improved the data-aware replayer
Support for configuring configurable processes
Improved event-transition conformance
XES Cost extension
Improved use of classifiers for heuristics miner
Smart binding of classifier keys
Improved PNML import w.r.t. CPN Tools models
Caching of latest log info
Improved alignments between traces and Petri nets
CSV export for key-value sets
Improved import/export of transition systems
Import of BPMN 2.0 models from .bpmn files
Improved model repair
Established vs. runner-up packages
Stochastic Petri nets

Support for configurable Petri nets
Discovery and replay using divide-and-conquer techniques
Discovery of services
A new dotted chart
A new genetic miner for structured processes
General bug fixing
ProM 6.3 can be downloaded from

www.promtools.org/prom6 or
sourceforge.net/projects/prom.
PAGE 21
Commercial Alternatives
Disco (Fluxicon)
Perceptive Process Mining
(before Futura Reflect and BPM|one)
ARIS Process Performance

Manager
QPR ProcessAnalyzer
Interstage Process Discovery
(Fujitsu)
Discovery Analyst (StereoLOGIC)
XMAnalyzer (XMPro)

22
Challenges
PAGE 23
Distributing
process
mining
problems to
cope with
big data
PAGE 24
streaming
event data
(sensors, RFID, messages, etc.)
PAGE 25
On-the-fly
process mining
Operational
support
PAGE 26
Concept drift
Concept drift
PAGE 27
Cross-organizational mining
cross-organizational /
comparative process mining
PAGE 28
Example: Dutch Municipalities

(see Coselog project www.win.tue.nl/coselog/)
+/- 430 Dutch

municipalities
PAGE 29
Another Example: Suncorp case

End to end process has between 250-1000 process steps
Product
Dev
25+ steps
Sales
Service
500
Claims
steps
50+ steps
75+ steps
100+ steps
Sources: Guidewire reference models, GIO CISSS Project, CI US&S P4PI Project
Home
Motor
Commercial
Liability
CTP / WC
30
variations
PAGE 30
Thanks to Marcello La Rosa (QUT)
Two variants of the same process
PAGE 31
context aware
process mining
PAGE 32
Process
Cube
PAGE 33
Process Cube: Dimensions and Cells

New behavior
process cube
process cell
event
case type
ct
tw
tim
ew
ind
ow
ec
cell sublog
cell model
dimension
cases
event class
time
event
case
activity
timestamp
resource
type
.
PAGE 34
Example Dimensions
clustering and
classification
group
acbe
abce
ade
acbe
abce
acbe
abce
ade
ade
cross- location
organizational
process
mining
concept
time drift
analysis
cases may
be
distributed
over
multiple
cells, the
same event
may belong
to multiple
cells
PAGE 35
Example
gold
silver
normal
PAGE 36
Another Example
>100k
>50k & 100k
50k
PAGE 37
Another Example
PAGE 38
Roll up and drill down
gold customer
roll up
case type
2: BC
3: BC
7: BC
8: BC
2: CFG
3: CFG
7: CEDG
8: CDEG
D
C
C
B
E
sales
event class
tim 201
ew 3
ind
ow
D
A
20
12
1: CDEG
4: CEDG
5: CFG
6: CEDG
silver customer
D
C
tim 201
ew 3
ind
ow
20
12
case type
1: ACDEG
2: BCFG
3: BCFG
4: ACEDG
5: ACFG
6: ACEDG
7: BCEDG
8: BCDEG
1: AC
4: AC
5: AC
6: AC
delivery
event class
drill down
PAGE 39
Two foundational ways of spitting event

data: horizontal or vertical
A
1: AC
2: BC
3: BC
4: AC
5: AC
6: AC
7: BC
8: BC
split
horizontally
split
vertically
1: CDEG
2: CFG
3: CFG
4: CEDG
5: CFG
6: CEDG
7: CEDG
8: CDEG
A
1: ACDEG
2: BCFG
3: BCFG
4: ACEDG
5: ACFG
6: ACEDG
7: BCEDG
8: BCDEG
merge
horizontally
G
start
complete
B
merge
vertically
D
1: ACDEG
4: ACEDG
6: ACEDG
G
E
D
7: BCEDG
8: BCDEG
G
E
5: ACFG
2: BCFG
3: BCFG
PAGE 40
Ingredients
Process Cube View (PCV)

based on PCS that can be
linked to real events
Event Base (EB) with

(raw) event data
Process Cube Structure (PCS)

defining dimensions
independent of content
PAGE 41
Event Base (E,P, )

Set of events E
Set of properties P
Partial mapping
(p(e)=v means event e has property p with value v)
PAGE 42
Process Cube Structure (D,type,hier)

D is the set of dimensions
type defines the type of each dimension (string, int,
etc.)
hier defines the hierarchy per dimension (does not
need to be a tree, elements at the same level may be
partially overlapping)
Process Cube
Dimensions
case
activity
resource
type
total
time
PAGE 43
Compatible: D P and correct types
re
so
ur
ce
case type
Process Cube Structure

(PCS) defining
dimensions D
independent of content
activity
Event Base (EB) with

(raw) event data
PAGE 44
Process Cube View (Dsel , sel)
ur
ce
re
so
re
so
ur
ce
case type
View on a process cube structure (D,type,hier).

Dsel D are the selected dimensions.
sel is a function determining the structure of each
dimension in Dsel .
activity
activity
PAGE 45
Process Cube View Example (1/3):

Dimension resource is not selected
PCV = (Dsel , sel)
case type
re
so
ur
ce
case type
PCS = (D,type,hier)
activity
activity
PAGE 46

Dimension activity is coarsened
PCV = (Dsel , sel)
re
so
ur
ce
case type
re
so
ur
ce
case type
PCS = (D,type,hier)
activity
activity
e.g., per subprocess rather than per activity
PAGE 47

Dimension case type is filtered
re
so
re
so
ur
ce
case type
PCV = (Dsel , sel)
ur
ce
case type
PCS = (D,type,hier)
activity
activity
part of the case type dimension is not selected
PAGE 48
Materialized Process Cube View
re
so
activity
ur
ce
case type
case type
3 x 2 x 1 = 6 event logs
activity
re
so
ur
ce
case type
activity
PAGE 49
An event may be part of multiple cells!!
case type
departments may
overlap
subprocesses share
interfaces
contracts involving
multiple parties
.
activity
events are unique,

but may be part of multiple cells
PAGE 50
OLAP (Online Analytical Processing)
PAGE 51
OLAP operations can be formalized

using the definitions in the paper
PAGE 52
Initial Implementation
(work of Tatiana Mamaliga)
PAGE 53
Architecture
PAGE 54
Results Generated Using ProCube
PAGE 55
Sparsity is a problem when unloading

cells to ProM
Process cubes tend to

be sparse. Palo does
not handle this well.
PAGE 56
Big Data: Opportunities and Challenges
PAGE 57
Remember
A
1: AC
2: BC
3: BC
4: AC
5: AC
6: AC
7: BC
8: BC
split
horizontally
split
vertically
1: CDEG
2: CFG
3: CFG
4: CEDG
5: CFG
6: CEDG
7: CEDG
8: CDEG
A
1: ACDEG
2: BCFG
3: BCFG
4: ACEDG
5: ACFG
6: ACEDG
7: BCEDG
8: BCDEG
merge
horizontally
G
start
complete
B
merge
vertically
D
1: ACDEG
4: ACEDG
6: ACEDG
G
E
D
7: BCEDG
8: BCDEG
G
E
5: ACFG
2: BCFG
3: BCFG
PAGE 58
Decomposing Conformance Checking

e.g., maximal decomposition, passage-based
decomposition, or SESE/RPST-based decomposition
decomposition
technique
SN
decompose
model
process
model
L
event log
decompose
event log
e.g., A* based alignments, token-based replay, or

simple replay until first deviation
conformance
checking
technique
yields a (valid) activity partitioning
M1
M2
Mn
submodel
submodel
submodel
conformance
check
conformance
check
conformance
check
L1
L2
Ln
sublog
sublog
sublog
conformance diagnostics
PAGE 59
Example of a valid decomposition

f
c7
t7
a
start
t11
t2
c1
c8
t8
c6
c9
t9
c3
t3
t1
t5
c5
Log can be split in the same way!
end
h
c2
c
t4
c4
t10
e
t6
SN1
SN2
SN5
a
start
t2
t1
c1
t7
t8
t8
c6
t9
t9
t10
t10
c3
f
c7
t3
t1
c8
t11
c9
t5
d
t5
e
t6
end
c5
SN6
e
t6
a
SN4
t1
c2
c
t4
SN3
d
t5
t6
t4
c4
PAGE 60
Example of alignment for observed trace

a,b,c,d,e,c,d,g,f
a,b,c,d,e,c,d,g,f
SN1
f
c7
t7
t8
SN2
SN5
c8
start
t11
t2
t1
c1
c1
a
start
c6
t9
c5
t6
end
h
c4
f
t8
t9
t9
t10
t10
c8
t11
c9
c9
d
t5
t4
c6
f
t8
t5
t5
t3
t3
t1
c3
t1
c2
c3
t2
c7
t7
SN6
e
t6
t10
end
c5
SN4
t1
c2
t4
t6
SN3
t5
t6
t4
c4
Etc.
PAGE 61
Conformance checking can be

decomposed !!!
General result for any valid decomposition: Any
event log or trace is perfectly fitting the overall
model if and only if it is also fitting all the individual
fragments
SN1
f
c7
t7
t8
SN2
SN5
c8
start
t11
t2
t1
c1
c1
a
start
c6
t9
c9
e
c5
t6
end
h
c4
t10
f
t8
t9
t9
t10
t10
c8
t11
c9
d
t5
t5
t4
c6
f
t8
t5
t3
t3
t1
c3
t1
c2
c3
t2
c7
t7
end
c5
SN6
e
t6
a
SN4
t1
c2
t4
t6
SN3
Wil van der Aalst, Decomposing Petri nets for process mining:
A generic approach. Distributed and Parallel Databases,
Volume 31, Issue 4, pp 471-507, 2013
t5
t6
t4
c4
PAGE 62
Decomposing Process Discovery

e.g., language/state-based region discovery,
variants of alpha algorithm, genetic process mining
e.g., causal graph based on frequencies is

decomposed using passages or SESE/RPST
decomposition
technique
decompose
event log
event log
M
process
model
compose
model
process
discovery
technique
yields a (valid) activity partitioning
L1
L2
Ln
sublog
sublog
sublog
discovery
discovery
discovery
M1
M2
Mn
submodel
submodel
submodel
PAGE 63
Learn more about decomposing process

mining problems?
W.M.P. van der Aalst. Decomposing Petri Nets for Process Mining: A Generic
Approach. Distributed and Parallel Databases, 31(4):471-507, 2013.
W.M.P. van der Aalst. A General Divide and Conquer Approach for Process Mining. In M. Ganzha,
L. Maciaszek, and M. Paprzycki, editors,Federated Conference on Computer Science and
Information Systems (FedCSIS 2013), pages 1-10. IEEE Computer Society, 2013.
W.M.P. van der Aalst. Decomposing Process Mining Problems Using Passages. In S. Haddad and
L. Pomello, editors, Applications and Theory of Petri Nets 2012, volume 7347 of Lecture Notes in
Computer Science, pages 72-91. Springer-Verlag, Berlin, 2012.
J. Munoz-Gama, J. Carmona, and W.M.P. van der Aalst. Hierarchical Conformance Checking of
Process Models Based on Event Logs. In J.M. Colom and J. Desel, editors, Applications and
Theory of Petri Nets 2013, volume 7927 of Lecture Notes in Computer Science, pages 291-310.
Springer-Verlag, Berlin, 2013.
J. Munoz-Gama, J. Carmona, and W.M.P. van der Aalst. Conformance Checking in the Large:
Partitioning and Topology. In F. Daniel, J. Wang, and B. Weber, editors, International Conference
on Business Process Management (BPM 2013), volume 8094 of Lecture Notes in Computer
Science, pages 130-145. Springer-Verlag, Berlin, 2013.
E. Verbeek and W.M.P. van der Aalst. Decomposing Replay Problems: A Case Study. In D. Moldt
and H. Roelke, editors, Proceedings of the International Workshop on Petri Nets in Software
Engineering (PNSE 2013), volume 989 of CEUR Workshop Proceedings, pages 213-232. CEURWS.org, 2013.
PAGE 64
Conclusion
Challenges in process mining:
concept drift, comparing processes,
cases, and organizations, need for
contextual analysis, limited
scalability, performance problems.
Unifying view: Process Cubes.
Similar to OLAP but also many
differences.
Related to decomposing process
mining problems.
Initial implementations, but just the
starting point.
PAGE 65
www.processmining.org
www.win.tue.nl/ieeetfpm/
PAGE 66

Process Mining Trends

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Process Mining Trends

Uploaded by

Copyright:

Available Formats

Process Cubes

Slicing, Dicing, Rolling Up and Drilling

Positioning Process Mining

Example Process Discovery

Example Process Discovery

Example Process Discovery

Models are like the glasses required to see

Alignments are essential!

conformance checking to diagnose deviations

Example: BPI Challenge 2012

Loops of W_Completeren aanvraag and

Many moves on log of

Work of Arya Adriansyah (Replay project)

O_DECLINED and W_Wijzigen

Many moves on log of

O_DECLINED and W_Wijzigen

The average waiting time for the input place of

Many moves on log of

O_ACCEPTED has average sojourn time of 27.07 minutes,

Activity W_Wijzigen contractgegevens is the

ProM (2004 now)

Overview of ProM releases

ProM 4.1 (149 plug-ins)

ProM 6.0 (170 plug-ins

ProM 4.2 (175 plug-ins)

ProM 5.0 (228 plug-ins)

ProM 6.1 (322 plug-ins

ProM 3.1 (91 plug-ins:

ProM 5.1 (254 plug-ins)

ProM 4.0 (142 plug-ins)

ProM 6.2 (standard,

ProM 5.2 (274 plug-ins:

ProM 6.2 (extended,

ProM 4.1 (149 plug-ins)

ProM 6.0 (170 plug-ins

ProM 4.2 (175 plug-ins)

ProM 5.0 (228 plug-ins)

ProM 6.1 (322 plug-ins

ProM 3.1 (91 plug-ins:

ProM 5.1 (254 plug-ins)

ProM 4.0 (142 plug-ins)

ProM 6.2 (standard,

ProM 5.2 (274 plug-ins:

ProM 6.2 (extended,

ProM 1.1 ( 29 plug-ins:

ProM 4.1 (149 plug-ins)

ProM 6.0 (170 plug-ins

ProM 4.2 (175 plug-ins)

ProM 5.0 (228 plug-ins)

ProM 6.1 (322 plug-ins

ProM 3.1 (91 plug-ins:

ProM 5.1 (254 plug-ins)

ProM 4.0 (142 plug-ins)

ProM 6.2 (standard,

ProM 5.2 (274 plug-ins:

ProM 6.2 (extended,

ProM 4.1 (149 plug-ins)

ProM 6.0 (170 plug-ins

ProM 4.2 (175 plug-ins)

ProM 5.0 (228 plug-ins)

ProM 6.1 (322 plug-ins

ProM 3.1 (91 plug-ins:

ProM 5.1 (254 plug-ins)

ProM 4.0 (142 plug-ins)