Process Mining Trends PDF

Process Cubes
Slicing, Dicing, Rolling Up and Drilling

Down Event Data for Process Mining
prof.dr.ir. Wil van der Aalst

www.processmining.org
PAGE 1
PAGE 2
Positioning Process Mining
process model analysis

(simulation, verification, optimization, gaming, etc.)
performance- compliance-
oriented process oriented
questions, questions,
problems and mining
problems and
solutions solutions
data-oriented analysis
(data mining, machine learning, business intelligence)
PAGE 3
Example Process Discovery
(Vestia, Dutch housing agency, 208 cases, 5987 events)
PAGE 4
(ASML, test process lithography systems, 154966 events)
PAGE 5
(AMC, 627 gynecological oncology patients, 24331 events)
PAGE 6
Models are like the glasses required to see
and understand event data!
PAGE 7
PAGE 8
Alignments are essential!
• conformance checking to diagnose deviations

• squeezing reality into the model to do model-based
analysis
PAGE 9
process event log
model
synchronous
move
move on move on log

model only only
PAGE 10
Example: BPI Challenge 2012
(Dutch financial institute, doi:10.4121/uuid:3926db30-f712-4394-aebc-75976070e91f)
“O_DECLINED” and “W_Wijzigen

Loops of “W_Completeren aanvraag” and contractgegevens” are often skipped
“W_Nabellen offertes” are often performed
Many moves on log of

“O_CANCELLED”,
”O_CREATED”,
”O_SELECTED”,
“O_SENT” occurred Many moves on log of
with the same “W_Afhandelen
frequency value (i.e. leads” ( > 2200 times)
60) before parallel occurred in the end of
branch traces
Work of Arya Adriansyah (Replay project) PAGE 11

“O_DECLINED” and “W_Wijzigen
Synchronous moves of
Loops of “W_Completeren aanvraag” and
Move on log of contractgegevens”
“Completeren are often skipped
aanvraag”
“Completeren“W_Nabellen
aanvraag”offertes” are often performed
The average waiting time for the input place of

“W_Nabellen offertes+START” is very long (2.83 days)
compares to the average waiting time of other places
Many moves on log of
“O_CANCELLED”,
”O_CREATED”,
”O_SELECTED”,
Move onoccurred
“O_SENT” log of “O_CANCELLED” and “A_CANCELLED” Many moves on log of
with the same “W_Afhandelen
frequency value (i.e.
Moves on model towards end of traces
leads” ( > 2200 times)
60) before parallel occurred in the end of
branch traces
“O_ACCEPTED” has average sojourn time of 27.07 minutes,

while “A_REGISTERED”, ”A_ACTIVATED”, and
“A_APPROVED” have average sojourn time of 29.56 minutes
Activity “W_Wijzigen contractgegevens” is the

bottleneck, but it occured rarely (onlyPAGE
4 times)
12
event logs
“pre “post
Process Mining Spectrum
current mortem” historic mortem”
data data
machines
people
business “world” organizations
processes
navigation documents
auditing cartography
recommend
information system(s)
diagnose
compare
enhance
discover
promote
explore
predict
provenance
detect
check
event logs
“pre “post
mortem” current historic mortem”
data data
navigation auditing cartography

recommend
diagnose
compare
models enhance
discover
promote
explore
predict
detect
check
de jure models de facto models
models
de jure models de facto models
control-flow control-flow
control-flow control-flow
data/rules data/rules
data/rules data/rules
resources/ resources/
organization organization
resources/ resources/ PAGE 13
ProM (2004 – now)
See www.processmining.org PAGE 14

Overview of ProM releases
ProM 1.1 ( 29 plug-ins:

7 analysis, 3
conversion, 9 export, 4
import, 0 filter, 6 ProM 4.1 (149 plug-ins)
mining) 4/16/2007 ProM 6.0 (170 plug-ins
1/1/2005 distributed over 50
packages)
ProM 2.0 ProM 4.2 (175 plug-ins)
9/27/2010
7/1/2005 9/14/2007
ProM 6.1 (322 plug-ins

ProM 3.0 ProM 5.0 (228 plug-ins) distributed over 57
2/6/2006 6/17/2008 packages)
9/5/2011
ProM 3.1 (91 plug-ins: ProM 5.1 (254 plug-ins)
25 analysis, 10 10/16/2008 ProM 6.2 (standard,
conversion, 22 export, 468 plug-ins distributed
ProM 1.0 10 import, 9 log filter, over 68 packages)
9/1/2004 15 mining) ProM 5.2 (274 plug-ins:
90 analysis, 44 8/16/2012
5/18/2006
conversion, 49 export,
ProM 2.1 ProM 4.0 (142 plug-ins) 22 import, 32 filter, 37
mining) ProM 6.2 (extended,
9/1/2005 11/23/2006 676 plug-ins distributed
9/3/2009 over 99 packages)
5/27/2013
2004 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2013
Today
PAGE 15
ProM 1.1

7 analysis, 3
import, 0 filter, 6 ProM 4.1 (149 plug-ins) SNA
packages)
7/1/2005 9/14/2007
9/27/2010 MFM
2/6/2006 6/17/2008 packages)
9/5/2011
90 analysis, 44 8/16/2012
5/18/2006
9/3/2009 AlphaM
over 99 packages)
5/27/2013
2004 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2013
Today
PAGE 16
ProM 4.0 conformance checking

7 analysis, 3
packages)
9/27/2010
7/1/2005 9/14/2007

2/6/2006 6/17/2008 packages)
9/5/2011
25 analysis, 10 10/16/2008
performance analysis
conversion, 22 export, ProM 6.2 (standard,
10 import, 9 log filter, 468 plug-ins distributed
ProM 1.0 over 68 packages)
90 analysis, 44 8/16/2012
5/18/2006
5/27/2013
2004 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2013
Today
PAGE 17
ProM 5.2

7 analysis, 3
packages)
9/27/2010
7/1/2005 9/14/2007

2/6/2006 6/17/2008 packages)
9/5/2011
9/1/2004 15 mining) ProM 5.2 (274 plug-
ins: 90 analysis, 44 8/16/2012
5/18/2006
5/27/2013
2004 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2013
Today
PAGE 18
ProM 6.0: A new start …

7 analysis, 3
packages)
9/27/2010
7/1/2005 9/14/2007

2/6/2006 6/17/2008 packages)
9/5/2011
90 analysis, 44 8/16/2012
5/18/2006
5/27/2013
2004 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2013
Today
PAGE 19
ProM Today

7 analysis, 3
packages)
9/27/2010
7/1/2005 9/14/2007

2/6/2006 6/17/2008 packages)
9/5/2011
90 analysis, 44 8/16/2012
5/18/2006
mining) ProM 6.2 (extended, 676
9/1/2005 11/23/2006 plug-ins distributed over
9/3/2009 99 packages)
5/27/2013
2004 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2013
Today
PAGE 20
ProM 6.3
ProM New features in ProM 6.3

The process mining framework ProM provides a · Petri nets with data
versatile and extendible environment for process
· Stochastic Petri nets
mining. It provides plug-ins to extract different types
of models from event logs, e.g., the construction of a · Support for configurable Petri nets
process and organizational models. Moreover, it · Discovery and replay using divide-and-conquer techniques
supports the conversion and analysis of models.
Using conformance checking techniques models can · Discovery of services
also be compared with reality and existing models · A new dotted chart
can be enhanced with additional information, e.g.,
indicating bottlenecks in a process. · A new genetic miner for structured processes
Improved features in ProM 6.3 · A new inductive miner for structured processes
· Improved the data-aware replayer · Support for configuring configurable processes
· Improved event-transition conformance · XES Cost extension
· Improved use of classifiers for heuristics miner · Smart binding of classifier keys
· Improved PNML import w.r.t. CPN Tools models · Caching of latest log info
· Improved alignments between traces and Petri nets · CSV export for key-value sets
· Improved import/export of transition systems · Import of BPMN 2.0 models from .bpmn files
· Improved model repair · Established vs. runner-up packages
· General bug fixing
ProM 6.3 can be downloaded from

www.promtools.org/prom6 or
sourceforge.net/projects/prom. PAGE 21
Commercial Alternatives
• Disco (Fluxicon)
• Perceptive Process Mining
(before Futura Reflect and BPM|one)
• ARIS Process Performance
Manager
• QPR ProcessAnalyzer
• Interstage Process Discovery
(Fujitsu)
• Discovery Analyst (StereoLOGIC)
• XMAnalyzer (XMPro)
• …
22
Challenges
PAGE 23
Distributing
process
mining
problems to
cope with
big data
PAGE 24
streaming
event data
(sensors, RFID, messages, etc.)
PAGE 25
On-the-fly
process mining
Operational
support
PAGE 26
Concept drift
Concept drift
PAGE 27
Cross-organizational mining
cross-organizational /
comparative process mining
PAGE 28
Example: Dutch Municipalities
(see Coselog project www.win.tue.nl/coselog/)
+/- 430 Dutch

PAGE 29
municipalities
Another Example: Suncorp case
End to end process has between 250-1000 process steps

Product
Sales Service Claims 500
Dev steps
• 25+ steps • 50+ steps • 75+ steps • 100+ steps
Sources: Guidewire reference models, GIO CISSS Project, CI US&S P4PI Project
Home      
Motor         30
Commercial      variations
Liability     
CTP / WC     
PAGE 30
Thanks to Marcello La Rosa (QUT)

Two variants of the same process …
PAGE 31
context aware
process mining
PAGE 32
Process
Cube
PAGE 33
Process Cube: Dimensions and Cells
New behavior process cell
process cube
event
ct
case type
tw
ec
ow
ind
ew
tim
cell sublog cell model
event class
cases
dimension
time
event
case
activity
timestamp
resource
type
….
PAGE 34
Example Dimensions
clustering and
classification
cases may
group be
distributed
acbe over
abce
ade
acbe multiple
abce
acbe cells, the
abce
ade
ade concept same event
time drift may belong
cross- location analysis
to multiple
organizational cells
process
mining PAGE 35
Example
gold
silver
normal
PAGE 36
Another Example
>100k
>50k & 100k

50k
PAGE 37
Another Example
PAGE 38
Roll up and drill down
roll up
1: AC 1: CDEG
4: AC 4: CEDG
gold customer
5: AC 5: CFG
6: AC 6: CEDG
1: ACDEG
2: BCFG D
3: BCFG A
4: ACEDG C F G
5: ACFG C
case type
case type
6: ACEDG E
7: BCEDG
8: BCDEG
2: BC 2: CFG
ow
ow
3: BC 3: CFG
silver customer
ew 3
7: BC 7: CEDG
ew 3
ind
A D
ind
tim 201
8: BC 8: CDEG
tim 201
C F G
D
C
B E
C F G
12
12
B
20
20
E
sales delivery
event class event class
drill down
PAGE 39
Two foundational ways of spitting event
data: horizontal or vertical
A D
1: AC 1: CDEG
2: BC 2: CFG
3: BC C 3: CFG C F G
4: AC 4: CEDG
5: AC 5: CFG
6: AC 6: CEDG
7: BC 7: CEDG
8: BC B 8: CDEG E
split A D
merge
horizontally horizontally
1: ACDEG
2: BCFG
3: BCFG C F G
4: ACEDG
5: ACFG
6: ACEDG start complete
split merge
7: BCEDG
vertically 8: BCDEG B E vertically
1: ACDEG A C G
4: ACEDG
6: ACEDG
E
7: BCEDG B C G
8: BCDEG
E
A C F G
5: ACFG
B C F G
2: BCFG
3: BCFG
PAGE 40
Process Cube View (PCV)
Ingredients based on PCS that can be
linked to real events
Event Base (EB) with
(raw) event data
Process Cube Structure (PCS)

defining dimensions
independent of content PAGE 41
Event Base (E,P, π)
• Set of events E
• Set of properties P
• Partial mapping π
(πp(e)=v means event e has property p with value v)
PAGE 42
Process Cube Structure (D,type,hier)
• D is the set of dimensions

• type defines the type of each dimension (string, int,
etc.)
• hier defines the hierarchy per dimension (does not
need to be a tree, elements at the same level may be
partially overlapping)
Process Cube
Dimensions
case
activity
resource
type
total
time
PAGE 43
Compatible: D P and correct types
Process Cube Structure

(PCS) defining
dimensions D
independent of content
case type
ce
ur
so
re
activity
Event Base (EB) with

(raw) event data
PAGE 44
Process Cube View (Dsel , sel)
• View on a process cube structure (D,type,hier).

• Dsel  D are the selected dimensions.
• sel is a function determining the structure of each
dimension in Dsel .
case type
ce
ce
ur
ur
so
so
re
re
activity activity
PAGE 45
Process Cube View Example (1/3):
Dimension resource is not selected
PCS = (D,type,hier) PCV = (Dsel , sel)
case type
case type
ce
ur
so
re
activity activity
PAGE 46
Dimension activity is coarsened
PCS = (D,type,hier) PCV = (Dsel , sel)
case type
case type
ce
ce
ur
ur
so
so
re
re
activity activity
e.g., per subprocess rather than per activity
PAGE 47
Dimension case type is filtered
PCS = (D,type,hier)
PCV = (Dsel , sel)
case type
case type
ce
ce
ur
ur
so
so
re
re
activity activity
part of the case type dimension is not selected
PAGE 48
Materialized Process Cube View
3 x 2 x 1 = 6 event logs
case type
case type
activity
ce
ur
so
re
activity
case type
ce
ur
so
re
activity
PAGE 49
An event may be part of multiple cells!!
• departments may
overlap
• subprocesses share
interfaces
case type
• contracts involving
multiple parties
• ….
activity
• events are unique,

• but may be part of multiple cells
PAGE 50
OLAP (Online Analytical Processing)
PAGE 51
OLAP operations can be formalized
using the definitions in the paper
PAGE 52
Initial Implementation
(work of Tatiana Mamaliga)
PAGE 53
Architecture
PAGE 54
Results Generated Using ProCube
PAGE 55
Sparsity is a problem when unloading
cells to ProM
Process cubes tend to

be sparse. Palo does
not handle this well.
PAGE 56
Big Data: Opportunities and Challenges
PAGE 57
Remember
A D
1: AC 1: CDEG
2: BC 2: CFG
3: BC C 3: CFG C F G
4: AC 4: CEDG
5: AC 5: CFG
6: AC 6: CEDG
7: BC 7: CEDG
8: BC B 8: CDEG E
split A D
merge
horizontally horizontally
1: ACDEG
2: BCFG
3: BCFG C F G
4: ACEDG
5: ACFG
6: ACEDG start complete
split merge
7: BCEDG
vertically 8: BCDEG B E vertically
1: ACDEG A C G
4: ACEDG
6: ACEDG
E
7: BCEDG B C G
8: BCDEG
E
A C F G
5: ACFG
B C F G
2: BCFG
3: BCFG
PAGE 58
Decomposing Conformance Checking
e.g., maximal decomposition, passage-based e.g., A* based alignments, token-based replay, or
decomposition, or SESE/RPST-based decomposition simple replay until first deviation
conformance
decomposition
checking
technique yields a (valid) activity partitioning
technique
SN decompose
model M1 M2 Mn
process
model submodel submodel submodel
conformance conformance conformance

check check check
L decompose
event log L1 L2 Ln
event log sublog sublog sublog
conformance diagnostics
PAGE 59
Example of a valid decomposition
f
t7 c7 t8 c8 t11
g
t2
c6 t9 c9
c1 c3
b
a d
start t1
t3
t5 c5
h
end Log can be split in the same way!
c2 c c4 t10
t4
e
t6
SN1 SN2 SN5

f f
a c8
t2 t7 c7 t8 t8 t11
start t1
c1 c3 g g
b
a d c6 t9 t9 c9
t3
t1 t5
d
t5 c5 end
h h
e
t10 t10 SN6
t6
e
t6
a
t1 SN4
d
c2 c t5
t4
e c c4
SN3 t6 t4
PAGE 60
Example of alignment for observed trace
a,b,c,d,e,c,d,g,f
a,b,c,d,e,c,d,g,f f
SN1
a
SN2 SN5
f f
t2 c7 t8 t8 c8 t11
t7
start t1
t7 c7 t8 c8 t11 c1 c3
b g g
a d c6 t9 t9 c9
t3
g t1 t5
t2 d
c6 t9 c9
c5 end
c1 c3 t5
b h h
a d e
t10 t10 SN6
t3 t6
start t1 t5 c5 end e
h a
t6
c2 c c4 t10 t1 SN4
d
t4
e c2 c t5
t4
t6 e c c4
SN3 t6 t4
Etc.
PAGE 61
Conformance checking can be
decomposed !!!
• General result for any valid decomposition: Any

event log or trace is perfectly fitting the overall
model if and only if it is also fitting all the individual
fragments
SN1 SN2 SN5
f f
f a
c7 t8 c8
t2 t7 t8 t11
start t1
t7 c7 t8 c8 t11 c1 c3 g
b g
a d c6 t9 t9 c9
t3
g t1 t5
t2 d
c6 t9 c9
c5 end
c1 c3 t5
b h h
a d e
t10 t10 SN6
t3 t6
start t1 t5 c5 end e
h a
t6
c2 c c4 t10 t1 SN4
d
t4
e c2 c t5
t4
t6 e c c4
SN3 t6 t4
Wil van der Aalst, Decomposing Petri nets for process mining:
A generic approach. Distributed and Parallel Databases,
Volume 31, Issue 4, pp 471-507, 2013 PAGE 62
Decomposing Process Discovery
e.g., causal graph based on frequencies is e.g., language/state-based region discovery,

decomposed using passages or SESE/RPST variants of alpha algorithm, genetic process mining
process
decomposition
discovery
technique
yields a (valid) activity partitioning technique
L decompose
event log L1 L2 Ln
event log sublog sublog sublog
discovery discovery discovery
M compose
model M1 M2 Mn
process
model submodel submodel submodel
PAGE 63
Learn more about decomposing process
mining problems?
• W.M.P. van der Aalst. Decomposing Petri Nets for Process Mining: A Generic
Approach. Distributed and Parallel Databases, 31(4):471-507, 2013.
• W.M.P. van der Aalst. A General Divide and Conquer Approach for Process Mining. In M. Ganzha,
L. Maciaszek, and M. Paprzycki, editors,Federated Conference on Computer Science and
Information Systems (FedCSIS 2013), pages 1-10. IEEE Computer Society, 2013.
• W.M.P. van der Aalst. Decomposing Process Mining Problems Using Passages. In S. Haddad and
L. Pomello, editors, Applications and Theory of Petri Nets 2012, volume 7347 of Lecture Notes in
Computer Science, pages 72-91. Springer-Verlag, Berlin, 2012.
• J. Munoz-Gama, J. Carmona, and W.M.P. van der Aalst. Hierarchical Conformance Checking of
Process Models Based on Event Logs. In J.M. Colom and J. Desel, editors, Applications and
Theory of Petri Nets 2013, volume 7927 of Lecture Notes in Computer Science, pages 291-310.
Springer-Verlag, Berlin, 2013.
• J. Munoz-Gama, J. Carmona, and W.M.P. van der Aalst. Conformance Checking in the Large:
Partitioning and Topology. In F. Daniel, J. Wang, and B. Weber, editors, International Conference
on Business Process Management (BPM 2013), volume 8094 of Lecture Notes in Computer
Science, pages 130-145. Springer-Verlag, Berlin, 2013.
• E. Verbeek and W.M.P. van der Aalst. Decomposing Replay Problems: A Case Study. In D. Moldt
and H. Roelke, editors, Proceedings of the International Workshop on Petri Nets in Software
Engineering (PNSE 2013), volume 989 of CEUR Workshop Proceedings, pages 213-232. CEUR-
WS.org, 2013.
PAGE 64
Conclusion
• Challenges in process mining:

concept drift, comparing processes,
cases, and organizations, need for
contextual analysis, limited
scalability, performance problems.
• Unifying view: Process Cubes.
• Similar to OLAP but also many
differences.
• Related to decomposing process
mining problems.
• Initial implementations, but just the
starting point.
PAGE 65
www.processmining.org
www.win.tue.nl/ieeetfpm/
PAGE 66

Process Mining Trends PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Process Mining Trends PDF

Uploaded by

Copyright:

Available Formats

Process Cubes

Slicing, Dicing, Rolling Up and Drilling

prof.dr.ir. Wil van der Aalst

process model analysis

• conformance checking to diagnose deviations

move on move on log

“O_DECLINED” and “W_Wijzigen

Many moves on log of

Work of Arya Adriansyah (Replay project) PAGE 11

The average waiting time for the input place of

“O_ACCEPTED” has average sojourn time of 27.07 minutes,

Activity “W_Wijzigen contractgegevens” is the

navigation auditing cartography

de jure models de facto models

See www.processmining.org PAGE 14

ProM 1.1 ( 29 plug-ins:

ProM 6.1 (322 plug-ins

ProM 1.1 ( 29 plug-ins:

ProM 1.1 ( 29 plug-ins:

ProM 6.1 (322 plug-ins

ProM 1.1 ( 29 plug-ins:

ProM 6.1 (322 plug-ins

ProM 1.1 ( 29 plug-ins:

ProM 6.1 (322 plug-ins

ProM 1.1 ( 29 plug-ins:

ProM 6.1 (322 plug-ins

ProM New features in ProM 6.3

· Improved the data-aware replayer · Support for configuring configurable processes

· Improved event-transition conformance · XES Cost extension

· Improved model repair · Established vs. runner-up packages

· General bug fixing

ProM 6.3 can be downloaded from

+/- 430 Dutch

End to end process has between 250-1000 process steps

Thanks to Marcello La Rosa (QUT)

>50k & 100k

Process Cube Structure (PCS)

• D is the set of dimensions

Process Cube Structure

Event Base (EB) with

• View on a process cube structure (D,type,hier).

PCS = (D,type,hier) PCV = (Dsel , sel)

PCS = (D,type,hier) PCV = (Dsel , sel)

e.g., per subprocess rather than per activity

part of the case type dimension is not selected

• events are unique,

Process cubes tend to

conformance conformance conformance

SN1 SN2 SN5

• General result for any valid decomposition: Any

e.g., causal graph based on frequencies is e.g., language/state-based region discovery,

discovery discovery discovery

• Challenges in process mining:

You might also like