Professional Documents
Culture Documents
and Resilience
SenseMaker:
Evaluating Complex
Adaptive Systems
Keil Eggers
May 9. 2016
On Big Words
Will be using new vocabulary throughout
the presentation
If something is confusing, please stop and
ask
Learning this for the first time together!
2 | May 2016
Organizing a Party
This Lunch and
Learn is a
celebration
Blindfolded and
swinging
Lets break this
bad boy open.
3 | May 2016
What is Evaluation?
Why do we do it?
What outputs are we expecting from an
evaluation?
In what contexts does CPPR help with
evaluation and what do partners expect?
5 | May 2016
Where We Are
Drive to anticipate the future.
Fail-safe programs to achieve set
outcomes
Robustness: prediction and prevention of
negative outcomes
Hypothesis-based
6 | May 2016
Resources
(Inputs)
People:
Trained
Lemonade for
Life Trainers
(TtT model)
Experienced
MIECHV
professionals
Knowledge:
Adverse
Childhood
Experiences
research
Hope &
resilience
research
7 | May 2016
Program
Components
(Activities)
Lemonade for Life:
Prerequisite
online ACEs
module
1 day training
Participant
resource
materials
(Handbook &
handouts)
Coaching
Number of
professionals
trained in
Lemonade for
Life
Number of
families engaged
in meaningful
ACEs discussion
Intermediate
Outcomes
Professionals
have
hope/mindset
Increased family
engagement in
services
Improved
programmatic
outcomes
Long-Term
Outcomes
The VISION
Anticipatory awareness
Understand the evolutionary potential of
the present
Safe-to-Fail experiments
amplify success
dampen failure
SenseMaking
Understanding actions and their
consequences that emerge from efforts to
create order
Making sense from knowledge of the past
Generate future action
Cynefin (Kun-evin)
The place of your multiple belongings,
religious, tribal, spiritual, geographical,
historical, etc that profoundly influence
what you are (individually and collectively),
but of which you are only ever partially
aware
11 | May
2016
What is SenseMaker?
Software developed by Dave Snowden
and Cognitive Edge
Based on complexity theory, ecology, and
distributed ethnography
Creates human sensor network through
capturing self-identified narrative
fragments
Features
Distributed cognition
Fine-grained material
Disintermediation- original material accessed
directly, not mediated by analysts to decision makers
Evidence-based hard and soft data for actionable
results
Identifies weak signals of hidden opportunities
Semi-constrained signification to not limit to
hypothesis
Mix of soft and hard data to make sense of complex
realities, to distill patterns and to respond in a timely
manner
13 | May
2016
The SenseMaker
Survey
Analysis
Clusters- what is important to look at
based on patterns
Vector analysis
Movement of stories over time
Fitness Mapping
17 | May
2016
Features of SenseMaker
Self-signification
Generates quantitative (visual) data
Patterns & trends against topics of
interest
Analysis, interpretation and use for
action/interventions
Mix of soft and hard data to make sense
of complex realities, to distill patterns and
to respond in a timely manner
20 | May
2016
Outputs
Fine-grained data
See clusters of stories, and read
individual narrative data points
Original material directly accessed by
decision makers
Explorer software for easy analytics
Tableau output- visual representation of
quantitative data
Self-Signification
The process of tagging narrative
Old Approach: Hierarchical taxonomy,
marking key words and assigning the
material to a category
Heavy work for analysis
But semantically, very difficult due to
cognitive science and the nature of
language
22 | May
2016
Why Micro-Narratives?
Not a constructed story to prove a point
Large amounts of fragmented narratives
Unstructured, anecdotal way of seeing
world
Tell specific anecdotes, self-signify what it
means.
You cant ask generally Does this work?
because sometimes it does, sometimes it
doesnt.
Continuous free capture of narrative with
instant feedback.
23 | May
2016
24 | May
2016
How do we go from
where we are to
where we want to
be?
27 | May
2016
Thinking Big
Could change landscape of social service
reporting
LfL as a first experiment leading to wider
applications
Develop expertise in innovative narrative
evaluation
Questions and
Discussion