E-Assessment & Learning Analytics

8. Dashboards & Visualization

Prof. Dr. Sven Strickroth

SS 2021
 An Example: Anscombe’s quartet

• Four data sets of 11 points

• Practically identical descriptive
statistical properties
• mean(x)=9, mean(y)=7.5
• var(x)=11, var(y)=4.125
• correlation(x,y)=0.816
• linear regression line:
https://en.wikipedia.org/wiki/Anscombe's_quartet

y = 3.00 + 0.500x
• Very different when shown
visually
• Published 1973 to motivate
visual representations
2
 When/Why do we need visual representations?

Visualizations allow us to
• Understand patterns
• Gain insights
• Make decisions
• Communicate data

Only needed if no perfect computational solution exists

• Ill-defined problems
• no single optimal solution
• no clear objective measure

3
 The Power of the Visual Sense

How do we get from start to end?

Example courtesy of Michael Sedlmair 4

 The Power of the Visual Sense

How do we get from start to end?

Perception beats Cognition!

Example courtesy of Michael Sedlmair 5
 Outline

Dashboards

Data Visualization

Evaluation

6
 Learning Objectives

You
• know what a dashboard is
• know target groups of dashboards
• know commonly used indicators
• know design recommendations for dashboards
• know different visualization idioms
• know when to choose which visualization
• know how to evaluate visualizations
• know how to evaluate Learning Analytics dashboards

7
Dashboards
 Definition

Learning Analytics dashboards are “single displays that aggregate

different indicators about learner(s), learning process(es) and/or
learning context(s) into one or multiple visualizations.”
(Schwendimann et al., 2016)

• Goal: “transfer” information efficiently and effectively

• developed for different stakeholders
• need to be properly designed
• at best: designed to be interactive!
• indicators need to be properly selected

9
Target groups (Schwendimann et al., 2016)

10
 Indicators used in Dashboards
(Schwendimann et al., 2016)

Indicator group name Examples of indicators in the group

Learner-related Prior education, Competences, Age, Prior courses they

took, University entrance grade
Action-related Number of page visits, Number of file downloads, Time
spent on tasks, Login time, Timeline
Content-related Sentiment of the messages, Topics covered and omitted
in the report, Number of concepts and links in a concept
map
Result-related Average grade, Grades distribution in a group
Context-related Location of learners around a tabletop, Placement in a
classroom, Geographical location
Social-related Network showing communication with others in a
group forum, Direction of interaction in a group around
a tabletop

11
Activities, indicator, and metrics (Ahmad et al., 2020)

12
 Recommendations for dashboard design
(Jivet et al, 2018)

1. LA dashboards should be designed as pedagogical tools that enhance

awareness and reflection as a means to catalyze changes in the
cognitive, behavioral and emotional competences.

2. Educational concepts from learning sciences should be used to

motivate design decisions.

3. Comparison with peers should be used cautiously.

4. Do not assume the dashboard will have the same effect on all its
users, but rather seek to determine which group of learners benefit the
most and how to customize the dashboard to provide the same support
to all its users.

5. The dashboard should be seamlessly integrated into the online

learning environment and into the usual learning activities of the
learner.

13
Type of visualization in dashboards
(Schwendimann et al., 2016)

14
Data Visualization
 Foundations of Human Perception and Cognition

Human Visual Perception

• Physiology of visual perception
• Perception of colors
• Gestalt laws
• …

Human Cognition
• Types of human memory
• Limited Attention
• Cognitive load
• …

➔ cf. lectures on HCI, Information Visualization, …

16
 Idioms

• Idioms for low-dimensional table data

• 1-dimensional data
• 2-dimensional data
• 3-dimensional data
• Idioms for high-dimensional table data
• geometric transformation
• glyphs
• Idioms for Trees/Hierarchies
• node-link diagrams with geometric distortions
• tree maps
• sunburst displays
• Maps
• Metaphors
• Multiple Coordinated Views & Interaction
• dynamic queries
• animated Transitions
• …

17
 Idioms

• Idioms for low-dimensional table data

• 1-dimensional data
• 2-dimensional data
• 3-dimensional data
• Idioms for high-dimensional table data
• geometric transformation
• glyphs
• Idioms for Trees/Hierarchies
• node-link diagrams with geometric distortions
• tree maps
• sunburst displays
• Maps
• Metaphors
• Multiple Coordinated Views & Interaction
• dynamic queries
• animated Transitions
• …

18
 Line chart

Primary (ordered) attribute (here: time) on the horizontal axis

Secondary (ordered) attribute (here: # activities) on the vertical axis
Uses a line mark to convey continuous values

Example: Activities over time (LeMo, Elkina et al., 2014)

• Focus on a selected course
• Shown: hit rates on all course objects & number of users

19
 Bar chart

Primary (categorical) attribute on one axis (here: horizontal)

Secondary (ordered) attribute on the other axis (here: vertical)
Uses line or area marks to convey separate data points

Example (LeMo, Elkina et al., 2014):

– Usage of Learning Resources: Which learning resources are used how often?
– Shown: hit rates on resources (moodle objects)
20
 Box and Whisker Plot

Values Median Mean

(3,3,3,3,3) 3 3

(1,2,3,4,5) 3 3

(1,1,2,3,50) 2 11,4

Transports information of data and some descriptive statistical measures

Example Activities over time (LeMo, Elkina et al., 2014):

• At different days
• Shown: hit rates on all course objects

21
Word cloud

22
 Idioms

• Idioms for low-dimensional table data

• 1-dimensional data
• 2-dimensional data
• 3-dimensional data
• Idioms for high-dimensional table data
• geometric transformation
• glyphs
• Idioms for Trees/Hierarchies
• node-link diagrams with geometric distortions
• tree maps
• sunburst displays
• Maps
• Metaphors
• Multiple Coordinated Views & Interaction
• dynamic queries
• animated Transitions
• …

23
 Scatterplot

Most common visualization for displaying 2D data

Each axis encodes an ordered (usually quantitative) attribute
Each item is encoded by a mark
- Position of the mark according to the attribute values
Users can easily identify correlations, global trends, local trade-offs, outliers

24
 Idioms

• Idioms for low-dimensional table data

• 1-dimensional data
• 2-dimensional data
• 3-dimensional data
• Idioms for high-dimensional table data
• geometric transformation
• glyphs
• Idioms for Trees/Hierarchies
• node-link diagrams with geometric distortions
• tree maps
• sunburst displays
• Maps
• Metaphors
• Multiple Coordinated Views & Interaction
• dynamic queries
• animated Transitions
• …

25
 3D-Scatterplots

• Tempting: map
each attribute to a
dimension of a 3D
scatterplot
• Occlusion of marks
with different
positions and
general difficulty to
visually map the
(x,y,z)- coordinates
of the single marks
• → 3D rotation can
convey spatial
layout better
26
Interactive, rotating 3D-Scatterplot

27
 Scatterplot using additional channels

• Mark for each item uses a

channel to convey 3rd
attribute
• Shows the relationship
between all three attributes
• Assumes one categorical
attribute
- binning of quantitative
variable

- potential loss of
information
28
https://de.mathworks.com/help/examples/stats/CreateScatterPlotsUsingGroupedDataExample_01.png
 Heat maps

Example Activities over time (LeMo, Elkina et al., 2014):

• In different courses (each column one course)
• Shown: hit rates on all course objects
29
 Idioms

• Idioms for low-dimensional table data

• 1-dimensional data
• 2-dimensional data
• 3-dimensional data
• Idioms for high-dimensional table data
• geometric transformation
• glyphs
• Idioms for Trees/Hierarchies
• node-link diagrams with geometric distortions
• tree maps
• sunburst displays
• Maps
• Metaphors
• Multiple Coordinated Views & Interaction
• dynamic queries
• animated Transitions
• …

30
 Parallel Coordinate Plot

• n attributes means n axes (grows linearly)

• All attributes receive uniform treatment
• Allows to find correlations

31
 Parallel Coordinate Plot for sets (cf. Sankey diagram)

1st 2nd 3rd crew

Female (s) 141 93 90 3
Female (d) 4 13 106 20
Male (s) 62 25 98 670
Male (d) 118 154 422 192

Parallel coordinates for categorical data

Substitute items by a frequency-based representation
32
 Radar (Spider) Chart

Cicular coordinates
2D patterns can easily be spotted

33
 Idioms

• Idioms for low-dimensional table data

• 1-dimensional data
• 2-dimensional data
• 3-dimensional data
• Idioms for high-dimensional table data
• geometric transformation
• glyphs
• Idioms for Trees/Hierarchies
• node-link diagrams with geometric distortions
• tree maps
• sunburst displays
• Maps
• Metaphors
• Multiple Coordinated Views & Interaction
• dynamic queries
• animated Transitions
• …

34
 Chernoff Faces

Humans are sensitive to a wide range of facial characteristics

(e.g., eye size, length of a nose, etc.)
18 characteristics to encode data by stylized faces
distinction vs. interpretation
35
 Idioms

• Idioms for low-dimensional table data

• 1-dimensional data
• 2-dimensional data
• 3-dimensional data
• Idioms for high-dimensional table data
• geometric transformation
• glyphs
• Idioms for Trees/Hierarchies
• node-link diagrams with geometric distortions
• tree maps
• sunburst displays
• Maps
• Metaphors
• Multiple Coordinated Views & Interaction
• dynamic queries
• animated Transitions
• …

36
 Idioms

• Idioms for low-dimensional table data

• 1-dimensional data
• 2-dimensional data
• 3-dimensional data
• Idioms for high-dimensional table data
• geometric transformation
• glyphs
• Idioms for Trees/Hierarchies
• node-link diagrams with geometric distortions
• tree maps
• sunburst displays
• Maps
• Metaphors
• Multiple Coordinated Views & Interaction
• dynamic queries
• animated Transitions
• …

37
 Tree

• Example decision tree (Asif et al., 2017)

38
 Node-link diagrams

Graph can represent any information modeled as items and links, e.g. social networks

Example Learning Paths: What are the typical paths? (LeMo, Elkina et al., 2014):
• Shown: different resources and the transitions

39
 Idioms

• Idioms for low-dimensional table data

• 1-dimensional data
• 2-dimensional data
• 3-dimensional data
• Idioms for high-dimensional table data
• geometric transformation
• glyphs
• Idioms for Trees/Hierarchies
• node-link diagrams with geometric distortions
• tree maps
• sunburst displays
• Maps
• Metaphors
• Multiple Coordinated Views & Interaction
• dynamic queries
• animated Transitions
• …

40
 Tree map

Example Usage of learning resources (LeMo, Elkina et al., 2014):

• Shown: by size – hit rates on different resources

41
 Idioms

• Idioms for low-dimensional table data

• 1-dimensional data
• 2-dimensional data
• 3-dimensional data
• Idioms for high-dimensional table data
• geometric transformation
• glyphs
• Idioms for Trees/Hierarchies
• node-link diagrams with geometric distortions
• tree maps
• sunburst displays
• Maps
• Metaphors
• Multiple Coordinated Views & Interaction
• dynamic queries
• animated Transitions
• …

42
 Sunburst

Circular visualization to give each element more space

Radial visualizations may better depict the structure of the tree, but are not
as space-efficient as treemaps

43
 Idioms

• Idioms for low-dimensional table data

• 1-dimensional data
• 2-dimensional data
• 3-dimensional data
• Idioms for high-dimensional table data
• geometric transformation
• glyphs
• Idioms for Trees/Hierarchies
• node-link diagrams with geometric distortions
• tree maps
• sunburst displays
• Maps
• Metaphors
• Multiple Coordinated Views & Interaction
• dynamic queries
• animated Transitions
• …

44
 Maps

Visualization of geographic location

Example: Spatial distribution of learners

(Strickroth & Pinkwart, 2014)

45
 Idioms

• Idioms for low-dimensional table data

• 1-dimensional data
• 2-dimensional data
• 3-dimensional data
• Idioms for high-dimensional table data
• geometric transformation
• glyphs
• Idioms for Trees/Hierarchies
• node-link diagrams with geometric distortions
• tree maps
• sunburst displays
• Maps
• Metaphors
• Multiple Coordinated Views & Interaction
• dynamic queries
• animated Transitions
• …

46
Metaphors

…
47
 Idioms

• Idioms for low-dimensional table data

• 1-dimensional data
• 2-dimensional data
• 3-dimensional data
• Idioms for high-dimensional table data
• geometric transformation
• glyphs
• Idioms for Trees/Hierarchies
• node-link diagrams with geometric distortions
• tree maps
• sunburst displays
• Maps
• Metaphors
• Multiple Coordinated Views & Interaction
• dynamic queries
• animated Transitions
• …

48
 Multiple Coordinated Views & Interaction

Might be helpful to display the same data with different

visualizations at the same time

Make visualizations interactive

• allow filtering, zooming, highlighting, …

Consider animated transitions

49
 Chart chooser

https://extremepresentation.typepad.com/blog/2006/09/choosing_a_good.html
50
 Frameworks

D3.js (https://d3js.org/)
HTML, CSS, SVG
https://observablehq.com/@d3/gallery
Chat.js

https://en.wikipedia.org/wiki/Comparison_of_JavaScript_charting_libraries

51
 And it goes on…

more examples, theory, …

cf. lecture on Information Visualization by Prof. Butz

52
Evaluation
 General (Ellis & Dix, 2006)

• Explorative
• Qualitative
• expert interviews
• semi-structured interviews
• insightful findings
• Iterative

54
Evaluated LA dashboards (Jivet, 2018)

55
Used evaluation techniques (Jivet, 2018)

56
 Evaluation recommendations for LA dashboards
(Jivet, 2018)

1. Dashboard evaluation should focus (primarily) on whether its goals are fulfilled,
(secondarily) on the impact on learners’ affect and motivation, and (finally) on the usability
of the tool.

2. The evaluation of a tool’s usability and usefulness should not be limited to whether users
find the tool usable and useful, but in order to build trust and confidence in learning
analytics tools, it should also assess whether learners understand the data, how much they
agree with it and how they interpret it.

3. Dashboard evaluation should use data triangulation to validate its effects with self-reported
data, tracked data as well as assessment data.

4. The evaluation should include an assessment of the design features that rely on educational
concepts.

5. Validated measurement instruments should be used to assess if the dashboard had any
impact on the learner or if learner characteristics as measured through these instruments
play a role in how learners perceive the dashboard and how they respond to it.

57
 Evaluation Framework for Learning Analytics
(Scheffel, 2017)

Full questionnaire & evaluation steps: Scheffel, 2017: p. 138 58

Wrap up
Wrap up

60
 Next Lecture

• Learner Modelling
• Recommender Systems

61
Prof. Dr. Sven Strickroth
Ludwig-Maximilians-Universität München
Institut für Informatik
Lehr- und Forschungseinheit für
Programmier- und Modellierungssprachen
Oettingenstraße 67
80538 München

Telefon: +49-89-2180-9300
sven.strickroth@ifi.lmu.de

62
 References & Further Reading

• Schwendimann, B. A. et al. (2016). Understanding learning at a glance: An overview of learning

dashboard studies. In Proceedings of the sixth international conference on learning analytics &
knowledge (pp. 532-533).
• Jivet, I., Scheffel, M., Specht, M., & Drachsler, H. (2018). License to evaluate: Preparing
learning analytics dashboards for educational practice. In Proceedings of the 8th international
conference on learning analytics and knowledge (pp. 31-40).
• Beuster, L., Elkina, M., Fortenbacher, A., Kappe, L., Merceron, A., Pursian, A., ... & Wenzlaff, B.
(2013). Learning analytics und visualisierung mit dem lemo-tool. DeLFI 2013: Die 11 e-Learning
Fachtagung Informatik.
• Asif, R., Merceron, A., Ali, S. A., & Haider, N. G. (2017). Analyzing undergraduate students'
performance using educational data mining. Computers & Education 113, 177-194.
• Strickroth, S., & Pinkwart, N. (2014). Engineering Hybrid Learning Communities: The Case of a
Regional Parent Community. Systems, 2(4), 393-424.

• Ellis, G. & Dix, A, (2006). An explorative analysis of user evaluation studies in information
visualisation. In Proceedings of the 2006 Conference on Beyond Time and Errors: Novel
Evaluation Methods For information Visualization. BELIV '06. ACM Press, New York, NY, 1-7.
• Scheffel, M. (2017). The evaluation framework for learning analytics. Heerlen, The Netherlands:
Open Universiteit.

• https://en.wikipedia.org/wiki/Data_visualization#Techniques

63