Syddansk Universitet

Datadriven design for learning

Chongtay, Rocio; Johnsen, Lars

Publication date:
2014

Document Version
Early version, also known as pre-print

Link to publication

Citation for pulished version (APA):

Chongtay, R., & Johnsen, L. (2014). Datadriven design for learning: multimodality as a service. Abstract from
Designs for Learning, Stockholm, Sweden.

General rights
Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners
and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.

• Users may download and print one copy of any publication from the public portal for the purpose of private study or research.
• You may not further distribute the material or use it for any profit-making activity or commercial gain
• You may freely distribute the URL identifying the publication in the public portal ?
Take down policy
If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately
and investigate your claim.

Download date: 09. jun.. 2016

Category:
research
study
in
progress


Title
of
proposal:
Data‐driven
design
for
learning:
multimodality
as
a
service


Author(s):
Rocio
Chongtay
and
Lars
Johnsen.


Department
of
Design
and
Communication,
The
University
of
Southern
Denmark,
Engstien
1,
DK‐
6000,
Kolding,
Denmark.
{rocio,
larsjo}@sdu.dk


Keywords:
Data‐driven
design
for
learning;
semantic
web;
linked
data;
multimodality;
design
as
a

service;
semiotic
enzymes.




Research
questions,
project
aims
and
theoretical
framework


This
paper
explores
issues
related
to
data‐driven
design
for
learning,
i.e.
the
use
of
structured
data

in
digital
design
representations
for
learning,
especially
so‐called
linked
data
exposed
on
the

Semantic
Web
(also
known
as
Web
3.0
or
the
Web
of
Data).
It
describes
ongoing
work
attempting

to
develop
conceptual
as
well
as
technological
tools
for
data‐driven
designs
for
learning
and,
in

due
course,
to
investigate
the
usefulness
of
such
tools
in
concrete
learning
materials
and
in
actual

learning
contexts1.



The
work
is
underpinned
by
the
idea
of
trying
to
adopt
central
notions
and
principles
from
social

semiotics
and
multimodality
(Bezemer
and
Kress
2008;
Van
Leeuwen
2005)
and
multimedia

learning
theory
(Mayer,
2009)
in
design
approaches
based
on
methods
and
technologies
of
the

Semantic
Web.
Thus,
more
narrowly
it
seeks
to
examine
to
what
extent
the
organization
of
virtual

learning
content
can
be
made
"semiotically
explicit"
and
how
this
information
may
be
exposed
on

the
Web
of
Data
to
enhance
the
discoverability,
reusability,
accessibility
and
adaptability
of
the

learning
content.



An
important
aspect
of
the
project
concerns
the
concepts
of
mobility
and
mobile
learning
content

(O’Malley
et
al.,
2005).
In
the
present
context,
truly
mobile
learning
content
is
defined
as
learning

resources
that
can
not
only
be
aptly
delivered
on
mobile
devices
like
smartphones
and
tablets
but
































































1

The
work
is
part
of
a
larger
project
recently
begun
at
the
Department
of
Design
and

Communication
at
the
University
of
Southern
Denmark
entitled
"Mobile
Designs
for
Learning

(MD4L)".


also
be
moved
across
media,
materials
and
semiotic
modes
(text,
graphics,
layout,
typography,

colour,
etc.)
for
purposes
of
learning
and
teaching.


One
central
research
question
of
the
project
focuses
on
multimodal
meaning
making
and

remaking
in
data‐driven
information
spaces
and
more
specifically
on
how
data‐driven
approaches

may
facilitate
greater
multimodal
richness
and
variability
in
learning
designs.


Multimodality
is
often
conceived
of
as
a
positive
quality
in
learning
materials
for
a
number
of

reasons:
it
may
support
different
learning
styles
among
the
intended
audience
(Stull
and
Mayer

2007);
it
may
provide
visual
scaffolding
for
the
comprehension
of
text
(Mayer
2009);
and
it
may

offer
divergent
perspectives
on
subject
matter
or
convey
different
levels
of
details
in
the

presentation.
Up
till
now
multimodality
has
most
often
been
"hard‐wired"
into
digital
learning

materials
through
the
inclusion
of
(moving)
images,
diagrams
and
graphics
and
the
application
of

fixed
style
sheets.
Increasingly,
much
multimodality
is
created
via
interactive
functionality
enabling

learners
to
customize
various
aspects
of
the
content
and
the
user
interface
(viewing
more
or
less

content,
setting
layout
and
typography
parameters
and
so
on).
Dynamic
multimodality
done
this

way,
however,
is
costly
and
is
still
fixed
in
the
sense
that
it
is
constrained
by
a
single
programmatic

design.


In
the
project,
we
propose
what
we
think
is
a
fairly
novel
approach
to
the
construction
of
certain

forms
of
multimodality
in
learning
resources.
It
hinges
on
the
concepts
of
"multimodal
design
as
a

service"
and
"semiotic
enzymes".
The
underlying
idea
is
that
learning
materials,
enriched
and

encoded
using
a
common
metadata
vocabulary
as
the
one
mentioned
below,
may
freely
be
linked

to
so‐called
semantic
widgets,
web‐based
or
local
software
modules
providing
means
to
convey

the
content
in
different
ways
(Haller
et
al.,
2011).
The
embedded
meta(data)
in
the
content
thus

functions
as
a
kind
of
latent
enzymes
ready
to
trigger
transformation
processes
of
various
sorts.




The
vision
is
that
subject
matter
experts
like
teachers
and
professors
can
"plug
in"
design,
so
to

speak.
They
can
concentrate
on
subject
matter
and
content
structure
and
less
on
design
choices

and
learners
can
have
a
greater
say
in
what
design
options
they
want
for
the
materials
they
study

(visual
support,
interactivity,
etc.).



The
first
part
of
the
project
has
been
to
adopt
and
test
a
vocabulary
for
describing,
structuring
and

encoding
digital
content
with
a
view
to
its
semiotic
potential
as
learning
material.
The
vocabulary

allows
learning
resources
to
be
modelled
and
represented
on
the
basis
of
their
constituent

multimedia
objects,
their
subject
matter
and
any
external
learning
model
or
taxonomy
to
which

they
may
be
aligned.
Using
these
dimensions,
educational
technologists
and
designers
can
add,

and
embed,
relevant
metadata
intended
to
reflect
semiotic
categories
like
medium,
ideational
and

textual
meaning,
genre
and
so
on
(Vorvilas
et
al.,
2011).
The
vocabulary,
maintained
at

schema.org,
comprises
a
set
of
types,
properties
and
relations
used
by
the
major
search
engines

including
Google's,
to
optimize
discoverability
and
search
results.
Also,
schema.org
elements
are

compatible
with
current
semantic
web
technologies
such
as
Microdata,
RDFa
and
Json‐LD.

The
vocabulary
is
extensible,
meaning
that
its
descriptive
categories
can
be
extended
or

specialized
for
specific
domains.
Further,
it
allows
for
layering
learning
architecture.
For
example,

an
educational
video
may
be
specified
as
being
about
a
certain
topic
‐
person,
event
or
place,
say.

This
topic
may
in
turn
be
described
in
terms
of
its
relations
to
other
topics
or
be
semantically

linked
to
external
resources
such
as
open
data
stored
in
Wikidata.









Currently,
we
are
exploring
how
semantic
widgets
may
be
employed
to
transform
educational

texts
marked
up
using
schema.org
metadata.
We
are
experimenting
with
semantic
widgets
to

carry
out
transduction,
i.e.
moving
meanings
from
one
mode,
or
genre,
to
another.
Here
we
focus

on
the
generation
of
graphic
organizers,
synoptic
visual
genres
like
info
boxes,
time
lines
and

concept
maps,
to
facilitate
text
comprehension.
Prior
to
this
work
we
have
tried
to
describe

semantic
widgets
in
more
theoretical
terms
(Johnsen
&
Chongtay,
in
press).


In
the
project,
we
are
also
looking
into
ways
of
marking
up,
in
a
standardized
way,
visual

components
in
genres
like
geographical
maps,
drawings
or
concept
maps
making
them
amenable

to
processing
by
semantic
widgets
and
to
enable
them
to
be
linked
to
external
resources

elsewhere
on
the
web.



Methods/methodology


The
project
is
first
and
foremost
an
analytic
"desktop"
endeavour
ending,
hopefully,
in
a
set
of

technological
proofs
of
concept
demonstrating
the
potential
of
our
ideas
for
real‐world

application.
These
proofs
of
concepts
will
in
the
first
instance
be
based
on
imagined
use
case

scenarios.




However,
we
also
wish
to
empirically
test
some
of
these
proofs
of
concept
in
follow‐up
studies
"in

the
field".
These
are
yet
to
be
planned
in
detail,
though.
Right
now,
the
following
possibilities
are

being
considered:



One
idea
is
to
assess
the
meaningfulness
of
the
metadata
vocabulary
to
learners
in
mobile
Web

2.0
learning
scenarios.
The
plan
is
to
test
if,
or
to
what
extent,
learners
can
utilize
the
vocabulary

when
creating,
tagging,
linking
and
uploading
content
in
specific
learning
contexts,
and
once

uploaded
how
this
content
may
exposed
on
the
Web
of
Data.



Another
plan
is
to
investigate
the
degree
to
which
existing
learning
materials
can
be
"semiotically

enriched"
using
the
vocabulary
and
how
this
semiotic
enrichment
may
be
put
to
use
in
authentic

contexts
through
the
application
of
pedagogically
designed
semantic
widgets
in
order
to
enhance

accessibility
and
adaptability
and
hence
learnability.

A
third
idea
is
to
explore
opportunities
and
constraints
in
using
open
data
from
repositories
such

as
Wikidata
as
the
primary
basis
for
the
generation
of
multilingual,
multimodal
resources
for

learning
for
specific
domains
or
audiences.


Expected
results


In
general,
it
is
hoped
that
the
project
will
provide
insights
into
how
we
may
go
about
constructing

data‐driven
designs
for
learning
and,
equally
importantly,
what
issues
are
involved.
In
more

practical
terms,
we
hope
to
be
able
to
present
a
conceptual
framework
for
building
data‐driven

learning
materials,
including
mobile
ones,
which
may
be
communicated
to
teachers,
designers
and

students
alike
in
a
set
of
accessible
guidelines,
as
well
as
a
set
of
software
tools
to
support
this

goal.


References


Bezemer,
J.
&
Kress,
G.,
2008.
Writing
in
Multimodal
Texts:
A
Social
Semiotic
Account
of
Designs

for
Learning.
Written
Communication
25,
2,
166‐195.


Haller,
A.,
Groza,
T.,
&
Rosenberg,
F.
Interacting
with
Linked
Data
via
Semantically
Annotated

Widgets.
In
Proceedings
of
JIST,
pages
300‐317,
2011.



Johnsen
&
Chongtay
(2014):
Defining
and
exemplifying
semantic
widgets
for
learning.
(In
press).



Mayer,
R.E.,
2009.
Multimedia
Learning.
Cambridge
University
Press.
2nd
edition.


O’Malley,
C.,
Vavoula,
G.,
Glew,
J.,
Taylor,
J.,
Sharples,
M.,
&
Lefrere,
P.,
2005.
MOBILearn
WP4
‐

Pedagogical
methodologies
and
paradigms:
Guidelines
for
Learning/Teaching/Tutoring
in
a
Mobile

Environment,
No.
4.




Stull,
A.T
&
Mayer,
R.E.,
2007.
Learning
by
Doing
Versus
Learning
by
Viewing:
Three
Experimental

Comparisons
of
Learner‐Generated
Versus
Author‐Provided
Graphic
Organizers.
Journal
of

Educational
Psychology,
Vol.
99,
No.
4,
808–820.


Van
Leeuwen,
T.,
2005.
Introducing
Social
Semiotics.
London:
Routledge


Vorvilas,
G.
&
Thanassis,
K.
&
Ravanis,
K.,
2011.
A
Genre‐based
Framework
for
Constructing

Content
for
Learning
Objects.
Asian
Journal
Of
Computer
Science
And
Information
Technology
1:1.