You are on page 1of 59

Media and Mind

Mid-quarter Recap
Steen & DeLiema - Winter 2015

Transformation through
metacognition

The thrust of this course has been to study the mind
through a blend of cognitive science, attention to
experience, and media

Conventions in media help us understand our own
minds and studying our own minds helps us
understand media

“To transform the world, there must be regeneration
within ourselves” (Krishnamurti)

Complex multimodal
cognition

Because cognition ropes together so many resources—five
modalities, imagination, emotion, narrative—the complexity of
the system prevents once-and-for-all solutions to hard problems

Consciousness itself is designed to react to a changing external
world—very little of the dynamics of our internal organs are
experienced; they run automatically; consciousness ignores
routine

Consciousness, then, is geared toward novelty, toward change

Having simple instructions on how to conduct one’s life are
simply impossible

Perception

Think of the medium as the message in terms of the evolution
of the senses (close your eyes and as you slowly open them,
try to understand the power of visual experience)

How does the medium of the senses (sight, smell, sound, etc.)
change the message (scale or pace of human life) regardless of
the content (what is actually seen)? You all argued:

Division between self and external world

More urgency to act

Caught up in the future

Perception continued

The eye takes in light… (Plato’s extromission model, Al
Haytham’s star model, current neuroscience models)

The brain processes that light…

And only then do we become aware of the external world

The experience of seeing—the sights of all the things
around us—are 3D moving pictures produced in the brain

All media rest on this core conscious infrastructure

“…rendering the world before them onto that curved sheet of paper,
in fact of tracing the world onto that page freehand…” (caption about
the Oakes twins’ artwork at the Museum of Mathematics in NYC)

Imagination

Imagination is also geared toward novelty and
change (can’t imagine a still apple for more than
a few beats)

Imagination best handles vision and sound (the
two senses most developed in media)

Imagination can render new scenes, create new
realities

Impermanence of imagination

“If one looks at the surface on which this book is
held-perhaps your hands, or a lamp-lit table, one will
find that now...seven seconds later...the thing still
sustains itself. If one instead imagines a lamp-lit table
in some distant room, it is probably now... seven
seconds later...already beginning to become lost to
you, and can be held onto only by a palpable labor of
sustaining it” (Scarry, 1995, p. 19)

Technological revolutions

Technology—in trains, telegraphs, and photography—
begin to erase space and time

These technologies are extensions of our embodied
faculties (e.g. bodies in trains, hearing in the telegraph,
and eyesight in photography)

These technologies (media) allow people to come
together, to experience the same event, to become
displaced in time, etc. (messages) in powerful ways
regardless of what these technologies show (the content)

Picasso and Jay Mark
Johnson

Communication as simulation

The upshot is that communication is an act of
simulation

Sometimes these simulations are imagined, as in the
case of the recursion diagrams.

Communication as simulation

The upshot is that communication is an act of
simulation

Sometimes these simulations are imagined, as in the
case of the recursion diagram.

Or…in the case of what Tim Roth’s character
experiences in Reservoir Dogs while telling a story

Commode Story - Reservoir Dogs
1

3

6

2

4

5

7

8

Communication as simulation

And other times these simulations are give to viewers
in the format of perception, as in the case of the march
from Selma to Montgomery

Power of television

Television has much of the power of imagination—
editing, displacing time, showing multiple
perspectives, slowing down events, speeding up
events, etc.—but it presents these experiences in the
format of perception

Television is (some else’s) imagination delivered to
any number of audience members in the format of
perception

We respond as if we are seeing for ourselves

Bearing witness?

Manohla Dargis, New York Times film critic, describes this process in the
following way:

“A film image is created by light that leaves a material trace of something that
exists — existed — in real time and space. It’s in this sense that film becomes a
witness to our existence” (September, 2012)

Mark Leibovich recently critiqued the idea of bearing witness:

“In today’s social media- driven news cycle, however, “bearing witness” has
largely become an easy rhetorical snack food — a mass-market trope available
to anyone who wants to weigh in, or show up.

Instead, all that matters is that something happened, that someone identified it
and that the principals are to be congratulated for talking about it or seeing it up
close. That’s why we have news cycles — to bear witness in a circular
blur.” (September, 2014)

Viewpoint in simulation
“Viewpoint permeates human cognition and communication - predictably, since
we never have experience of the world except as a viewpoint-equipped embodied
self among other viewpointed embodied selves” (Sweetser, 2011, Viewpoint
Volume)
1. Perceived viewpoint
2. Imagined and remembered viewpoints
3. Distant perceived viewpoint
4. Gestural viewpoint
5. Linguistic viewpoint

Andrew Wyeth, Cristina’s World

Viewpoint in simulation
1. Perceived viewpoint
Where you are sitting relative to others?
How are you listening? With headphones on? With some
eavesdropping?
How are you moving? Standing still in a rally? Moving with everyone?
!

Viewpoint in simulation
2. Imagined and remembered viewpoint
Memories from the field or observer perspective? Are you seeing
yourself in the mental simulation (observer perspective) or are you
seeing the scene from your own eyes (field memory)? (See Nigro and
Neisser, 1983)
Observer memories tied to objective; distant past; inauthentic
feelings; life-changing self-concept, self-awareness, social phobia,
trauma
Field memories tied to emotional; recent; authentic feelings; same
self-concept
Another possibility we could consider is the intelligibility of the scene
(which angle is most informative)

Viewpoint in simulation
3. Distant perceived viewpoint (television = distant +
seeing)
the visual angle on the scene…
the distance between subject(s) and the camera…
the movement of the camera…
the focus of the camera…
the person from whose eyes the camera looks out…

(1) Angle - Overview
ANGLE ON SCENE IS FROM ABOVE

(1) Angle - Side-view
ANGLE ON SCENE IS FROM THE SIDE OR BELOW

(1) Angle - Side-view
SCENE FROM CITIZEN KANE

FLOORBOARDS REMOVED
TO FILM SCENE

(1) Mixed overview and sideview

(1) Mixed overview and sideview
Mixed angles (from
Gone With the
Wind)

(2) Distance
DISTANCE BETWEEN CAMERA AND SUBJECT(S) (COULD
BE OVERVIEW OR SIDE-VIEW)

(3) Movement - Overview
WAYS TO MOVE THE CAMERA FROM THE OVERVIEW ANGLE

(3) Movement - Side-view
WAYS TO MOVE THE CAMERA FROM THE SIDE-VIEW ANGLE

(3) Movement - Side-view
Steadicam (from The
Shining)

Handheld (from Mean
Streets)

(4) FOCUS

(4) FOCUS
DOLLY ZOOM FROM JAWS

(4) Whose eyes?

other options?

(4) Whose eyes?
POV (from
Halloween)

Political implications
Distant perspective in coverage of the Iraq War (Schwalbe, 2006)
Conflict in Misurata, Libya during the Arab Spring
Refugees in war-torn Congo
Decisions juries make in response to camera angles in police
interrogations

Entertainment
Viewpoint switching happen fast—evidenced by our
analysis of the opening sequence to Finding Nemo
1

2

4

5

7

8

3

6

9

10

Viewpoint in simulation
4. Gestural viewpoint
Character versus observer viewpoint (Parrill, 2009)

Kleinrock on packet
switching

A=C@13
23AB7</B7=<
=@23@

BVS^OQYSba`SQSWdSbOUabVObb`OQYa]c`QS
RSabW\ObW]\O\R]`RS`

BVS^OQYSba[Ogb`OdSZOZ]\URWTTS`S\b\Sbe]`Y
^ObVeOga

Participant viewpoint
Science
Math education!
education!
Historical examples!
Narrative!
When
A
middle
Dina
schooler
explains,
performs
“I
didn't
the
really
motion
want
of
to
Einstein on the train,
McNeill (1996) on cartoons,
stay
unzipping
at the
his of
human
skin
and
stepping
et
1998)
out,
McClintock
as bottom,”
part
the (Nemirovsky
Young
(2002)
onal.,
dreams,
!
genes,!
&
(2004)
on
while
the “I”exclaiming,
refers both"Do
to herself
IParrill
change
inSweetser
the
my physical
skin
like
Salkthis,
on becoming
the
virus,!
computer
science,!
world
and
vloop,
to the
vloop?"
virtual(Warren
entity inet
the
al.,graphing
2001)!
Ochs’ studies of domainParrill (2009) on cartoon
world
state physicists

storytelling

Science education!
Math education!
Dream
state!
Barbara
McClintock!
Complex systems(Wilensky
Children graphing
Physicists!
“...When
makes
the
gesture
of I
“...when
I wasVictoria
reallySalk!
working
with
[genes],
& Resnick,
1999),!
(Nemirovsky
et
al.,
1998),
“...[Scientists] also express involvement more
cupping
her(Warren
ear
as
she
says,
she of
is
Metamorphosis
et
“...I
wasn't
would
outside,
also
ask
I was
questions
down
there.
as“Iifheard”,
interrogating
I was part
Teachers’
algebra
extremely
by taking curriculum
the perspective
of
al.,
2000),
Solar
systems
(Hall,
1996),
instantiating
her
own
body
an
iconic
the
myself:
system.
"Why
I was
would
right
I do
down
that
there
if as
I were
with
a them,
virus
(empathizing
object being
(Crowder,
1996), Physicswith) some
math diagrams
(de Freitas
representation
ofgot
a cell?”
character
in
thein
taleworld”
and
everything
or
a
cancer
big.”
(Salk,
(as
cited
1983)
Keller,
(diSessa, 1993)!
Sinclair,
2012)
analyzed” (Ochs& et
al., 1996)

(Young,
1984)2002)

Immersing in forests

Eduardo Kohn (2014)

“Vultures, because of their
“In
their
interactions
with
“Some
people
in
Avila
species-specific habitats and
“One man
took
delight
in
animals,
the
Runa,
in
many
dispositions,
inhabit
apuma
different
become
runs
by
“People
in
Avila
try
to
make
explaining
to me
how thethe
giant
ways,
try
to
emulate
world
from
that
of thebile;
Runa.
drinking
jaguar
this
sense
of
these
various
selves
anteater
adopts
the
perspective
anteater.
Theysubjective
attempt to
Yet
because
their
that
inhabit
the
forest
by them;
trying
helps
them
adopt
a
ofcapture
ants
in order
to
fool
the
perspective
of
point
of view
issee,
that
of by
to
see
how
they
and
predatory
point
of
view,
when
the
anteater
sticks
its
another
organism
as
part
of a
persons,
they
see
this
different
imagining
how
different
tongue
into
any
nests,
ants
and
it
facilitates
thethe
passage
larger
whole…People
in
Avila
world in the interact”
same way(p.
the
perspectives
96)
see
it
as
a
branch
and,
of
their
souls
into
the
take
great
pleasure
in
finding
Runa see their own world” (p. a
unsuspecting,
climb
on”when
(p. 96)
viewpoint
that
encompasses
bodies of
jaguars
95)
multiple
perspectives”
(p. 97)
they
die” (p. 107)

Truth and viewpoint
Meaning in the OkGo music video (aptly titled, “The
Writing’s On The Wall”) is revealed only by seeing the
scene from a single point of departure

In contrast, the meaning of a park
in London can only be understood
by factoring in its history and the
viewpoints of all who experience
the park

Play and pedagogy

Play in childhood offers the opportunity to simulate perennially
intractable problems—repeating problems that do not have once
and for all solutions

Play takes advantage of multi perspective cognition—in chase
play, sometimes you run as the fleer and other times as the chaser

Play gives the chance to experience new points of view and can
open up channels of empathy (think of violent youth in the Last
Chance in Texas)

Play can also investigate darker human experiences (think of the
children from Afghanistan depicting a suicide bombing)

Combinatorics

Part of the reason
we need play is
because there are so
many possible
courses of action
for actors with lots
of resources

The state space of
possible action is
immense

Problem solving

Running simulations of perennially intractable
problems offers the chance to find effective solutions

How do we know what counts as an effective
solution?

Human cognition rests on a narrative infrastructure

Narrative

One approach to understanding narrative structure:

Actors

Goals

Resources

Obstacles

Strategies

Importantly, narrative guides/constrains the search for good outcomes in
enormous state spaces of possible action

Without narrative, the search could be paralyzing; we wouldn’t know what
counted as success or failure

Narrative at multiple levels

These narratives intersect at different levels of experience:

Students might have their future careers in mind at the reallife level, their GPA in mind at the school level, or even the
experience of a photon or division sign at the virtual level

Video gamers may have their hunger or homework in mind
at the real-life level, their points in the game in mind at the
gamer level, or the raw experience of jumping Mario in
mind at the virtual level

These levels interact; decisions at one level modulate
decisions at another level

Affordances

Resources afford courses of action

A paperwork is bendable, throwable, pokeable, etc.

Which of these affordances the agent sees is dependent on his/
her goals, such that…

if you want to reach something behind a counter, you may
bend the paperclip

if you want to lend a paperclip to a friend, you may throw it
to him

if you want to hurt someone, you might bend and then poke

Affordances

The number of affordances in a tree are vast and which
you see depends on your goals and your abilities.

Think of The Giving Tree

Past, present, future

When things go wrong, we often investigate why

We attribute a moment of failure in the present to a range of factors
in the past. Are those past factors something internal/external,
controllable/uncontrollable, and stable/unstable?

How we explore the past defines what we think will be possible,
likely, and valuable in the future? These exploration of the past and
our predictions about the future organize our actions in the here and
now.

News and film explore these possibility spaces in areas of
perennially intractable problems (dating, war, violence, resource
management, etc.)

Time

Possible
Past
attributions
of failure (e.g.
what do I
blame?)

Failure
in the here
and now

Likely
Valuable

Attribution Theory

What do we credit for our failures and successes? In
other words, an outcome in your life happened because
_________________.

The capacity to reflect back on the past and assess what
caused a given outcome rests on the technology of the
imagination, roots itself in narrative structure, and
involves the search for affordances.

We ask ourselves: Why did this happen? What could
have happened that would have lead to a different
outcome? And how do our answers to those questions
affect what we try next?

We credit outcomes to causes. For instance, you
might credit “lack of sleep” or “mean professors” or
“hard tests” to a bad grade.

Each of these causal factors can be understood
along three dimensions:

Locus: Is this factor in you or in the external
world?

Stability: Will this factor persist or change?

Controllability: Can you control this factor?

5 stages of causal reasoning
1. identifying that you're not reaching your goal
(evidence)
2. explaining why you don't reach your goal (explanation)
3. determining where you made a mistake (causal
surgery)
4. determining where you can intervene (agency - what
becomes responsibility and blame in the media)
5. planning

Veil of Ignorance
John Rawls!
You have to plan a society where…

“...no one knows his place in society, his class
position or social status; nor does he know his
fortune in the distribution of natural assets and
abilities, his intelligence and strength, and the
like.”