Wanda Strauven (Amsterdam)

The Archaeology of the Touch Screen

On October 16, 2011, the online version of the Dutch daily paper NRC Handelsblad
posted a link to the YouTube clip »A Magazine is an iPad that does not Work«, whe-
re we see a one-year old girl interacting with both an electronic tablet and a paper
magazine.1 To her frustration, the magazine is not responding to her touch com-
mands. But she perfectly understands how the iPad works … In Dutch we use the
expression »dat is kinderspel« (that’s child’s play) to say that something is very easy
to do, so easy that even children can do it. The YouTube clip very literally illustrates
this saying regarding new media. Technology is not just becoming user-friendlier, but
also truly infant-friendly, by developing an intuitive, almost natural language. But the
video also shows the technophobes among us the drawback of such a development:
the next generation will no longer be able to read ›normal‹ papers (i. e. texts printed
on paper).
The idea of adolescents being ahead of their time and surpassing the generation
of their parents in technological terms is nothing new. It is a recurring commonplace
or topos of our media history, as media-archaeologist Erkki Huhtamo would put it.2

This text was originally delivered as a lecture at the Institut für Theater-, Film- und Medien-
wissenschaft der Universität Wien on January 12, 2012. Some passages are taken from my article
»Early cinema’s touch(able) screens: From Uncle Josh to Ali Barbouyou«, published later that
year in the online journal of the European Network for Cinema Studies, NECSUS, 2/2012:
http://www.necsus-ejms.org/early-cinemas-touchable-screens-from-uncle-josh-to-ali-bar-
bouyou/.
1 http://www.nrc.nl/nieuws/2011/10/16/video-baby-snapt-wel-een-ipad-maar-niet-een-tijdschrift/.
I would like to thank Markus Stauff for having drawn my attention to this video. The YouTube clip
was uploaded on October 6, 2011. It provoked a lot of responses, many of which doubted the true
»digital« intuition of the baby (or the »OS upgrade«, as the maker and father of the baby calls it).
See also part 2 of the clip: http://www.youtube.com/watch?v=kJZSLvTK4pw.
2 On the notion of »topos«, see Erkki Huhtamo, »From kaleidoscomaniac to cybernerd: Towards
an archaeology of the media«, in: Electronic Culture: Technology and Visual Representation, ed. by
Timothy Druckrey, New York: Aperture 1996, p. 296–303, p. 425ff.

Brought to you by | Universiteit van Amsterdam - UVA Universiteitsbibliotheek SZ

Authenticated
Download Date | 2/15/18 4:00 PM
70 wanda strauven

From the 19th-century optical toys to video

games, the smartphone and the tablet, eve-
ry time a new technological device is int-
roduced, children are not only portrayed
as potential new customers, but they also
become immediately very skillful users – a
tendency which always worries the conser-
vatives.
Here I would like to remind us that,
before the introduction of the iPhone and
the iPad, there was a lot of concern about
the mutation of the thumb that young
mobile phone users were manifesting due to
Fig. 1: Source: NY Daily News, October 13, 2011: their very skillful and very addictive use of
http://www.nydailynews.com/life-style/baby-doesn- the communication protocol SMS.3 The his-
understand-magazine-concept-manipulate-ipad-
tory of the Short Message Service (SMS) is
article-1.962945
a story à part. It was initially conceived of as
a one-way communication system (limited
to 160 characters) that call centers could use to send short messages such as voice
mail notification to their subscribers. The industry could never have foreseen the
immense success that this ›unsexy‹ protocol was to have, especially among young
people, as a point-to-point application.4 SMS turned phone-call receivers into text-
message senders; or, even better, it turned the telephone (transmission of the spoken
word) back into a telegraph (transmission of the written word). At the beginning of
the 21st century one could therefore claim that the cell phone belonged to the Kittle-
rian genealogy of writing systems, from Morse’s telegraph to Remington’s typewriter,
to Edison’s phonograph and the cinematograph.5 Today, however, this picture has
changed. The cell phone has become a screen, that is, a surface to be looked at and to
be touched, and is therefore to be inscribed in a genealogy of screens. Of course, the
smartphone still allows us to make phone calls and send text messages, but these ope-
rations have become somehow secondary. We now use the cell phone to take pictures
and to make short clips of our child, who in turn wants to see the result immediately

3 See for instance the following article in a Belgian newspaper: »Mobiele telefoon doet duim mute-
ren«, Gazet van Antwerpen, March 24, 2002.
4 Alex S. Taylor and Jane Vincent, »An SMS history«, in: Mobile World: Past, Present and Future, ed.
by Lynne Hamill, Amparo Lasen and Dan Diaper, London: Springer 2005, p. 75–91, here p. 79.
5 Friedrich Kittler, Gramophone, Film, Typewriter, Stanford: Stanford University Press 1999.

Brought to you by | Universiteit van Amsterdam - UVA Universiteitsbibliotheek SZ

Authenticated
Download Date | 2/15/18 4:00 PM
 The Archaeology of the Touch Screen
71
on the very same device and who very intuitively uses the pinching gesture to zoom
in or zoom out. It is this fundamental link between seeing and touching that I am
concerned with and that I am exploring in my media-archaeological project of the
touch screen, of which this essay is a part.
As a film scholar, I became interested in the touch screen through my fascination
with the so-called rube films of early cinema. For many years now this specific genre
of early cinema has been haunting me, one might say. I frequently come back to them
in both my teaching and my research. What are these rube films about? The genre
consists in depicting the naive attitude of country folks visiting the city. My specific
interest lies in a subgenre of the rube film, which takes place at the movie theater,
proposing thus an early form of self-reflexivity. In terms of the touch screen, the
rube film that intrigues me most is Edwin S. Porter’s Uncle Josh at the Moving Picture
Show (US 1902).6 This film shows Uncle Josh at a vaudeville theater where he gets to
see three moving pictures: Parisian
Dancer, Black Diamond Express and
The Country Couple. The first two
moving pictures are pre-existing
Edison films made in 1897, whe-
reas the third one was most likely
shot for the occasion. Taking place
in the countryside, it is a courtship
scene between another rube and
a young girl, whom Uncle Josh
takes for his daughter. He decides
Fig. 2: Edwin Porter, Uncle Josh at the Moving Picture
to intervene and jumps on stage to
Show (1902)
punish the man on the screen. The
screen is touched, and torn down.
Following Thomas Elsaesser, on several occasions I have read this specific rube
film not as a lesson in cine-literacy (»don’t touch the screen«), but as a form of inter-
nal (or textual) disciplining: thanks to Uncle Josh’s ›unsophisticated‹ attitude, the
spectator’s attention is drawn to the screen, that is, to what happens on the screen.7

6 Porter made this film for the Edison Company. It is a remake of an earlier British rube film by
Robert W. Paul: The Countryman and the Cinematograph (1901).
7 See Thomas Elsaesser, Filmgeschichte und frühes Kino. Archäologie eines Medienwandels, München:
edition text + kritik 2002; Wanda Strauven, »Touch, don’t look«, in: The Five Senses of Cinema, ed.
by Alice Autelitano, Veronica Innocenti and Valentina Re, Udine: Forum 2005, p. 283–291; and
Wanda Strauven, »The Observer’s Dilemma: To Touch or Not to Touch«, in: Media Archaeology:

Brought to you by | Universiteit van Amsterdam - UVA Universiteitsbibliotheek SZ

Authenticated
Download Date | 2/15/18 4:00 PM
72 wanda strauven

In other words, it is a remedy against the distraction that characterized early shows
of moving pictures. But even if we are not reading it as a counterexample of the »look
but don’t touch« rule, Porter’s rube film remains a very overt thematisation of the
relation (or tension) between seeing and touching: because, by touching the screen,
the image disappears.8
Whether or not Uncle Josh at the Moving Picture Show was successful as a discip-
lining instance (in attracting attention to the screen and provoking laughter) is dif-
ficult to verify. It equally produced a mimetic impulse in the spectator. According to
Leonardo da Vinci, the incitement of such an impulse was a sign of a good artwork.
In Trattato della Pittura one can read:

One painter made such a figure that whoever saw it immediately yawned and kept
repeating this accident as long as his eyes were on the painting which, like him, feigned
someone yawning. Others have painted libidinous acts, and so much lewdness that
[the paintings] have incited spectators to the same celebration. This, poetry will not
do.9

For directly involving the spectator in an (spontaneous) act of imitation, Leonardo

considers the art of painting superior to poetry. Accordingly, with his own drawing
Five Grotesque Heads, the great Renaissance master intended not only to show dif-
ferent stages of laughter, but also to »engender laughter in the beholder«.10 In sum, a
good artwork is contagious!
This is precisely what happens in Porter’s film when Uncle Josh is watching the
first moving picture of the female dancer. He jumps on stage to dance with her, or
rather to imitate her dancing movements. The film-within-the film might become
infectious to the external spectator as well, making him or her want to dance with (or
like) Uncle Josh. And if we push this reasoning further and apply it also to the third
moving picture, in which Uncle Josh grabs the diegetic film screen and eventually

Approaches, Applications, and Implications, ed. by Erkki Huhtamo, Jussi Parikka, Berkeley/Los An-
geles: University of California Press 2011, p. 148–163.
8 This is due to the principle of rear-projection, which I discuss more in detail in »Early cinema’s
touch(able) screens«.
9 Leonardo da Vinci, Paragone: A Comparison of the Arts, translated by I. A. Richter, London/New
York/Toronto: Oxford University Press 1949, p. 66. I would like to thank Caroline van Eck for
having suggested looking into Leonardo da Vinci’s writings for this reading.
10 Patricia Trutty-Coohill, »Comic rhythms in Leonardo da Vinci«, in: Enjoyment: From Laughter
to Delight in Philosophy, Literature, the Fine Arts, and Aesthetics, ed. by Anna-Teresa Tymieniecka,
Dordrecht: Kluwer Academic Publishers 1998, p. 185–202, here p. 185.

Brought to you by | Universiteit van Amsterdam - UVA Universiteitsbibliotheek SZ

Authenticated
Download Date | 2/15/18 4:00 PM
 The Archaeology of the Touch Screen
73
Fig. : Phénakistiscope (1833)

tears it down, we can read it as a direct invitation to touch the screen. Because, after
all, why should film screens not be touched?
First of all, it cannot be stressed enough that moving pictures for public consumption
made their first appearance among slot machines and fairground attractions, that is, in
a context where bodily interaction with the apparatus was common or even necessary.
Of particular interest are the hand-cranked viewing machines, which allowed for some
manipulation on the part of the viewer. A good example is the mutoscope, patented in
1894 by Herman Casler. In contrast to Edison’s kinetoscope that was launched earlier
that year, the mutoscope was not motor-driven. As pointed out by Erkki Huhtamo, the
mutoscope allowed the viewer to »freely adjust the cranking speed, and interrupt the
session at any point to observe a particularly interesting frame (perhaps a half-naked
lady)«.11 This manual operation, which implies not only a direct, physical contact with
the apparatus but also the possibility of manipulating or altering the view, is a feature
that late-19th-century, hand-cranked, viewing machines share with early-19th-century
optical toys, such as the thaumatrope (mid-1820s), the phenakistiscope (1832) and the
zoetrope (1833). According to traditional film history, these optical toys belong to the
prehistory of cinema. They are generally considered as »pre-cinematic« devices because
of their application (or illustration) of the so-called persistence of vision, discovered by
the Belgian physicist Joseph Plateau in 1829. In this dominant – and teleological – view,
the focus is of course on the visual, on the spectacle for the eye(s). Film historians rarely
underline the importance of the hand, of the manual operation, which lies at the basis
of these 19th-century forms of spectacle and which actually points to another more obvi-
ous and straightforward lineage: from arcade games to computer games.12

11 Erkki Huhtamo, »Slots of Fun, Slots of Trouble: An Archaeology of Arcade Gaming«, in: Hand-
book of Computer Game Studies, ed. by Joost Raessens and Jeffrey Goldstein, Cambridge, MA: The
MIT Press 2005, p. 3–21, here p. 9.
12 See Nicolas Dulac and André Gaudreault, »Circularity and Repetition at the Heart of the At-

Brought to you by | Universiteit van Amsterdam - UVA Universiteitsbibliotheek SZ

Authenticated
Download Date | 2/15/18 4:00 PM
74 wanda strauven

Beside the lineage of the 19th-century optical toys and viewing machines, there is
another connection worthwhile exploring, which is early museum culture with its
hands-on practices. Touching artworks on display was a rather common practice in
the period of early museum culture, which runs roughly from the mid-17th century
to the end of the 18th century (that is, before the gradual institutionalization of the
museum in the 19th century). As explained by Constance Classen, it was a practice
that museums had inherited from private art collections and that was an almost man-
datory aspect of the guided tour, with the curator acting as »gracious host« and the
museum visitors as »polite guests«. According to the logic of this hospitality, museum
visitors were supposed to »show their interest and goodwill by asking questions and
by touching the proffered objects«. And, as Classen adds: »To be invited to peruse
a collection of exotic artefacts or objets d’art and not touch anything would be like
being invited to someone’s home for dinner and not touching the food.«13 Impor-
tant to note here is that this hands-on practice was not limited to three-dimensional
objects, but also applied, although to a lesser degree, to paintings. People would touch
paintings to feel the texture of the canvas and/or the paint, or »simply to exercise
their right to touch«.14 It was above all a matter of closeness, of direct contact.
By the mid-1840s such a hands-on practice had become taboo in the art world.
This was a direct result of the institutionalization of the museum and the disciplining
of the museum visitor, which basically meant: lowered voices, a measured pace, and
hands-off behavior. However, as Classen points out regarding the Ashmolean Muse-
um of Art and Archaeology of Oxford, which is the first public museum in Britain,
founded in 1683: »as late as 1827 the Ashmolean regulations allowed visitors to handle
artefacts with the curator’s permission«.15 So it is a practice that lasts until the mid-
1820s, and this is the moment when the first optical toys, such as the thaumatrope,
appear on the market. One could therefore say that the early-19th-century optical toys
ensure a continuation of the hands-on practice that for more than a century pervaded
the semi-private/semi-public sphere of the early museums. The hands-on practice
will then move back from the private sphere (optical toys as home entertainment)

traction: Optical Toys and the Emergence of a New Cultural Series«, in: The Cinema of Attractions
Reloaded, ed. by Wanda Strauven, Amsterdam: Amsterdam University Press 2006, p. 227–244. On
the importance of the manual operation of magic lanterns, see Lisa Cartwright, »The Hands of the
Projectionist«, Science in Context, vol. 24, 3/2011, p. 443–464.
13 Constance Classen, »Touch in the Museum«, in: The Book of Touch, ed. by Constance Classen, Ox-
ford: Berg 2005, p. 275–286, here p. 275.
14 Ibid., p. 279.
15 Constance Classen, »Museum Manners: The Sensory Life of the Early Museum«, Journal of Social
History, vol. 40, 4/2007, p. 895–914, here p. 899.

Brought to you by | Universiteit van Amsterdam - UVA Universiteitsbibliotheek SZ

Authenticated
Download Date | 2/15/18 4:00 PM
 The Archaeology of the Touch Screen
75
Fig. 4: Interactive art installa-
tion at Open Studio, JCCAC,
Hong Kong, September 2008

to the public sphere with the advent of the hand-cranked viewing machines at the
end of the 19th century, precisely when cinema emerges as a new form of (visual)
spectacle. Here one should insist once more on the importance of manual operation.
Hand-cranked projectors, like the one in Uncle Josh at the Moving Picture Show, are
necessarily linked to the body of the projectionist. They also allow for some mani-
pulation of the image projected, precisely as in the case of the optical toys. This and
other ›interactive‹ features of early cinema will gradually disappear with cinema’s ins-
titutionalization. As with museum culture, the hands-on practice will become taboo.16
Whereas in the museum and arts context touch was to be reintroduced on a regu-
lar basis over the years, from Marinetti’s tactile boards of the 1920s to today’s interac-
tive art installations and entirely touch-interface-designed museums, the film screen
has remained basically an untouchable element of the cinematographic dispositif.
Here I should specify that when I talk about the film screen I mean the theatrical
film screen, the surface for projection that we find in the movie theater, that is, in the
public sphere. I distinguish the film screen from the television screen (or set) and the
computer screen (or monitor), which are both much more tangible as objects within
our private sphere. I also leave aside the phenomenon of home cinema and the fact
that today we can watch films on various portable touch screens, which somehow
annul the gap between private and public. Those touch screens (iPhone, iPad, and so
on), like the 19th-century optical toys, belong to the history of gadgets and ownership.
The theatrical film screen, on the other hand, is a screen we do not own. This is also
true, as I will discuss below, for the first generation of touch screens.

16 See also Strauven, »The Observer’s Dilemma«.

Brought to you by | Universiteit van Amsterdam - UVA Universiteitsbibliotheek SZ

Authenticated
Download Date | 2/15/18 4:00 PM
76 wanda strauven

Within the context of (institutionalized)

theatrical film viewing, the screen remains
untouched. The forbidden act of touching the
screen is, however, a recurring motif in so-called
meta-filmic films. One of my absolute favori-
tes is Jean-Luc Godard’s scene-at-the-movie in
his obscure and absurd war epic Les Carabiniers
Fig. 5: Jean-Luc Godard, Les Carabiniers (FR/IT 1963), which is a beautiful homage to
(1963) the early meta-filmic rube films.17 Godard’s film
is about two countrymen, Ulysse and Michel-
Ange, who join the army to get rich. In between
their war actions, Michel-Ange goes to the cinema for the first time in his life. Like
Porter’s Uncle Josh, he gets to see three moving pictures, which are remakes of early
cinema classics: Lumière’s L’Arrivée d’un train (FR 1895), Lumière’s Repas de bébé (FR
1895) and Georges Méliès’ Après le bal, le tub (FR 1897), retitled by Godard, Le bain
de la femme du monde (The Society Lady’s Bath).18 During the projection of the third
film, Michel-Ange jumps on stage and starts caressing the bathing naked lady. While
trying to join her in the bath, he tears down the diegetic film screen. In contrast with
Uncle Josh at the Moving Picture Show, there is no rear projection, and there is no
projectionist behind the screen to punish the rube. Unlike Uncle Josh, Michel-Ange
is unable to interrupt the scene: the projection continues on the wall. And, this is the
main point, Michel-Ange does not interrupt his own act of touching. He goes on
touching the projected image of the society lady’s leg, as if it were a solid 3D body, a
sculpture (maybe his sculpture? his name is Michel-Ange after all …). It seems as if
Godard was trying to convince us that the 2D film screen is indeed touchable, that
Uncle Josh and the other rubes were right, and that the cinema audience has wrongly
unlearned the sense of touch.
Interestingly enough, in those years, the early 1960s, the first touch screen tech-
nology was developed. Allegedly, the first touch screen was invented in 1965 by E. A.

17 See also Wanda Strauven, »Re-disciplining the Audience: Godard’s Rube-Carabinier«, in: Cinephi-
lia: Movies, Love and Memory, ed. by Marijke de Valck and Malte Hagener, Amsterdam: Amster-
dam University Press 2005, p. 125–133.
18 As pointed out to me by Alexandra Schneider, it is likely that Godard’s source of inspiration for
this third film was – instead of Georges Méliès’ first »nudity« scene – Buster Keaton’s One Week
(1920), where the bride played by Sybil Seely is taking a bath as well. In Keaton’s film the setting is
more alike (a proper bath room with a tub, instead of a bucket in a dressing room) and, interestin-
gly enough, the lens of the camera is covered, literally touched, by a hand to block our view when
the lady is reaching out for the soap that has fallen on the floor.

Brought to you by | Universiteit van Amsterdam - UVA Universiteitsbibliotheek SZ

Authenticated
Download Date | 2/15/18 4:00 PM
 The Archaeology of the Touch Screen
77

Fig. 6: E. A. Johnson’s touch screen (1967) Fig. 7: A touch-screen keyboard designed by the
Human-Computer Interaction Laboratory at the
University of Maryland (1991)

Johnson at England’s Royal Radar Establishment. It was a touch screen for air traffic
control. Gradually, touch screens were also successfully applied in sales kiosks, public
information services, as well as in museums.19 These early touch screens, which were
screens in the public sphere (precursors to today’s ATM machines and the like), did
not have a good reputation: they were imprecise, slow, and poorly designed. There-
fore, in the late 1980s to early 1990s, research focused on, so-called, high-precision
touchscreens.
From today’s perspective, seven years after the launch of the iPhone, it is difficult
to imagine that in the early 1990s the touch screen was looked upon as old-fashioned.
In 1991, an article on the touch screen began as follows: »If you thought touch screens
were a thing of the past, this essay will bring you up to date on improvements to this
input device’s user interface. I suspect we will be seeing touch screens used for more
applications than ever before.«20 The author was Ben Shneiderman, who had founded
the Human-Computer Interaction Lab at the University of Maryland in 1983. This
lab conducted many touch screen experiments in the early 1990s with the main goal
of making the selection strategy more accurate. All those experiments, which ran
from keyboard applications to finger-paint programs, used the monitor (or computer
screen) as touch screen.21 In those very same years, Xerox PARC was developing the
so-called digital desk, which was a user interface projected onto a real desk to be

19 Catherine Plaisant, »High-precision touchscreens: Museum kiosks, home automation and touch-
screen keyboards«, January 1999: http://www.cs.umd.edu/hcil/touchscreens/
20 Ben Shneiderman, »Touch screens now offer compelling uses«, IEEE Software, March 1991, p. 93f.,
p. 107, here p. 93.
21 Andrew Sears/Ben Shneiderman, »High Precision Touchscreens: Design Strategies and Compari-
sons with a Mouse«, International Journal of Man-Machine Studies, vol. 34, 1991, p. 593–613.

Brought to you by | Universiteit van Amsterdam - UVA Universiteitsbibliotheek SZ

Authenticated
Download Date | 2/15/18 4:00 PM
78 wanda strauven

Fig. 8: Digital Desk (Xerox PARC, 1991)

Fig. 9: SixthSense (MIT Media Lab, 2009)

manipulated by touch. This digital desk was to replace the ›desktop‹ metaphor, which
Xerox PARC themselves had introduced in 1970. Xerox PARC aimed to overthrow
their electronic desk by turning the physical desk into a workstation, into a surface to
be touched and physically worked upon. In other words, their touch screen disposed
of the computer screen and relied on the principle of projection, a principle typical
of, and essential to the film screen. In this application, because of the use of front
projection, the hand of the engineer who touched the projected desk became a screen
itself, upon which data are ›imprinted‹ or projected in an echo of Michel-Ange’s skin
which served as a screen for the ongoing projection of the bathing society lady in
Godard’s rube film-with-in-the-film,.22
In a further leap, in 2009 (one year before the launch of the iPad), researchers at
the MIT Media Lab presented their SixthSense project, which was – according to
their own website – a »wearable, gestural interface that augments our physical world
with digital information, and lets us use natural hand gestures to interact with that
information«.23 The main motivation behind the project, run by Belgian professor
Pattie Maes, was to address the point that smartphones were still cumbersome in
terms of accessing data. Relying again on the principle of projection, the project pro-
posed that access could become more direct. And that all surfaces could function as
screens, even, or especially, the hand. The promotional video of SixthSense shows,
for instance, how the hand becomes a phone display with buttons to be pushed in
order to dial a number. Here again, as in the previous example of the Xerox PARC
engineer’s hand at the digital desk, the skin becomes a screen. It becomes a surface

22 See the 1991 demonstration video »Tactile Manipulation on a Digital Desk« by Pierre Wellner:
http://www.youtube.com/watch?v=laApNiNpnvI.
23 http://fluid.media.mit.edu/projects/sixthsense.

Brought to you by | Universiteit van Amsterdam - UVA Universiteitsbibliotheek SZ

Authenticated
Download Date | 2/15/18 4:00 PM
 The Archaeology of the Touch Screen
79
Fig. 10: Coal screening
around 1895

onto which data are projected, a surface similar to the theatrical film screen’s but one
with the possibility of interaction and manipulation due to tactile technology. Sixth-
Sense illustrates one way in which touch screen technology »touches« our own skin.
In doing so, it reminds us of one of the many possible etymological origins of the
word »screen«, one deriving from the Latin corium (Greek korion), meaning »skin«.

Brought to you by | Universiteit van Amsterdam - UVA Universiteitsbibliotheek SZ

Authenticated
Download Date | 2/15/18 4:00 PM