LEONARDO

Designia,g Infurmatioo 'J:ecbnofogy, RichardC,ryille,

1995
Tecbno[l)mmnicism: Oigical Narrative,, Holi:smi,an,d che Romance of the Real, T'he Language of
RichardCllfJ.w,.1999 .
The Visual Mind, editedbyJ.licheJe Emmer,,1994 New Media
The R.oooci.n !theGarden: Telerobotii:sand Telep,is:temologyin the Age ,ofthe Wn-
ternet,. editedb),Ke:,i,Go./dberg,
2000
LeonardoAlmam:,,edittdbyCraig Ha11Ti:s, 1'994
[n ,Seaoch oHnm:iuttion: The XeroKPARC PAIR 1Pr,o,j1eirt,
editedby CraigHarm,, 1999'
The Digital Dialeui,c: New Essayson New Media,, ,edited/i;yPeterL1me1JfelJ,,
1999
The I.anguage of New Media, Let, Ma11miich,,
20()'}
Immersed in Techn,o,lo:gy:.
Art anJ Virtual Environments,. editedby Mary llt1ne'/llilos,er
with Douglas.Mm:l.etid,
1'996

Lev Manovich

TheMITPress Cambridge,
MassachusettsLondon,England
© 2001 Massachusetts Institute ofTechnology 17D
Ntm1tmtKlem/PeterL1menfeldJVivi<111
So/Jchack

AU rights reserved. No part of this book may be reproduced in any form by any elec-
uonic o.r mechanical means (in.dmllilg photocopying, recording,, or information
storage and retrieval) without permission in writing from the publisher.

This book was set in Bdl.Gothic and Garamood 3 by Graphic Composition,Jnc.

Printed and bound i10 the UnimedStates of Amerka.

Library of Go!l11gress
Caal.ogill11g-in-PubUration
Data
Manovich, Le,,.
The lianguage,ofnew media J LevMlloo'l'id-1.
p ..cm. - (lwnimlo),
r.efereru:esand index.
Includes biblio;grapliiicai.l
ISBN 0-.262-BlNl-1 (he: a1k.paper), 0°262-63255-t (pb)
L .Mas med.ia~Technological innovations. L Title. IL Leonardo
(Series)Camhridge, Mass.)

P96.T42M35 2000
302.2-clc21 00-0578:82

1098765
 •
What is new media? We may begin answering this question by listing the
categories commonly discussed under this topic in the popular press: the In-
ternet Web sites, computer multimedia,. computer games, CD-ROMs and
DVD, virtual reality. Is this all there is to new media? What about television
What Is Nlew IMedia?
1
programs shot on digital video and edited on computer workstations? Or
feature films that use 3-D animation and digital compositing? Shall we also
count these as new media? What about images and text-image composi-
tions-photographs, illustrations, layouts, ads--created on computers and
then printed on paper? Where shall we stop?
As ca.n be seen from these examples, the popular understanding of new
media identifies it with the me of a computer for distribution and exhibition
rather than production. Acordlingly, texts distributed on aa computer (Web
sites and electronic books)are considered to be new media whereas texts dis-
tributed on paper are nor. :=Similarly,photographs that are put on a CD-ROM
and require a computer to be viewed are considered new media; the same
photographs printed in a book are not.
Shall we accept this definition? If we want to understand the effects of
computerization on culture as a whole, I think it is too limiting. There is no
reason to privilege the computer as a machine for the exhibition and distri-
of media over the computer as a tool for media production or as a me-
bution of
dia storage device . All have the same potential to change existing cultural
languages. And aU have the same potential to leave culture as it is.
The last scenario is unlikely, however. What is more likely iS that just as
the printing press in the fourteenth century and photography in the nine-
teenth century had a revolutionary impact on the development of modern
society and culture today we are in the middle of a new media revolution
the shift of all culture to computer-mediated forms of production, distribu-
tion, and communication. This new revolution is arguably more profound
than the previous ones, and we are just beginning to register its initial ef-
fects Indeed, the introduccion of the printing press affected only one stage
of cultural communication the distribution of media. Similarly, the in-
troduction of photography affected only one type of cultural communica-
tion--still images. In contrast the computer media revolution affects all
stages of communication induding aquisition manipulation storage and
distribution; it also affects all types of media--texts, still images moving
images, sound, and spatial constructions.

10
What Is New Media?
 How :shall we begin to
to map out the effects of this fundamental shift? HowMedia Became N,ew
What are the ways in which the use of computers to record store, create, and
distrubtue media makesit "new?"
In the section "Media and Computation," II show that new meida repre-
sents a coavergence of two separate historical trajectories: computing and
media technologyes Both begin in the 1830s with Babbage's Analytical Eng-
gine and Daguerre's dagguereotype. . Eventually, inin the middle of the twenti-
ethcentury, a modem digital comupter isis developed to perform calculations
on numerical data more efficiently; it takes over from numerous mechanical
tabulations and calculations widely employed by companies and covernments
since the turn of the century. In a parallel movement we witness the rise
of modern media technologies, that allow che storage of images, image
sequences, sounds, and text using different material forms--photographic On Aug1ISt19, 18.39,the Palace of the Institute in Paris Wills; fiilledwith cu-
plates, film stocks, gramophone records, etc. The synthesis of these two his- rious Parisians who had come to hear the formal descriptfo,,n of the new re-
tories? the translation of all existing media into numerical data. accessible production process invented by Louis Daguerre, Daguerre, well
through computers. The result is new media-graphics, moving images, known for his Diorama, called the new process dagllerreotype. According toa
sounds,shapes, spaces,and texts that have become computable; that is, they contemporary, "a few days later, opticiims' shops were crowded with ama-
comprise' simply
. y another set or computer dara. In Principles
• I of NewMedia.i~-
teurs panting for daguer!'eotype apparatus, and everywhere cameras were
dia," I look at the key consequences of this new status of media. Rather than trained on buildings. Everyone wanted to re<::ordthe view from his wi,ndow,
focusing on familiar categories such as interactivity or hypermedia, I suggest and he was lucky who at first trial got a silhouette orroof tops against the
a different list. This list reduces all principles of new media to five-- sky."1 Themedia frem:y had begun. Within five months more d~an thirty dif-
numerical representation, modularity, automation, variability, and cultural ferent descriptions of die redmig ue had been published ar,ouml th,e world-
transcoding. In the last section, "What New Media Is Not," I address other Bacc.elona, Edinbu1::gh, NapJes., Philadelphia, St. Petersburg, Stockholm. Ar
principles chat are often attributed to new media. I show that these prin- .first, daguerreotypes of architecture and lam:lsca,pesdominated the
ciples can already be found at work in older cultural forms and media tech- imagination; two yea.rslat,er,,afoer various technical improvements co the
nologies such as cinema, and therefore in and of themselves are in suflicient pmc,ess had been poru:a.it galleries bad opened everywhere-and
to distiguish new media from old. eve.ryooe rushed co have her pinure taken lb:, 11:he new media machine.~
In 1833 Charles Babbage began designing a device he calied '"[he Ana-
lyitic,dEn,gine.vThe Engine ,comained most ofrhe key feacures of the modern
di,gi,t:alcornpmer. Punch cards, were used ro enter bmh da,mand instructions.
T.'"tisinfmmarion was stQ:red in rhe Eagjne'smemory . .A pmcessing unit,

l. Q~•otecin Beaumom
Newhall., Tl"' ll'""-'"'-''''J'
llfoseum of Modero Arr. I %,4),,!.8,
ed. (N,e.,,'i:'imrk:
2. Newhall, 'll,,Hi1t"")••f l'/io1,~g~t1pbJ•,
17-22.

- -
 whid1 Babbage referred mas a "mi,11,"performed operations on the data and
wmr,e die :resuhs to memory; final results ,ve:rem be primed out primer; IHlJa
The Engiine was designed to be capable of doing any m:arhernatical opera.-
tion; nm only would it foll.ow the program fed into it by cards,, but it would
also decide· which inscrucrions m ei,i:,ecurenext, based 0111
imermecliue re-
su Its. However, .incontrast 1t,o ,ofdie En-
the daguerreotype, not a single c,01J1f'
gine was rnmp,.lered. While rhe iinvemi,on of rhe daguerret::1t),pe,,a modern
media tool fur the reproductii:m ofrea.lity,, impacted society iimm.iediue.ly,the
i mpacr of the computer was ye·:tm be seen.
Interestingly, Babbage :oono,.,,,ed
d11e.idea of using punch c:uds m,store
information from an earlier prog.rammed machine. Around U:100,.J. M.
Jacqu~..r,dim•,e1neda loom rhar was amom:aticaUy commHedl byp,lllm:hedpa-
per cards. The loom was used m weave i 111trkarefigurative images, im:luding
Jacquard "sportrait. This specia.lized graphics computer,, ro m speak, foispired
Babbag,e i,111
his work on the Analytiical Engine, a general computer lfm nu-
merical cakul.a1tions. As Ada Augus.ir:11Babbage's suppo.n:er a111dChe Jirsr
1 camera/pmjeccion hybrid, first m a scientific audieoce and lacer, in Decem-
computer pmgra.mmer, pu1t it,, ~Th,e A:n.dyticalEngine weaves a.lgebrnical
Thus a pro-
patterns, j,mt as :the Jacqua.rd .loom weaves flowers and leavesi,,"'~
grammed ma.chine was al:~eadys~·nrhesizing irn~gieseven liefo:reiit was pur to
prncess.in,g numbers. The cornn,ecirionbetween the Jaoqwudl ]!,oomand rhe
A!llalyitica[Engine is nor something historians of co:mpu1t,ers
make much of,
since for them computer ima,ge synthesis represents jw;r one application of
rhe modem digiraJ compttit,er among thousands of odllec:s,bur for a hismrian
of rne\\•media, ir i1sfull of significance.
We should! not be surprised that both trajecrories-the deV"elopment of
modem media and the development of computers-begin around the same
time. Both media machines and computing machines were absolutely nec-
essary for the functioning of modem mass s:ocieEies.The ability to dissemi-
nate rhe same rexts, images, and souru:lsco millions of citizens-thus
assuring the same ideological beliefs-was as essential as the ability to keep
track of their birth records, empfoyment rooords, medical records, and police
records. Photography; liiilm, the offset priming press, radio, and television
Persp.ctit't:&.."kgro,,ndto,theCom/>lller
3. Cha;rles lla,ines, .A C""1/J11ler A,~·,(Cambriidlge,,
:i,,1a,ss,
Harvard U11h•e1rsicy Press, 1990), l8.
- -
Chapter l
made the fo,rmer possible while computers m:adie possible the latter. Mass
media and d~t:a processing are complementacy n:cl:mologies; they appear to-
gether and devdop side by side, makin,g modem mass society possible.
For a long 1timed1e two trajectories ran i!llparallel without ever crossing
_Jaths. Throughout the nineteenth and the early twentieth centuries, nu-
merous mechanical and electrical tabulators and calculators were deV"eloped;
they gradually became faster and their use more widespread. In a parallel
movement, we witness the rise of modern media that allow tlhe storage ,of
images, image sequences, sounds, and texrs in different material forms-·
photographic plates, film stock, gramophone records, etc.
Let us continue tracing this joint history .. In the 1890s: modern media,
rnok another step forward as stiU photographs were pm iiirnmorion. InJam1.-
ary 1893, the first movie studio-Edison•s:'Black :Mraria"-statted pr,od.uc-
ing twemy-second shorts that were shown in special .Kineroscope parlors.
Two yeall'Slater che Lumiere brothers showed their new Cinemacogmp,hie
ber 1895,,m the paying public Wfrhin a yea..r,audiences: foJohannesbl.ll"g,
Bombay,Rio de Janeiro, Melbourne, Mexico Ciry; aod Osaka.we.res:ubjeclled
to the· new media machine, and they found it irres:fatible.4' Gradually soenes
grew ]ooger, the staging of realiry before the camera and dte subsequent ed-
iting olFsamples became more inuicate, and copies mul'.tiplied. In Chicago
and Cak1m:a,,london ad St. Petersburg, 'E'oky,oand B,edin, and thousands
of smaller places, film images would soothe mo'lie atu:lieooes.,who we.re fac-
ing an increas:ingly dense· information environmem m:ttside the theatler;,an
em•iron1mel!]td:mt no longer could be adequatiely handled. by their own sam-
p.lin,g;and data prooessing sys.ems:(i.e .., their bra.ins),.,Peri,ooic rdps into the
d,aii::k
melaxation chamber.; of mmrie cheaters became :a mruc.inesurviru c,ech- 1
li]i,q,lle
for the subjects ,ofmodem sociery.
The Ul90s was die cmcfa.l decade n.ot on.ly for the development of me-
d,ia,b111c also for computing. If indiividual brains wel!'e'ov,erwhelmed bytlrn1e
ammnu ,of information they had to process, d1e same was,tme of coi:po-
rario.ns andl of governments., fo 1887, the U.S. Ce,nms:Bureau was st:iU
4. D,,.,id Bordwell and Kn.still, Timmp,sim,.Film A,t: A.11Tn~'im1, 5,tb eel. (New Voik:
McGnw-Hill), 15.
V~hal h New Media1'
interpmeti.ng fig:ures.&om the 1880 censm. For the 1890 census, the Censw.
BW'ealil adopm:1 electric tabulating maclunes designed by Herma111Hol-
leridt. The da:ta ooUecred011every persion was punched into cards; 46,804
enwner.m:i,i::s,oom:pfoted forms for a total population of 62,979,766. The
Hollerith mbwlato[ opened the door for d1e adoption of calculating ma-
chines by husines,s;.dw:fog the next decade eJecuic tabulators became stan-
dard equipment in rnswance companies, public utility companies, railroad
puter ..As we know, the inv,e:cuors of cinema •eventually serded on using dis-
crete images recorded 011 a scr.ip of celluloid; tire im<entors,of the compurer-
wruch needed much grea,t,er speed of access as weU as ,the albil.fryto quickly
read and write data-evem1.1aUy decided to store it ,eiectmnical.ly.in a binary
code.,
The histories of media and eiomp111tingbecame forther ,emwined when

offices, and accoormlingdepattmems. In lSH li, HoUerith's Tabulating Ma- German engi11eerKonrad Zu:11ebega.n building a ,c,omp1.1mer
.in the fo,ing

chine Company was me!ged with three other companies to form the Com- room of his pacerus' apattmem fo Bediin-the same )'•earthat Turiing wrote

puting-Tabulating-Recording Company; in 1914, Thomas J. WatSon was hiis sem.ioal. paper. Zuse's compmer was the first working digital computer.

chosen as its head. Ten years later its busi11ess tripled, and Watson :r,enamed One of h.is inoi:wations was, using punched tape to cot1itml computer pro-
the company the ~International Business Machines Corporation.," or IBM.' grams,, The tape Zuse used was acmaUy discarded 3,5mm movie film. 6
JMov.in,ginto the twentieth century, the key year fo.rdue histmy of media Ot1e of rJhes,unriviag pieces of this fiiim shows binary ,code punched over
and ,oo,mpudng is 1936. British mathematician Alan Turing wrote a semi- d11eoriginal frames of an interior shot. A typical movie scene-two people

nal.paper ,e111dded "On Computable Numbers." In it:he provided a theoreti- in a room in'llolved in some action-becomes a support fur a.set of computer
cornmarn:k Whatever meaning and emotion was rontaiimed in d1is movie
cal descripti.on.,ofa general-pw:pose computer later named afrer .it:sinventor:
scene hasbeen wiped out by itS new function as data carrier. The pretense of
Tmi~g Machine ..~ Evendwugh it "'l.'l!as
gthe Uni'fers:ail capa:lbI,eof only Cmuop-
mooem media to create simulations of sensible reality is similarly canceled;
entioirui ,,me machine could perform :my calculation d•t could be done by
' .a
hum:aa and ,could also i.:micat,eany od1e.r computing madlin,e. Tlite madune media are reduced to their original condition as information carrier, nothing

operated by r,eading and w.ritin,g numbers, on an endless ope ...A•tevery step less, nothing more. fo. a technological remake of the Oedipal complex, a son
murders his father. The iconic code of cinema is discarded in favor of the
the tape 'll,r,olJ!ld
be advanced to rretri,evetlite next command,, read d:ie data, or
more eHiciem binary one. Cinema becomes a slave ca the rnrnputer.
write the ie:sul.t. ItS diagram looks.s:usp,.ic:iously
like a film p,rojiecitll'r.Is this a
But this is not yet the end of the story. Our StoEJ,,·hai5, a imewtwist-a
coincidence?
happy one. Zuse's film, with its strange superimposition of binary over
If we beJieve the word ci1101ll:Ztogr:~pb,,
which means "writimg mllv,em.ent,"
iconic code, anticipates the convergence that wiU follow half a century later.
the essienoe:,ofoinema is recmdiimg and storing visible data io a maoe[ial form.
A film camera rec;ords data 011fill:m;a ,film projector reads i t olf. This doe-
1
The two separate hisnorical trajectories finally meet. Media and computer-

is ~milar m a c11m.p1uner
maitic app1Ui31:l.1S i.11one key respect: .A c,omput,er's Daguerre's daguefreotype and Babbage's Analytical Engi:nie, the lumicre
Cinematographie aad Hollerith's tabulator-merge into orne.. All existing
program and ,data also have to be stored io some medium. This :is why the
U nivet:Sal 1iuri.ng Machine l.oob like a film projector. h is a kin.d of film media are translated into nwnerical data accessible fordae computer. The re-
sult: graphics, moving images, sounds, shapes, spaces, and texts become
camera and lilm proj.ector at ,onc,e,,readin.g iruitructions an.ddata stored on
In
endless tape and writing diem in odl1er locations OD this tape .. fact, the
computable, that is, simply sets of computer data. In short,, media become
new media.
developmem.tof a suitable .s1tor:a,g,1e
medium and a method for ,cooing data
rep,esiem important parts ,of tthe pr,ehistory of both ,cinema ,a111d the com-
1
This meeting changes the identity ofborh media and the cornpmer frsei£
No longer just a calculator, control mechanism, or comm1.micatil!ln1devke,

5.. Eames.A Compuier

Perrpeawe,22-27., 46-51, 90-91.
6. Ibid.,, 12!0.

- -
Chaple.- l
~hat is Ni,ew1\/ledia?
the computer becomes a media processor. Before, the computer could read a Principlesof NewMedia
row of numbers, oucpuning a srncistical result or a gun 1tni:iiecrn,ry.
Now .it
can read pixel values, blurring the image, adjusting its co1umsr, or checking
whether ir contains an outline of an object. Buikling on these lower-level op-
erations, it can also perform more ambi1cfotJ1S
ones-searching image data-
bases for images simifar in rnmpos,irion or comtent co an input ]mage,
detecting shoe changes in a mm•ie, m symhesizing tlrte movie sllmt icsdf,
complete with setting and actors .. hi ,a hisro.rical loop, the computer has, re-
rurned to ics,mii!!]:ins.No longer jus,t an Aaalyrical Engine, suiHhil,e ooly· fur
crunching numbers, it has become Jac,q11ard's.
loom-a media sy,nchesizer
and manip1.darnir:

The identity of media has changed ev,en more dramaticaUy than that of ithie
computer. Below I summarize some of 'the key differeru:es between oM al!ld
new media. In compiling this list of differences, I tried to arrange them in a
logical order. That is, the last three principles are dependent on the first two.
This is not dissimilar to axiomatic logic, in which certain axioms are taken
115sain:ing points and further theorems are proved on dieil' basis.
Not every new media object obeys these princ~ples .. They showd be
,mnsidered nor as absolute laws but rather as g,e.11e1raJ
tendencies of a
c1J1.hureuDdergoing compmerizadon. As compu~e·~i:zatfon affects d,eeper
md deeper layers of culrnre, these tendencies will'increasingJy mallifest
thiem.s,el.ves.

I.. Numerical Representation

Al.I new media objects, whether created from scratch 011 computers or con-
ven:ed from analog media sources, are composed of digital rode;.they are nu-
merical representations. This fact has two key consequences,::

1. A new media object can be described fonnal]y (mathematically). For

instance, an image or a shape can be described wing a math,ematical
function.
2. A new media obj&t is subject to algorithmic man.ipwarion. For i.11-
srance,. by app[ying appropril!Jte algorithms., we can oor,omatically rem.ove
"lloise" from a phom,graph, improve its contaisc, b:are die edges of the
In s,hort, media~ programmah!e.
shapes, or change :iitspropC1,ni,011s.

-
Ct.apter] ll~hialls Ne1y!Media:?

Ill
Wheio new media objects are created on computers, meyoriginate in nm-
um.is spectrum of the colors is verbally reduced co,a. seri,es of discontinuom
meriia1lfo,m1.Hut many new media obj,ects.are converted from various forms
tenn:s) .."'9 ]n assuming that any form of communication requires a discrete
ofotd mediia..Altbougb most readers:u111.detstandthe ditierence between am,
repr,es.entation, semi:otkians took human language ais the prototypical ex-
lqg md ,digital media,.a few noces sho1dd he added on the terminology anrl:
:aimpl.eof a commuDkation syst•em. A human language is discrete OD mos:t
tbe oonversi.onprocess itself. This proces.sassumes that data is originally am-
scales::We .speak in sen1t,ences;a sentem::e is made from w·ords; a woird cornists
that
ti1111.0Rs, ~the axis or climensfoo that is measmed has no apparent in-
of morp.lhiemes,and so on.. If w,e fol.low this assumpti•on, we may expect that
divisible uruit from which it is compo,sed.'.''1' Goavening continuous data into
a numerical repmesentationis ,called.digit.fa11Ztin. Digitization consists of two
m,edia used in culm.raJI1commuointion wm
have di.scret,e Ievds. At first ,ch.is
d1100.ry
seems to work. Indeed, a. film samples the contiouou:s rime of human
steps: samplfoig:and quantttarion. First, data is SfmJletl,most often at regu-
into discrete fume.s;;a drawing samp]es vis:ibier,ealli
,eXJist,ence ty into discrete
lar intervals.,md:i as tbe grid ,ofpi::re.ls11Wd to 1epresent a digital image. The
1

dlors. This assump-

liDes; and a primed photog1r,aph samples it into di.scre.c,e
frequency of sampli11gis refecred mas mdutirn1.. Sampling turns continum1s
tion does !IIOt unive:rsallJy wmk, however: Photogrmp!i:s, fur instance, do not
data into di!mte data.,, that is, data occwriing i11distinct units: people, the
have aay apparent units. {Jodeed, in the 1970s:semiotiics was criticized :for
pages of a book, pi..ul's ..Second,. eaichsample is (J't/4.ntified,
that is, it is assigned
ireslinguistic bias,. and m.06,t :semioti.cians came 10 ll'e<)Ognizetha1t a iaaguage-
a numerical value drawn from a defined raimge(s:1111:h as 0-25 5 in the case of
of meaniiragcannot be applied to inany kinds
based model ofclisttnct 1.1.ttirs
an 8-bit greysc:ale image).. 8
.ofcwtural comm1mica1tioD,..) ])iifor,eimportant, the discrete units of modem
While some old media such as plhotograpby and scu.lpnu:e are trullyoon-
media are us:ually not units of memings in the w:ay morphemes are. Neither
tinuous, moo invoh,e the combiaiarion of continuous and discrete coding.
lilm f.t:ame:snor halftone dots bi!Jveany refation m how a film or photograph
One example is motion picture film:ea.ch frame is a continuous photograph,
:affects the viewer (except in modem art and avant-gai:rde lilm-think of
,t ti.me is broken, im.o a number of samples (frames). Video goes one step
b 111
paintings by Roy 1.ichtensteiD a111dlilms of Paul Sharits-which often make
fwthe.r bysampling tthe frame along the vemt.ical.
dimension (scar,lines). Sim-
d1ie~ma:t•erial"unircsof media into units of meaning).
ilady, ,l!Jphotograph printed using a halftone process combines discrete
Tille most likely reason mod'.eramedia bas discrete level$ .is because it
and continuous representations. Such a photograph consists of a number of
em,e~g,edduring the lndusuiai] Revolution. In ,the nineteenth rentur),, a new
orderly dots (i.e., samples), although the diameters and areas of dots va.y
o~g,mi:z:ation ofproductio,:a known as the fuctory :sysc,em
,gradually replaced
continuously.
illltisan labor. fr reached '''"~"'•~u, fonn when Hemy ford i11s:talledthe first
As the last example demonso:ates, while moo.em media contain levels of
:l!SlSemb,ly line in lil.isfuctocy .in 19 I 3,.The assembly line ridiiedon two prin-
disc.rete representaicio,n,,
the samples are never·qwm.tified. This quantification
,cip.les.The first was standilltdfaacion of pans, ,employ,ed in tine pro-
of samples is the crftcial step acmmplisbedl by digjcization. But why, we may
The second, newer
duction ofmilmury uniforms. ia tthe nineteenth ,cent1J1ry.
ask, are modem media technologies often foipart discrete? The key assump-
p,.rim::.iplewas the separntio11oftlh:eproduction process into a set of simple,
tion of modern semi.otics is that commut1icuion requires discrete units.
rrepetiitive, and sequential activiitties dmt could be ex,eruted by worikeirs who
Without discrete 1!.ltlits,there is no lu.gwg,e. ,o\s Roland Barthes put it,
did .aot have to master the entiil-'e[Process and could be e«1s,ily
replaced.
"Language is,, a:s,it were, that whlch div.ides r,eafoty(for instance, the comin-
Not surpirismgly, modem media lioUowsthe logic ofrhe facto.ry, not only in
terms; of division oflabor as witnessed ,in Hollywood film studios, anima,cion

7. ]saac Vim 11r !Kalovand Juclsol!Iliwsebush,Gw,,pact<I'

Graphiaf~r De.rig,rmand Amin {New
NostG1ndReinhold, 191116),
York: Vll!l!I. l•t
'9·..ll.c,landBa,,rhes,Elemmtsof Sew1iology,
,tuns, Annette Livers and Colin Smith !New York:
8. Ibidl...,21.
Hill ~nd Wang:, 1968}, 64.

- -
 I
studios, and television prodlucciun, but also on die level of ma.rerial or-
ganization. The invention of rypeseuin,g machines in the U3..80sim:lusnial-
ized pub.iishing while leading to a standar<lizatioD ofooth t)'pe dlesign and
fonn{nlillmiberand types). fo d1e ]890sc:inemacombined aummuically pro-
duc,e,d.im:a,ges{via pho:co,graphy) wiid1a mechanical pmjiecror. This required
.atrdi:zatfonof both image d:imensi,ons(size, frame ratio, connasr) and
sram::1
1
cernpota:I :sampling rare. ~,;en ea:rlie~, .in the 1880s, d1,efuse television sys-
tems alrelldy involved srand!ardiizaiion of sampling both in time and spa.ce.
These modlern media systems a.lso followed facrory logic in du1:t,once a new
"model" (a film, a photograph, an audlio .recording) was in:rrodu,c,ed,numer-
ous idemkd media copies. would be produced from this master .. As I wiJI
s.ho,w,,
1newm,,ediiafoUows, or acrua]ly mns ahead of, a qui:t,ediflierenc logic of
posr-i11dus:uial s:ociety-1rha.r ofil]dividual mstomizatio111,,1i:itcherthan mass
stauru:laJ11d!ization.
2:,.M,odl1Ulall'ity
This princip]e can be called d11e'"firacralurm:rure of new media.,'"Jl.!lstas a
fractal. h:asd1e same structl.!lreon different scales, a new media object has
rhe :same modular sm.11omr,erhrol.!lghout. Med.ia dements,, he they im-
:a,ges,:sounds, shapes, or behavio.rs, a.re represented as ,ooUe,criio,ns
ofdliscrere
samp]es (pixels., poly:g,ons, voxds, d1araccers, scrip:cs). Thesie elements are as-
sembled! .imo lilll'ger-scal.eobjects but continue co maima.in their .separate
idemities .. The objens ,themselves can be combined inim ,e,.,en brger ob-
jects-agail!l.,, without losing their independence. For example, a multime-
dia "mo'll.ie"'authored in popular Macromedia Director software may consis1t
of hundreds of stiU images., QuickTime movies., and sounds chat are stored
separately and loaded at run time. Because air elements are scored independ-
ently, they can be modjfied at any time w frhom having to change the Direc-
tor ''movie~ itself. These ''movies'' can be assembled.into a larger "movie," and
so 011. Another e·xampfo of modularity is [he concept of "object" used in Mi-
,crosafcOffi1oeapplications. When an '"obje'Ct'"is imened into a document (for
instanoe, a.med.iadip insened into a Worddocument), it continues to maintain
its iadependlenoe :and can alw:aysbe edited with the program ,originally used to
creatie ir..¥er a1mcher example of modullaJriit)'is.the structure of an HTML doc-
um,enc:With the exemption of cext,,it ,mllsis1cs ofa number ofsepm.te oojecrs-
GIF andJPEG .iimages.,media dips, V'iinualReality Modeling Lang:uaige(VRML)
scenes,. Sboc.lk:wa.~'e
and Flash ma,vies-whid!i are all stored ind~ndently,
- -
Chpter 1
locally, and/or on a network. In shore, a new media object consists of inde-
pendent pares, each of which consists of smaller independent parts, and so
on down m the l,evel of the smallest "atoms"'-pixels, 3-D points, or teXt
'
characters ..
The World Wide Web as a whole is al.so complerely modular. It consists
of numerous Web pages, each in its rum cm:1:siscingof separate media ele-
ments. Every element can always be acce--~ed,cmits own. Normally we chink
of elements as !belonging to their corresponding Web sites, but this 1s just a
convention, rein:foroed by commercial Web browsers. The Netomat browser
by artist Maciej Wisnewski, which extracts elements of a particufar media
type from different Web pages (for instance, images only) and displays them
together without identifying the Web sires from which they are drawn,
highlights for us chis fundamentally d:iscreteilllcndnonhiera.n:h:ical organiza-
tion of the Web.
In add.ition to using the metaphor of a fractal, we can also make an arud-
ogy between the modularity of new media and scrucmr:m ,computer pr~
gramming. Structural computer programming, which became· s:tffldard m
rhe 1970s,.involves writing small and self~suflident modules (ca]led in dif-
ferent computer languages s11hroutines,
ftmctirms,
procedttreS,
scripts},
which are
rhelil assembled into larger programs. Many new media objecn are in fact
computer programs that foUow strucmral programming style ..Fo,r example.,
mosc interactive multimedia app]ications are wriu:en in Mac.romedia Di:rec-
oor'sl.ingo ..A l.ingo program defines scripts that oon:c~I.vruriolllSrepeated ac-
tioos, s,.ac·has dicking on a button~ these scripcs are 1J1Ssembledinto larger
S!Ctipts.]n the case of new media objects that are nor oompute:rprograms, an
an.alo,gywith. srrucrural programming still can be maclle,because their pans
ca.n be 11,cessed,, modified,, or substirured without affecting d1e overall struc-
nme ,of,ainobject. This ana]ogy, ho,weve1, has its limits. Ifa. p111ticular mod-
ule o:f:acomputer program is ddeted, the program wiU not run. ]n contrast,
as with u:aditional media,. deleting parn of a new media objien does, not reri-
der it meanfog]ess. In facr, rliie modular stroCCUI'eof nevv media makes such
del,edon and substituti.on of par11S particularly easy.. For examp]e, si.nce an
HT:MI. ,document consists of a 1:1wnberof separate objiects:each represented!
by a ]il]e of HTML code, it .is'l,lleryeasy to delete, substitute,. o,radd new ob-
j~n. Similarly, since in Pllmtod1op 1theparts of a digita] forage mually kept
placed on separate layers,. these parts can be deleted and substituted with a
dick of a button.
Is New Media?
W111al
 3,.Auromatio.B
Toe n1i11mer.iall
coding of media (princ.iple 1) and the modular struJCtureof a
media object (principle 2) all.ow for the all1tomationof mmy ope1rationsin-
voh,ed in .mediacreation, manipwa.tfoo, and access. Thus hwnan i1nteJ111tion-
semantics. This research can be Sttn as pan of a larg,e,i:·
projiect of artificial in-
telligence (AI}. As is well km.1,wn,the AI project has ochiev,edonly llimited
success :sinceits beginnings in the l 950s. Correspomliiing:ly, work on media
genera cion that requires an understanding of semantics is;also,in tlileresearch
1

al.ity ,canbe removed from 11:he creativ,ep,rocess,at teasein part. 10 alll!dis rarely ind1.1dedin commercial software. Begim1fag i 11the 1970s,
Stlllg,e
!FbIILowing ,ofwhat can be called mlow-level"amom.ation
,a1esome ex:a.mp.les ,compumerswere often used 1togen~ate poe·tryand fiction..In the 1990s,,fre-
,ofmedia creation, in which the ,oompUlter user modifies or creates from scratch •Cj;Lleu1rem:s.
of Internet cha1t1,o;;nnsbecame familiar with "'botts"~computer
a media ,obj,ectusing templ.aies,orsimp]e ~!?Prithms. These techniqlLlles are ro- pm,g.am:sthat simulate human conv,ersation.Researchers attNe'II;'Yo,rkUai-
bu.s1t enoug)li.so that they are included in m!D5t commercial software for image v,ers:i,tydesigned a "virruai]theater'" composed of a fo·w'"vi.m.1al actors" who
editing, 3-D graphics.,word processing, graphics fayout, andso forth. Image- ,adjusted their behavior in real-time in response to a wer's actions.,1,1 TlheMIT
editing programs such as Pbocoshop can automatically correct scanned im- Media Lab developeda number ,ofd,iffe.rentprojects dev,i:nedto "high-level"
ages, improving contrast range and removing noise. They also come with a1.1toma.ti1:1n ,ofmedia creati,cmand use: a "smart camera"' d11at,when given a
filters that can automatically modify an image, from creating s.implevariations script,,aumomaticallyfoUowstlile.acitionand frames the slht,oics;' 2 ALIVE, a vir-

of color to changing the whole image as though it were painted by Van Gogh,
Sewat, or another brand-name artist. Other computer programs can automat-
icaUygenerate 3-D objects such as trees, landscapes,andhuman figures as well
as detailed ready-to-use animations of complex natural phenomena such as fire
imdWl1Cerfalls. In Hollywood films, flocksof birds, ant colonies,and crowds of
people are automatically created by AL (arrificfa1life) software. Word pro-
cessing, page layout, presentation, and Web a.:eatian programs come with
"agents" that can automatically create the layout of a document. Writing soft.-
ware helps the user to create literary narratives using highly formalized genre
•ICl!llJ'll'elltions. Finally, in what may be the most familiar experienceof auto-
mated media generation, many Web sites auto.maticai.lygenerate Web pages
on me fly when tlre user reaches the site. They assemble the information from
databases and format ic using generic temp]ates and scripts.
Researche.raa!e also working on wlra.at can be called "high-level" automa-
tion of media crea.tion,whi.ch requues a computer to understand, to a certain
ma.l,envimnment whe11etlheme·r imeroc·cswith animated ch~i!!ctea:s;H and a
o,ewlicincl,c1fhuman-compu,oer i.11toerface where the comp,t11ter presents itself
to a useras an animated talking ch.m:ract,er. The character,,genera·cedbya com-
put,er i1nreal-time, commun.icaceswith the through us,er natural language;
it al:110ui,es to guess the user:s emotio,nal state and to adj lll!St the style of in-
te.racti.10nacc,01dingly.14
the av,erngecomput,er user encountered AI
'The area ,ofnew media 111rhe1:1e
i..n ,!:he1990s was not, howev,er,tlilehuma111-computer interface, but computer
games ..Almost every comme:l:'cialgame included a comioonem ca.lledan ''Af
enigiine,"'which stands for the part of the game's computer rnd,e dmt controls
its characters-car drivers in a ,carroce simulation, enemy fumes in a strate!,'y
game :suchas.CommandandC011q1,1;a; sicgLe attackers .in l'irst-pe:r:sonshooters
such as Qnake.AI engines ,usea variety of approaches oo :simulate human in-
teUig,eoce,from ru1e-baseds:ystem,sw,neural netwod:s. Like AI expert sys-
te:rnis,..
d:1,e characters in computer games have expettiS1ein :samewell-defined

degree,, the:meanings embedded .iridllleobjects being generated,. dm is,.their bur narrow area such as attackin,g:the user. But because ,oomputer games are

m. I discuss particwar cases,ofcmmpmster

auoomarioo of vis.mi.Ioom1Dumicru1ion
im.moie deciil 11. http://www.mr.l.nyu.edu!u1Dpro~/.

;n "'A11Jlllm3lt.imn Technology
of Sight foam PbotognpllJ'!/to,Computer Vision; Ele<tl"li111~r•Clj,l1m;e; 12. http://www-whice.media.miudulvismod/demoslslllrurocam/.
-,,rid Vi'111,:;f
~talion, ed. by Ti~thy Dsruckreyand Mlchae[ Simd (New '\t'brk: Aperture, 13. http://patcie,www.media.mit.edu/ peopJe/pactie/CACM-95/alire-cacm95 .html.
1'9916},, ?erspective, Radrur,
229-239; and in "Mapping :5pi.oe:: aad <Ci:111111,pum
Gnphics," SIG- !4. This researchwas pursued ar different groups at the MIT lab. See, for instance, the home
GRAPH '93 Vis11al
Proceedings,
ed. by Thomas Linehan (New York: liiCM, 1'993)1,14~-147. page of the Gesture and Narraci,·eLmguage Group, hctp:1/gn. v,-ww.media.mit.edu/groups/gn/.

- -
~: f's :1\1,e•s.•
Wlh.al. Medi;,?
highly codifiedand rule-has,ed,. these characters function very effectively; that
is, d1ey ,effectively respond to the few things the user is alfo,wed m ask them
,codo: run forward, shoot, pick up an object. They cannot do anything else,
bnc then the game does not provide the oppormtfr1:y for the user ro test this.
For instance, in a martial am fighting game, I can",task ,qu,esdons of my op-
ponent, nor do I expect him or her to start a convers~l'tion with me. AU I can
do i:s~attack" my opponent by pressing a fewbuttons, and within this highly
codified situation che computer can "fight" me back very effectively. In short,
computer charaners can display intelligence and skills only because pro-
grams place severe limits on om possibfo imeractions with them. Pm differ-
ently, comput,er:s can pretend m be irmellige:ru.only by tricking us inro usfog
a very smmUpart of who we are when we lll)mmunicace with them. At the
1997 S]GGRAPH (Special foteres:c Gro,up on Compm1er Graphics of the
Associario1111 for Computing Machi11ery)ronvemion, for ,exampLe,,I played
against both human and computer-,ooncmlled characters in a VJ!!.simulation
ofa nonexistent sporu game. All my opponems appeai,ed.assimple blobs lll)V-
ering a few pixds, of my VR display;. in this resolution, i.rmade ,ab:soh111tely no
difference who was human am:1who was not.
A.lon,g w.iirb•·1ow-le1,11el" and "'hig.11-leveJ-automation of media crea.tion,
anmher area of media use subj,ec1t 1 ed ro increasing aurnmarii,onis m,edia access.
The SVl'i~dtw compute.rs as a m,e!l:rns of storin,g and aocessfa,g enormous
amounts of media material,, exempl,ifiedby the "'medm assets," s,mred i:n the
databases ofsrn,ck agencies and global emenainment conglome'I·ines, as welt
as public '"media assets'' di:srribllltedacross numerous Web s.i:c,es, created the
need c,o ffiindmore effici,ent ways m classify and seaJ~chm,edia ,objens. Word
proce,sso,rs a11docher rex:t-managemem software has I,o.ngprolll'.idled the ca-
pacity to search for specilic s1tr.in,gsof text and aummar:i,c.aUyindex docu-
mem:s. The UNIX operating .sysremalso included powe.rfill.l commands to
seaDchand fihe:r text files ..In tire 1990s software des.igne~s :st:artedlito provide
media use.rsw.ith similar abi.i!ities..Vi.rageintroduced Virage VIR Imag,eEn-
gine, w!Jich alllowsone ro ,seairchlioirvisually similar image c:wnem among
millions: of.images as weU as a :seitofvideo search tools to allow i.nde,x:in,gand
:seam::hi
..ng video .liies.15 By the erndl,of the ] 990s,, the k,ey Web sea~ch engines:
15. Seehctp::l>'"www
..vimge.com/pmducrs.
- Ill
already included die ,opci,onto search the Internet by speci'lic media such as
images, video,.and aud1o. . ...
The Internet, which can be thought of as one huge dis:tributed medi1a
also, cryscalli 21ed the basic condition of the 11eV1r
damb111:Se, iDformation soc.i-
e:ri,: o'lleJabundance of inlioirmaciionof all kinds. One respoo:sewas the popu-
lar idea of software "age:nt:s" des:igoedJto automate seuclhing for relevant
i,nfonn:ation. Some agents act as llilters thac deliver small amounts ,of infor-
ma.tion given the user's, crireriia. Others allow users to tap, into the ,expertise
d1.eirsel.eaions and choi1::e:s.
of other users, followi.111,g Fm eDrnple,, the ll,ITT
Software Agents Group, de1"efoped such agents as BU.ZZwarch, which "'dis-
riUs and cracks trends,.,,themes, and topics widli.111,c,oUe,ctiolJIS
oftexts acooiss
time" :such as Ioremet discussions and Web pages:;,Letizia, "a user i1nerface
agent mat assists a u.ser bmws,ing the WolllldWiid,e Web by - ... somuiog
current position m fi11dWeb pages of possible inter-
ahead from the 11.1Ser':s
est"; and Foorprinrs, which "uses infonn:arioo [eft by other people to help
you find your way around." 1'6 ..
By the end of the ,cwentieth cencmy, the problem was no longer bow ro
create a new media object such as an image; the new prob]em was how to fiDd
an object rhat already exists somewhere. If you want a panicul!ar image,
chances are it already exists-but it may be easier to create one from scratch
dun co find an existing one. Beginning in the nineteenth century, modem
society developed technologies that automa·ced media creation-the ph~m
camera,, film camera, rape recorder, videorecorder,, etc,, These technolll:g11,es
allowed us, over the course of 15 0 years, to acoomwlate an u:npre,cedented
amount of media materials-photo archives, film ]ibrar:ies,., audio archives.
This led to the next stage in media evolution-the need for new technol-
ogies ro store, organize, and efficiently ,accessthese materials. The new tech-
nologies are all computer-based-media d:atabases; hypermedia ancl ot.lier
ways of organiz:in,g media material such as the hierarchical file system itself;
cext management .,oftware; pro,gnms for co1t11tent-based search and re-
trieval. Thus ,auromu:ioo of media access became dmenext logical stage ,of
the process that ]hrad!been put into motion when the first photograph was
raken. The emerg:enc,e ,of new media coinddes with rllhliis seoond stage of a
a,6.,htq,d/agemtS.www.media
..m.it..ed!llgroupslaigents/projecr:s/.
media society, now concerned as m1111d:1
w.ith accessing am:1reui.11g existing (a ~erm coined by Theodor Adm1110in d-1e 1930s) .is actually ahead of most
,othte·rindustries. The idea that a mstomer might determine the exact fea-
media objects as with ,creating new ,ones.17
tl11Jes, ,of her desired car at the sho"1.•niom,,uansmit the specs to the factory,
4i.,Va:riability aJDdhours, later receive the car, reml!l:insa dream, but in the case of computer
A .new media object is not :something fixedonce and for an,. but something media. such immediacy is reality. Because the same machine is used as both
that ,can ,ex.ist in different., poioentiaU:y.infinite ve.rsioDs.This i.sa.out.her,con- showroom and factory, that is, the same computer ge.oerates and displays
sequeDceof die numeriatl ,cod.iD,g,of media (pri,ncipie l), aod th,e modular med.fa-and because the media exists not as a material object but as d:na
tliiat can be sent through wires at the speed of light, the cusromi.:iJedversion
slDJCltWeof a media object (ptincipk 2).
Old m,edia involved a l:111mancreator who manually assembled 1~ext1L1al,
vi- 1
created in response to the user's input is delivered almost immediately. Thi.IS,
sual,, andJ\or audio elements .into a pMticu.lar composition or seque11ce. This to continue with the same example, when you access a Web she, the senr,er
sequ.e:1-u::e
was stored in some mat,erial,, .itsorder determi:ru:dmice and for all. immediately assembles a customized Web page.
Numetom 11:o_p,iies
11:onldbe nut off fi:om the master, and,. io perliect rnrre- Here are some particular cases of the variability principle (most of them
spondence witb the logic ,of a11 .indmtrial society, tbey wel.'e aU iden.tical. will be discussed in more detail in later chapters):
by variability. {Odier mermsd1at are
New media, .incontrast, is cl:iara:nexfa:ed
oltelil.used in relation to new media :and that might serve as IIIPIP'r,op,riarte
syn- 1. Media elements are Stiored in a mediad.ttaba1e;a variety of end-mer ob-
onyms of ~ar.iabie
are 11111table
a11dlitf1tid.) Imtead of ide11ti11:al
,copies, a new jects, which vary in resolution and in form and content, can be generated., ei-

media object typically gives rise to many different versions. An.dratlber than ther beforehand or on demand, from this database. At lfurst,we might think

being created comp1etely by a.l:n1ma1:1 these ve.rs.i0,nsllil'e,aifoeoiil:Ipart

a11tth1Jr, that this is. simply a particular technological implemenmion ,of the vari-
automa.tical]y assembled bya com[Plli~er
..,(The example of Web p:ages auw- ability principle, hut, as I w.iUshow in the "Database" section, in a computer
matically g1eneraced from databases ming templates created by "i/:eb design- age the database comes to function as a cultural form in .its own right. lt of-
ers can be i1m1,okedhe:re as well.) Tbm me principle oh:arialbilitjl ii:s,
dosely fers a particu.lar modeI of the world and of the human experience. It also af-
fects how the use.rconcei'l'eS the data it contains.
connected m ,automation.
Variabiliicy would also not be p,ossibLewi thout modwtarity. Stored digi-
1
2. It becomes possible to separate the levels of "content" and inter-

tally, rather thao in a fixed medium,, media elements maiotain thei.r :se,p.u:ate face..A rnmzber beci·eated
of,diffarmti:riteif,mm::at1 fi·omthesa111e· data. A new media
identities and. am be assembled in,oo ou.merom; sequences wu:ler pmgram object can be defined as ,one o:rmore interfaces to a multimedia d!a.tab.ise ..18
control. In ad.d:itio,n, beca1.J1Se
the iel1ement:s.
themselves are bmken inm, dis- 3. lnfomiation,a/)11/!ll can ,,eti!led by a ,C01i1p11ter
,the11.J;;u pr~g;r:am to msto111ize ,au-

crete :samples.(for ilfflstance,ao image·is repmesentedas an array of pixels),.th.ey tumaticallythemedia ,co1i1p11i.it'ioJ!I'

,QS weltas to createelr:1,1,~'IIIX
thWM<rlves ..Examples:
Web si:t,esuse information abo,1.u d11etype of hardwill!ieand browser or user's
can be aeaud md •Customilied,c1n the liljl·
..
The log;ic of new media thm couespornds tu the postindusuial 1og;icof network address to cwmmize automatically the site the user will see; imer-
"production 011 demand" and. "just in. ti.me" delivery fogies that were them- activ,e computer i:mrallation,s,use information about the user's body move-
selves made by the use o.f co1mpl!ll1tiers
and ,computer netvmrks at all mient.s;to generate and images, or to ,control the behavior of
stages of ma11ulacturing and dis,uibll!tio11. Here, the ~cwtmre i.ndustry" att.ifid:al.creatmes.

17. See mt"'Jli'll11111t-Garde

as Sofc,.,a;,e,," ,ed. Stephen Kowts (Frankfurt aJf!Jd
imO!!ir.,11•<11ie, New difre•~en1t
1:8.. For an ,experiment in ,crea.ci:11g multimedia interraoe.sro ithe :same,oe><t,.
see my
l!',euJ,tiJsi,.tkyNavigator(hc1p:.llvisam..ucsd..edu/ ~manovichlfLN).
York: Campus,Verllag, 1999) (htrp:.llwis.arts,oc~d..edu/-manovich).

-
 4. A pani,cular case of this customization iis:br:a11chir1g-t)!J!,e
interactivity
imerncth•fry"). The ,~erm refers ro pro-
(sometimes: al'so called "1;ze11'11-bamd
grnms. in wh.ich all the possiblie objiecrs the user can visi.t form a brnioclliing
nee stmcn.ire. \When the user readies a particular object,, the prqgram pres-
ems: her wirh choices and al.lows her ro dmose amon,g them. IDtpe,lldin,g on
the vah1e chosen, the user advarices along a particular branch of the rree. In
this case the informa,cion t!ISedby a program is die ou1tpu1tof the user's cog-
nitiv,e p,1,ocess,rather than the network address or body pos.irion.
5. Hypml1£diais another popular new media structure, which is concepw-
dose to branching-type inceracriviry {because quite often the elements
are connected using a branch tree scmcmre). fo hypermedia, the multime-
dia elements making a document are connected through hyperlinks. Thus
the elements and the srrructure are indepem:lem of each other-ra[her than
hard-wired wged1er;, as in traditional media. The World Wide Web is a par-
ticular implementation ofhypermedfa in whkh rile elements are disrri bured
througl1out the network. Hypene:u is a pa.niicula, case of hypermedia that
uses 0 111ly one media type-text. Ho',!,!'
does the principle of variabiliqr work
in this case? We can think of all poss.ible parru through a hypermedia docu-
ment as being dliffe,rem Yer-sions.of it. Hy foUowing the links,, the use:r re-
trie,,es: a partirnJ!ar versiorn of a document..
6. Allllother way in which difre~enr ,,e1~si,onsof che same media obje,crs are
1.1pdam.
commonli• gel!llerated in oompurer rnlture is ,chmugh periodic Fm in-
stance, modem software applications ,canperiodically check. for updates on
rhe foremet and then download and ins.can these updates, sometimes with-
om any action on the part of ,che user .. M1:1s:1c Web sites are also perioclically
updated either manual]y or amoma1ticaUy;. when the data. in the cla.rabases
tha:t driiYe the sites cha.nges.. A particularl.y interesting case of this; "update-
:abili1ry"feature is chose sites rrnat ,c,cmcinuously update information such as
stock prices or 'l\•,eather_
7. One of die most basic cases of die 1•arfobiliry p,rinciple is scalability,in
which different versions of the same, media objien can be generated at vari-
ous sizes or l E'lr,elsof detait The metaphor of a map is useful in thinking
1
~bout clheKalabili.ty principle. If we equate a new media object with a phys-
ical territory, differem versions of this obfecc are Iike maps of this territory
generated at dHiferem scales ..Depending on the scale chosen, a map provides
more or less d,ecai] about rhe territory. Indeed, different versions of a mew
media object may va.ry srricdy quantitatively, that is, in the amount of de-
- -
Ch.apte,rI
tai] present: For instr.mot, a full-size image and its: .icm:i,.automatically ,gen-
erated by Phocosho,p;; :ai full text and its s!hon:er ver:s.ion, genera,c,edlby che
"Aumsummarize'" ,command in Microsoft Word;. or the different ve.rsions
that can be cr1eat,edusing the "Outline" com:mam:1in Word. Beginning with
version 3 (1'997), Appk:'s QuickTime format made it possible to embed a
number of different versions diat differ in size within a single QuickTime
movie; when a Web user accesses the movie, a version is aummatically se-
!1ectecldepending on connection speed. A conceptually si.mHar techn.ique
called '"distancing" or ~]eve! of detail" is used in fot,ei::acdve vircual wmld:s
as VllML scenes .. A. des,igner creates a n1.1.mbe.rof models of the same
s1.11ch
each with progres:s:ive]y·]ess:detait When the vi,riamlcamera is dose
o!l:ij~,ect,
m, die object, a highly detai.led. model is us.eel; if d1e object is far away, a [,ess
derailed version is aummati.1caHy subsriruted by a p,iogram to save unneces-
sary computation of detail ichat cannot bc!'seeD any,;v:ay.
New media also, allow mm create versions of tbe .sameobject tbat differ
from each other in mme sE1bs:ttantialways .. Here t.he comparison with maps
IDDger works. Examples: ,of commands in oommoruy
of different scales 1rmD,
used software packages: rlliattallow the creacioD ofsuch qualitatively different
veBions are "Variatfom,'" and "Adjustment l.ay,ers" in Photoshop 5 and the
"writing scyle~ op,tion in Word's "SpeHing and Grammar" command. Mme
examp,Eescan be follllld 0111dte foternet wher,e, begi1m:ing in the mid-1990s,
it become common m ,create :a.few,different ¥er:sfoosofa 'Website. The usei
with a fast connection can ,cboos,ea rich multimedia 'lllfrs:ion,
whereas the user
wi.tth a slow com1ecciion caa cboos:ea. more bare-bo•oesversion that loads
laster.
Among new media a.tt'lllorks, David Blair's Wax Weh,a Web sfoe that is an
''adaptation" of an hou:r-1.oog ,;,iideonarrative, off"ers :ai more radical imple-
m.em:arion of che scalability principle. While i.n.tei::accingwith the n.arrative,
the Ills.ercan change the scale of representation at any po,fot, going fmm an
i.mage,-based outline of the mo'l'ie to a complete script or a pairticular slhot, or
a VR.Ml. scene based on this shot,. and so on. 19 Another example of how use·
ohhe scalability principle can create a dramatically new experience of an old
i.9. http://jeffierm111
..villag,e.virginia.edu/wax/ ..
What ls New :Media?
media obj;,ectis Smephe.ira.
Mamber's datahas.e,-,i:ib:iv,en
ll'epresentation of Hitch- c,1Jntain~ngnodes interco111n,enedby refational links." 20 Because fo nev,1 media
cock's; The .IJi,:d,, Mamheir's software ge1oerates a stiU rot
every shot of the .i111dividual
media elements (images, pages of text, eoc.) al'i/i•ay.s
,~etain their in-
film; it mmautomatically combines: all tbe s,tiUs into a rectangular matrix dividual .identity (the pr.inciple of modularity), they ,can be "wi:red" together
one shot per ceU. As,a result, time is spadali2ied, similar to th.e process in Edi- into, more than one object. Hyperlinking is a particular wa.y of.achieving this
son's eady Kinetoscope cylinders. Spatiali.::i:iag the film allows us to study its wiri:ng..A hypedink Deilltll:'S
a connection between two eleme,i;n:s, for example,
diffe.ren.t c,emponl structures, whid:i woutd he·hard to observe otherwise. As betw,een two words in n'ilo different pages or a sentence on one page and an im-
in Wli:i:Wei!I,,
the user om a,t any point chan.ge the scale of represematioo, ,g!O- age in another, or two different ploces within tire same page. Elements con-
ing from a.complete film to a pankular shot. nected thro1.1gh hypedi:nks can exist on the same computer or on different
As can be seen,.the principl,e ,ofv,ar.iability is useful in alllowing us w con- computers connected on a network, as in the case of the World \'i7ide \l'eb.
necit immy impo.nant cha:racterisdcs of niew media that 011 fill'.St
si,gh1tmay If in old media elements are "hardwired" ineo a unique structure and no,
appear wnrebted. In part.ic1.1lar, sud1 popular new media s;nuct,ur,es as longer maintain their separate identity, in hypermedia elemems and struc-
branch.~n,g ,(,ormenu) interactivity and hypermedia can he seen as particular ture are separate &om each other. The structure of hyperlink:s-t),pical!y a
instances of the variability p.rindpl,e .. In the case of branching interactiv- branching tree-can be specified independemly from the come,ms of a cloc-
ity, tbe us,er plays an active rol,e in det,ermining the o.~der in which aheady umem. To make an analogy with the grammar of a natural language as de-
gen.era,t,edlelements are acc,essed. This is the simplest kirid of i1111t,eracti1vity;, scribed in Noam Chomsky's early hngtJJistic theory,2 1 ,ve cam rn,mpare a
more oomplex kinds ai:e also p.oss.iMe in which both. rbe elem,enu and the lliypermedia structure that specifies. connections betw,em nodes with the
structure of tbe wh.ole object are ei.ther modified o.r geoot:ated ,on the fly in deep stmauire· of a sentence; a particular hypermedia t,ext can then be com-
response tio the user's interaction with a progxam. We ,can refer m s11Jch par,ed 'l'l,l'id:i
a p,articular sentence in a na.rural lang1m,ge. Another usefoI anal-
as epe,i.
impleme,11.t:atio1JS ,iwer:activity
to distinguish them fi-om tile 1d0Sed
in- ogy is rnmpmer programming. fo pmgramming,, there is dear separation
ter:aaiv:a,1tbwtlJSe.5 fixed elemea:cs arranged in a fued bmnching :structure. ibet-...
1een idgorithms and cLua.,An al.gorithm sped.fies the sequence of steps
Open. ioreiractivity can be impLemented using a variety of approaches, in- t,c1be performed on any data,, jw.t as,a h}rperrnedia stm,cmr,especifies a set of
dudit1g procedural and object-oriented computer progtamming., AI, AL, na~·i.gu,i.011paths (i.e., connectio111s;between nodes) that portentially can be
and neural t1e.wo,rks. i1pp[ied to any set ,of media objiects.
As long ,as.there exis.ts some kernel, some structu,e, some pr,oro:type that 'the pr.indple of variabiHty e·xemp]ifi:eslio,w, hi:storicaU,·, cha,rigiesill me-
remains unchanged throughout the interaction, open interactivity can be dia ted1111o[ogiesal)e couelaoed 'ili•id1social cha:n,ge. If the l1:1gi.,c
,of.old media
thoughit ,of as a subset of the variability principle. Here a useful analogy ,om:i~espondedt!o the log.ic of inidh.1suia]
mass sodety, du:!'logic of new media
can be made with Wiy:gernstein's theory of family resemblance, later de- fits d11e.l1:1gicof the poscind1.1nrial society;, which vallue:sindividuality over
vel,oped into the theory of prototypes by cognitive psychologists. In a fam- rnnlionnity. Jn industriall m:ass so6ety everyone was sup,po.sed to enjoy the
ily, a number of relatives will share some features, although no single same good:s-and to sha1:1e
the same beHefs...This was also the logic of media
family member may possess all of the features. Similarly, accordfag to the t,ecbnoLo,gy.. A media object 'll•as,asse·mbfod in a media. fa.ctmy {such as a
theory of prototypes, the meanings of many words in a natural .language Ho.Uywood studio). Millions of identical copies we1Je pmduoed from a
derive not through logi.cal definition but through proximity to a certain
prototype.
Hypermedia, the other popular structure of new media, can also be seen as a
particular case of the:more general principle of varfa:bili.ty. According to the 20. frank Halasz and Mayer SchMJrtz,,"The Dexter Hnierrext IReferenoeIMDll!el,"
C,-,,i,i,_.,;_
definition by Halasz and Schwartz, hypermediai. s;;1stems "provide their users ,,;~Jiow,of.d,e
AClll (New York: /\CM, 1994), 30.
21. N,= Chomsky,SyntacticSm,.m,,,l!J
,('TheH~gDe u<i Paris: MmmlmJ,,i '9'Ji7}
with the ability mcreaite, manipulate andJor examine a network of information-

-
Cha11ll!rl

-
mas:ter am:I d.is;irr.ihmed co all rhe citizens. Broadcasting, cinema, and prinr onl.y art. Second, Ippol.itlo fullows the tradition of conceptual a.rt in whkh
media al] followed this logi.c. an artist can vary any dimension ,of the anwork, el•en i.ts ,content; my use of
fo a pos.1tim:lusuial sodeqr, ,every citizen can. ccmstruet her own cusrom the term aims to .reflect the lo:gic of maillStneam culmre in thar versfo.ns of
1

lifest)•l.eaml "sdecr" her ideology from a farge (hue nor infinite) number of which can be a welJ-
the object share some weU-defined "data .." This "dlata.,"
choices. Rather than pushing the same obj:ects/informarioni tom. mass,audi- known narrative {.PiJthl:l),an it.con (Coca-Colla si.gm),,a chairacter (Mkkey
ence, mad::ecing now tries, to target each individual separate.I)'· The logic of Mouse), or a famous: s:tar ()fadonna), is refer:red m itn the media industry as
new media technology .~eaecrs this 11ewsocial logic. Every visito:r ro a Web "property:' Thus aU cul1tW.1ml projects prod1w:ed by Madonna will be auto-
sic,eaur,ornnically ge:ts her own ,cos.comversion ofthe s:.iteicreated!,on the Hy matically unit,ed by her name. Using die !theory ,of prototypes, we can say
fmm a datab!ISJe.The .!a:rn,g11a,ge
of the text, rhe comenu, the ad!sdispfayed- that the property acrs as a prototype, and different versions a.rederived from
all these um be cusromized. According ro a report in USA Today (9 No- this prototype. Moreover, when a number of versions are b,ejng commerc.iaUy
••emlber ]999), "Unlike ads in magazines or ocher !t1ea!-wor!dp1:.1blications, re·]easedlbased. on some "property;' usually one of these versions is rreaced as
'banner' ads on Web pages change with every page view. And most of the the soooce of the "data,." with others positioned as being derived from this
companies chat place the ads on the Web sire track your movements across ,s.ouroe.Typically; the version rhat is in the same media. 111s.th.e o,rigfoal "prop-
the Net, 'remembering' which ads you've seen,. exactly when you saw them, ,erty'" is created as chie s01.1roe. f'o,c inn.ance, when :111
movie s,rodio rele!ISlesa
whether you dicked 011 rhem, where i•ou were ar ·the rime,. and the site you game based ,on i.t,.,product tie-irni, music
0 ,ew:film,along witl1 a ,comp1:.11ter
have visited just before:• 22 written for the movie, ,et,c.,the film is usually prese1med as the "base~ objea
version of the oomp]eni texr by
Every hyperteirc reader gets her 01111n from which other objecu are derived. So when Geo~g,e Lucas releases a new
'.Sdeccing a paniruUllr path through it. Similarly, every user ofan il!lteraetiive Star Wan movie, tllt.eodgiinal property-the origi.ll'lalStar War;trilogy-is
in.stall:ati,cmge•,tsher own version of the work. And so mi. fo this way new referenced. The new mo,.,i,e becomes the "base" objea., ru1dall other mttl ..ia
media tech11oito;gy
acts as the moin pe.1rfoctrealization ofrhe utopia of:an ideal objects rel.easedal,cmgwid1 it refer to this object., Go11Ner.sely, when oomputer
s,o,cieirycomposed of unique i.ndi.viduals .. New media obj1e·c,csassu1re users .Raiderare remade into m,rnvies.,
games such as l'tll!'llb rhe original computer
chat d1eir d1oi,c:,es-and therefo,re, thei.r underlying tfu.01..1;gl-us
and desiJ11es- game is presented as die "base"object •
are urniqMe,ra.ther than preprogra.m:med and shared with others,. As though .Although I deduce the principle of v:ariability from mo~e basic pri1[11cipl,es
trying to compensate for d1eir e.adier role in making us anrime·same,, de- of new media-muner.ical representation arndmodular:icy ,of information- .
scenda1m ,of cbeJacquard loom, thie Hoilerith tabulato,t,. andlZuse's cinema- the principle can also be seen as a .comeq11ence of the computer's way of rep- ·
computer are now working rroconvince us that we are all uniqrue. resenting dara-and modeling the wodd itself-as va.riables rather dun
The princ.ip[e of variabai1ty as presented here has som,e parallels m the oonstanrs. As new media theorist and an:hit,ect Mar.cos No,,,,ak notes, ,a,ro,m-
concept of "variable media," developed by the artist arnd,curamr Jon Ip- pmer'--and computer culture in its w:ake-substirutes every constant with ''
polir,o. 23 I believe that we differ in ,rwo key respects. Fi1rst, [ppo.1:itouses vari- a wriable. 24 In designing all functions a:nd data .smicrures, a computer pro-
abilitJ' ro describe a characteristic shared by recent corweptrual. all'ldlsome grammer tries :always ro use variables rather dun constants. On the level of
digital arr, whereas I see variability as a basic condition ofaUnew media, not the human-romp111c,erinterface,mis princip]e mm:ns ·ltbatme user is g.iven many
oprjom to mod,ify che performance of a progmm. or a media object., be it a

22. "How Marketers "Profile'Users," USA Today9 November 1999, 2A..

23. See htrp://www.three.org. Our conversations helped me ro cLar.ifyrnr ideas.,and [ ,un ·,oery 24. MaKOSNowk,, lec~w:e at die '1nreractive Frictions" oonferem:e, University ofSol.l!tbern
grateful ro Jon for the ongoing exc.han,g,e
.. California,LosAngeles,6 J111l1111e
1999.

What Is New Media?

compute.r gam,e,, Web site, Web browser, or the ope·rating system itself. The
tb.eir genders, or otlhecwi.se alter them, I should make her boyfriend a husband, I
user can c.bim,gethe profile of a game chara.crer, .modifyhow folders appear
should e:i:p,lkate all the tributaries of my extended family (its remarriages, irs in-
on the desktop,,, how filesare displayed, wlhat icons are used, and so forth. If I shouM novelize the whole thing, I sh1ml.dmake it m1.1ltigener-
p,oliric:s),,
te1me,ci11e
we apply thi:s,pri:11cipI,eto culture at large, h 'ili1ould mean that every choice
a1:i1mal:,I should work in my forefu•then(stonemasons and rnews.papermen),,I should
resportSible tfor gi.vi1t1ga cwrura.l
object a unique identity can potentially
let anific,ecreate an I should make the events ,o,rderl)'',
:surfac,e., I should wait
remain always, ,open .. Size, degree of detail, formait, oolor, shape, interactive
and w·rite about it later, I should wait until I'm not angry, [ :s.nouMn'itduner a nar-
trajectory, trajec1:1orythrough space, durat1on, dlJ11tbm, point of view, the
rati"e w·i1thfiragments, with mere recollections of good times, m with regrets, 1
presence or absence of panicular characters, tlhe devdopment of plot-to
should mail:e Meredith's death shapely and persuasi11e, not blunir and disjum:tive,]
name just a rewdi.mensio,ns ,ofrulrorall,objects iD diffe,rent media-can all
sh•uild•1"thwveto think the untbi,nkahle, JIshouldn't hwvem suHe.r,,I .~ho,:.1ld
address
be defined as varfab1es:,,oo,be freeiy modified by a user:
bierh,,er,edirectly (these are ,the ways] miss }'DU), I sliiouldlw•ri,te0111lyofaffecrion, I
Do we want, or need,, such freedom? As the pioneer of interactive film-
make our travels in ,clhiserut:hlylandscape safe and sernme·,
.sho111d I shouJd have a bet-
making Grahame Weinbren argues, in relation ro interactive media, making a
itere11di.a,g,.,
I shouidn't say her lifo W3Sshort and often sad,, I shm.!ldn't say she had
choke involves a moral responsibility. 2$ By passing on tlhese choices to the user, demi:ms,as.I do, coo.
the author a!.sopasses on tlne responsibility to represent the world and the hu-
man condition in it. (A parallel is the use of phone or Web--basedautomated
'5,,Trans,coding
mem1 syst,ems by big companies to handle their customers; whil,e companies
with the basic, "mate1rial"' principles of new media-mimeric
iBie;g.inni:IJlg
have tumed m such systems in the name of"choice~ and ~freedom,"Olle of the
coding amidmodular organiz:1uion-W1,r,e mmred to more "'deep" :and far-
effects of thi:sqrpe of aummation is that labor is passed from the ,c,ompany's em-
1:1eaching,ones-automation allld ~•aJ"iability. The fiftlh and bst principle
ployees ro the cwtome!. Ifbefore a customer would get the imonnatioiri or buy
of cidtu:ral.:transcod.ing aims to deocribe wliat in my view is the most sub-
the product by 1nt•emtting with a company employee, now she 'lmas
m s[Pend
of the oomputerfaation
s,twmi:al co,ns,equellllCe of media. As I have suggested,
her own rime· and energy navi,gad:rl}gthrough numerous menm c,o acc,omplislb.
cornpme,rimtfon turns media into oomp1.1ter data. While firomone point of
the same result,), Tire moral anxi,ety mait
ilvcompanies the shift from corista11ts
view·,oomputeriz.edl media still displays s,tmcmral organ ..ization that mwkes
to variables, from traditions ro d1oi,c,es,
i.n all w:eas oflife in a ,001111t
1emporary so-
sense ui, i.ts human users-images feature recognizable objecu;; text 6ies
ciety, and the ,wll'.,esponding am:i,ety of a writer who has m port:ray i.·t,.is,well
consist of grammatical sentences; virtual spaces are defined along the famil-
rendered in the dosing ~ge ofa shon :story by rbe contempomey Ame:rican
iar Cartesian coordinate system; and so on-from another point of view, j cs
writer Rick Moody(the story is about d11edeath ,ofhiis sister): 216
4
strucmre now follows the established conventions of the computer's org:ani-
Zlltion of data. Examples of these conventions a:re ciiffere111tdata srmctures
it more, I should ,con,cf1lll
I should liic:t.ionali2le m:~•sdf..I should ,coos,ide,rthe· respon-
such as lists,. records, and arrays; the already-mentioned substi1t1UtionofaU
her rwo children i1uo,ooe,, o,r [lf'l'l!ne
sibilities of c!baracterization, I s.hould ,c,o,nJlht,e
constants by variables; the separation between algorithms and dw1rastruc-
tures; and modularity.
The structure of a computer image is a case in point. On the level of rep-
resentation, it belongs on the side of human culture, automatically entering
~1n ,the Ocean of Sueu:M lllf Story; l'llillemiium•FilmJo,,,naf 28:
25. Grai:mne Weinble111,,
in dialog with other images, other cultural "semes" and "mythemes." But on
{Spring t'Sl'!l'.i).,
lil[tp:llwww.sva.,edu!Ml'J.ljounialprug,esl'Ml'J28IGWOCEAN,HTIML
another level, it is a com put er file that consists of a machine-readable header,
2'6. Rid; .1¥[,oodl1, first published in Conjunctions,
Dem011olog,1, reprint,edin TheKGIi LY Reah,
qoooooli,n 'li'inc,ell'assaro,"Unlilie!J' St11.rie1,."
H"'~I Magazinevol. 299', no.. l 191 (A1.1g1JSt followed by numbers representing color values of its pixels. On this level it
1'99'9).,
:BS--8'9. enters into a di:tlog w ich ocher computer files. The dimensions of this dialog
rure not the image's content, meanings, or formal qu:tlicies, but rather fil,e

- -
What ls New Media?
 size, file t) 1'Pe,,tivpe'ofcompression used, file format,, and so on. In short, these
dimensions belong to the computer's own cosmogony .rather than ro human
culru,re.
Similady,, new media in general can: be thought of as consiis:dngof rwo
dininn layea;s-the "rnl1rura.l 1!!1.ye.r"
and the "computer E:xamp.les of
rnregodes belonging ro the cu!1rural layer are the encydoped.iia at1cl,the shore
story; story and plot; composition and point of view; mimesis and catharsis,
corned): and tragedy. Examples of categories in rhe comput,e:r la.yer are pro-
cess: and packet (as. in da:ra packers transmitted through the network); son:-
ing and marching; function and variable; computer language and data
srrm:mre.
Because new media is created on compmers, dis:tributed. via computers,
and stored and arrhivecl cm computers, ,he fogfa::of a. computer can be ex-
pected ro signifirn.nd;, infft.1encethe traditional cuhuraI fogic of media; that
is, we may expect that the computer layer wm.affect the rnl.tuta! layer. The
ways in which the, computer models d:ie world, represernitsdat:ll.,,andl alfows:
us ro ope.rate on it; the key operat.ions, beh,iod all computer programs (s:uch
as s,ea11rch,
march, sorr, and nl,~er);;the ,mn.ve.mions ofHCI-i 11s:ho,n,,what
can be called the mmputer's ontology, epistemology, and pragrn,arics-
influence the cuhural layer of new media, its organization, .its ,emem:ging
genres,,, fos:contents.
Of mu;rse, What I can "the computer layer" is not itself fo:iedbur rather
d1ange$o,ver rime ..As hardware and sofrware keep evolving and as t:be com.-
purer is used 1for new rasks and in 111ew
ways, this layer undergoes, ,ooaidn,11ous:
mmsfmmatio•ai. 'The new use ofd1e comput,er as a media machine i.sa cue, in
point. This use, .iis:
having an efkct o.n rh.e computer's hardware and s:oifmrare,
especially on tbe l,evel of che huma,11-computer inrerfuce, whkh inc1:1eas:ingly
resembLes: the iimerfaces of olde,r med.iia.machines and rnl1cural. mechnol-
ogies-V1(R,, tape player, photo camera.
fo summary, dhe computer lay,er and the culture layer influence each
otl1er: 'i'o u,se, another concept from new media, we can saJ• that rhey are
being ,c,om[POSi!~ed together. The, res,1.dt,ofd'JiS composi~e is a new compurer
culture-a l!:ll!endof human and ,oompmer meanings., of tracli.tional ways in
which !m.ma,11C!L!lm.re
moclelecl the 'i'i~ori.d
and the compm:et's own means of
represemi1:1g i,t.
Thmugbm1t dhtebook, we wiU encounter mani• exampl.es of the principle
of transood:i111,g
at work. For instance, in "TlrneLanguage ,of Cultural Inter-
faces,'.'we will look at ho,w conventions of the primed pagie,,
ci.n.e,ma,and crn:-
dhi,onal HCI interaa i..11 d1e interfaces of Web sioes., CD-ROMs, vinuat
discuss how ,adata-
spaces,. and compumer games,. The "Database" seaito,n w:i11.
oase, originally ,a comp11t,er technology to organize and access data, is be-
coming a new cuhural form in iits own right ...But we am also reinterpret
some ofche principles of new media already di:scussedas consequences of the
transcoding principle. For instance, hypermedia can be undemood as one
cul.mral effect of the separation between an algorithm a!ild a dJat:a.scmcrure,,
ro computer programming. Just as in programmfog., where al.go-
esse:l!l.ti.al.
ridims: and data structures exist independently of each 01tber,, fo hypermedJia
data is,separated from ,chemrwigatfonstructur,e. Similarl;i':,,
the modular scmc-
,rumeof new media can be se,en as aim eflecc of the modularity in struaural
c,omputer progra:mmi1ng. J1JJ:stu a s:tmcruml c,ompu~er program consists of
sm,aller modules that i1Dtum ,oonsist ofeven small'ler modules, a new mediia
,object has a modular s:uucrume.
In new media li,ngo,, to ''minscode" something is: to translare it inm an-
other format. The comp1Ltt,e:riza.tfon
of culture graduaUy accomplishes s,imi-
lar trariscodin,g in relation to alllicultural categories and concepts,. That is,
cultural caregori,es: .and con,cepts are subsdrurecl, on the level of meaning
and/or language,,, by new o.11esthat deriv,e from the computel"s ontology,,,
epistemology, and p:ral/gmatics:.,New media rhus acts as a forerunner of this
more general pmc,ess ofculturaI 1econceptmli:mr:io-n.
Given the p.rncess:of '''conceptual transfer~ from d11ecomputer wodd co
culture at large,,aind gi:ven die new status of media as computer data,, wba.t
'theoretical framewmk caia w,e use tc understand i.t?'On one levd new med.ia
is old media that has: beel'.Idig,itized. so it seems: appropriate to look at new
media using the perspective ,of med.ia studies .., We may compare new media
and oikl media . .such as pd.111t,
photography, or te:I~ision. We may also ask
about rile conditiom of dis1trib1.ttionand recepdoa and patterns of use ..We
may also ask about simiiari!Cies:a.nd differences in thie material prope.1t1cies
of
eaidl medium and how these, Jfea their aesd1edc possibilities.
This perspective i.s .important and I am using it frequently in this book,,
!but it is not sufficient. h ainnot acldress the most fund.amental quality of
new m.edia that has no, hi:smrical precedent-programmabi]icy. Compar--
fog aiew· media rt!oprint, plhornigraphy, or television wil] never tell us the
wbole story. Fo1 akhoogh from one point ofview new media is indeed another
type of media, from another it is simply a particular type of compu,oer data,

- -
Cih'(pter 1 What Is NewMedia?
something sto~ed in files and databases, r1etr~ev1ed and .sorted, run through al-
goritlhm.s and written, to the output d.evioe. That the data represent pixels
and that:this devke happem to be an orutprutscreen is beside the point. The
compat!er may penorm perfectly the :llole•of,the Jacquard loom, but under-
neath it .is fundamentally Babbage's Amtallytical
Engine-after all, tltis was
its identi.ty fo,.rno years. New media may look like media, but this is only
the surface.
New med.ia ,calls for a new sta,ge .in media theory whose beginnings mn
be traced back ,~othe revolutionary works,ofHarold Innis in the 195·0sud
Ma:csball McLuhim in the 1960s. 'fil:1)u11dentand the logic ,ofnew media, we
to oomputer science. k i:s tbere that we may expect to find tbe
need tio t1.11t111
new tei:ms,, categi,ries, and operations, that cruu:acterize media that becaime
programmable . .from mediastudies,we mowt,os,omething that canbe·,called~j~ft-
warejtudies"'-from mediatheoryto softrwirethef)ry;The principie ,of tn.11JS,cod.-
ing is oaie way to :start thinking a:bout software theory. Anod11erway, wll1uich
this book experimet1ts with, is to, use c,onc,eprsfrom compucer scieBoe :ascait-
eg,mriesof new media theory. Examples ltefe are "interfuoe" and "database:'
And last but oot le.ast, along with .atllillyzing"material" allcl l,ogical prin-
ciples of computer batdwai:e aind so,£i::,.,i;are.,
we can also look at the human-
,computer in1:1edaoeand the interfaces of :so,ftwareapplications used to author
and access new media objects .., Tbe two ,chap~ers that follow a11ede,r,ooed'00
these topics ..
- -
Clhil!ipierl
Having proposed a list of the key differences between ne"wand o!.dmedia, l
now would like to address other potential candidates. Folfowi.111:g
are some of
the popularly held notions about the difference berween new and okl media
that I will subject to scrutiny:
1. New media is analog media converted to a digital representation. In
contrast co analog media, which is rnnti1:n10us,digitally encoded media is
discrete.
2. All digital media (texts, still images, visual or :audio time data, shapes,
3-D spaces) :share the same digital code. This allows. different media types to
be displayed wing one machine-a computer-which a,ns as a multimedia
dispfay device.
3. New media allows for random access. In contrast m film m videotape,
which store daca· sequentially, computer storage devices make it possible to
access any data element equally fast.
4. Digitization i.nevitably involves loss of information. In contrast to an
a11alog representation, a digitally encoded representation. contains a fixed
amount of information.
5. In contrast to analog media where each successive ,mpy loses quality,
digitally encoded media can be copied endlessly without degradation ..
6. New media is interactive. In contrast to old media where cbe order of
presentation is fixed, the user can now interact with a media object .. fo. the
process of interaction the user can choose which elements to display or which
paths to follow, thu:s generating a unique work. Im this way the user becomes
the ro-amhor of the work.
WhaI h New Media?
 Cinema as New Media (whether the intertides ,ofd1e silent era or the tide sequences of the lacer pe-
If we place new media within a longer historical perspective, we wmsee that riod) for illwhole cenclll1'· Cinema was thus the original modem umukime-
many of the principles above ar,e .not unique to new media, but GUl be found I dia.~ We can a.Isa poimitto much earlier examples of multip]e--mediadispfays,
in older media technologies as wdl. _[will ilJustrate this face by using thieex- such as medieval illuminated mm1uscripts that combine rext, graphics, and
ample of the technofogy of cinema. represemationa! images.

0) New media is analog media convened to a digital represenrntion. fo con- (3} New media allow for random access .. In contrast m £iillmor videompe,
trast to analog media, which is continuous, digirally encoded media is discrete. which s;rore data sequentially, computer storage devices make it _poss.ibleto ac-
cess any dam element equally fast.
Indeed, any digital representation consists of a limited number of
samples. For exampfo, a digital still image is a matrix of pixds-a 2-D sam- For example, once a film is digitized and loaded in 1he compmer's mem-
pling of space. However, cinema was from its beginnings based ,on sam- ory,, any frame can be accessed with equal ease. Therefore, if cinema sampied
pling-the sampling of time. Cinema sampled time twe.111ty-fu11r
times a time bur stiU preserved its linel!Jrordering (rubseguent moments of rim,e be-
second. So w,e cam say that cinema prepared us for aew media., AU char re- come subsequent frames), new media abandons this "t1111mU1-<centeredn
rep-
mained was to rake chis already discrete representati<em and to quantify it. resentation altogether-co put represented cime fully under hwnan control.
Bm thi1sis,simply a mechanical step; what cinema accomp.lisihed ·wasillmuch Tfme is mapped onto two-dimensional space, where it can be managed, an-
more difficult conceptual break-from the continuous w rh,e disc:r,ete., alyzed,. and manipulated more easily.
Cinema is not the only media technology emerging mw:11rdthe end of the Such mapping was already widely used in the ninereemh-cenrury cinema
nineteenth century that employed a discrete represeorado:rn. ]f cinema sam- machines. The Phenakisticope, the Zootrope, the Zooprmcisrope, the
pled time, fax transmission ofimages, starting in 1907, sampled :a..2-Dspace; Tachyscope, and Marey's photographic gun were all based ,on the same prin-
even earliier, the fim td,evisim1 experiments (Carey 1875; Niplk:oiw Ul:.84)al- ciple-placing a number of slightly different images aro1u.m:I perimeter of a
1th,e
ready invol\11edsampling of both rime and space.2 1 However, :reillchi.11,g
mass circle. Eve.n more striking is the case ofThomas iEdison'sfirst ci.nema apparatm.
populari t)• much earlier than these other technologies, cinema WllS the fas:rto In 1887 Edison and his assistant, William Dickson, began experiments co
of the vi:suallpublii.,clk:now.ledge.
make the principle of discrete 1,eprese11t11tion adopt the already proven technology of a plhiom:ig.raphrecord fur recording and
displaying motion pictures.,Using a special pia:1.11re-recordingcamera, tiny pin-
{2) Ail digital media {c,exts, :sciU images, vjsual or a1:1di111
time data, shapes, point-size phoo~graph.s we.replaced in spirals on a qdiru:Iricalcell similar in size
3·-D spares) share the same digital code. This allows d~frer,em miedia types co to the phonography cyUnder. A cylinder was m bold 42,000 images, eaclh so,
be displ:a.yedusing one machine--a compucer~which 111!:ES a
:1115 :nr1111ki,media small (X, inch wide) toot a viewer would ha'l!lero Iook at them d:irough a mi-
disp!ary devioe. croscope. 28 The ,storage 01.pacity of this medium was twenty-eight minures-
twenty-eighic minu,tes ,ofrnminuous rime taken apart, .llattened on a surface,
Abhou,gh computer multimedia became commonplace o1r:ily amullld and mapped ,oncoa i:wo-,dimensioool grid. (In sho,n,,ri.mewasprepared fur ma-
1990, filmmakers had been combining moving ima;g,es, s,m:md, and ,text nipulation and reooclerin,g,,:something soon to be accomplislhed by film edimtS.)

27. Albm Albramson, E.learoni,;Alru':imPia,,,.l!l1:II ffotory of 1he'C,t,.,;,;,,,,

c~...,,, (Be,rl.:d'e.,, 28. Charles Mmser, TheEmr,ge,uo/Cimm.a:TheAmerica'S'ma to 190'7{Berkeley: U nivas:i,cy
Unive-rsiry,ofCalifornia Press, 1955), 1:5-24. of CaliforniaPress, 199'1),,65,.

- -
C~a,pte, 1 What Is New Media?
$?.·.,~.,,: -------
f

.. . and
ohuio11 .'-.. coma.ins
. a. fi:x,ed amm.mt of ..mformation."'9' from a logical p .
The Myth of the Digital o f view , tuts .--nruic·...c1
1~,e is· a correct dedulC'tion from the 'd fd' . l 01nc
'sentaf Ad' · al!• · 1 ea O 1g1ta repre-
Discrete Rp,rese11tarion,random access, muhimedia-cimema al.readycon- . d', _mD. .1g1t .11mageconsist:rnfaliDite1mmberofpi·x,els eachh .
tained these princip]es. So they ,cannot help us to separatlenew media from :a is.u11ct c,olor or tonal value a d . . .. , , ' avmg
det ·1 . . , 11 chis number determrne:s the amount of
old media. let w co,n1tin11e int!errogating the 1:1emaining pri11ciples.If many B. 1111
11an, image
d f can rep.resenic. '£et in r ea11.ty · this
. . d1fforence
.. not matter
,d.o,es:
prindptes ,of.new media turn out to be not so Dew,what about the idea of II t ,e;- f'.11
sam . 1990s,,,fT!<e.11
. o. . the . ,cheill . .
p consumer scarme.rs were capable of.
digital 11epresentatiion?Surely,dus is the one idea that radically redefinesme- m,ng images at resohm,on:s of 1,200 or 2 400 .. j, . , . .. .
a digin.l]y s,mrecl ima .. , .... . . ' pui:e s pe1 mch. So while
dia? The answ,e[ is 1mt so straigbtfurwacd,, lm,we~~ became this idea acts ge m:ss,t111 .. rnmpnsecl of a finite in1.mber of . .
as an umbrella fo•r 1tluieeunrelated roncepcs-a:11alog-to-digital rnnversion such. rr,es,olution
.. ii::can contain much.. finer. d etiu·1 tnan
'- w1:s,e~•,er
, possib]
pn:els,,.. ath
numedcal represenra- d
(digitization), ,11.
,common representat~onaI code:,, era, itm.u.l photography; Tbi.s 11ull.ili h . .. .· . ewn
"irr'e·" . . es t e whole d1s,u11(UOll between an
tion. Whene'll1er we ,claim that some q11ialicy·
af new media is dllleto its di.gi- .'-' ut11eeamount of.. mfunnat.
fixed, . um, ma.. commuou:s-tone
. photograph" and a
tal status, we need to specify which of tbese tbree concepts is ac work.,For amountofdet&lmadig·i•11l .. , · Th e more re1eva
'· · ·1"·mage ' · ·
much information i . , .. . , · · · · nt question, 1s:how
ai::aropte.,dJ1e.factthat different media c!ILl'.!I
be rnmbined into a single digi.tal n ,Ml imag,e ca11be useful m £he viewer By t'" d f
file is due ~o the use of a comm.on feprese,11tatio,ml code, whec!!easthe ability new m d" ' fi d · ' ue e.11 o
digital ei;:asge ~:u'edcade,.lt,eclhnol~gy had akeady reached the poim where a
to copy media without intmd111dn,g degradation is an effect of numer~cal • eas1 Y contain much · fi .
would ever want. more m ormatmn than anyo,ne
represe111tatioa.
Because •of this ambiguity, I tcy' ro av,oid using the wo:td. digit.alin this But
. even t he pixe · I-based representation wh. l
essence of digital i . , ic 1 appears to be the very
book. In "Pri,Ddp,lesof New Media" I showed that m,merical r1epresenu1tion magmg, cannot be taken for g d So
is the one i,eallycrucial concept ,ofthe tl:u:,ee.NwnericaJ repres,e1rm1.tio11
turns graphics: sofrware h.as bypassed th . . . ranee . . me wmpmer
grid-liixed. 'res I . . . . ' , e mm.n i.umitation ohhe nad.itional pixel
media foro computer data, thus making it programmable. And this indeed . . , , o ut1on. Ltve P.u:111n',. im image-edi .
pixel-based ima e · . trng program, converts a
radically changes the nature of media. . g m!Joa set of m,athemari,calequatioM 'T'-. II
•t,o,wo,·•11. •h . . . · • uas a ows the user
In contrast, as I will.show below,. the alleged principles of a.ewmedia that , 1·
.....,wu.. .an1mageofv1[1tlllall..'.I'u.11 .. d resolutmn
1m1te . ..Another paint
are ofoen dediu~edfrom the concept of digitization-that analog-to-digital gram,,, J\l,l.atadoll'; , ,.. 1ng.,ona uny
makes possibl.e pa1·:,n,.: . image
. . h.. a. . pro-
o:fjust a fe · . ]s ,_ . . • , ,,w IC1,1may consist
wnv,ersion inevitably results in a loss of informat:iun llllld that digital copies w pcre , as tnou,g;.bl.tt were a hi h- I .. . 1..
this by bfeakin each . g reso unmi mill/g,e ..Ut achieves
are 1dentical to the original-do not hoid up uoder closer e'Xaminati.on;,that 1
g . m1toam1mberofsmal[enulb- . ,1
is, although these principles are indeed ~ogkall ,coaseq11ences,ofdigitfa.ati.on, grams, the prnel is no lmi<>era '"linal fi·•ont' "· f: ptxe s.) In both pro-
. . " · • 1er as ar as the u ·
tlley do not apply m concrete co,mputer t,eclhn.ofogies in the 'lli'alli1t1 whi.ch Jt simply does not exist li . .' . ser is rnncemed,
fixed r l . . . exture-mappmg algorithms make the notion of a
they are rurreody used.
"' ima e esoa utmn .. meanmgless . in a diffierent way. They ofoen score the same
g t a number of chfferent resolutions. During r d . , 1h
,(4) Digiti.zation inevit:ablyin'i'•oh,esloss ll(finfurmation.11• comrast to an anas map of arbitrary resolution is produced b . . en en~,g, t. e texture
1og rept,e~entation,a d(gic:allyenc,odedrepresentation comnmins,
w fi11:daimount
dosest w,this resolution (A . ·1
·
~ mte~polatmg two ima.ge'Sd1at 3.IJe
sim1 ar technique 1sused b VR. __JI:.... 1 , ..
stores the, number of . f . . Y ""-"u, are,, which
of i,rulio:rmacion. detail.) ' vers10ns
. o 31.sm<>ula
· " r 0 .b'iect at d'ffi
l erem ,degrees of
cettam compress. wO,n tee h mques
· elimi.11a~epi11el-based
fo. his important study of digital photography The ReconfiguredEye,
William Mitchell explains this principle as, follows.:"There is an indefinite
amount of info,rmatioD in a continuous-•t:ooe pbomgraph. so enlargement
more detail but yields, 11.
usually 1C'll1eals fuzzi,er and grainier picture .... A ,J..Mitchell '. Tix R'°""'l/ig'" cedE·y·,·((am.bridge, Mass: Mff li'.[ess,,1982), 6.
2Sli.~"i,l!llii~m
digital image,,,on tlte other hand,. has,pre1ciselylimited spatial and tonal res-

- -
 repres.emation a!mgerher, instead repr:esenting an image via different mad1.-
emati.cal ,comrrucu (such as trans,fonns). c,omp,ression becoming mo,re and more the norm for li'ep,resenting visual io-
fmrruuion. If a single digital image already contains a lot of data, this
amount increases dramatically if we want to pmd11c,e and distribute mov-
,('5), 'In conmm to analog media where each successive,cop)•looesqu~l.iil!J,'.,
dig-
ing images in a digital form. (One second ofvideo, for instance, consists of
iitallyencoded med.iaci.11 be mpi:,edendlessly wirhouc degmda:tii,0111.
thirty still images.) Digital television with its hundreds of channels and
video on-demand services, the distribution of fuU-lengrh films on DVD or
.il\,Hc,cheU
summari2ces :rlhisas follows: "The rnntin1u:ius: sparfal and tonal
over the Jncemet, fully d igiral post-production of feature fi]ms;--all of these
v:uiation of a1:1alogpictures .is not exactly replicab[,e,. :so suclh ima,g,es cannot
developments are made possible by lossy compression .. h wm be a number
be t:ran:smiued or copied wi chout degradation .... Buie disc:rete srar,es can be
of ,'f'lll"S before advances in smrage media and communfoatfo,n bandwidth
repli.cated precisely, so a digital image that is a thousand generations away
'lll•ijl
,diminate the need to oomp,ress audio-visual dam. So,ratber than being
from the o,.riginal is indistinguishable in quality foam any ,one of its progen-
od:Je.m·ise pure and perfect w,odd of the di,gital,
:im a.berration, a Haw in ,rll:tie
itors.''3°' Therefore in digitaJ culture, "an image file can be copied endlessly,
wh1ere not even a si:ngtebi,t of information is ever .loot,iossy compression is
and the copy is distinguishable from the origin-al by its date since there is no
the v,e:ryfoundation of ,computier,cuhure, al!"least fur now. Therefore, while
fos:sof,qmtlity." 31 This is all true-in principJe ..In reality, however, there is
in dreo:ry, computer techooto,gy en.taHs the 8aw[,ess::rr,eplkarion of data, its
acruaUy much more degradation and. loss of info~tion between copies of
anual use in contemporaty oode:cy ischaractieri,2ced by l,oss of data, degrada-
digitaJ images, than between copies of traditimal photographs. A single dig-
t:ioo,, and noise.
ital image consists of millions of pixds. AU ofthis data requires considerable
storage space in a computer, it also rakes: a lcmg time (in contrast to a text
TlimeMyth of Interactivity
1mtransmit over a network. Because of d1iis:,rhe software and hardware
111,e)
We lrnaveonly one principle still remaining from rhe o.rigjnallist: inreractiv.icy.,
used to acquire, store, manipclate,, and transmit digiraJ images rely uni-
formly on lossycomjJreS.Iion---the
tedru1iq1111e
,of maldng image files smaller by
(6) New media is:i111erac:riYe.
fo contrast coold .mediawhere the order of pres-
deleting some infmmation. fa:amples: ,of the rechni,que induide the JPEG
,aa,onow interact with a media ,object. In the process
em:ation is fixed,,the 1.1.Sl!r
form111t,'lll1l1kh iis:used to smrestill ima,gies:,and MPEG, which is used to store
of interaction the user ,c:acn
ch.oosewhich ,elements ,to,display ,or which pad1s to
digital vidoo Oil DVD. The technique involves a compromise between im-
a u1•Iqu.e work. In this w11;11·
follow, thus generat:i11,g the· ·mer becomes the i:;o-
age quali:ty a11dfile size-the srnal le.r the si.i'lle
of a compressed :lile, the more
aurbor of the work.
visible the visual an:ifacts initrodUJoed ill deleting information become. De-
pending on the· ]eveJ of compression, 1th,es,ean:ifacts range from bardy no-
ticeable m qui.te' pronounrnd. As wiith digital I av,oid using rbe word interacti1ltli.n thiis book without qwal-
i11j,,i..ng
it, for the same reas:on-I find the ,concept ro, be· t100, broad tiO be truly
One nuw argue that chis situation is ~em,por:ary,that once cheaper com-
pur,er sror:a,g,e
uu:1 faster networks: beoome oornmonpface, Jossy compression
usieful
fo refation to computer-based media, the concept of interactivity is a tau-
wil.1disappear. Presently, however, die ue,nd is:quite the opposite, with lossy
1

mlogy.Modem HCI is by definition interactive. In contrast to earlier inrer-

fuces :rud1as batch processing, modern HO allows the user to control the
computer in real-time by manipulating infur:mation displayed on the screen.
30. Ibid., 6. Once an object is represented in a compute[, i1tauromat:icaHy becomes in-
:H. Ibicl., 49. teractive. Therefore, to call computer media "inten.ct:ive"' is meaningl,ess-
it simply means srating the most basic fact abourt compute.rs.

- -
Chapter 1
What J:s,NewMedia?
 dia theorists, prepared the ground for the imeractive ,c,ompmer installations
Radrer than ,evoking this concept. by itself, I use a numlber of o,ther
that appeared in the l980s.JJ
,concepts,such as menu-based interactivity, scalability, sim,1daticm,image-
When we use the concept of" interactive media" e:1:dL1Sively
in relation co
interface, and image-ins,crument, to describe diffei,ent kinds ,1Jfi,nteractive
computer-based media, there is the danger that we wiH interpret "incer-
sttucmres and operations. The dlis;tinct:ion between "do:s,ed'' a:1Jd'"open"
action" literally, equating it with physical interaction between a user and a
interactivity is just one ex.ample of 'tbis approach. media object (pressing a button, choosing a link, moving the body), at che
Although it is relatively eiisy co spoci:fydifferent interacid'lle' :structures
expense of psychologicaI interaction. The psychological processes ofifiJling-in,
used in new media objects, it is mUJchmore difficuk to deal theoretically
hypothesis formation, recall, and identification, which aEe required for us to
with users' experiences of these sttucmres. This aspect ,ofinten:cti.viqr re-
any text or image at all, aEe mistakenly ide11tiliedwitth an ob-
,compre.h.em:11
mains one of the most difficult theoretical. questions raised by 111,ew
media.
jectively existing structure ofirateracch,e li.nk:s;..34
Without pretending m have a complete answer, I would like to address some
T'hi:sminalk.ei.snot new; on thie cormuary,it is a snucmr:al foatu.reof the
;aspects of the question here. hisncnJ'of modern media. The literal. in.~erpretation of innel1aictivit)r
is just the
AH dassical, and even moreso modem, art is "interactive" in .a number of
~aves~.. of a larger modem t1:1endno externalize meniuil life,,a process
ways. Ellipses in literary narration, missing details of objectS in visual arc.,
u11w 1luch media technologies-pho~o:graphy, film, VR-have played a key
and other representational "shortcuts» require the user to fill in missing in-
10,Ie. 35 8 egm111ng
. ·. m· t h,e nmeteend11
· ,c,einury,we witness re:::w:rentclaims by
formation.32 Theater and painting also rely on techniques of staging and
tbe users and 'theorists of new media. technologies, from Francis Galton (the
compo,sition to orchestrate the viewer's attention over time, requiring her to
inventor of ,composite pllotogl1aphy in the 1870s) to, Hugo Munsterberg,
forus 1iu different parts of the display. With sculpture and architecture, the
Se,rgei Eis.ens;ceinand, recendy,Jamn Lanier, that these tecb1mfogies exter-
viewer h:asto move her whole body to experience the spatial structure.
nalize and objectify tile nund. Galto11not only claimed diu '',che ideal faces
M,odern media and art pushed each of these techniques further, placing
,obtaiin,edby the method of comp,osi,meportraiture appear m llta,oea great deal
new cognitive and physical demands on the viewer. Beginning in the 1920s,
new narrative techni.gues such as film m1intage forced audiences to bridge
quickly the mental gaps between unrelated images. Film cinematography
actively guided the viewer to switch from one part of a frame to another. The 33.. The 1lllllltionclllUt
computer,intetactiveatt!bas,
its originsin new an lorms of11he1960sis expJru,ool
n,ewrepresentationa] style of semi-abs,tract:ion, which along with photogra- ,inSiillreDilllilda,
"The History oftlle:lnierlii(ein InieracriveAJ:c;·ISEA(InternatioailllSwmposiwnon
phy became the "imemational style" of modem visual culture, requited the El,~ctnlai,c
&£), 1994 Proc~,ings(l111::11p;i~.u.iah.!6Jbookshoplisea...Jlr«lt:1atgenl08.html;.
"From
v.iewer to reconstruct represented objects firom a bare' minimum-a contour, l?utii:Jip!llioa Origimsof InteractiveAnt in lJmlTI
m Interaction:Towardll:lb,e Ha,sliimanlre,oo., ed.,
a fewpatches of oolo~~
shadows casiclbydrl.e<1bjeci:snot represented direcdy. Cl~~ In, HasLi,,li:uo,aDigilalCuJl:zmii,
!Seanle.:BayPress, 1996)127'9'-29'0.
SeealsoSimon Penny,
Finally, in the, 1960s, continuing wheJC F'm:ucism and Dada left off, new ",G:imumerGulrureamdthe 'Techoollll!lical
]mperative:The An:is1in Dara.space;·in Simon Penmy,

forms of art such as h.appenings, performimce,, a:ndlinstallation turned art ex- ,ed•.,,Cri1.ir.rl
ln,ie,ie E/Et:tro11k
hli!llli:,,
(Albany:Sute Unl:ve,:sii:y
of New '!fmrkl':ress,1993},47-74.

plicitly participatio,nal-a transfonnatioui thlllt,.according to some new me- 34. ''Jfhiis:

~,gument relies on a cogniciv,istpe:i:spec,cive
that stresses the .activ,emental processes
1t,ex1.
im,,.oh,edin •comprehens.ionof any co,'l:ru,i:al ]for examples of a ,rogni,tivis1.mpproachin film
..see B:ord:welland Thompson, Fil"'·JI.rt,aod David Bordwell,,Narn,,t:iol'I
:~rndli,u., in 1heFicti•"
F,iJi,,,(~[adison; University ofWisoonsin Piress,19:!19).
3'.)• for II more·detailed analysis ohbis: nend, see my article "From ,the E><1ema!lia:ation
of the
"mebeholder's sbaure"illllderoding the missiog informarioo in
32. Ernst Gtimbrich llllltdyzes
II) chieImplantation of Techndl',c,gy,:"
lf'lsyd1ie in il1i,1dRwa!Ntiol'I:
[,,1:erf,rce
8,,"i 11 ed.
JColllJ>llle,;
mthePs:,d;ology
visual images i10.bis,dass.ic A.rt aml ll/.mio,r;A St:ud]II of PictorialRepnsrnt,rtion
Florian Ro1aer,(Munich:Ak,,demie Zum Drinen Jahrtausend, 199'5), '90-1,00,.
(Priocemn, NJ..: Princeton University Press,, 191/l0,)1
..

-
What Is INewMedia?
in common with .... so,-called ahsrn1ct ideas» bu.t i11:fact be proposed w re- 1 ra,rber than being locked into a '"prison-l'u:mseof language" (Fredric Jame-
name absuaa ideas: ~rumulative ideas:' 36 Accruding m Munsterbe.~g, who
w115a Professor of Psycl:wlogy at HaurvardUniversity and an aud10r of 1meof
the earliest theoretical treatments ,ofc.inema ,entitled The Film:A P9"Ch,rJ~gi-
calStady (l9 l 6), the essence offiLmlies .in.in ability to reproduc,eor "'objec-
tify" ,;,arlous mental functions ,on du::scre,eo::"The photoplay ,abey:s, laws the
of the mioo radrer than those ohbe ,outt,erworkl;n3:1 In tbe 1'920:sE,ise111ste:.io
1 having to use a picture ,of the word 'cup."' 42 . Here, as with the earl.ier tech-
specu1i:lltedtlmat:lilm could be used t,o ,ext,emalize-and control-thinking.
As an exper.im,eot :iDthis direction, he ooMl.yconceived a sneen :aicbp,ta.tioo
of Marx's Capital."The cont,ent of CAPITAL{its,aim) is now form1di1ted:to
teach the work,er to think dialectic:a.Uy," i11
Eis,eostein writes end1us,iasttic,aully
April of 19'28:..~8 fo accordanoe with die:prmdpEes of"Marxist diaulectics'' as
canonized by tbe official Soviet phliosaphy, Eisenstein planned to .1uese11t the
viewer witb rbe visua.Eeq:uiv:alen·ts,ofthesis a.milanti-iliesis sa,that the viewer
could then proceed to arrive at sy1nb,esis,dmt is,. the correct com::Iw,i1m,as
pre-programmed by llisensrein. . . .
VR. pioneer Jaron Lmie.r similarly saw VR 1c,ec.lil110,]ogy
In the 19:80.s.,, as
capable of campletely objecti:fying-better yet, tmnsp:illl'1eodymergi~g
with-me,ntall p,rocesses. His descriptioru; of its capabilities did J[ljOt dis-
tinguish 'between internal mentai foncttions, events, :md proi:1ess•esand ex.-
llemaUy presented images. This is how, according to Lanier, VR can itake
over bumlllll memory: "You cam play back your memory through time and
classify yoiu memories in various ways. You'd be able to run bad:: through
the experi.emial! places you've been in order to be able to find people,
toots:'''-' La.nier a.Esoclaimed that VR will lead to the age of uposc-symbolic
commi.mication," commurucai:ion without language or any other symbols.
Indeed, why should tbere be any need for linguistic symbols if everyone
36. Quoted in JillaimSekula, "The Body ancl tme.Aochi.,,e,." 39 (1987):51.
1(11;1,obet'
The Pbotoplay:.II. Pzyd,ol~gju,JSu,dy (New York: .D. Awtem- ""'d
37. Hugo Miins.lll!llberg,.
Compan)', l'9•t,6),,4].
38. Sergei Eisenstein,.,. [taos. Maciej Sbwowslei,Ja.i•11.euda,.
"Notes,fura Film.11f'1Capit1l,""
Ola~llw
Annette Michelsa11., 2 (1916): UI.
39_ Timollhr Dmdaey, "Reveogeof ,cii,e wid,Ja1ron
Neids: lln lnci:,i;,,•iew lanier;· llftenm.,,g•
(May 119'91),
9.
- -
Sion),tOwill happilt· live in the ukimate nigbrmare of democracy-the
sin,gle mental space that is sharecl by eve·Jr)•one,and where every com-
ml.lUllicariveace is always ideal (J,ii,rgen Haberrnas}. 41 This •is Lanietr's ex-
ample of how post-symbolic communication will function: "You can make
a cup that someone eJse can pick when there wasn't a cup before, without
nology of film, the fantasy of objectifying and at:igmenting co,nsieiousness,
extending the powers of reason., goes hand in hand with the de.s:ir,em see in
1 technology a return to the primitive happy age of pre-12m,guage, pre-
misunderstanding. Locked in virtual reality caves, with langoog e taken 1
away, we will communicate thmugh gestures, body movements,. and gri-
maces, like our primitive ancestors ...
The recurrent claims that new media technologies externalize and:objec-
tify reasonin.g,.and tha.r they can be used to augment or comro] it, aue based
on die assu.mpti.on of the isomorphism of mental .representaJtionsand opera-
tioos with e·xte.11mI
visual effects such as diissol'lles,composite· images, and
editied sequences:. This assumptiolil is shared .n.ot only by modern media
arcists,.aad critics but als.o,by modlem psychofogists ..Modem psy-
invento:c:s,,
cholqg.ical tbeories of i:he mind, from !Freud co,cognitive psycholo,g;i1,,
~epear-
e<.llyequate men,calprocesses with exoem.d, technologically genera1t,edvisual
forms:...'Thus f.reud in Thel12,terpi:eta,ti!i11t
,ofDrea11zs
(1900) compared the pro-
cess ofco,od,ensation with ooe ofFr~[ljcisGabon's procedures. that became es.-
pecially famm1s:making famii.i•portia,its by overlaying a different negarh,e
im111,g,e
for each member of the famiily,md then making a :s.ingleprint. 4 ~ Writ-
i1:1the :same decade, t.he .American, psychola:g.isr Edward Titchener
i1111,g
40. Fredric Jameson, The Pri,on-bt,,,se
efl.-,,"1/r,age:
A Cr;ricalA.croumof Str11cturalitm·
,wd Rm-
• U11iversiry Press, 1972).
,;,,,. Fo,.,,,,,/;J,,,(Prinoeton, NJ.: l'r,urn,oeto
4 t • Ji.irgen Habermas, The Tb«,ry of Com:1wmicati1li!
Action: Reasonand Rati,m,,,!'.izatio"
ef.So,:iety
(The Theory of Communicative Action, V:01.·I), trans. Thomas McCart1'y (Bosmom:Beacon
Press, 1985).
and
42. Druckrey, ~Revenge of the Nerds," 6.
43'. Sigmund Freud, Standard Edition of lhe CompleteP,ychologicalWo,-ks(London: Hog,mh
Press, 1953), 4: 293.
Wha,t Is, l!l•ew~ledia?
opened the di:scUISlSiu111
of the, nature of abso:act ideas in his textbook of psy-
chology by noting ·that "the suggestion bas been made that an abstract idea
is a sort of composite photograph, a mental. pi.ccurewhich resullts from the
superimpositi1:1,r1of many particular perceptions or ideas,. and which there-
fore shows the ,oommrnrielements distinct and ,the individual elements
blurred.''4'1 He then p:rooeeds to consider the pros and cons of this view. We
slm,ulld not wo1Dderwby Titchener, Freud, and other psychologists take the
comparison for gramed :rather tbm preseming i t u a simple meraphor-
1
conrempo.rary cogniti.ve psychologists aim do not question why their
ds of the mind are so similar to the c,ompuoer 'lli'o,rks:tations,on which they
are constructed. TJbe linguist George Lakoffass,erted that "natural reasoning
makes use of at least some uru:onsdous aniellamomatic image-based processes
such as superimposfog images, scanni.ng them, fo1cusiogon part of t:hem,~4~
and the psychologist PhiHp Jobrnson-,L.aird pro,posed tba.t logical reasoning
is a matter of scanning: 11isua1modlels. 46' Such notio,ns ,11ouldhave been. i.m-
possibie before the emergence ohetevision and computer graphics. These 'i'i-
sual technologies made operati,ons on images s:uch a:ssca.mJJing.focusing,. and
superimposition seem na.tural.
What to make ofthis modem desire to ene,raafoe the, mind? It can be re-
lated to the demand] of modern mass sodety for standardization.
jects have to be sraralardized, and the mean:s by wh~ch they are standardized!
need to be standardized as well. Hence the ,oibj;,ecti.lication
ofinternal, private
mentaI processes, and their equation \\•i.tb e'.ll:te,n:ial
visual forms that
easily be manipulated, mass produced, 11110d
s:taodardi2l!ldon their own. The,
private and im:lividm[ a[1etranslated iom the publiic and become regulated ..
What befo.re bad been a mental process,, a 111n.i1qudy
individual scare, now
became pan of the pubiiic sphere. Uoobsen111blemd interior processes 111nd
,<!'
rep,resentatioos were t:akieo out of indiv~du:IL!bead:s and placed outs,ide-:ll!S:
dl'awings, pbocognphs, arid other visual forms,. Now they ,could be di:scus,51ed
in public, employed in 1teaching and propaga1111da,
standatdized,. and m=-
44.. Edward 18.radfo.rd
TI,ichener,A Begi1112eri !New York:Macmillan, 19I S},,114..
P'scychalogJ1'
45. Georgelakof(, "Goi;1rucive
Linguistics,:·Vmllt 44il4:5
0986): 149.
.a ,Cogniti"'Sa,,,,.
46. PhilipJ o!mw1:1-laird,,1'lifmtalA!od'Js:Toi,,a,:dii of Ltmgr,age,.l.'1/=-• .,,,,,r
~Cambridge: Cambridge Univetsiti,•P,ress, 19,B,3,).
,C.,,zs,:iousner,
-
Chapter l
disuibuted .. What was private became public. What was uni,que, became
mass-produced .., What was hidden in an individual's mind became shi.ired.
Imeranive computer media perfectly fits this trend to. ,euen:iali.ie and
objectify the mind's opemdons. The very principle of hypedi111'dng, which
forms dre b:lllScis
of interactive media, objectifies. the proce:ss ofusocia:tion, of-
ten taken ro be central to hum1111rnthinking. Mental pmce<ssesof reflection,
problem soh1:ing,,recall, aud association are, exremalimd, e,quillltedwith fol-
lowing a folk, mo111ingr,o,a 111,e,.v
pa,g,e,.choosing a new image, or a new scene ..
moo- Before we would look at an im:ill,g,e
and mentally follow our own priva[e as-
sociation.s to od1e.r images. No1i'i'intera,ctive computer med.ia asks u:si nuead
to dick on m image in o~d,er to go to another image. B,efore, we would read
a sernc,em::,e
of a story or a hne of a poem and think of ,other lines, images,
memori,es. Now inte:racdv,e media asks m to dick on a hi,ghlighted sentence
m ,go to a!lother sentence. In shore,,.w·eare asked to fulio'i11l'
pre-pmg~ammed,
ubjectively existing associations .. Put diffe.rently, in what can be read as an
updated version of frend1 phil,o.sopher Louis Althus:ser's concept of'" inter-
pellation;• we are asked ,c,omistake: the structure of somebody's dse mind for
ourown. 47
This .isa new kind ofidemifo::ation appropriate for the information
The Sll]b- cognitive labor. The cultural technologies of an industrial society-cinema
and fashion-asked us 'to identify with someone else~sbodily image. Inter-
active media ask us to identify with someone else"s mental structure. If the
Gllll cinema viewer, male and femal,e,, lust,ed after and tried to emulate the body
of the movie star, the computer us,er .isaslkiedto foHow the mental trlljectmy
of the a.ew media designer.
11orionof ideological inte.tpei!,il.l!i,on
47. Louis Altlmsser introduced his i:rnlluem.1tiall in ··ideol-
ogy moo.IdeologicalState Appilll.mtusie5, an lmrestigaition) .." in Le~;,,and Philm-
1(No1,rsmw:ordis.
ophy,trans. Ben B.rewsrer(New York: Monrhlr it.ec~.i.ew
Press, 1971).
What Is NewMe.idia?