Imagination in Media Arts: Technological Constraints and Creative Freedom

Imagination in Media Arts:
Technological Constraints and Creative Freedom
Dissertation defense:
Rama C. Hoetzlein
Nov 19th,2010
Media Arts & Technology

University of California Santa Barbara
Committee:
Tobias Höllerer, Chair (MAT / CS)

George Legrady (MAT)
Laurie Monahan (Art History)
Imagination in Media Arts: Technological Constraints and Creative Freedom Rama C. Hoetzlein
Dissertation Defense. Committee: Tobias Hollerer, George Legrady, Laurie Monahan Media Arts & Technology, UC Santa Barbara
Research question:
Media artists explore a wide range of techniques for developing art. These
techniques, found in current tools for artists, come with inherent limitations
which, over time, may separate creative communities into groups that are
familiar with particular methods. The question examined here is whether
these techniques can be integrated into shared, multi-dimensional
frameworks. Are our current technical constraints inherent to particular
kinds of media, or are they (human) constraints resulting from the
evolution of goals in creative communities?
Creative Dimensions:
1) Programming and Language

Artists may or may not wish to program. Some tools require programming, others do not.
2) Modality and Media

Media artists work with images, surfaces, audio and video, which may come
from various input devices.
3) Live Performance and Computation
Some tools focus on high quality offline rendering, while others allow artists to interact
with the artwork during a performance.
4) Motion, Dynamics & Autonomy
Media artists are often interested in the motion and dynamics of
systems, as an outcome of an emergent process.
5) Structure and Surface
Structure is the explicit representation of form, which can be distinguished from behavior.
Surface is the visual style of a structure. Current tools give different weight to each.
6) Image and Idea

Certain approaches to the image consider it as a symbolic, or semantic object.
How might tools for the media artist support these interests in relation to the others?
Early History of the Digital Object
Digital Image as Shape: Michael Noll, Bridget Riley's Painting Currents, 1966 (Bell Labs)
Image Processing: Manfred Schroeder, Ken Knowlton and Leon Harmon, Studies in Perception I, 1966 (Murray Hill)
Motion & Behavior: William Reeves, Fuzzy Objects (1983), Craig Reynolds, Flocks, Heards, Schools (1987)
Surfaces: Jack Bresenham (1965), Pierre Bezier (1962), Ed Catmull (1973)
Rendering: Bui Phong (1973), James Blinn (1976), Don Greenberg (1971), and Turner Whitted (1980)
LUNA, A Language for Media Artists
Introduction of a novel visual dataflow language for media artists, based on resolving
various creative dimensions into a single tool
Graphical Interface Design
LUNA is a visual dataflow language, inspired by Scrabble, in which the

combinatoric arrangement of high level nodes allows one to create dynamic,
structural objects without the need to program.
Graphical Interface Design - History
Sketches for the interface (2006).
Monarch, /w Jorge Castellanos (2007)
MINT/VFX. NSF IGERT Project, 2007-2009
Graphical Interface Design
Colored objects and input tabs represents the type of media

flowing through the system. Tool bars show available media and behaviors.
LUNA: Interface example – Building the woven sphere reference model
Interface Evaluation
Interface comparisons
performed by building the
same object in Houdini 10
and in LUNA.
LUNA – Interface Implementation
Overall, the interface of LUNA is constructed as a graph of nodes which can

render in 2D and 3D. LUNA implements a GUI system in which events flow
down the graph from the right.
A Language for Procedural Modeling
Formally, LUNA is a procedural language containing two graphs:

1. The procedural graph is a DAG defining behavior, in which each
node can input and output a scene sub-graph.
2. The scene graph is a DAG defining media, which can be generated
dynamically by the procedural graph. Proxies manage rendering.
A Language for Procedural Modeling
Novel contributions:
1. A real-time dataflow language for procedural modeling
2. Objects are stored using geometry buffers which can be

directly copied to the GPU for rendering.
3. Geometry buffers also allow objects to be modified in a uniform

way regardless of type. E.g. Spherify points, curves or meshes.
4. Scene graphs are rendered using proxy objects that efficiently

rendering many objects of the same type.
5. LUNA integrates the flexibility of procedural modeling with

the performance of scene graphs.
A Language for Interactive Procedural Modeling
Performance Evaluation
How well does LUNA support media artists?
“Constraints in creativity are both limiting and liberating. They

are used to impose boundaries upon the creative space we
occupy and at the same time enable us to grapple with inherent
tensions between different demands, which may lead to a new
idea, direction or artifact. When we choose particular forms,
materials and tools for our creative work, we are also choosing
the kinds of constraints that will shape our process and its
outcomes."
Linda Candy, 2007. Constraints and Creativity in the Digital Arts. Leonardo, 40(4):366-367.24
Inherent constraints – Implicit in the tool. Not a matter of choice.
Creative constraints – Chosen by the artist to resolve tensions in the work.
Evaluation:
In general, there are no generally accepted ways to measure the ability of a

tool to support creativity.
Evaluation is based on subjective criteria resulting from

the Creative Support Tools workshop held by the NSF in 2005.
1) Low threshold – “the interface should be easy to use”
2) High ceiling – “the system should be powerful and complete”
3) Wide walls – “the system should suggest a wide range of expressions”
Programming and Language
Processing. Casey Reas & Benjamin Fry
LUNA. Node authorship.
Text-based languages like Processing were designed to teach programming, and to allow
experimentation with behavioral rules. This focus on behavior allows the artist to experiment with
emergent forms, but limits ones ability to explore structure. LUNA supports text-based interaction
through node authorship, but its primary mode of interaction is through visual, high level objects which
represent structures. Behaviors are coded into each of the nodes in the graph.
ConMan, Paul Haeberli. 1988

Max/MSP, Cycling ’74 and Soundium, Pascal Muller
& Stefan Muller Arisona.
LUNA is a visual dataflow language. Early systems such as ConMan expressed procedural
relationships visually in the graph (similar to LUNA). Max/MSP was developed as a system for signal
processing, so its language is stream oriented with expressions. Soundium uses design trees to
express behavioral transformations of simple shapes. Both of these support interactive feedback.
Other visual dataflow languages focus specifically on three dimensional forms, such as Xfrog and
Houdini. Yet neither of these provides real time output for live performance.
Modality and Media
LUNA was designed to support many different kinds of media.

Currently implemented nodes cover materials, images, points, lines, curves and surfaces.
Modality and Media
Presence (2008), collaboration with

Dennis Adderton and Jeff Elings, at
UC Santa Barbara Davidson Library.
Modality and Media
Blocks, with Mark Zifchock, Abram Connelly & Marty White.

The LUNA API was used to do both 3D rendering and the graphical user interface.
Modality and Media
Modality and Media
Synthetic Rendering of retinal astrocyte cells, from imagery produced by the

Neuroscience Research Institute, Univ. of California Santa Barbara, Gabe
Luna, Geoffrey Lewis, and Steve Fisher, and with Tobias Hollerer and B.S.
Manjunath (Dept. of Electrical and Computer Engineering.
Simulations created using LUNA.
Live Performance and Computation
LUNA is the first procedural modeling language for artists to offer:

1. Real-time profiling while modeling
2. High quality, deferred shading for installations and performances
3. A mixing interface for interactive editing of procedural models.
Motion, Dynamics and Autonomy
Ned Kahn, Basin of Attraction (1989) Protrude Flow, Kodoma (2000) Sluice, Kate MccGwire Digital Sluice, R. Hoetzlein. LUNA
Protrude Flow, Kodoma
``Chaos is the irregular, unpredictable behavior of deterministic,

nonlinear dynamical systems.'' Roderick Jensen, Yale
University.
Artists are interested not only in representation, but in the ideas and meaning
behind a system or behavior, and its expression through a particular work.
Lapis, John Whitney Sr. (1967)
Spira, R. Hoetzlein (2010). Created in LUNA
LUNA presently explores behavior through the manipulation and combination of whole systems. In the future, LUNA
could be extended to include a scripting language similar to Processing’s interface.
Structure and Surface
Some artists wish to explore structure explicitly, not as

an outcome of motion but as an object in itself. (Images from LUNA)
Structure and Surface
When a structure is present, there is a creative choice as to how to reveal the object, and how to shade or stylize its
surface. The structure can be made more or less visible. (Images from LUNA)
Image and Idea
Behavior and structure may be considered as aspects of the digital model,

the representation of the form of ideas and objects by symbols (machines).
Jörg Schirra relates the digital model to the digital image, but does not distinguish
between the model and word (both are ‘not-images’)
Image and Idea
SHRDLU, Terry Winograd (1968) Aaron’s Garden. AARON, Harold Cohen (1989)
Image and Idea
Social Evolution, 2007-2009
Fragments, R. Hoetzlein. 2010. Automatic Drawing, André Masson, 1924. Ink on paper.
Water color and ink on paper. 9 1/4" x 8 1/8". Museum of Modern Art, New York
Sequence, 2010. Water color and ink on paper.
Fragment Collage, 2010. Computer generated compositions by LUNA combined with hand sketched fragments.
Soft Sketches, 2010. Collaborations with LUNA.
Review – Tool Survey
Luna
2010
Rama
Hoetzlein
Media arts
OpenGL
free demo,
LGPL API
Conclusion
LUNA demonstrates that artists working with different techniques do not

necessarily need to work with different tools. An often-heard view is that having
many tools is beneficial to the creative community as each tool supports a
different method of working, with each artist choosing a tool to engage in a
particular format. A problem with this perspective is that it requires media artists
to learn many systems to become proficient in their field across a range of
techniques. Additionally, some projects may require techniques that can only be
found in two different tools, and cannot be easily combined through data
passing. While exposure to multiple tools may still be beneficial for educational
reasons, the idea that media artists must learn many tools because this is the
only way to explore the digital arts is refuted by LUNA.
LUNA is an environment in which several, previously disparate methods in

media arts are integrated into a single framework, as demonstrated through the
creative dimensions explored here.
Demo version and tutorial available at:
http://www.rchoetzlein.com/luna/
Thank you !
Committee:
Tobias Höllerer, Chair (MAT / CS)

George Legrady (MAT)
Laurie Monahan (Art History)

Imagination in Media Arts: Technological Constraints and Creative Freedom

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Imagination in Media Arts: Technological Constraints and Creative Freedom

Uploaded by

Copyright:

Available Formats

Imagination in Media Arts:

Technological Constraints and Creative Freedom

Media Arts & Technology

Tobias Höllerer, Chair (MAT / CS)

1) Programming and Language

2) Modality and Media

6) Image and Idea

LUNA is a visual dataflow language, inspired by Scrabble, in which the

Sketches for the interface (2006).

Monarch, /w Jorge Castellanos (2007)

MINT/VFX. NSF IGERT Project, 2007-2009

Colored objects and input tabs represents the type of media

Overall, the interface of LUNA is constructed as a graph of nodes which can

Formally, LUNA is a procedural language containing two graphs:

1. A real-time dataflow language for procedural modeling

2. Objects are stored using geometry buffers which can be

3. Geometry buffers also allow objects to be modified in a uniform

4. Scene graphs are rendered using proxy objects that efficiently

5. LUNA integrates the flexibility of procedural modeling with

“Constraints in creativity are both limiting and liberating. They

Inherent constraints – Implicit in the tool. Not a matter of choice.

Creative constraints – Chosen by the artist to resolve tensions in the work.

In general, there are no generally accepted ways to measure the ability of a

Evaluation is based on subjective criteria resulting from

1) Low threshold – “the interface should be easy to use”

2) High ceiling – “the system should be powerful and complete”

3) Wide walls – “the system should suggest a wide range of expressions”

Processing. Casey Reas & Benjamin Fry

LUNA. Node authorship.

ConMan, Paul Haeberli. 1988

LUNA was designed to support many different kinds of media.

Presence (2008), collaboration with

Blocks, with Mark Zifchock, Abram Connelly & Marty White.

Synthetic Rendering of retinal astrocyte cells, from imagery produced by the

LUNA is the first procedural modeling language for artists to offer:

``Chaos is the irregular, unpredictable behavior of deterministic,

Spira, R. Hoetzlein (2010). Created in LUNA

Some artists wish to explore structure explicitly, not as

Behavior and structure may be considered as aspects of the digital model,

Social Evolution, 2007-2009

LUNA demonstrates that artists working with different techniques do not

LUNA is an environment in which several, previously disparate methods in

Tobias Höllerer, Chair (MAT / CS)

You might also like