MODULE 8

KEY CONCEPTS IN AUTHORING AND PROGRAMMING MULTIMEDIA

AUTHORING SYSTEMS
 Authoring System is a program which has pre-programmed elements for the
development of interactive multimedia software titles.
 Authoring systems vary widely in orientation, capabilities, and learning curve.
 There is no such thing (at this time) as a completely point-and-click automated
authoring system; some knowledge of heuristic thinking and algorithm design is
necessary.
 Authoring is actually just a speeded-up form of programming; you don't need to
know the intricacies of a programming language, or worse, an API, but you do need
to understand how programs work.

WHY SHOULD YOU USE AN AUTHORING SYSTEM?

 It generally takes about 1/8th the time to develop an interactive multimedia project,
such as a CBT (Computer Based Training) program in an authoring system as
opposed to programming it in compiled code.
 This means 1/8 the cost of programmer time and likely increased re-use of code
(assuming that you pass this project's code to the next CBT project, and they use a
similar or identical authoring system).
 However, the content creation (graphics, text, video, audio, animation, etc.) is not
generally affected by the choice of an authoring system;
 Any production time gains here result from accelerated prototyping, not from the
choice of an authoring system over a compiled language.

MULTIMEDIA PROGRAMMING VS MULTIMEDIA AUTHORING

 Programming involves low-level
assembly and construction and control
of Multimedia and involves real
languages like C and Java.
 Quicktime may also be programmed in C
Authoring involves the assembly and
bringing together of Multimedia with
possibly high-level graphical interface
design and some high-level scripting.

While authoring and programming both result in

the creation of multimedia products, they are
not the same thing.
In the BAMS program, you will be expected to be
able to author digital media and to do some
multimedia programming in MMS 143
(Introduction to Multimedia Computing). You will
not be expected to create new types of
Figure 8.1. The QDesign Player
software that can be used to play or display digital
An audio player developed in the late
media. That is, you won't be expected to create an
1990s/early 2000s. I developed some of the
alternative to, say, iTunes, VLC, or Windows
code behind this player,... a long time ago!
Media Player.
AUTHORING SOFTWARE
 Authoring software - software programs you do use to create digital media
materials.
 Each type of authoring software can create specific types of multimedia materials.
 For example, if you were designing multimedia educational materials, you can't make
an interactive quiz using a word-processing program because the program wouldn't have
all the right tools you have for making an interactive quiz.
 However, other tools do, eXeLearning, a software program that allows teachers to
create interactive instructional materials with media content embedded in it.
 Solution the problem of not being able to create the kind of multimedia content that you
want: Use another software program for authoring your content.

ADVANCED MULTIMEDIA PRODUCTION: MULTIMEDIA PROGRAMMING

However, it is possible that as you become more experienced with producing multimedia, you'll
find that no authoring software exactly suits your needs, in which case you might consider...

PLUGINS OR MACROS
 Modifications like adding, removing, or editing functionality in an existing authoring
software or digital media software program.

PYTHON, JAVASCRIPT, C++, C#, JAVA, OR THE ARDUINO LANGUAGE

 But even then, you might eventually find that even if you use plugins and macros, no
available software meets your needs and that you need to do some software or
hardware programming using languages such as these to name a few.
 This is particularly true when you create unusual or bespoke multimedia products
and interfaces, which you might do in MMS 194 (New Media Art) or MMS 150 (UI/UX
Design).

PUREDATA, OPENFRAMEWORKS, AND PROCESSING

 Some of the most interesting examples of multimedia content are created using
these programming languages that are built on top of more basic but more
powerful programming languages.
 These tools allow you the most amount of flexibility in deciding what content to
present and how users can interact with your content, but they also take more time
to learn.
 PureData and Processing are free and open source programs, and tutorials for both
are available on the Web Videos.

PUREDATA (https://puredata.info/)
 Pure Data (or just Pd) is an open source visual programming language for
multimedia. Its main distribution (aka Pd Vanilla) is developed by Miller Puckette.
 Pd-L2ork/Purr-Data is an alternative distribution (originally based on the now
unmaintained, dead and deprecated Pd-Extended project), with a revamped GUI and
many included external libraries.
 Pure Data is an open source visual programming environment that runs on anything
from personal computers to embedded devices (ie Raspberry Pi) and smartphones
(via libpd, DroidParty (Android), and PdParty (iOS).
  It is a major branch of the family of patcher programming languages known as Max
(Max/FTS, ISPW Max, Max/MSP, etc), originally developed by Miller Puckette at IRCAM.
 Pd enables musicians, visual artists, performers, researchers, and developers to
create software graphically without writing lines of code.
 Pd can be used to process and generate sound, video, 2D/3D graphics, and
interface sensors, input devices, and MIDI.
 Pd can easily work over local and remote networks to integrate wearable
technology, motor systems, lighting rigs, and other equipment.
 It is suitable for learning basic multimedia processing and visual programming
methods as well as for realizing complex systems for large-scale projects.
 Algorithmic functions are represented in Pd by visual boxes called objects placed
within a patching window called a canvas.
 Data flow between objects are achieved through visual connections called patch
cords.
 Each object performs a specific task, which can vary in complexity from very low-
level mathematical operations to complicated audio or video functions such as
reverberation, FFT transformations, or video decoding.
 Objects include core Pd vanilla objects: external objects or externals (Pd objects
compiled from C or C++), and abstractions (Pd patches loaded as objects).

THREE EXAMPLES OF WHAT YOU CAN MAKE USING PUREDATA

 A sequencer made using PureData
 An experimental video made using PureData (You can adjust the overlay and
transparency.
 Creating animations and sound during a live performance using PureData

PROCESSING (https://processing.org/)
 Processing is a flexible software sketchbook and a language for learning how to
code within the context of the visual arts.
 Since 2001, Processing has promoted software literacy within the visual arts and
visual literacy within technology. There are tens of thousands of students, artists,
designers, researchers, and hobbyists who use Processing for learning and
prototyping.
 Processing uses Android 2.3.3

THREE EXAMPLES OF WHAT YOU CAN MAKE USING PROCESSING

 An interactive multitouch-interface made using Processing (Max Planck Research Ntwrk)
 An Android game made using Processing
 A work of video art based on motion capture datamade using Processing
(Soundsculpture)
In the meantime, you will probably find that the wide variety of software available for
authoring content is sufficient to meet your needs. You will be introduced to various
authoring and production tools throughout the duration of your tenure at BAMS, especially when
you take your MMS 14x and 17x courses.
 AUTHORING PARADIGMS
 Another way to classify authoring software is by the authoring paradigm (also known
as an authoring metaphor) used in order for the user to understand how to manipulate
information to achieve a desired result.
 Have you ever edited an audio file before? If you have, have you noticed how similar it
is to editing a video? This is because the software you use to edit these use similar
authoring paradigms.
 In contrast, editing audio or video is different from editing a slideshow presentation,
or a text document, or an image because the editing software use different authoring
paradigms.

Multimedia Authoring Paradigms

(https://users.cs.cf.ac.uk/Dave.Marshall/Multimedia/node24.html)

 Authoring paradigm or authoring metaphor is the methodology by which

the authoring system accomplishes its task.

THERE ARE VARIOUS PARADIGMS

SCRIPTING LANGUAGE

 The Scripting paradigm is the authoring method closest in form to traditional

programming.
 The paradigm is that of a programming language, which specifies (by filename)
multimedia elements, sequencing, hotspots, synchronization, etc.
 A powerful, object-oriented scripting language is usually the centerpiece of such a
system; in-program editing of elements (still graphics, video, audio, etc.) tends to be
minimal or non-existent.
 Scripting languages do vary; check out how much the language is object-based or
object-oriented.
 The scripting paradigm tends to be longer in development time (it takes longer to
code an individual interaction), but generally more powerful interactivity is
possible.
 Since most Scripting languages are interpreted, instead of compiled, the runtime
speed gains over other authoring methods are minimal.
 The media handling can vary widely; check out your system with your contributing
package formats carefully.
 Examples of a Multimedia scripting language: Apple's HyperTalk for HyperCard,
Assymetrix's OpenScript for ToolBook and Lingo scripting language of
Macromedia Director
Here is an example lingo script to jump to a frame:
global gNavSprite

on exitFrame
go the frame
play sprite gNavSprite
end
Iconic/Flow Control

 This tends to be the speediest (in development time) authoring style; it is best suited
for rapid prototyping and short-development time projects.
 Many of these tools are also optimized for developing Computer-Based Training
(CBT).
 The core of the paradigm is the Icon Palette, containing the possible
functions/interactions of a program, and the Flow Line, which shows the actual
links between the icons.
 These programs tend to be the slowest runtimes, because each interaction carries
with it all of its possible permutations;
 The higher end packages, such as Authorware (Figure below) or IconAuthor, are
extremely powerful and suffer least from runtime speed problems.
 AUTHORWARE

MACROMEDIA AUTHORWARE ICONIC/FLOW CONTROL

EXAMPLES
FRAME

 The Frame paradigm is similar to the Iconic/Flow Control paradigm in that it

usually incorporates an icon palette; however, the links drawn between icons are
conceptual and do not always represent the actual flow of the program.
 This is a very fast development system, but requires a good auto-debugging
function, as it is visually un-debuggable.
  The best of these have bundled compiled-language scripting, such as Quest (whose
scripting language is C) or Apple Media Kit.
CARD/SCRIPTING

 The Card/Scripting paradigm provides a great deal of power (via the incorporated
scripting language) but suffers from the index-card structure.
 It is excellently suited for Hypertext applications, and supremely suited for
navigation intensive (a la Cyan's "MYST" game) applications.
 Such programs are easily extensible via XCMDs and DLLs; they are widely used
for shareware applications.
 The best applications allow all objects (including individual graphic elements) to
be scripted; many entertainment applications are prototyped in a card/scripting
system prior to compiled-language coding.
CAST/SCORE/SCRIPTING

 The Cast/Score/Scripting paradigm uses a music score as its primary authoring

metaphor; the synchronous elements are shown in various horizontal tracks with
simultaneity shown via the vertical columns.
 The true power of this metaphor lies in the ability to script the behavior of each of
the cast members.
 The most popular member of this paradigm is Director, which is used in the
creation of many commercial applications.
 These programs are best suited for animation-intensive or synchronized media
applications; they are easily extensible to handle other functions (such as
hypertext) via XOBJs, XCMDs, and DLLs.

 Macromedia director uses this method and examples can be found in figs 2.2-
- 2.4:

MACROMEDIA DIRECTOR SCORE WINDOW

 MACROMEDIA DIRECTOR CAST WINDOW

MACROMEDIA DIRECTOR SCRIPT WINDOW

HIERARCHICAL OBJECT
The Hierarchical Object paradigm uses an object metaphor (like OOP) which is visually
represented by embedded objects and iconic properties. Although the learning curve is
non-trivial, the visual representation of objects can make very complicated constructions
possible. (Shopee)
HYPERMEDIA LINKAGE
The Hypermedia Linkage paradigm is similar to the Frame paradigm in that it shows
conceptual links between elements; however, it lacks the Frame paradigm's visual linkage
metaphor.
TAGGING
The Tagging paradigm uses tags in text files (for instance, SGML/HTML, SMIL
(Synchronised Media Integration Language), VRML, 3DML and WinHelp) to link pages,
provide interactivity and integrate multimedia elements.
 Think about any media authoring tool that you already know how to use. What authoring
paradigms does it use?
 Earlier, two advanced multimedia production tools were mentioned, PureData and
Processing. Look at the interface that they use (Figures 8.4 and 8.5). What authoring
paradigms is used by each tool?
AUTHORING SOFTWARE CAN BE CLASSIFIED IN DIFFERENT WAYS.
Thick Client (Or Fat Client) Applications.
 Some authoring software need to be installed in your computer's operating system;
word-processing programs like Open Office Writer and Microsoft Word are
examples of such software.
 Thick client applications require your computer to do most of the work

Thin Client Applications

 Other software programs are run from within a web browser. For example, this course
was created using Google Sites, a Web-based tool provided by Google.com.
 Whereas thin client applications rely on some other computer to do the work.
 With thin client applications, your computer is mostly a "window" into the application
which is really being run by another (and usually much more powerful) computer.
WHY IS IT IMPORTANT TO LEARN ABOUT AUTHORING PARADIGMS?

 A great number of different authoring tools exist, but many share the same authoring
paradigm. So, if you know how to use Final Cut Pro, you'll be able to quickly learn how
to use Adobe Premiere and all other video editing tools because they use the same
timeline-based authoring metaphor.
 If you know how to use Audacity, you'll know how to use Pro Tools. If you know how to
use PureData, you will find it easy to understand how to work with MaxMSP.
 Knowing about authoring paradigms means you can easily transfer your
knowledge about one authoring tool to another authoring tool.

MULTIMEDIA PRESENTATION METAPHORS

Don't confuse authoring metaphors with the metaphors
you use to play multimedia content, which I'm calling
"multimedia presentation metaphors" for now.
The authoring metaphor refers to how you manipulate
media information so you can create content, while
the presentation metaphor refers to features of the
tool that you need to use so you can play back,
navigate, or otherwise experience a multimedia
product. For instance, going back to Figure 8.1, the
audio player uses a multimedia presentation metaphor
that is based on the analog cassette tape player (Figure
8.6).
Figure 8.6. The analog cassette tape
https://www.slideshare.net/gauravc7/multimedia- player
authoring-and-methaphors
The inspiration for the presentation
https://ftms.edu.my/v2/wp- metaphor used by many common media
content/uploads/2019/02/MMGD0101-chapter- player software (including the one
presented in Figure 8.1)
3_062015.pdf
https://www.itma.vt.edu/courses/appliedid/instructor.php