Metaphors and Styles of

Human-Computer Interaction
The psychology of metaphors
 Like affordances, having a metaphor can tell you
how you might use something.
 Unlike affordances (which are direct), metaphors
are indirect (require more inferences)
The psychology of metaphors
 a metaphor provides a lot of information
 it enables the transfer of skills
 good metaphors provide natural mappings
 metaphors are not taken literally
 they can highlight underlying assumptions
 metaphors are not symmetrical
 they can be violated
The psychology of metaphors
 metaphors can be mixed (e.g., windows and
desktops)
 metaphors can be misleading (putting a disk
in the trash)
 some things don’t seem to have any obvious
metaphor (ex: UNDO)
 one metaphor is better than another if it leads
to more correct predictions about a system’s
behavior.
Popular metaphors for computers
 computer as vast library (Memex, 1945)
 computer as giant calculator (ENIAC, 40s-50s)
 computer as intelligent assistant (Licklider, 1957)
 computer as sketchpad (Sutherland, 1962)
 computer as tool or typewriter (Engelbart, 1963)
 computer as human pretender (Weizenbaum, ‘60s)
 computer as network (Taylor, 1968)
 computer as book (portable) (Kay, Xerox PARC)
 computer as desktop/windows (Xerox PARC, ‘70s)
with objects
 computers for the rest of us (Apple, 1984)
More or less successful metaphors
 text editing as using a typewriter
 voice mail as answering machine or
mailbox
 data as files (in folders or directories),
represented as icons on desktop/in windows
 deleting a file as throwing it in the trash
 applications as tools (sometimes w/ icons)
 programming as building objects
 programming as directing actors on a stage
 applications as agents
 Metaphors often suggest
styles of interaction
Some styles of interaction:
 Commands (& queries, keyboard shortcuts)
 Menus (and W.I.M.P. interfaces)
 Direct manipulation
 Conversation (often w/ speech or natural language)
 Browsing
 Forms and spreadsheets
 Immersive environments
 Metaphors often suggest
styles of interaction
Some styles of interaction:
 Commands (& queries, keyboard shortcuts)
 Typing in choices to a vending machine
 Telling a dog to go fetch
 Interacting with VMS, DOS, Unix, or protocols such as
Telnet, ftp, etc.
 Programming your VCR to record a TV show
Command interface advantages
 Domain-specific language
 Similar to scripting language
 The user tells the system what to do
 e.g. tell the time, copy a file, save a file
 The user is in control (takes the initiative)
 Very common conceptual model for operating a
device (little or no intelligence in the device)
 Main benefits:
 Powerful (especially for experts and frequent users)
 Supports fast and efficient interaction
 Good for repetitive kinds of actions
Example: Copy command (DOS)
 Copies files from one location to another. The destination
defaults to the current directory.
> copy file1 [destination]
 If multiple files are to be copied, the destination must be a
directory, or an error will result.
> copy file1 file2 file3 [destination]
 Files may be copied to devices. To send file to printer:
> copy file lpt1
To display file on screen ("console"): (alternatively: type file)
> copy file con
>
No feedback given after one of these commands; just a prompt >
Equivalent Unix command: cp
Command interface drawbacks

 The user needs to know exactly where the

information is and how it’s organized
 The user needs to know command syntax
 No tolerance whatsoever for errors
 Can be cryptic, especially if little feedback is given
Another example: SQL queries
Which companies sell everything FabCo sells?

(List companies such that there does not exist

an item sold by FabCo that they do not sell.)
“Which companies sell everything FabCo sells?”

select distinct X.name

from suppliers X
where 0 = (select count (*)
from suppliers Y
where Y.name = “FabCo”
and 0 = (select count (*)
from suppliers Z
where Z.name=X.name
and Z.item=Y.item))
“Which companies sell everything FabCo sells?”

There must be an easier way!!

 (Schneiderman, 1986)
“Since computers can display information 1000
times faster than people can enter
commands, it seems advantageous to use
the computer to display large amounts of
information and allow novice and intermittent
users simply to choose among the items.”
 Styles of interaction

 Commands (& queries, keyboard shortcuts)

 Menus (and W.I.M.P. interfaces)
 Direct manipulation
 Conversation (often w/ speech or natural language)
 Browsing
 Forms and spreadsheets
 Immersive environments
Menus
 Users are given predetermined choices
 Are often part of a
WIMP interface (point & click)
 Windows and window managers
 Icons
 Menus
 Pointing devices
Menus
 enforce a hierarchy on the user’s goals
 represent well-trodden paths - you can do
only what the designer envisioned
 You cannot filter and combine the world in
novel ways
Decisions in menu design
 What is the task hierarchy?
 How to order or group items?
 How to trade off depth vs. breadth?
 What labels to choose?
 Whether to include shortcuts?
Menu types (not mutually exclusive)
 Permanent  Binary
 Pull-down  Extended
 Fall-down  Multiple selection
 Pop-up  Radio buttons
 Embedded
 Styles of interaction

 Commands (& queries, keyboard shortcuts)

 Menus (and W.I.M.P. interfaces)
 Direct manipulation
 Conversation (often w/ speech or natural language)
 Browsing
 Forms and spreadsheets
 Immersive environments
Direct Manipulation Interfaces
Representations behave as if they were the
objects they represent.
This reduces the distance between users and
their goals.
What You See Is What You Get
(WYSIWYG)

(Hutchins, Hollan, & Norman, 1986)

Direct manipulation
 Invented by Ivan Sutherland (Sketchpad, 60’s)
 Term was coined by Ben Shneiderman (1983)
 Theory developed by Hutchins, Hollan, & Norman
 Involves dragging, selecting, opening, closing
and zooming actions on virtual objects
 Exploit’s users’ knowledge of how objects behave
in the physical world (various metaphors)
 Consistent with “what you see is what you get”
(WYSIWYG)
Features of direct manipulation
 continuous representation of the objects of
interest
Features of direct manipulation
 continuous representation of the objects of
interest
 physical actions, not complex syntax
Example: copying a file to a diskette:
> copy file.txt to b:\file.txt
versus
dragging a file icon to a diskette icon
(move vs. copy confusion)
Features of direct manipulation
 continuous representation of the objects of
interest
 physical actions, not complex syntax
 rapid, incremental, reversible operations with
immediate visual feedback
Features of direct manipulation
 continuous representation of the objects of
interest
 physical actions, not complex syntax
 rapid, incremental, reversible operations with
immediate visual feedback
 easy to learn, easy to remember
 can operate upon output representation and
use it as input back to the system
Direct Manipulation
(semantic distance) What do I
Gulf of Execution do now?

User’s
System
Goals

Gulf of Evaluation What does

(articulatory distance) that mean?

Hutchins, Hollan, & Norman

Why are DM interfaces enjoyable?
 Novices can learn the basic functionality quickly
 Experienced users can work extremely rapidly to carry
out a wide range of tasks, even defining new functions
 Intermittent users can retain operational concepts over
time
 Error messages are rarely encountered
 Users can immediately see if their actions are furthering
their goals and if not, do something else
 Users experience less anxiety
 Users gain confidence and mastery and feel in control
Some problems with D.M.
 Sometimes: information overload or screen clutter
 Not all tasks can be represented by objects
 it’s hard to represent abstract things
 DM must function in the “here” and “now”
 icons can be just as cryptic as words (the Vocabulary Problem:
stay tuned)
 With DM, user is responsible for doing everything; but
some tasks are better achieved by delegating!
 e.g. spell checking
 repetitive actions are tedious!
 Moving a mouse around the screen can be slower than pressing
function keys to do same actions
 Some DM metaphors are taken too literally
Debate about styles of interaction
Direct Manipulation (tool metaphors)
v.
Conversation (agent metaphors)
 (Schneiderman, 1986)
“People are different from computers, and human-
human interaction is not necessarily an
appropriate model for human operation of
computers. Since computers can display
information 1000 times faster than people can
enter commands, it seems advantageous to use
the computer to display large amounts of
information and allow novice and intermittent
users simply to choose among the items.”
 (Schneiderman, 1986)
“People are different from computers, and human-
human interaction is not necessarily an
appropriate model for human operation of
computers. Since computers can display
information 1000 times faster than people can
enter commands, it seems advantageous to use
the computer to display large amounts of
information and allow novice and intermittent
users simply to choose among the items.”
But what about:
“Which companies sell everything FabCo sells?”
 Styles of interaction

 Commands (& queries, keyboard shortcuts)

 Menus (and W.I.M.P. interfaces)
 Direct manipulation
 Conversation (often w/ speech or natural language)
 Browsing
 Forms and spreadsheets
 Immersive environments
Dialogue systems
 May take initiative
 Underlying metaphor is having a conversation
with another person
 Ranges from simple voice recognition systems
to more complex ‘natural language’ dialogues
 Examples include dialogs about timetables,
search engines, advice-giving systems, help
systems
 May (or may not) consist of anthropomorphic
agents who converse with you or are part of an
interface, e.g. Microsoft’s Bob and Clippy
Agents
A metaphor for an “intelligent” process that takes
some initiative and perform tasks on the user’s
behalf
The user delegates responsibility to the agent.
This feels very different from direct manipulation!
Agents can be amplifiers or alternatively,
prosthetics (2 distinct metaphors for agents)
Tasks for Agents (Laurel; Maes)
 Delegation
 Information retrieval - Web “bots”
 Sorting, organizing, filtering
 Coaching, tutoring, providing help
 Reminding
 Programming, doing repetitive things
 Advising
 Entertaining
 Navigation
Agents (anthropomorphic or not)
The most obvious metaphor for an interactive
system that seems intelligent or takes
initiative may be a human-like character.
(But note: agents can be non-anthropomorphic
as well… a form can take initiative and act
“intelligent” by filling in the right values
automatically.)
 What agent interfaces have you used?
Food for thought:
To what extent is dialog with a person
similar to interacting with a computer?

(Shneiderman: not at all!)

(Others: If the system can take the
initiative and act “intelligently”, then
conversation is a good metaphor for
interacting with an agent.)
Should representations ever
be anthopomorphic?
CON: Ben Shneiderman
Sherry Turkel
Joseph Weizenbaum (Eliza)

PRO: Brenda Laurel

I say: It depends on the situation!!

Drawbacks of delegation
 “Intelligent” interfaces can be
unpredictable.
 How do you come to “trust” your agent?
 Are anthropomorphic interfaces honest?
 There may be problems with
anthropomorphizing an interface
 (Shneiderman’s arguments)
Theater and stage metaphors
(Brenda Laurel; Laura Gould)

Interfaces are like theater -

we suspend disbelief.
Claim: Anthropomorphizing an interface is
not a problem: People aren’t so easily
fooled.
(This contrasts w/ Shneiderman’s claims.)
 (What do you think?)
Initiative
 Some kinds of human conversations
are scripted - who takes the initiative is
relatively fixed.
 In other kinds of conversation,
neither person takes all the initiative.
 Ideally, initiative should be flexible.
Human-Computer Dialog
What can go wrong?
 A dialog can be under-determined.
when the user is mystified about
what to do next and is
forced to take the initiative.
(a blank screen, cryptic commands)

(Harold Thimbleby)
Human-Computer Dialog (cont.)
 A dialog can be over-determined
when a system is authoritarian
and takes too much initiative.
(unnatural constraints)
Advantages:
 reduces the consequences of errors
 what the user does is predictable
Dialog Design
 Avoid under-determined dialogs by
providing help and by reducing options.
 Avoid over-determined dialogs by
letting users undertake tasks in flexible
orders.
Don’t force them to do unnecessary
things.
 Support alternative styles of interaction
whenever possible
Different users are different!
 Interfaces that take the initiative are
better for novices.
 Interfaces that let the user take the
initiative are better for experts.

An interface is over- or under-determined

with respect to a particular user.
 Direct manipulation and menu
interfaces are sometimes (not always)
over-determined.
 Command languages and natural
language are sometimes (not always)
under-determined.
Pros & cons of conversational style of
interaction
 Dialogue allows users, especially novices and
technophobes, to interact with the system in a
way that is familiar.
 makes them feel comfortable, at ease, not scared
 Misunderstandings arise when the system fails
to parse what the user says.
 Sometimes interfaces pretend to use natural
language, which can lead to problems.
 e.g. child types into a search engine, that uses
natural language the question: “How many legs
does a centipede have?” and the system responds:
 Styles of interaction

 Commands (& queries, keyboard shortcuts)

 Menus (and W.I.M.P. interfaces)
 Direct manipulation
 Conversation (often w/ speech or natural language)
 Browsing
 Forms and spreadsheets
 Immersive environments
Exploring and browsing
 Similar to how people browse information
with existing media (e.g. newspapers,
magazines, libraries, pamphlets)
 Information is structured to allow flexibility in
way user is able to search for information
 e.g. multimedia, web
 Browsers, hypertext, links
Forms and spreadsheets
 Forms can vary from over-determined
(enforcing a fixed order) to quite flexible
(allowing user initiative)
 Spreadsheets are extremely flexible,
allowing “user programming”. They can
sometimes be underdetermined
Immersive environments
 Virtual reality (stay tuned)
 Often mimic real life (representations
are highly literal; go well beyond
metaphors)
Different styles of interacting lend
themselves to very different metaphors
 Commands
 typing commands via keyboard; function key shortcuts
 Menus
 selecting from pre-determined choices
 Direct manipulation
 acting on objects and interacting with virtual objects
 Conversational dialogue systems
 interacting with the system as if having a conversation
 Exploring, browsing, & foraging for information
 finding out and learning things
 browsers (different from menus)
Choose the right interaction style and
the right metaphor

 Understand how the system works.

(use the system yourself)
(ask engineers)
 Figure out what problems users have.
(watch them use similar systems)
(create prototypes and watch users)
 Generate metaphors and examine their
properties.
Ask these Qs:
 Does the metaphor have enough structure?
 Is the structure relevant to the application or
problem? (or is it superficial?)
 Is the metaphor easy to represent? Concrete
enough?
 Will the users “get it”?
 Can it be extended?
 Is the metaphor misleading? Annoying?
 Does the metaphor fit with the style of interaction?
Problems with interface metaphors
 Sometimes they break conventional and cultural
rules
 e.g. recycle bin placed on desktop
 Can overly constrain designers in the way they
conceptualize a problem space
 May conflict with principles of good design
 Forces users to understand the system in terms of
the metaphor
 Designers can inadvertently use bad existing designs
and transfer the bad parts into the new designs
 Limits designers’ imagination in coming up with new
conceptual models