What is Interactive Computer Graphics ?

Interactive Graphical Systems Computer Aided Design (CAD)

• interactive modeling (simple visual representations)
HT2002 • metrically correct constructions as a basis for automated manufacturing
• results : Technical sketches, simple rendered still images

Computer Graphics Animation

Lesson 2 : Graphics Primer • modeling & design of objects
• modeling of dynamic behavior
• results : HQ static pictures and static movies

Stefan Seipel Interactive Computer Graphics

• on-line rendering (real-time!)
• rendering as realistic as frame-rate allows
• results: user driven non static 3D scenario

What is Interactive Computer Graphics ? What is Interactive Computer Graphics ?

Example : Computer Aided Design (CAD) Example : Computer Graphics Animation

Richard Nordhaus
Tin Toy
School of Architecture and Planning
Copyright Pixar Animation Studios
University of New Mexico
 Interactive Graphical Systems: Structure
What is Interactive Computer Graphics ?
Example : Interactive Computer Graphics

Multimedia Real- Multimodal

time
Feedback Input

Computer-
simulation

Real world
Design and Simulation of a Robot Cell
Copyright Sandia National Libraries Replication

How are graphical objects defined ? Definition of “points”

Surface representations Based on the definition of a reference co-ordinate

=> infinite number of points to form object surface system by their components
(mathematical functions)

=> finite set of surface points 1) kartesian co-ordinate system { O; i, j, k } j

- rendered as point clouds - orthogonal
- rendered as polygons given a certain connectivity - orthonormal P(x,y,z)
.
O
=> finite set of volume points (voxels)
- rendered as point clouds i
- rendered as solid volume objects
k
2) polar co-ordinate system { O; r, OI } Relations between polar and cartesian co-ordinates

definition of O, y, and z
y

r x2  y2  z2 x = r ˜ cosM ˜ cosO

r .P(r, OI ) SO S

x
O
tan O = y = r ˜ sin M
I SI S z
z O y
tan M
2 2
z = r ˜ cos M ˜ sin O
x z

Application of the dot product - Backface Culling

Vector algebra (quick rehearsal of useful formulas)
Given: A polygon vertex P, and polygon normal vector N
Users viewing position V
1. Inner product / dot product of two vectors
Question: Is the polygon facing towards the observer ?

a xb a x b x  a y b y  a z bz y
.
~v N
b
a P. .
ba O
a axa a x2  a y2  a z2
V´ x
a xb D= ab
D cos  1
a ˜b z .V
Used for
ba a xb š a 1 a) calculation of length and distances Solution: The polygon is facing towards the observer if the normal vector of that face is facing towards
the observer. This is the case, if the length of the projection of the vector V’ onto N is
b) angle measurements
greater than zero. See next slide for two examples.
Application of the dot product - Backface Culling Vector algebra (quick rehearsal of useful formulas)

§ 1· §2 3·
¨ ¸ ¨ ¸ 2. Vector product or cross product of two vectors
Let: P ¨ 1¸ ; N ¨1 3¸ ; N 4 9 1 9  4 9 9 9 1
¨ ¸ ¨ ¸
© 0¹ ©2 3¹ aub c

§ 10· §  5·
a y   
˜ b z  b y ˜ a z i  a z ˜ b x  b z ˜ a x  j  a x ˜ b y  b x ˜ a y k
¨ ¸ ¨ ¸
Case1: Let V ¨ 7¸ Case2: Let V ¨ 7 ¸ § a y ˜ bz  b y ˜ a z ·
¨ ¸ ¨ ¸ ¨ ¸
© 15¹ ©  9¹ ¨ a z ˜ b x  bz ˜ a x ¸
¨ ¸
© a x ˜ by  bx ˜ a y ¹ c
§ 10  1· § 9· §  5  1· §  6·
¨ ¸ ¨ ¸ ¨ ¸ ¨ ¸
V' V  P ¨ 7 1¸ ¨ 6¸ V' V  P ¨ 7 1 ¸ ¨ 6 ¸ c is a vector perpendicular to a and b.
¨ ¸ ¨ ¸
¨ ¸
© 15  0¹
¨ ¸
© 15¹ ©  9  0¹ ©  9¹ b
.
Used for .
v~ N x V ' 2 3 ˜ 9  1 3 ˜ 6  2 3 ˜ 15 v~ N x V ' 2 3 ˜ 6  1 3 ˜ 6  2 3 ˜ 9 a) calculation of surface normal vectors
6  2  10 18 4  2  6  8 b) test for collinearity of two vectors (c = 0)
a
c) calculation of reference frames
18>0 -> The polygon is facing towards V. -8<0 -> The polygon is not facing towards V.

Application of the cross product - Polygon Normal Setup

Concatenation of Transforms : Matrix Multiplication
Given: A triangle given by vertices A,B and C
y C.
Question: How does the polygons normal vector look like ?
N Matrix A ; n x k elements
. .
§ 1· § 2 · § 1 · A B
¨ ¸ ¨ ¸ ¨ ¸ O
A ¨ 1¸ ; B ¨ 1.5 ¸ ; C ¨ 2 ¸ Matrix B ; k x l elements
¨ ¸ ¨ ¸ ¨ ¸
© 0¹ ©  1¹ ©  1¹ x
=> C = AB defined; and the resulting matrix C has n x l elements
Span vectors: z
§ 1 2 ·
¨ ¸
§ 1 ·
¨ ¸
§ 11 ·
¨ ¸
§ 0 ·
¨ ¸ => generally : AB not equal BA !
ab B A ¨ 1.5  1 ¸ ¨ 0.5 ¸ ; ac C A ¨ 21 ¸ ¨ 1 ¸
¨ ¸ ¨ ¸ ¨ ¸ ¨ ¸
©  1  0¹ ©  1¹ ©  1  0¹ ©  1¹

0.5 1 k

N ab u ac
1
1
1
0
§  0.5  (  1) ·
¨
¨ 0  (  1) ¸
¸
§ 0.5·
¨ ¸
¨ 1¸
c ij ¦a
s 1
is b sj ;1  i  n ,1  j  l
0.5 1 ¨ ¸ ¨ ¸
© 1 0 ¹ © 1¹
1 1
Transformation of Points : Matrix-Vector Multiplication Matrix representation of basic geometric transforms
Scale Translation Rotations
Transformation represented as a Mn,k where n=4, k=4.
§sx 0 0 0· §1 0 0 tx · §cos- sin- 0 0·
¨ ¸ ¨ ¸ ¨ ¸
Point represented by homogeneous coordinates { x, y, z, 1 } as column vector P ¨0
S ¨
sy 0 0¸ ¨0 1 0 ty¸ ¨sin- cos- 0 0¸
T Rz (-) ¨
where k = 4, l = 1. 0 0 sz 0¸ ¨0 0 1 tz ¸ 0 0 1 0¸
¨ ¸ ¨ ¸ ¨ ¸
©0 0 0 1¹ ©0 0 0 1¹ © 0 0 0 1¹
The resulting matrix has n=4 rows and l=1 columns
§1 0 0 0·
¨ ¸
¨0 cos-  sin- 0¸
Rx (-) ¨
0 sin- cos- 0¸
¨ ¸
m11 m12 m13 m14 x m11x + m12y + m13Z + m14 ©0 0 0 1¹
m21 m22 m23 m24 y m21x + m22y + m23Z + m24
P’ = M.P = m31 m32 m33 m34 z = m31x + m32y + m33Z + m34
m41 m42 m43 m44 1 m41x + m42y + m43Z + m44
§ cos- 0 sin- 0·
¨ ¸
¨ 0 1 0 0¸
Ry (-) ¨
 sin- 0 cos- 0¸
¨ ¸
© 0 0 0 1¹

Accumulated/composite transformations Accumulated/composite transformations

Any attitude of an object can be expressed/decomposed as an accumulation of

elementary transforms.
A)
Y 1. Rotate about Z, 50 deg.
Example : A = RzTS
2. Translate along X, 0.5
3. Rotate about X, 25 deg.
Observe : Matrix concatenation order (transformation sequence) is important.

B)
1. Rotate about X, 25 deg.
Example : A = RzT Example : A = TRz
2. Translate along X, 0.5
3. Rotate about Z, 50 deg.
y Z X
y y y y

x x x x x x
Calculation of Reference Frames (Attitude Matrix) Calculation of Reference Frames (Attitude Matrix)

Given: Three points in space identify the spatial attitude of an object´s reference frame
ªt11 t12 t12 t14 º ª1º ªt11 ˜1  t12 ˜ 0  t12 ˜ 0  t14 ˜1º ªax º ªa x ? ? 0º
«t t 24 »» «0» «t ˜1  t 24 ˜1»» «a » «a 0»»
Txa * x « 21 t 22 t 23
˜« » « 21  t 22 ˜ 0  t 23 ˜ 0
« y» Hence: Txa « x ? ?
y «t31 t32 t33 t34 » «0» «t31 ˜1  t32 ˜ 0  t33 ˜ 0  t34 ˜1» «az » «a x ? ? 0»
« » « » « » « » « »
¬t 41 t 42 t 43 t 44 ¼ ¬1¼ ¬t 41 ˜1  t 42 ˜ 0  t 43 ˜ 0  t 44 ˜1¼ ¬1¼ ¬0 0 0 1¼
y b C b
a . a ay
A. .B O ax
az
ªt11 t12 t12 t14 º ª0º ªt11 ˜ 0  t12 ˜1  t12 ˜ 0  t14 ˜1º ªbx º ª? bx ? 0º
c «t t 24 »» «1» «t ˜ 0  t 24 ˜1»» «b » «? b ? 0»»
O c Tyb * y « 21 t 22 t 23
˜« » « 21  t 22 ˜1  t 23 ˜ 0
« y» Hence: Tyb « y

x «t31 t32 t33 t34 » «0» «t31 ˜ 0  t32 ˜1  t33 ˜ 0  t34 ˜1» «bz » «? bz ? 0»
« » « » « » « » « »
x z ¬t 41 t 42 t 43 t 44 ¼ ¬1¼ ¬t 41 ˜ 0  t 42 ˜1  t 43 ˜ 0  t 44 ˜1¼ ¬1¼ ¬0 0 0 1¼
z
A, B, and C can be used to calculate A transformation matrix Txa rotates the unit vector
(1,0,0,1) into the new base vector a=(ax,ay,az,1) ªt11 t12 t12 t14 º ª0º ªt11 ˜ 0  t12 ˜ 0  t12 ˜1  t14 ˜1º ªcx º ª? ? cx 0º
a new orthonormal basis {A;a,b,c} «t
« 21 t 22 t 23 t 24 »» «0» «t ˜ 0
« 21  t 22 ˜ 0  t 23 ˜1  t 24 ˜1»» «c » «? ? cy 0»»
Tzc * z ˜« » « y» Hence: Tzc «
a CA ªt11 t12 t12 t14 º ª1º ªt11 ˜1  t12 ˜ 0  t12 ˜ 0  t14 ˜1º ªax º «t31 t32 t33 t34 » «1» «t31 ˜ 0  t32 ˜ 0  t33 ˜1  t34 ˜1» «cz » «? ? cz 0»
«t t 24 »» ««0»» «t ˜1  t 22 ˜ 0  t 23 ˜ 0  t 24 ˜1»» «a » « » « » « » « » « »
b ( B  A) u a « 21 t 22 t 23
Txa * x
«t31 t32 t33
˜
t34 » «0»
« 21
«t31 ˜1  t32 ˜ 0  t33 ˜ 0  t34 ˜1»
« y»
«az » ¬t 41 t 42 t 43 t 44 ¼ ¬1¼ ¬t 41 ˜ 0  t 42 ˜ 0  t 43 ˜1  t 44 ˜1¼ ¬1¼ ¬0 0 0 1¼
c aub « » « » « » « »
¬t 41 t 42 t 43 t 44 ¼ ¬1¼ ¬t 41 ˜1  t 42 ˜ 0  t 43 ˜ 0  t 44 ˜1¼ ¬1¼
Note: a,b, and c must be normalized!

Calculation of Reference Frames (Attitude Matrix)

Representations of 3D Models
Analoguously, a transformation matrix TOA translates the origin of the base system O=(0,0,0,0) into the new
Origin A=(Ax,Ay,Az,1)

ªt11 t12 t12 t14 º ª0º ªt11 ˜ 0  t12 ˜ 0  t12 ˜ 0  t14 ˜1º ª Ax º ª? ? ? Ax º Quantitative Models
«t t 24 »» «0» «t ˜ 0  t 22 ˜ 0  t 23 ˜ 0  t 24 ˜1»» «A » «?
« 21 t 22 t 23 « 21 « y » Hence: T ? ? Ay »» Mathematical, physical, chemical... function define vertex data
TOA * O ˜« » OA
«
«t31 t32 t33 t34 » «0» «t31 ˜ 0  t32 ˜ 0  t33 ˜ 0  t34 ˜1» « Az » «? ? ? Az »
« » « » « » « » « »
¬t 41 t 42 t 43 t 44 ¼ ¬1¼ ¬t 41 ˜ 0  t 42 ˜ 0  t 43 ˜ 0  t 44 ˜1¼ ¬1¼ ¬0 0 0 1¼
Procedural Models
Since Tax,Tby, and Tcz operate independantly on the respective base vectors, they can be Declarative or procedural system describes vertex data and connectivity
combined into on single transformation matrix, that rotates a all base vectors simultaneously. (rule based, language based, recursive and fractal functions)
The translation matix operates independently on to trnaslate the origin, regardles of the
rotational components. Hence, the combined transformation matrix can be denoted as:
Hierarchical Geometric Models
ªax bx cx Ax º Explicit definition of objects by vertex data and connectivity
«a Ay »»
T « y by cy Re-Usage of geometric prototypes
«az bz cz Az » Defining hierarchies and geometric relations between instances
« »
¬0 0 0 1¼
 How are graphical objects defined ?
How are graphical objects defined ?
Finite set of volume points (voxels) : Voxelviewer
Mathematical functions : Example MATLAB

How are graphical objects defined ? Object representations in interactive CG

Finite set of surface points : Submarine
Graphic rendering engines are traditionally optimized for
• drawing of 3D points
• drawing of 3D vectors
• drawing of 3D polygons (mostly triangles)
• (some can draw spheres e.g. SGI Extreme)

=> objects are represented as composite clumps of vertices

and polygons.

=> more advanced, analytical descriptions of objects are usually

not found in interactive computer graphics
(see 3D animation programs and raytracing s/w)
 Explicit geometry representations in interactive CG
Object representations in interactive CG

Example : Scene with a teapot Organization of geometry data: Vertex List

1 x,y,z,...u,v
2 x,y,z,...u,v
1. explicit definition of vertices&polygons 2. combined approach: explicit vertex data and 3 x,y,z,...u,v
rendered using flat shading mathematical polynomial functions define 4 x,y,z,...u,v
curved surfaces. 5 x,y,z,...u,v
6 x,y,z,...u,v
v7

x,y,z,...u,v
Polygon List
n
p4 v6 1 4 { 1, 2, 3, 4 } { ......... }
y
v2 2 3 { 6, 4, 3 } { ......... }
Vertex Index Vertex Data
v3 Cartesian Coordinates 3 3 { 5, 4, 6 } { ......... }
p2 Normal Vectors 4 4 { 3, 2, 7, 6 } { ......... }
Texture Coordinates
p1 p3 Color Values
v5
v1 k 3 { l, m, n } { ......... }
v4
x Polygon Index
Number of Vertices
z
Pictures: Copyright (c) 1990 Pixar Objects reference frame List of Vertex Indices
Other data e.g. Texture Reference

Rendering Methods Rendering Methods

• Point Clouds • Object vertices are drawn as singular pixels
• Color of the pixels can be constant (object color)
or be shaded (depending on lights and object colors)
• Determination of discrete screen-coordinates from
model vertex coordinates
• Very fast rendering method
Examples:
• Wire Frames • Edges of object polygons are drawn as lines
• Color of the lines can be constant (object color)
or be shaded (depending on lights and object colors)
• Determination of discrete screen-coordinates for
model vertices
• 2D line drawing (scan conversion) between the vertices
• Very fast rendering method if used without hidden
surface removal
Examples:
Rendering Methods
Surface Shading Models
• Surface Shading • Object polygons are drawn as filled areas
• Color of the polygons can be constant (object color)
or be shaded (depending on lights and object colors)
(constant or ambient shading, diffuse shading)
• Determination of discrete screen-coordinates for model vertices
• 2D line drawing (scan conversion) of the edges
• 2D area fill inside the polygon
• Can be very slow depending on shading method
Examples:
• Constant / Flat Shading
) one shading value is calculated per polygon
) entire polygon is filled only with that color value
) fast but not nice looking
• Gouraud Shading (Color Interpolation)
) shading values are calculated for each vertex
) color values for each pixel inside polygon
are linearly interpolated
) good visual appearance at cost of reduced rendering speed

• Phong Shading (Normal Vector Interpolation)

) for each pixel the surface normal vector is interpolated
from the given vertex normals
) shading calculation is performed for each pixel
) extremely slow rendering, very good visual results (specular properties)

Example: Flat Shading vs. Gouraud Shading Requirements: Flat Shading vs. Gouraud Shading

Flat-Shading:
n1
One surface normal n2
vector per face/polygon.

n3 n4
n4

Phong-Shading:
n1
fn1 n2
Surface normal vectors fn2
per vertex.
Usually calculated from n1
n2
surrounding polygon
normal vectors. fn3
fn4

Example : n1 = (fn1 +fn2 +fn3 +fn4)/4

Gouraud Shading and Phong Shading Differences : Gouraud Shading vs. Phong Shading
Example: Ik (nk x l) O ˜ [ §  1· § 2· §  2·
¨ ¸ ¨ ¸ ¨ ¸
l l n1 ¨ 4 ¸ ; n21 ¨ 4 ¸; l ¨  4¸
O 10
. ; [ 10
. ¨ ¸ ¨ ¸ ¨ ¸
©  1¹ © 05
.¹ © 1¹
n2 nn n2
n1 n1
v2 C2 v2 C2
n nx ny nz arccos(nl) I(phong)I(gouraud)
D(n) D(n) 0,00
1,00
-1,00
-0,85
4,00
4,00
-1,00 -0,77152
-0,93 -0,79243
0,771517 0,771517
0,792431 0,78
Intensity accross span
C1 v Cn C1 v Cn 2,00 -0,70 4,00 -0,85 -0,81264 0,812638 0,79
1 1 3,00 -0,55 4,00 -0,78 -0,83198 0,831983 0,80 1,2
4,00 -0,40 4,00 -0,70 -0,85032 0,850316 0,81
5,00 -0,25 4,00 -0,63 -0,86749 0,867493 0,82
6,00 -0,10 4,00 -0,55 -0,88338 0,883381 0,82
7,00 0,05 4,00 -0,48 -0,89786 0,897862 0,83 1
Gouraud v3 Phong v3 8,00
9,00
0,20
0,35
4,00
4,00
-0,40 -0,91084
-0,33 -0,92224
0,91084
0,922239
0,84
0,85
10,00 0,50 4,00 -0,25 -0,93201 0,932007 0,86
Illumination and color calculation at vertices: Illumination and color calculation for each pixel analytical: 11,00 0,65 4,00 -0,18 -0,94012 0,94012 0,87 0,8
12,00 0,80 4,00 -0,10 -0,94658 0,946578 0,88
O For vertex pixels: 13,00 0,95 4,00 -0,03 -0,95141 0,951406 0,88
C1 ( n1 x l ) ˜ [ 14,00 1,10 4,00 0,05 -0,95465 0,954652 0,89 I(phong)
C1 ( n1 x l ) O ˜ [

I(n)
15,00 1,25 4,00 0,13 -0,95638 0,956382 0,90 0,6
16,00 1,40 4,00 0,20 -0,95668 0,956681 0,91 I(gouraud)
C2 (n2 x l) O ˜ [
17,00 1,55 4,00 0,28 -0,95564 0,955645 0,92
C2 (n2 x l) O ˜ [ 18,00 1,70 4,00 0,35 -0,95338 0,953379 0,93
19,00 1,85 4,00 0,43 -0,95 0,949997 0,94 0,4
Color calculation between vertices by (bi)linear 20,00 2,00 4,00 0,50 -0,94561 0,945611 0,945611

interpolation: For pixels between vertices, interpolation of normal vector l -2 -4 1

and calculation of illumination and color: 0,2
Cn (1  D ) ˜ C 1  D ˜ C 2 š > 0 d D d 1@
nn ( 1  D ) ˜ n 1  D ˜ n 2 š > 0 d D d 1@
0
cn (n k x l) O ˜ [ 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

More on illumination : see Foley, chapter 16.1 k

Explicit geometry representations in interactive CG

Hierarchical modeling

Organization of geometry data: Vertex List

Purposes:
1 x,y,z,...u,v
2 x,y,z,...u,v
3 x,y,z,...u,v
4 x,y,z,...u,v Construct complex objects in modular fashion
5 x,y,z,...u,v • reusability of building blocks
6 x,y,z,...u,v
v7
• ease of modeling
Polygon List
y p4 v6
n x,y,z,...u,v
1 4 { 1, 2, 3, 4 } { ......... } Increase storage efficiency
v2
v3
Vertex Index Vertex Data 2 3 { 6, 4, 3 }
{ 5, 4, 6 }
{ ......... }
• build once
Cartesian Coordinates 3 3 { ......... }
p2 Normal Vectors
Texture Coordinates
4 4 { 3, 2, 7, 6 } { ......... } • execute/re-use many times (display lists)
p1 p3 Color Values
v5
v1 k 3 { l, m, n } { ......... }
Allow easy update of components
v4
x Polygon Index

z
Number of Vertices
Allow for easier animation of behavior
Objects reference frame List of Vertex Indices
Other data e.g. Texture Reference
 Hierarchical modeling Simple Robot : Modeling Hierarchy

Example: simple robot

Base

Trunk

Head Left_Arm Right_Arm

Tree of involved instances List of generic objects used

Three different generic objects Several instances of objects arranged in hierarchica
order

Creating the Model Hierarchy Creating the Geometric Context

Base VRT_NodeScale(base,0.1,0.2,0.1);
base = VRT_NodeNew(root,"Robot Base");
Trunk trunk = VRT_NodeNew(base,"Robot Trunk"); VRT_NodeTranslate(trunk,0.0,2.0,0.0);
head = VRT_NodeNew(trunk,"Robot Head"); VRT_NodeScale(trunk,1.5,1.0,1.5);
left_arm = VRT_NodeNew(trunk,"Left Arm");
Head Left_Arm Right_Arm
right_arm = VRT_NodeNew(trunk,"Right Arm"); VRT_NodeTranslate(head,0.0,1.2,0.0);
Tree of involved instances VRT_NodeScale(head,0.5,0.5,0.5);
cylinder = VRT_Cone(1.0,1.0,1.0,20);
cube = VRT_Cube(1); VRT_NodeTranslate(right_arm,-1.4,0.8,-0.2)
box = VRT_Box(0.2,1.0,0.2); VRT_NodeScale(right_arm,2.0,2.0,2.0);
VRT_NodeRotate(right_arm,180,0,0);
VRT_NodeSetGeometry(base,cylinder);
VRT_NodeSetGeometry(trunk,cube); VRT_NodeTranslate(left_arm,1.0,0.8,-0.2);
VRT_NodeSetGeometry(head,cube); VRT_NodeScale(left_arm,2.0,2.0,2.0);
Generic objects VRT_NodeSetGeometry(left_arm,box); VRT_NodeRotate(left_arm,180,0,0);
VRT_NodeSetGeometry(right_arm,box);
Simple Robot : The Scene Graph

“base”
nnn
nnn
nnn

Start robot.exe

“trunk”
nnn
nnn
nnn

“head” “left_arm” “right_arm”

nnn nnn
nnn
nnn nnn
nnn
nnn nnn
nnn