An iOS Developer’s OpenGL Primer

Jamie Montgomerie @th_in_gs

Things Made Out Of Other Things

OpenGL ES 2.0

OpenGL ES 2.0
A transparent, lightweight, interface to modern graphics hardware.

Why Use OpenGL?

Why Use OpenGL?

Flexibility

Why Use OpenGL?

Flexibility Performance

How OpenGL uses your hardware to put stuff on the screen.

How you can use OpenGL to put your stuff on the screen.

Demo

Demo

What makes all that stuff happen?

A transparent, lightweight, interface to modern graphics hardware.

A transparent, lightweight, interface to modern graphics hardware.

A transparent, lightweight, interface to modern graphics hardware.

Where do the pixels come from?

Demo

Where do the pixels come from?

Where do the pixels come from?

Fragment shader.

Where do the pixels come from?

Fragment shader. • Simple program that runs on the GPU.

Where do the pixels come from?

Fragment shader. • Simple program that runs on the GPU. • Computes the shade for each fragment, or pixel.

Where do the pixels come from?

Fragment shader. • Simple program that runs on the GPU. • Computes the shade for each fragment, or pixel. • Run once for every pixel simultaneously.

Where do the pixels come from?

Fragment shader. • Simple program that runs on the GPU. • Computes the shade for each fragment, or pixel. • Run once for every fragment independently.

Where do the pixels come from?

Fragment shader. • Simple program that runs on the GPU. • Computes the shade for each fragment, or pixel. • Run once for every pixel simultaneously.

OpenGL ES 2.0 Shading Language

OpenGL ES 2.0 Shading Language
uniform lowp sampler2D sTexture; varying highp vec2 vTextureCoordinate; void main() { gl_FragColor = texture2D(sTexture, vTextureCoordinate); }

Shading Language Basic Types
Scalars Vectors Matrices Samplers
float int bool [i,b]vec2 [i,b]vec3 [i,b]vec4 mat2 mat3 mat4 sampler2d samplerCube

Shading Language Variable Initialisation
float i = 1.0;

Shading Language Variable Initialisation
float i = 1.0; const float too = 2.0;

Shading Language Variable Initialisation
float i = 1.0; const float too = 2.0;

Shading Language Variable Initialisation
float i = 1.0; const float too = 2.0; int j = 1;

Shading Language Variable Initialisation
1 2 3 4

v =

Shading Language Variable Initialisation
vec4 v = vec4(1.0, 2.0, 3.0, 4.0);

Shading Language Variable Initialisation
vec4 v = vec4(1.0, 2.0, 3.0, 4.0); mat2 m = mat2(1.0, 0.0, // 1st Column 0.0, 1.0); // 2nd Column

OpenGL ES 2.0 Shading Language
uniform lowp sampler2D sTexture; varying highp vec2 vTextureCoordinate; void main() { gl_FragColor = texture2D(sTexture, vTextureCoordinate); }

Colours.fsh
uniform lowp sampler2D sTexture; varying highp vec2 vTextureCoordinate; void main() { gl_FragColor = texture2D(sTexture, vTextureCoordinate); }

Simples.fsh

void main() { gl_FragColor = vec4(0.0, 0.0, 0.0, 1.0); }

Simples.fsh

void main() { gl_FragColor = vec4(0.0, 0.0, 0.0, 1.0); }

Colours.fsh
uniform lowp sampler2D sTexture; varying highp vec2 vTextureCoordinate; void main() { gl_FragColor = texture2D(sTexture, vTextureCoordinate); }

Colours.fsh
uniform lowp sampler2D sTexture; varying highp vec2 vTextureCoordinate; void main() { gl_FragColor = texture2D(sTexture, vTextureCoordinate); }

Colours.fsh
uniform lowp sampler2D sTexture; varying highp vec2 vTextureCoordinate; void main() { gl_FragColor = texture2D(sTexture, vTextureCoordinate); }

Colours.fsh
uniform lowp sampler2D sTexture; varying highp vec2 vTextureCoordinate; void main() { gl_FragColor = texture2D(sTexture, vTextureCoordinate); }

Colours.fsh
uniform lowp sampler2D sTexture; varying highp vec2 vTextureCoordinate; void main() { gl_FragColor = texture2D(sTexture, vTextureCoordinate); }

Colours.fsh
uniform lowp sampler2D sTexture; varying highp vec2 vTextureCoordinate; void main() { gl_FragColor = texture2D(sTexture, vTextureCoordinate); }

Colours.fsh
uniform lowp sampler2D sTexture; varying highp vec2 vTextureCoordinate; void main() { gl_FragColor = texture2D(sTexture, vTextureCoordinate); }

Colours.fsh
uniform lowp sampler2D sTexture; varying highp vec2 vTextureCoordinate; void main() { gl_FragColor = texture2D(sTexture, vTextureCoordinate); }

Colours.fsh
uniform lowp sampler2D sTexture; varying highp vec2 vTextureCoordinate; void main() { gl_FragColor = texture2D(sTexture, vTextureCoordinate); }

Colours.fsh
uniform lowp sampler2D sTexture; varying highp vec2 vTextureCoordinate; void main() { gl_FragColor = texture2D(sTexture, vTextureCoordinate); }

Colours.fsh
uniform lowp sampler2D sTexture; varying highp vec2 vTextureCoordinate; void main() { gl_FragColor = texture2D(sTexture, vTextureCoordinate); }

Colours.fsh
uniform lowp sampler2D sTexture; varying highp vec2 vTextureCoordinate; void main() { gl_FragColor = texture2D(sTexture, vTextureCoordinate); }

Demo

Demo

Where do the pixels come from?

Where do the pixels come from?

Where do the pixels come from?

Where do the pixels come from?

pixels

Colours.fsh
uniform lowp sampler2D sTexture; varying highp vec2 vTextureCoordinate; void main() { gl_FragColor = texture2D(sTexture, vTextureCoordinate); }

Colours.fsh
uniform lowp sampler2D sTexture; varying highp vec2 vTextureCoordinate;

Colours.fsh
uniform lowp sampler2D sTexture; varying highp vec2 vTextureCoordinate;

Where are these inputs coming from?

Where are these inputs coming from?

Where are these inputs coming from?

uniforms

Where are these inputs coming from?

uniforms
• Your

app code

Where are these inputs coming from?

uniforms
• Your

app code • Objective-C (or other language - C, C++, Java etc.)

Where are these inputs coming from?

Where are these inputs coming from?

varyings

Where are these inputs coming from?

varyings

Vertex shader

Where are these inputs coming from?

varyings

Vertex shader • Simple program that runs on the GPU.

Where are these inputs coming from?

varyings

Vertex shader • Simple program that runs on the GPU. • OpenGL ES 2.0 Shading Language code.

Where are these inputs coming from?

varyings

Vertex shader • Simple program that runs on the GPU. • OpenGL ES 2.0 Shading Language code. • Run once for every vertex simultaneously.

Where are these inputs coming from?

varyings

Vertex shader • Simple program that runs on the GPU. • OpenGL ES 2.0 Shading Language code. • Run once for every vertex independently.

Where are these inputs coming from?

varyings

Vertex shader • Simple program that runs on the GPU. • OpenGL ES 2.0 Shading Language code. • Run once for every vertex simultaneously.

What’s a Vertex?

What’s a Vertex?
•A

vertex is a point

What’s a Vertex?
•A

vertex is a point • Represents

What’s a Vertex?
•A

vertex is a point • Represents • Disconnected point

What’s a Vertex?
•A

vertex is a point • Represents • Disconnected point • Line end point

What’s a Vertex?
•A

vertex is a point • Represents • Disconnected point • Line end point • Triangle corner point

What’s a Vertex?
•A

vertex is a point • Represents • Disconnected point • Line end point • Triangle corner point • Two or three dimensional (vec2, vec3)

What’s a Triangle Strip?

What’s a Triangle Strip?

• An

array of points...

What’s a Triangle Strip?

• An

array of points... ...an array of vec2s or vec3s, in Shading Language terms.

What’s a Triangle Strip?

1

1

2

1

2

3

1

2

3

1

2

3

4

1

2

3

4

1

2

3

4

1

2

3

4

5

1

2

3

4

5

6

1

2

3

4

5

6

7

1

2

3

4

5

6

8

7

1

2

3

4

5

6

8

7

9

Where do the Fragment Shader inputs come from?

varyings

Vertex shader • Simple program that runs on the GPU. • OpenGL ES 2.0 Shading Language code. • Run once for every vertex simultaneously.

Where do the Fragment Shader inputs come from?

varyings

Vertex shader • Simple program that runs on the GPU. • OpenGL ES 2.0 Shading Language code. • Run once for every vertex simultaneously. • Calculates the position of the vertex...

Where do the Fragment Shader inputs come from?

varyings

Vertex shader • Simple program that runs on the GPU. • OpenGL ES 2.0 Shading Language code. • Run once for every vertex simultaneously. • Calculates the position of the vertex... • ...and arbitrary other inputs to pass to the fragment shaders.

Vertex Shader

Vertex Shader
attribute vec4 aPosition; attribute vec2 aTextureCoordinate; varying vec2 vTextureCoordinate; void main() { vTextureCoordinate = aTextureCoordinate; gl_Position = aPosition; }

Colours.vsh
attribute vec4 aPosition; attribute vec2 aTextureCoordinate; varying vec2 vTextureCoordinate; void main() { vTextureCoordinate = aTextureCoordinate; gl_Position = aPosition; }

Colours.vsh
attribute vec4 aPosition; attribute vec2 aTextureCoordinate; varying vec2 vTextureCoordinate; void main() { vTextureCoordinate = aTextureCoordinate; gl_Position = aPosition; }

Colours.vsh
attribute vec4 aPosition; attribute vec2 aTextureCoordinate; varying vec2 vTextureCoordinate; void main() { vTextureCoordinate = aTextureCoordinate; gl_Position = aPosition; }

Colours.vsh
attribute vec4 aPosition; attribute vec2 aTextureCoordinate; varying vec2 vTextureCoordinate; void main() { vTextureCoordinate = aTextureCoordinate; gl_Position = aPosition; }

Colours.vsh
attribute vec4 aPosition; attribute vec2 aTextureCoordinate; varying vec2 vTextureCoordinate; void main() { vTextureCoordinate = aTextureCoordinate; gl_Position = aPosition; }

Colours.vsh
attribute vec4 aPosition; attribute vec2 aTextureCoordinate; varying vec2 vTextureCoordinate; void main() { vTextureCoordinate = aTextureCoordinate; gl_Position = aPosition; }

Colours.vsh
attribute vec4 aPosition; attribute vec2 aTextureCoordinate; varying vec2 vTextureCoordinate; void main() { vTextureCoordinate = aTextureCoordinate; gl_Position = aPosition; }

Demo

Demo

Demo

Where do the pixels come from?

pixels

Where do the fragment shader inputs come from?

Where do the fragment shader inputs come from?

Where do the fragment shader inputs come from?

varyings

Where do the shader inputs come from?

varyings

Where do the shader inputs come from?

varyings

Where do the shader inputs come from?

varyings

Where do the shader inputs come from?
uniforms varyings uniforms

Where do the shader inputs come from?
uniforms & attributes varyings uniforms

The OpenGL ES 2.0 Pipeline
uniforms & attributes varyings uniforms

Modern Graphics Hardware Pipeline
uniforms & attributes varyings uniforms

The OpenGL ES 2.0 Pipeline
uniforms & attributes varyings uniforms

What does the app code do?

What does the app code do?

What does the app code do?

• OpenGL

content is presented in a CALayer...

What does the app code do?

• OpenGL

content is presented in a CALayer... • ...specifically, a CAEAGLLayer.

What does the app code do?

• OpenGL

content is presented in a CALayer... • ...specifically, a CAEAGLLayer. • Use a UIView backed by a CAEAGLLayer...

What does the app code do?

• OpenGL

content is presented in a CALayer... • ...specifically, a CAEAGLLayer. • Use a UIView backed by a CAEAGLLayer... • ...like Xcode template and sample code’s EAGLView.

What does the app code do?

• OpenGL

content is presented in a CALayer... • ...specifically, a CAEAGLLayer. • Use a UIView backed by a CAEAGLLayer... • ...like Xcode template and sample code’s EAGLView. • Use the C OpenGL API to draw.

Integrating with UIKit

Integrating with UIKit

• You

can’t use the setNeedsDisplay model.

Integrating with UIKit

• You

can’t use the setNeedsDisplay model. • Draw explicitly when you have something to display.

Integrating with UIKit

• You

can’t use the setNeedsDisplay model. • Draw explicitly when you have something to display. • Draw on a timer (e.g. 30 frames per second).

Integrating with UIKit

• You

can’t use the setNeedsDisplay model. • Draw explicitly when you have something to display. • Draw on a timer (e.g. 30 frames per second). • Use the CADisplayLink timer class.

C OpenGL API

C OpenGL API

Context based

C OpenGL API

Context based • Like CoreGraphics’ CGContext.

C OpenGL API

Context based • Like CoreGraphics’ CGContext. • No state save/restore.

C OpenGL API

Context based • Like CoreGraphics’ CGContext. • No state save/restore. • Not thread safe.

C OpenGL API

Context based • Like CoreGraphics’ CGContext. • No state save/restore. • Not thread safe. • Use from one thread at a time.

C OpenGL API

Context based • Like CoreGraphics’ CGContext. • No state save/restore. • Not thread safe. • Use from one thread at a time. • Use share pools to share resources between threads.

C OpenGL API

C OpenGL API
• Functions

to upload things to the GPU, e.g.:

C OpenGL API
• Functions

to upload things to the GPU, e.g.: • Shader programs

C OpenGL API
• Functions

to upload things to the GPU, e.g.: • Shader programs • Textures

C OpenGL API
• Functions

to upload things to the GPU, e.g.: • Shader programs • Textures • Shader program uniforms, attributes

C OpenGL API
to upload things to the GPU, e.g.: • Shader programs • Textures • Shader program uniforms, attributes • Functions to set context state, e.g.:
• Functions

C OpenGL API
to upload things to the GPU, e.g.: • Shader programs • Textures • Shader program uniforms, attributes • Functions to set context state, e.g.: • Active texture
• Functions

C OpenGL API
to upload things to the GPU, e.g.: • Shader programs • Textures • Shader program uniforms, attributes • Functions to set context state, e.g.: • Active texture • Clear colour
• Functions

C OpenGL API
to upload things to the GPU, e.g.: • Shader programs • Textures • Shader program uniforms, attributes • Functions to set context state, e.g.: • Active texture • Clear colour • Functions to perform rendering
• Functions

Demo

Demo

Demo

Demo

Why Use OpenGL?

Why Use OpenGL?
Flexibility

Why Use OpenGL?
Flexibility Performance

Why Use OpenGL?
Flexibility Performance 3D Stuff?

What about 3D stuff?

What about 3D stuff?

• Upload

models as in triangle strips made of 3D vec3 points.

What about 3D stuff?

What about 3D stuff?

• Do

positioning maths in the vertex shaders

What about 3D stuff?

• Do

positioning maths in the vertex shaders • One uniform mat4 to position each strip - a model matrix.

What about 3D stuff?

• Do

positioning maths in the vertex shaders • One uniform mat4 to position each strip - a model matrix. • One uniform mat4 to position the world around the camera, calculate perspective - a projection matrix.

What about 3D stuff?

What about 3D stuff?
• Do

lighting maths in the vertex shaders.

What about 3D stuff?
• Do

lighting maths in the vertex shaders. • Upload

What about 3D stuff?
• Do

lighting maths in the vertex shaders. • Upload • light information

What about 3D stuff?
• Do

lighting maths in the vertex shaders. • Upload • light information • arrays of vertex normals

What about 3D stuff?
• Do

lighting maths in the vertex shaders. • Upload • light information • arrays of vertex normals • Calculate per-vertex lighting brightness, color

What about 3D stuff?
• Do

lighting maths in the vertex shaders. • Upload • light information • arrays of vertex normals • Calculate per-vertex lighting brightness, color • Formulas online

What about 3D stuff?
• Do

lighting maths in the vertex shaders. • Upload • light information • arrays of vertex normals • Calculate per-vertex lighting brightness, color • Formulas online • Pass as varying to fragment shader.

What about 3D stuff?

What about 3D stuff?

• Do

texturing, other per-pixel effects in the fragment shaders.

What about 3D stuff?

• Do

texturing, other per-pixel effects in the fragment shaders. • Multiply the pixel colour with the light colour.

Learning More

Learning More
• OpenGL

ES Programming Guide for iOS; Apple

Learning More
• OpenGL

ES Programming Guide for iOS; Apple ES 2.0 Programming Guide; Munshi, Ginsburg & Shreiner

• OpenGL

Learning More
• OpenGL

ES Programming Guide for iOS; Apple ES 2.0 Programming Guide; Munshi, Ginsburg & Shreiner PowerVR Insider web site

• OpenGL

Learning More
• OpenGL

ES Programming Guide for iOS; Apple ES 2.0 Programming Guide; Munshi, Ginsburg & Shreiner PowerVR Insider web site out for great new things in iOS 5!

• OpenGL

• Watch

github.com/th-in-gs/Colours Jamie Montgomerie @th_in_gs jamie@montgomerie.net www.blog.montgomerie.net th.ingsmadeoutofotherthin.gs

Things Made Out Of Other Things