Mobile 3D Graphics API (M3G)

Janne Levula Forum Nokia

Why 3D Graphics?
Phone screens may be flat but the world certainly isnt!

Objective
Gain an insight to the Mobile 3D Graphics API and take your games and applications to the next level

Whats Good to Know!

Fundamentals of 3D graphics

transformations, texturing, blending, OpenGL is the premier environment for developing interactive 2D and 3D graphics applications. Introduction in 1992, today industrys most widely used and supported 2D and 3D graphics application programming interface.

OpenGL or some other modern 3D API

Java technology, preferably MIDP 2.0

GameCanvas

Agenda
Overview? Getting started Scene graph Low level features Dynamic meshes Animation Demo New dimension of Java games

Agenda
Overview? Getting started Scene graph Low level features Dynamic meshes Animation Demo New dimension of Java games

Overview

Java M3G API (JSR-184) provides a scalable, small-footprint, interactive 3D graphics Java API for J2ME. For a wide range of applications, including games, animated messages, screen savers, custom user interfaces, product visualization Java M3G API is an optional package that can be adopted to existing J2ME profiles. The main target platform of the API is J2ME/CLDC, used with profiles such as MIDP 1.0 or MIDP 2.0. JSR-184 was selected by the Java Community Process (JCP) as The most innovative JSR in year 2003.

Mobile 3D Graphics API (M3G)

Also known as JSR-184 (Optional for J2ME) Designed for mobile devices

Primarily CLDC 1.1 / MIDP But also CDC

MIDlets

Mobile 3D Graphics API

Graphics Hardware

MIDP

Java Virtual Machine (CLDC 1.1)

The API Advantages

Benefits for developers

Improve rendering speed by using native Libraries to leverage the underlying hardware Acceleration Mask hardware differences (3D HW, FPU, DSP,). Exploit any new hardware automatically Makes easier application development

Benefits for hardware vendors

Gives a concrete target which features to offer

Benchmarked on an ARM9 processor

Why a New Standard?

OpenGL (ES) is too low-level, (OpenGL for Embedded Systems) is based on OpenGL)
Lots of Java code needed for simple things Bigger MIDlets, slower speed

Java 3D API is too bloated

A hundred times larger than M3G Does not fit together with MIDP

Now knew what we wanted!

Basic Java 3D ideas: nodes, scene graph Add file format, keyframe animation Remain compatible with OpenGL ES

Agenda
Overview? Getting started Scene graph Low level features Dynamic meshes Animation Demo New dimension of Java games

Intro
M3G can be described as a unified immediate/retained mode API with a configurable rendering pipeline at the low level and an animated scene graph layered on top of it.

Flexibility is guaranteed by rich set of parameters for configuring the rendering pipeline. There are dozens of mutually independent parameters in the pixel pipeline alone, yielding billions of unique configurations and different visual effects that can be applied to polygons.

Key Classes
3D graphics context Performs all rendering

Graphics3D

Loader

Can load individual objects and entire scene graphs (M3G and PNG files) Scene graph root node

World

Graphics3D

Contains global state

Rendering targets (Canvas, Image, Image2D, CustomItem) Camera, Viewport, Light sources, Depth buffer, Rendering quality hints

Rendering Modes:
Immediate mode Low level, allows the user to define every detail of rendering process Render a branch or an individual node at a time Explicitly give the Camera and Lights to Graphics3D Retained mode Hides low level from the user allowing animated 3-D content to be loaded and displayed with few lines of code Render a scene graph, rooted by the World Take the Camera and Lights from the World

Graphics3D

Immediate vs. retained mode Immediate mode

Render individual scene graph nodes or groups of nodes, or individual submeshes Used lights, active camera, and background are provided separately Render an entire world represented as a scene graph Used lights, active camera, and background are members of the scene graph

Retained mode

Difference between immediate and retained mode is vague as they can be freely mixed and matched

Graphics3D: How-to-Use

Bind a target to it, render, release the target

Tip: Everything is synchronous Tip: There are no callbacks Tip: Never mess with a bound target Tip: Graphics3D is a singleton (threads!) void paint(Graphics g) { myGraphics3D.bindTarget(g); myGraphics3D.render(world); myGraphics3D.releaseTarget(); }

Example: Hello, World

A simplified animation player
import import import import javax.microedition.midlet.MIDlet; javax.microedition.lcdui.Display; javax.microedition.lcdui.game.GameCanvas; javax.microedition.m3g.*;

public class Player extends MIDlet { public void pauseApp() {} public void destroyApp(boolean b) {} public void startApp() { PlayerCanvas canvas = new PlayerCanvas(true); Display.getDisplay(this).setCurrent(canvas); try { canvas.run(); } catch (Exception e) {} notifyDestroyed(); }

Example: Hello, World

class PlayerCanvas extends GameCanvas { PlayerCanvas(boolean suppress){super(suppress);} public void run() throws Exception { Graphics3D g3d = Graphics3D.getInstance(); World w = (World) Loader.load("/file.m3g")[0]; long start, elapsed, time = 0; while (getKeyStates() == 0) { start = System.currentTimeMillis(); g3d.bindTarget(getGraphics()); try { w.animate(time); g3d.render(w); } finally { g3d.releaseTarget(); } flushGraphics(); elapsed = System.currentTimeMillis()-start; time += (elapsed < 100) ? 100 : (int)elapsed; if (elapsed < 100) Thread.sleep(100-elapsed); } }

Example: Rotating Cube

Define raw data for a cube
// Corners of a cube as (X,Y,Z) triplets static short[] cubeVertices = { -1, -1, 1, 1, -1, 1, -1, 1, 1, -1, -1, -1, 1, -1, -1, -1, 1, -1, }; 1, 1, 1, 1, 1, -1

// A color for each corner as an (R,G,B) triplet static byte[] cubeColors = { (byte) 255, (byte) 0, (byte) 0, // red (byte) 0, (byte) 255, (byte) 0, // green }; // Define the cube as a single triangle strip static int[] indices = { 6,7,4,5,1,7,3,6,2,4,0,1,2,3 }; static int[] stripLen = { 14 };

Example: Rotating Cube

Copy raw data into Objects
// Fill in VertexArrays from short[] and byte[] int numVertices = vertices.length/3; VertexArray pos = new VertexArray(numVertices, 3, 2); VertexArray col = new VertexArray(numVertices, 3, 1); pos.set(0, numVertices, cubeVertices); col.set(0, numVertices, cubeColors); // Attach the VertexArrays to a VertexBuffer // Note the scale (1.0) and bias (0,0,0) vertices = new VertexBuffer(); vertices.setPositions(pos, 1.0f, null); vertices.setColors(col); // Fill in the triangle strip triangles = new TriangleStripArray(cubeIndices, stripLen); // Create a Mesh with default Appearance cube = new Mesh(vertices, triangles, new Appearance());

Example: Rotating Cube

Set up a Camera
Camera cam = new Camera(); // 60-degree field of view, screen aspect ratio // Near clipping plane at 1.0, far plane at 1000 float aspect = (float) getWidth() / (float) getHeight(); cam.setPerspective(60.0f, aspect, 1.0f, 1000.0f); // Place the camera at z=5.0 in world space // View vector is along the negative Z axis transform = new Transform(); transform.postTranslate(0.0f, 0.0f, 5.0f); g3d = Graphics3D.getInstance(); g3d.setCamera(cam, transform);

Example: Rotating Cube

Clear the buffers, render, animate
g3d = Graphics3D.getInstance(); g3d.bindTarget(getGraphics()); try { g3d.clear(null); g3d.render(cube, transform); } finally { g3d.releaseTarget(); } xAngle += 1; yAngle += 2; zAngle += 3; transform.setIdentity(); transform.postRotate(xAngle, 1.0f, 0.0f, 0.0f); transform.postRotate(yAngle, 0.0f, 1.0f, 0.0f); transform.postRotate(zAngle, 0.0f, 0.0f, 1.0f);

Agenda
Overview? Getting started Scene graph Low level features Dynamic meshes Animation Demo New dimension of Java games

The Scene Graph

Actually, its just a tree
World Group Group Camera SkinnedMesh Group Group Group Sprite3D Mesh Light

MorphingMesh

Not allowed!

Scene Graph: Nodes

The Scene graph itself is composed by Node objects: Camera Mesh: SkinnedMesh and MorphingMesh Sprite 3D Light Node is the type of object commonly manipulated in an application: translations, rotations, scaling, transformation Nodes can be grouped together, makes manipulation easier

Nodes of the Scene Graph

Screenshot from Swerve spy window in 3D Studio Max: showing topmost nodes of the scene graphs (Example).

Agenda
Overview? Getting started Scene graph Low level features Dynamic meshes Animation Demo New dimension of Java games

Renderable Objects
Mesh Made of triangle strips Base class for meshes (two subclasses: MorphingMesh and SkinnedMesh)

Sprite3D

2D image placed in 3D space Good for labels, etc.

Mesh
Composed of submeshes

Common buffer of vertex data Submesh has triangle strip indices to vertices One Appearance for each submesh Mesh VertexBuffer IndexBuffer Appearance coordinates normals colors texturecoords

Appearance Components
Material colors for lighting Can track per-vertex colors Blending, depth buffering Alpha testing, masking Winding, culling, shading Perspective correction hint Fades colors based on distance Linear and exponential mode Texture matrix, blending, filtering Multitexturing: One Texture2D for each unit

Material
CompositingMode

PolygonMode Fog Texture2D

Sprite3D
2D image with a position in 3D space

Scaled mode for billboards, trees, etc. Unscaled mode for text labels, icons, etc.

Sprite3D

Appearance Image2D Image2D

CompositingMode Fog

Agenda
Overview? Getting started Scene graph Low level features Dynamic meshes Animation Demo New dimension of Java games

MorphingMesh
For vertex morphing animation

R = B + wi (Ti B )
i

A base mesh and several morph targets Result = weighted sum of morph deltas Can morph any vertex attribute

Vertex positions Colors Normals Texture coordinates

MorphingMesh
Example: Animating a rabbits face

Base

Target 1 eyes closed

Target 2 mouth closed

Animate eyes and mouth independently

SkinnedMesh
Articulated characters without cracks at joints

Stretch a mesh over a hierarchic skeleton

v' = wiMiBiv
i

The skeleton consists of scene graph nodes Each node (bone) defines a transformation Each vertex is linked to one or more bones

SkinnedMesh
Weighted skinning
Neutral pose, bones at rest
Bone A Bone B shared vertex, weights = (0.5, 0.5) "skin" non-shared vertex

position in A's coordinate system interpolated position

Bone A

Bone B rotated 90 degrees

position in B's coordinate system

Bone B

SkinnedMesh Example

No skinning

Local skinning

Smooth skinning

Agenda
Overview? Getting started Scene graph Low level features Dynamic meshes Animation Demo New dimension of Java games

Animation: Relevant Classes

KeyframeSequence Storage for keyframes Defines interpolation mode Controls the playback of one or more animations A link between sequence, controller and target Base class for all objects that can be animated

AnimationController

AnimationTrack

Object3D

Demo

Creature in Wonderland
Document and example codes for this demo available at www.forum.nokia.com Remember also Nokia:

Nokia Reference Implementation, Binary For JSR-184 3D Graphics API For J2ME, Emulator Sun Wireless Toolkit (WTK) 2.2

40

Agenda
Overview? Getting started Scene graph Low level features Dynamic meshes Animation Demo New dimension of Java games

New Dimension for Java Games

Downloadable Java games will be literally transformed into a new dimension when the Mobile 3D Graphics API hits the market. This new standard will bring Java on par with the Nokia N-Gage game deck in terms of visual quality. Java is the most widely deployed platform for third-party applications in mobile terminals. Nokia alone expects to ship approximately 100 million color-screen Java devices in 2004. 3D graphics makes games look more impressive, but it also enables new genres by giving more freedom to designers and artists. It even helps to bring down costs: being inherently scalable, it allows games to be easily targeted at different display sizes and color depths.

New Dimension for Java Games

The difference in visual quality between state-of-the-art Java games ( left) and Symbian games (middle). The Mobile 3D Graphics API bridges this gap (right)

Developer Tools

Modeling tools for making *.m3g files = 3D scenes 3DS Max 7 includes a built-in M3G exporter. 3DS Max is the most popular 3D modeling and animation package in the game industry. Digital Element provides exporter that works also with 3DS Max version 6 HI Corporation has exporters for all versions of 3DS Max, as well as Lightwave 8. - utility for previewing and examining M3G files, useful for programmers and designers. Superscape: A basic version of their Swerve Exporter freely available. The exporter works with 3DS Max versions 5.1, 6, and 7. The professional version of Swerve Exporter is only available to game developers who sign a publishing deal with Superscape. Follow the open source projects: At least two independent developers (with no Web sites) are making exporters for Blender http://www.blender3d.com, an open-source 3D modeling package.

Summary

M3G is the 3D API for J2ME technology

Will be supported in millions of devices Currently supported in Nokia 6630, (6255i CDMA device, US model)

Not just an API

M3G also defines a binary file format

M3G is flexible

Using the immediate mode gives you full control High-level features make your code faster and smaller

More Information
http://java.sun.com http://www.forum.nokia.com

Thank You

Have a nice time with 3D development