Author: Meng Cheng, Lau

Project - comp7910/4060

Student#: 07634694

Simulation of Betty: A Humanoid Robot with OpenGL

COMP7910 Advance Graphics, Winter 2010 Meng Cheng, Lau (#7634694) Dept. of Computer Science University of Manitoba laumc@cs.umanitoba.ca Instructor: Dr. Dereck Meek

1. Introduction
This program illustrates the simulation model for the upper body of Betty, a humanoid robot with interactive moving joints which will rotate according to nine joint angles. One of the main objectives of this project is to illustrate various features in OpenGL libraries which will demonstrate model rendering, environment setting, visual effects and interactive control capabilities. Another objective of this simulation program is to avoid any miss-behaviour and collision of the robot in the physical world control. Therefore, this project implements DH-convention, a forward kinematics analysis method to calculate the final position of the end effectors, which refer to Bettys hands. This approach will enable the collision detection between the hand and the obstacle. It is demonstrated with Bettys left hand and a wall. This program is developed with QT creator, a cross-platform integrated development environment (IDE) for QT4.

2. OpenGL 3D Model Rendering

OpenGL is the industry's most widely used, supported and best documented 2D/3D graphics API, this makes it inexpensive and easy to obtain information on implementing computer graphics. OpenGL is supported and complete independence on most operating system and it is compatible with most programming languages. All OpenGL applications produce consistent visual display results on any OpenGL API-compliant hardware, regardless of operating system or windows system [4]. However, the basic OpenGL library is not capable of opening windows or responding to interrupts from a mouse or keyboard [5]. The GLUT (OpenGL Utility Toolkit) library by Mark Kilgard [6] and freeglut, a completely open-sourced alternative will provide such a facility [7]. Figure 1 illustrates the 3D model rendering structure of Betty in the simulation program which consists of four modules; model, environment, visual effect and user control.

Author: Meng Cheng, Lau

Project - comp7910/4060

Student#: 07634694

OpenGL 3D Modeling

Environment Window Perspective Camera View port

Models Torso Head Arms Joints

Visual Effect Lighting Texture Solid / wire Animation

Interactive Control Keyboard Mouse

Figure 1: Model rendering structure of Betty

2.1.

Environment Setting

To ensure the correct output on the model display, appropriate settings of OpenGL environment and GLUT variables in the program are essential. The first step to display the drawing is open a window with an initial size and position [9]. Figure 2 shows the basic code in the main() function to initialise the display window for GLUT.
glutInit(&argc, argv); // initialise GLUT toolkit glutInitDisplayMode(GLUT_DOUBLE | GLUT_RGBA | GLUT_DEPTH ); // initialise display mode glutInitWindowSize(w, h); // initialise window size glutInitWindowPosition(x,y); // initialise window position glutCreateWindow("title"); // open the window glutDisplayFunc(display); // register display callback function glutReshapeFunc(reshape); // set the reshape callback for the current window glutMainLoop(); // enter GLUT event processing loop

Figure 2: Initialise a GLUT window. The reshape callback is triggered when a window is reshaped and immediately after a window is created or whenever an overlay for the window is established [10]. Consequently setting up the correct viewport, viewing perspective and camera variables in the reshape() callback function as shown in Figure 3 is the next step. gluPerspective() specifies a viewing frustum into the world coordinate system. It sets the field of view angle in the y direction, the aspect ratio of w (width) to h (height), the distances from the viewer to the near clipping plane and far clipping plane [10].

Author: Meng Cheng, Lau

Project - comp7910/4060

Student#: 07634694

glMatrixMode(GL_PROJECTION); // set the coordinate system to projection matrix stack glLoadIdentity(); glViewport(0, 0, w, h); // Set the viewport to be the entire window gluPerspective(theta, w / h, near, far); // Set the correct perspective gluLookAt(0,0,size,0,0,0,0,1,0); //eye position (x,y,z), look at point (x,y,z), up direction (x,y,z) glMatrixMode(GL_MODELVIEW); // reset the coordinate system to modelview matrix stack glLoadIdentity();

Figure 3: Initialise a GLUT window.

2.2.

Model Rendering

In order to construct the upper body model of Betty, this program uses the basic object construction subroutine for each part as shown in Figure 4.
drawObject(){ glPushMatrix(); // push the current matrix stack glColor4f(r,g,b,a); // set the current colour glTranslatef(x,y,z); // translate current object glRotatef(theta,x,y,z); // rotate current object glScalef(x,y,z); //scale current object gluSphere(obj,radius,slice,stack) //draw a sphere glPopMatrix(); // pop the current matrix stack } // drawObject

Figure 4: Object construction subroutine

Figure 5 shows the basic scene generated by drawing three different objects as listed in Table 1. Table 1: Model Construction Object Sphere Cube Cylinder Parts Eyes, head, joints and hands Wall Arms, torso and neck

Cube is the basic shape which available in GLUT with glutSolidCube() and glutWireCube(). However, we need to use the GLUquadrics object from OpenGL Utility Library, GLU to draw cylinder and sphere. A sphere can be drawn with gluSphere(), but we need three quadric objects to

Author: Meng Cheng, Lau

Project - comp7910/4060

Student#: 07634694

construct a cylinder. On the other hand, gluCylinder() will draw the surrounding surface of a cylinder and gluDisk() will draw two flat disc's at each end to close the cylinder as shown in Figure 6.

Figure 5: The scene of Betty with joint indexes and axes

void myCylinder(GLUquadricObj* object, GLdouble topRadius, GLdouble baseRadius, GLdouble lenght, GLint slices, GLint stacks) { glPushMatrix(); gluCylinder(object, baseRadius, topRadius, lenght, slices, stacks); glTranslatef(0.0, 0.0, lenght); gluDisk(object, 0.0, topRadius, slices, stacks); // top cover glRotatef(180, 0.0, 1.0, 0.0); // flip glTranslatef(0.0, 0.0, lenght); gluDisk(object, 0.0, baseRadius, slices, stacks); // base cover glPopMatrix(); }

Figure 6: Method to draw a cylinder

2.3.

Different visual effects

For the purpose of realistic and flexible graphic output, this program implemented four different visual effects. It allows the model to switch between solid and wire as shown in Figure 7 and Figure 8. Table 2 compares the differences between the OpenGL variables in the code.

Author: Meng Cheng, Lau

Project - comp7910/4060

Student#: 07634694

Figure 7: Solid objects

Figure 8: Wire frame

Table 2: Comparison between solid and wire Solid

glEnable(GL_LIGHTING); // enable lighting effect glEnable(GL_DEPTH_TEST); // enables depth calculations with depth buffer for hidden surface removal glShadeModel(GL_SMOOTH); // smooth surface gluQuadricNormals(obj, GLU_SMOOTH); // smooth normal for quadric object gluQuadricDrawStyle(obj, GLU_FILL); // solid object draw style

Wire
glDisable(GL_LIGHTING); // disable lighting effect glDisable(GL_DEPTH_TEST); // disable depth calculations with depth buffer for hidden surface removal glShadeModel(GL_FLAT); // flat surface gluQuadricNormals(obj, GLU_NONE); // no normal gluQuadricDrawStyle(obj, GLU_LINE); // wire frame draw style

Figure 9 is the code to create the checker texture and texture mapping in the program. This code is adapted from the checker example in the OpenGL redbook [11] which produces the output as seen in Figure 10 when it is switched between glEnable(GL_TEXTURE_2D), gluQuadricTexture(obj, GL_TRUE) and glDisable(GL_TEXTURE_2D), gluQuadricTexture(obj, GL_FALSE). In contrast, It is simpler to turn on and turn off the light by simply call between glDisable(GL_LIGHT0) and glEnable(GL_LIGHT0) function after we initialise the lighting properties with glLightfv() and glEnable(GL_LIGHTING). The light-off effect is shown in Figure 11.

Author: Meng Cheng, Lau

Project - comp7910/4060

Student#: 07634694

// define storage for checker texture map GLubyte image[IMAGE_WIDTH][IMAGE_HEIGHT][3]; for (i = 0; i < IMAGE_WIDTH; i++) { for (j = 0; j < IMAGE_HEIGHT; j++) { c = ((((i & 0x8) == 0) ^ ((j & 0x8)) == 0)) * 255; image[i][j][0] = (GLubyte) c; image[i][j][1] = (GLubyte) c; image[i][j][2] = (GLubyte) c; } } // Set up Texturing // the texture wraps over at the edges (repeat) glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_REPEAT); glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_REPEAT); // when texture area is large, bilinear filter the first mipmap glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST); // when texture area is small, bilinear filter the closest mipmap glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST); // select modulate to mix texture with color for shading glTexEnvf(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_MODULATE); // build our texture mipmaps glTexImage2D(GL_TEXTURE_2D, 0, 3, IMAGE_WIDTH, IMAGE_HEIGHT, 0, GL_RGB, GL_UNSIGNED_BYTE, image);

Figure 9: Checker and texture mapping code

Figure 10: Checker texture

Figure 11: light-off display

Author: Meng Cheng, Lau

Project - comp7910/4060

Student#: 07634694

In order to use different colours for each object without calling glMaterialfv, the program uses ColorMaterial which makes it possible to change a subset of material parameters for each object using only the glColor command. Because if only such a subset of parameters is to be specified for each vertex, calling glColorMaterial is preferable to calling glMaterialfv [10].

2.4.

Interactive control

The robot joints and the visual effect that implemented in this program are controlled by the keyboard and mouse support callback functions which are provided by GLUT library. glutKeyboardFunc() and glutSpecialFunc() read num-keys from 0 to 8 to control different joints which are listed in Table 3. It also explains other shortcut key to switch between different visual effects as discussed in section 2.3. Other than keyboard control, this program allows user to use the right-click from mouse input to accommodate the flexibility of the user control as shown in Figure 12. The right-click menu and mouse input subroutines are enable with glutCreateMenu(), glutAttachMenu(GLUT_RIGHT_BUTTON) and glutAddMenuEntry(). Table 3: Keyboard control key description
Key Num-key (0-8) Shift + Num-key (0-8) s a t o l +/Up/Down Left/Right r Esc Responses and effects Increase joint resolution by one degree Decrease joint resolution by one degree Switch between solid and wire view Enable/disable animation Enable/disable checker texture Show/hide axes On/off light Increase/decrease animation speed Rotation by x-axis Rotation by y-axis Reset all joint to default position, 0 degrees Exit program

Author: Meng Cheng, Lau

Project - comp7910/4060

Student#: 07634694

Figure 12: Right-click mouse menu.

3. Forward Kinematics: DH-Convention

In the interest to calculate the global position of Bettys hands, we have to compute the forward kinematics. Although, a simple two-joint planar manipulator analysis is possible to solve with some simple transformation matrices. But the accumulated kinematics analysis of multiple-joint manipulator can be extremely complicated. Thus, this program applies Denavit-Hartenberg (DH) Convention to solve the forward kinematics analysis problem. DH-Convention introduces a simplified presentation to calculate the position and the orientation of the end effectors which is widely used in robotics and animation. In this convention, each homogeneous transformation, Di is represented as a product of four basic transformations matrices get the global position of any particular point at joint i [2].

Di = R z , i Tz ,di Tx ,ai R x , i
cos i sin i = 0 0 sin i cos i 0 0 0 0 1 0 0 0 0 1 1 0 0 0 0 1 0 0 0 0 1 0 0 0 1 d i1 0 0 1 0 0 0 a i 1 0 1 0 0 0 cos i 0 1 0 0 sin i 0 0 1 0 0 0 sin i cos i 0 05 0 0 1
(1)

Author: Meng Cheng, Lau

Project - comp7910/4060

Student#: 07634694

cos sin = 0 0

sin cos cos cos sin 0

sin sin cos sin cos 0

a i cos a i sin di 1

In equation (1), axis,

R z ,i is rotation matrix for angle i by z-axis, R x , i is rotation matrix for angle i by xis transformation matrix for length

Tz ,di is transformation matrix for length di by z-axis and Tx ,ai

ai by x-axis. Denavit and Hartenberg used screw theory in the 1950's to show that the most compact representation of a general transformation between two robot joints required four parameters. These are now known as the Denavit and Hartenberg parameters (D-H parameters) and they are the de facto standard for describing a robot's geometry [3]. Where the detail descriptions of the DHparameters are: ai - the distance between two joint axes measured along x-axis. i - the relative twist between two joint axes measured about x-axis. di - the distance between the two perpendiculars measured along the joint axis, z-axis i - joint angle about the z-axis 3.1.

Joints Construction

The upper body of Betty consists of a ten revolute joints system which produces 3 DOF (degree of freedom) to the head, 1 DOF to the jaw (not implemented) and 3 DOF to each arm as shown in Figure 5 in section 2.2 and Figure 15 in section 4. Figure 13 illustrates the coordinate frames attached and assigned to each arm. The origin is chosen at the torso of Betty. The DH-parameters are shown in Table 4. Table 4: The DH-parameters for both hands Joint 3 4 5 6 7 8

ai
0 l0 l1 0 l0 l1

3=90 4=-90 5=0 6=90 7=-90 8=0

di
0 0 0 0 0 0

3 4 5 6 7 8

Author: Meng Cheng, Lau

Project - comp7910/4060

Student#: 07634694

Figure 13: Joints construction diagram To establish the homogeneous coordinate between the origin and the arms, the shoulders coordinate frames are translated to the origin with the translation matrices, TL and TR for left and right shoulder respectively.

1 0 Where: TL = 0 0

0 0 t0 1 0 0 0 1 t1 0 0 1

and

1 0 TR = 0 0

0 0 t0 1 0 0 0 1 t1 0 0 1

Once the DH-parameters and relative frames are determined, we will combine the transformation matrices as the product of translation and DH-Convention matrices. The example shown in equation (2) and (3) is the calculation of the left hands global position which denoted as PL; based on the rotation angles of 3, 4 and 5 from the origin O with the transformation matrix ML.

10

Author: Meng Cheng, Lau

Project - comp7910/4060

Student#: 07634694

M L = TL D3 D4 D5
1 0 0 1 = 0 0 0 0 cos 5 sin 5 0 0 t 0 cos 3 sin 3 cos 3 0 0 sin 3 1 t1 0 1 0 1 0 0 sin 5 cos 5 0 l1 cos 5 cos 5 0 l1 sin 5 0 1 0 0 0 1 0 0
(3)

sin 3 cos 3 0 0

0 cos 4 0 sin 4 0 0 1 0

sin 4 cos 4 0 1 0

sin 4 cos 4 0 0

l 0 cos 4 l 0 sin 4 0 1

(2)

PL ( x, y, z ) = M L O
3.2.

Joints limit and simple collision detection

Simulation is a fundamental tool in humanoid robotics, fulfilling many important roles. For example, it is used to validate controllers with respect to safety and performance considerations; the controller development cycle of programming, testing and improvement is made more convenient through simulation, which should be easier, safer and quicker to execute than experiments on a real robot [8]. This program detects two crucial misbehaviours which may happen in the physical control to prevent any damage to the servos on the robot; they are joints limit control and collision detection based on the global positioning of hands and objects. Figure 14 illustrate the off-limit elbow on left arm is locked which indicate with red blinking background and grey coloured elbow. Figure 15 shows the warning detection when the left hand collides with the wall by moving the shoulder joint.

Figure 14: Off-limit joint warning

Figure 15: Wall and hand collision warning 11

Author: Meng Cheng, Lau

Project - comp7910/4060

Student#: 07634694

4. Conclusion and Future work

The current version of the simulation program only runs as a standalone application. But since it is developed in a cross platform environment, we will integrate it to Bettys motion controller program as the simulation module and incorporate with the physical control of the robot as Figure 16. Improvement on the collision detection is expected, it will be extended to the arms and consider the intersection between an obstacle and a cylinder or a sphere. Other visual effect enhancements like material, shadow and reflection will be implemented to make the model more realistic.

Figure 16: Upper body of Betty

5. References
Silicon Graphics International. Examples http://www.sgi.com/products/software/opengl/examples/glut/contrib/. 2009-2010 [2] Spong MW, Vidyasagar M. Robot Dynamics And Control. New York: John Wiley Sons, Inc. 2006. [3] Hooper R. Learn About Robots: Robot Forward Kinematics. http://www.learnaboutrobots.com/forwardKinematics.htm. 2010. [4] OpenGL. OpenGL API Documentation http://www.opengl.org/documentation/. 2010. [5] Whitrow R. OpenGL Graphics Through Applications. Springer-Verlag London Limited. 2008. [6] OpenGL. GLUT - The OpenGL Utility Toolkit. http://www.opengl.org/resources/libraries/glut/glut_downloads.php. 2000. [7] freeglut. http://freeglut.sourceforge.net/.2009. [8] Joshua G. H, Benjamin H. and Eduardo M. M. Robot simulation, collisions and contacts. IEEE/RSJ 2008 International Conference on Intelligent Robots and Systems. Nice, France. September 23-26 2008. [9] Hill F. S. and Kelly S. M. Computer Graphics: Using OpenGL. 3rd ed. NJ: Pearson Eduction Inc. 2007. [10] Silicon Graphics, Inc. OpenGL 2.1 Reference Pages http://www.opengl.org/sdk/docs/man/xhtml/gluPerspective.xml.. 2006 [11] Silicon Graphics, Inc. Code samples for the OpenGL v1.1 Programming Guide (Redbook). http://www.opengl.org/resources/code/samples/redbook/. 1997. [1]

12