Front cover

IBM BladeCenter JS21
The POWER of Blade Innovation
High-performance blade server ideal for extremely dense HPC clusters First IBM blade server with native virtualization for server consolidation Exceptional price-performance per watt for WebSphere with AIX or Linux OS

Stephen Hochstetler Benjamin Ebrahimi Tom Junkin Bernhard Zeller

ibm.com/redbooks

International Technical Support Organization IBM BladeCenter JS21: The POWER of Blade Innovation October 2006

SG24-7273-00

Note: Before using this information and the product it supports, read the information in “Notices” on page ix.

First Edition (October 2006) This edition applies to the IBM BladeCenter JS21, Linux, and IBM AIX 5L Version 5.3, product number 5765-G03.
© Copyright International Business Machines Corporation 2006. All rights reserved. Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.

Contents
Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi The team that wrote this redbook. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi Become a published author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii Comments welcome. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii Chapter 1. Introduction to BladeCenter and JS21 technology . . . . . . . . . . 1 1.1 BladeCenter terminology. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.2 Benefits of the BladeCenter family . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.3 BladeCenter versus an IBM rack server . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.4 Introduction to IBM BladeCenter JS21 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 Chapter 2. BladeCenter JS21: Hardware components . . . . . . . . . . . . . . . . 7 2.1 BladeCenter JS21 system specifications . . . . . . . . . . . . . . . . . . . . . . . . . . 8 2.2 Physical specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.3 Physical packaging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.4 BladeCenter JS21 models. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.5 Minimum and optional features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.6 Processor features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 2.7 Memory features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 2.8 Internal Serial-Attached SCSI disk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.9 BladeCenter chassis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 2.9.1 BladeCenter power modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 2.9.2 BladeCenter management modules . . . . . . . . . . . . . . . . . . . . . . . . . 24 2.9.3 Media features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 Chapter 3. BladeCenter JS21: Software environment . . . . . . . . . . . . . . . . 25 3.1 Operating system support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 3.1.1 AIX 5L . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 3.1.2 Red Hat Enterprise Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 3.1.3 SUSE Linux Enterprise Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 3.2 Virtual input/output Server. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 3.3 System management tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 3.3.1 BladeCenter Web interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 3.3.2 IBM Director . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 3.3.3 IBM Cluster Systems Management. . . . . . . . . . . . . . . . . . . . . . . . . . 33 3.3.4 IBM middleware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

© Copyright IBM Corp. 2006. All rights reserved.

iii

Chapter 4. BladeCenter JS21: Virtualization . . . . . . . . . . . . . . . . . . . . . . . 35 4.1 Virtualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 4.1.1 Advanced POWER Virtualization . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 4.1.2 Dynamic logical partitioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 Chapter 5. Planning considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 5.1 Documentation and planning resources . . . . . . . . . . . . . . . . . . . . . . . . . . 46 5.2 Network planning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 5.2.1 Minimal network requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 5.2.2 High-performance, low-latency network requirements . . . . . . . . . . . 51 5.2.3 Multiple BladeCenter environments . . . . . . . . . . . . . . . . . . . . . . . . . 53 5.3 Operating system support and installation . . . . . . . . . . . . . . . . . . . . . . . . 53 5.4 Systems management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 5.4.1 BladeCenter Web interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 5.4.2 IBM Director . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 5.4.3 IBM Cluster Systems Management. . . . . . . . . . . . . . . . . . . . . . . . . . 59 Chapter 6. Hardware setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 6.1 BladeCenter chassis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 6.2 BladeCenter internal network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 6.3 Management module configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 6.3.1 Connecting to the management module . . . . . . . . . . . . . . . . . . . . . . 72 6.3.2 Configuring the management module . . . . . . . . . . . . . . . . . . . . . . . . 75 6.3.3 Advanced management module: Management interface changes . . 79 6.3.4 Management module: Hardware vital product data . . . . . . . . . . . . . 81 6.3.5 Management module redundancy. . . . . . . . . . . . . . . . . . . . . . . . . . . 82 6.3.6 Management module: TCP/IP ports . . . . . . . . . . . . . . . . . . . . . . . . . 83 6.3.7 Resetting the management module . . . . . . . . . . . . . . . . . . . . . . . . . 84 6.4 I/O module configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 6.4.1 Setting the IP address of an I/O module . . . . . . . . . . . . . . . . . . . . . . 85 6.4.2 Setting external management over all ports . . . . . . . . . . . . . . . . . . . 87 6.4.3 Enabling the external I/O module ports . . . . . . . . . . . . . . . . . . . . . . . 88 6.5 Blade server configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 6.5.1 Assigning names to blade servers . . . . . . . . . . . . . . . . . . . . . . . . . . 90 6.5.2 Setting the boot sequence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 6.5.3 Assigning the media tray. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 6.6 Firmware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 6.6.1 Getting the latest firmware, tools, and support . . . . . . . . . . . . . . . . . 95 6.6.2 Identifying and updating the latest firmware in general. . . . . . . . . . . 96 6.6.3 Updating BladeCenter JS21 firmware using AIX . . . . . . . . . . . . . . . 99 6.6.4 Updating BladeCenter JS21 firmware (BIOS) using Linux . . . . . . . 101 6.6.5 Updating BladeCenter JS21 Ethernet adapter using Linux. . . . . . . 102 6.6.6 Updating the management module and advanced management module

iv

IBM BladeCenter JS21: The POWER of Blade Innovation

firmware using the Web interface . . . . . . . . . . . . . . . . . . . . . . . . . . 104 6.6.7 Updating I/O module firmware using a Web interface. . . . . . . . . . . 106 6.6.8 Updating IBM BladeCenter 2-port Fibre Channel switch module firmware using FTP. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 6.6.9 Updating the firmware using the stand-alone diagnostics CD . . . . 111 6.7 Providing a console for the BladeCenter JS21 . . . . . . . . . . . . . . . . . . . . 121 6.7.1 Configuring Serial over LAN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 6.7.2 Using Serial over LAN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128 6.8 System Management Services interface . . . . . . . . . . . . . . . . . . . . . . . . . 133 6.8.1 Activating the System Management Services interface . . . . . . . . . 133 6.8.2 Setting up remote initial program load . . . . . . . . . . . . . . . . . . . . . . 135 6.8.3 Selecting boot options. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140 6.9 Open Firmware interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152 6.9.1 Activating the Open Firmware interface . . . . . . . . . . . . . . . . . . . . . 152 6.9.2 Specifying IP parameters to Open Firmware . . . . . . . . . . . . . . . . . 160 6.10 Blade server Ethernet controller enumeration. . . . . . . . . . . . . . . . . . . . 162 6.11 Blade server I/O module routing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169 6.12 SAS hardware RAID configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170 6.12.1 RAID configuration methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170 6.12.2 Configuring RAID using the AIX stand-alone diagnostics . . . . . . . 171 6.12.3 Configuring RAID using iprconfig on Linux . . . . . . . . . . . . . . . . . . 183 Chapter 7. Installing and managing the Virtual I/O Server . . . . . . . . . . . 193 7.1 VIOS installation in a JS21 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194 7.1.1 VIOS image installation from a DVD . . . . . . . . . . . . . . . . . . . . . . . . 194 7.1.2 VIOS image installation from a NIM server . . . . . . . . . . . . . . . . . . . 194 7.1.3 VIOS image installation from a Linux server . . . . . . . . . . . . . . . . . . 202 7.2 Initial VIOS configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208 7.2.1 Virtualization setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209 7.2.2 Setting the date and time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210 7.2.3 Initial network setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210 7.3 VIOS management using IVM. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211 7.3.1 IVM graphical user interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211 7.3.2 IVM command-line interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213 7.4 Storage management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216 7.5 VIOS partition configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221 7.5.1 Creating the logical volume . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222 7.5.2 Creating logical partitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223 7.5.3 Updating the logical partition profile . . . . . . . . . . . . . . . . . . . . . . . . 231 7.5.4 Powering on a logical partition . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231 7.5.5 Opening a virtual terminal to a logical partition . . . . . . . . . . . . . . . . 231 7.5.6 Installing an operating system on a logical partition . . . . . . . . . . . . 232 7.5.7 Optical device sharing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233

Contents

v

7.5.8 Logical partition configuration changes . . . . . . . . . . . . . . . . . . . . . . 235 7.6 Network management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240 7.7 Advanced storage management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244 7.7.1 Virtual storage assignment to a partition . . . . . . . . . . . . . . . . . . . . . 244 7.7.2 Virtual disk extension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245 7.7.3 VIOS system disk mirroring. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249 7.8 Maintenance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253 7.8.1 VIOS maintenance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253 7.8.2 Logical partition maintenance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256 7.8.3 Command logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257 Chapter 8. Installing AIX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259 8.1 General topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260 8.2 Preparing AIX installation from CD/DVD . . . . . . . . . . . . . . . . . . . . . . . . . 260 8.3 Preparing AIX network installation using NIM . . . . . . . . . . . . . . . . . . . . . 261 8.3.1 Configuring AIX environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261 8.3.2 Installing the NIM filesets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262 8.3.3 Configuring the NIM master . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263 8.3.4 Defining the NIM client . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266 8.3.5 Allocating resources to a client . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268 8.3.6 Activating the client installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268 8.4 Installing AIX on the client . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270 8.5 Changing client from rsh to nimsh . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276 Chapter 9. Installing Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281 9.1 Installing Linux from a CD/DVD. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282 9.2 Basic preparations for a Linux network installation . . . . . . . . . . . . . . . . . 283 9.2.1 Installing Linux using the network: General remarks . . . . . . . . . . . 284 9.2.2 Configuring a BOOTP or DHCP service . . . . . . . . . . . . . . . . . . . . . 285 9.2.3 Configuring a Trivial File Transfer Protocol service . . . . . . . . . . . . 287 9.2.4 Configuring a File Transfer Protocol service . . . . . . . . . . . . . . . . . . 288 9.3 Installing SLES using the network. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290 9.3.1 Overview of SLES network installation . . . . . . . . . . . . . . . . . . . . . . 291 9.3.2 Preparing the installation source directory and service. . . . . . . . . . 293 9.3.3 Basic attended SLES network installation. . . . . . . . . . . . . . . . . . . . 299 9.3.4 Configuring the boot image file with mkzimage_cmdline . . . . . . . . 306 9.3.5 Unattended installation with SLES . . . . . . . . . . . . . . . . . . . . . . . . . 311 9.4 Installing Red Hat Enterprise Linux AS 4 Update 3 using the network . . 321 9.4.1 Overview of Red Hat Enterprise Linux network installation . . . . . . 322 9.4.2 Preparing the installation source for Red Hat Enterprise Linux . . . 324 9.4.3 Basic attended Red Hat Enterprise Linux network installation . . . . 326 9.4.4 Unattended installation with Red Hat Enterprise Linux . . . . . . . . . . 336 Chapter 10. System management scenarios . . . . . . . . . . . . . . . . . . . . . . 345

vi

IBM BladeCenter JS21: The POWER of Blade Innovation

10.1 BladeCenter Web interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 346 10.2 IBM Director . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 346 10.2.1 Installing the components for IBM Director . . . . . . . . . . . . . . . . . . 347 10.2.2 Using the IBM Director Console in a BladeCenter context . . . . . . 355 10.3 Cluster Systems Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362 10.3.1 Installing and setting up the CSM management server . . . . . . . . 363 10.3.2 Creating a CSM installation server . . . . . . . . . . . . . . . . . . . . . . . . 373 10.3.3 Defining cluster nodes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373 10.3.4 Adding AIX nodes to the cluster . . . . . . . . . . . . . . . . . . . . . . . . . . 377 10.3.5 Adding Linux nodes to the cluster . . . . . . . . . . . . . . . . . . . . . . . . . 379 10.3.6 Testing cluster installation and basic command . . . . . . . . . . . . . . 383 Chapter 11. Performance tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 385 11.1 Performance tuning for AIX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 386 11.2 Tuning options for Linux on POWER: Selected examples . . . . . . . . . . 386 11.2.1 Disabling unused processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 386 11.2.2 Disabling the graphical user interface . . . . . . . . . . . . . . . . . . . . . . 388 11.2.3 Tuning the kernel parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . 390 11.2.4 Disabling simultaneous multithreading . . . . . . . . . . . . . . . . . . . . . 394 11.2.5 Tuning memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 394 11.2.6 Tuning the file system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395 11.2.7 Selecting the disk technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . 396 11.2.8 Tuning in Transmission Control Protocol and User Datagram Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 399 11.2.9 Performance tuning tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 401 11.2.10 Tuning for applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 408 Chapter 12. Storage area network installation and booting . . . . . . . . . . 409 12.1 Storage area network setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 410 12.1.1 Assigning a disk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 411 12.1.2 SAN boot characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417 Appendix A. SUSE Linux Enterprise Server 9 AutoYaST XML file . . . . . 419 Appendix B. Red Hat Enterprise Linux 4 Kickstart file . . . . . . . . . . . . . . 427 Appendix C. SUSE Linux Installation Server setup . . . . . . . . . . . . . . . . . 431 Abbreviations and acronyms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 437 Related publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 439 IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 439 Other publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 440 Online resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 440 How to get IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 444

Contents

vii

Help from IBM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 444 Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 445

viii

IBM BladeCenter JS21: The POWER of Blade Innovation

Notices
This information was developed for products and services offered in the U.S.A. IBM may not offer the products, services, or features discussed in this document in other countries. Consult your local IBM representative for information on the products and services currently available in your area. Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any IBM intellectual property right may be used instead. However, it is the user's responsibility to evaluate and verify the operation of any non-IBM product, program, or service. IBM may have patents or pending patent applications covering subject matter described in this document. The furnishing of this document does not give you any license to these patents. You can send license inquiries, in writing, to: IBM Director of Licensing, IBM Corporation, North Castle Drive Armonk, NY 10504-1785 U.S.A. The following paragraph does not apply to the United Kingdom or any other country where such provisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of express or implied warranties in certain transactions, therefore, this statement may not apply to you. This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice. Any references in this information to non-IBM Web sites are provided for convenience only and do not in any manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the materials for this IBM product and use of those Web sites is at your own risk. IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring any obligation to you. Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. This information contains examples of data and reports used in daily business operations. To illustrate them as completely as possible, the examples include the names of individuals, companies, brands, and products. All of these names are fictitious and any similarity to the names and addresses used by an actual business enterprise is entirely coincidental. COPYRIGHT LICENSE: This information contains sample application programs in source language, which illustrates programming techniques on various operating platforms. You may copy, modify, and distribute these sample programs in any form without payment to IBM, for the purposes of developing, using, marketing or distributing application programs conforming to the application programming interface for the operating platform for which the sample programs are written. These examples have not been thoroughly tested under all conditions. IBM, therefore, cannot guarantee or imply reliability, serviceability, or function of these programs. You may copy, modify, and distribute these sample programs in any form without payment to IBM for the purposes of developing, using, marketing, or distributing application programs conforming to IBM's application programming interfaces.

© Copyright IBM Corp. 2006. All rights reserved.

ix

Trademarks
The following terms are trademarks of the International Business Machines Corporation in the United States, other countries, or both: 3090™ AIX 5L™ AIX® BladeCenter® Chipkill™ DB2® eServer™ i5/OS® IBM® iSeries™ Micro-Partitioning™ Power Architecture™ POWER Hypervisor™ Power PC® POWER5™ PowerPC Architecture™ PowerPC® POWER™ pSeries® Redbooks (logo) ™ Redbooks™ RETAIN® RS/6000® ServerProven® System p5™ System p™ System z9™ System z™ System/360™ Tivoli® Virtualization Engine™ WebSphere® xSeries® z9™ zSeries®

The following terms are trademarks of other companies: Java, Power Management, Solaris, and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. Microsoft, Windows, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both. i386, Intel, MMX, Intel logo, Intel Inside logo, and Intel Centrino logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States, other countries, or both. UNIX is a registered trademark of The Open Group in the United States and other countries. AltiVec is a trademark of Freescale Semiconductor, Inc. Linux is a trademark of Linus Torvalds in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of others.

x

IBM BladeCenter JS21: The POWER of Blade Innovation

Preface
Blade servers have shown themselves to be an efficient, powerful technology. They have captured the industry focus because of their modular design, which can reduce cost with a more efficient use of valuable floor space. They offer simplified management, which can help to speed up tasks such as installing, provisioning, updating, and troubleshooting hundreds of blade servers. You can do all of this remotely using one graphical console with IBM® Director systems management tools or cluster management software such as Cluster Systems Management (CSM). In addition, blade servers provide improved performance by doubling current rack density. By integrating resources and sharing key components, costs decrease and availability increases. The IBM BladeCenter® boasts innovative modular technology, leadership density, and availability. It is designed to help solve a multitude of real-world problems. This IBM Redbook takes an in-depth look at the IBM BladeCenter JS21. This is a 2-core or 4-core blade server for applications requiring 64-bit computing. It is ideal for computer-intensive applications and transactional Internet servers. This book helps you to install, tailor, and configure the IBM BladeCenter JS21 running either IBM AIX® 5L™ or the Linux® operating system (OS). This document expands the current set of IBM BladeCenter JS21 documentation by providing a desktop reference that offers a detailed technical description of the JS21 system. This publication does not replace the latest IBM BladeCenter JS21 marketing materials and tools. It is intended as an additional source of information that you can use, together with existing sources, to enhance your knowledge of IBM blade solutions.

The team that wrote this redbook
This redbook was produced by a team of specialists from around the world working at the International Technical Support Organization (ITSO), Austin, Texas. Stephen Hochstetler is a Senior I/T Specialist at the ITSO, Austin Center. He writes extensively and teaches IBM classes worldwide on all areas of IBM System p™ and Linux. Before joining the ITSO six years ago, Stephen worked in IBM Tivoli® Services, U.S., as a network management solution architect.

© Copyright IBM Corp. 2006. All rights reserved.

xi

Benjamin Ebrahimi is a Staff Engineer in the IBM Development Laboratory in Boeblingen, Germany, for the Open Systems Design and Development Group since 2006. He has more than 10 years of experience in planning, integration, installation, and development for multiple operating systems and architectures. He holds a degree in General Engineering Science and Electrical Engineering from the Hamburg University of Technology. His areas of expertise include Linux, Microsoft® Windows®, IBM AIX, 16-bit, 32-bit, and 64-bit computer architectures, different networks, for example, wide area network (WAN), local area network (LAN), wireless LAN (WLAN), storage area network (SAN), and development of programs, drivers, firmware, protocols, and hardware. Recently he has been involved in JS21/cell development supporting JS21/cell installations and infrastructure during system test and early customer installations. Tom Junkin is a Senior I/T Specialist in the IBM Americas Techline Organization. He has been with IBM for 35 years, with 15 years in the field variously as a Workstation Specialist, Systems Engineer, and Investment Territory Representative. He has also been a Program Manager for IBM federal contract management. He attended the University of South Carolina. His areas of expertise include pre-sales support for IBM RS/6000®, System p, AIX, and server consolidation in the federal industry. Bernhard Zeller is an I/T Specialist for Technical Sales System p in the IBM Systems & Technology Group (STG), IMT Germany, in Mannheim. He has 17 years of experience in the AIX 5L and IBM System p field. He holds a degree in electronics engineering from Fachhochschule Aalen. He has worked in IBM for 17 years. His areas of expertise include networking, operating systems, server management, and speech synthesis technology. Thanks to the following people for their contributions to this project: Betsy Thaggard ITSO, Austin Center Donn Bullock IBM, Raleigh Kerry C. Anders Walter Butler Catherine Kostetsky Frank Petsche IBM, Austin

xii

IBM BladeCenter JS21: The POWER of Blade Innovation

Michael Long Sanjay Verma IBM, Beaverton Torsten Bloth IBM Germany

Become a published author
Join us for a two- to six-week residency program! Help write an IBM Redbook dealing with specific products or solutions, while getting hands-on experience with leading-edge technologies. You'll team with IBM technical professionals, Business Partners and/or customers. Your efforts will help increase product acceptance and customer satisfaction. As a bonus, you'll develop a network of contacts in IBM development labs, and increase your productivity and marketability. Find out more about the residency program, browse the residency index, and apply online at: ibm.com/redbooks/residencies.html

Comments welcome
Your comments are important to us! We want our IBM Redbooks™ to be as helpful as possible. Send us your comments about this or other Redbooks in one of the following ways: Use the online Contact us review redbook form found at: ibm.com/redbooks Send your comments in an email to: redbook@us.ibm.com Mail your comments to: IBM Corporation, International Technical Support Organization Dept. HYTD Mail Station P099 2455 South Road Poughkeepsie, NY 12601-5400

Preface

xiii

xiv

IBM BladeCenter JS21: The POWER of Blade Innovation

1

Chapter 1.

Introduction to BladeCenter and JS21 technology
This chapter presents an overview of the IBM BladeCenter and blade server technology. It describes the following topics: 1.1, “BladeCenter terminology” on page 2 1.2, “Benefits of the BladeCenter family” on page 2 1.3, “BladeCenter versus an IBM rack server” on page 3 1.4, “Introduction to IBM BladeCenter JS21” on page 4

© Copyright IBM Corp. 2006. All rights reserved.

1

1.1 BladeCenter terminology
The term blade chassis refers to a chassis that can hold a number of hot-swappable devices called blades. Blades come in two varieties: blade servers and option blades. A blade server is an independent server that contains one or more processors and associated memory, disk storage, and network controllers. It runs its own operating system and applications. Each blade server within a system chassis slides into a blade bay. It plugs into a midplane or backplane to share common infrastructure components. These components can include power supplies, fans, CD-ROM, and diskettes, Ethernet and Fibre Channel switches, and system ports. Option blades can be sharable by blade servers. They provide additional features, such as controllers for external input/output (I/O) or disk arrays, power supplies, and so on.

1.2 Benefits of the BladeCenter family
For organizations that seek server consolidation, BladeCenter centralizes servers for increased flexibility, ease of maintenance, reduced cost, and streamlined human resources. Companies that have to install new applications for e-commerce and on-demand business can achieve speed while ensuring flexibility, scalability, and availability. For enterprise requirements such as file-and-print and collaboration, BladeCenter is designed to offer reliability, flexibility for growth, and cost-effectiveness. Clients with compute-intensive applications that require highly available clustering can use the BladeCenter to help achieve high degrees of scalability and performance. Lower cost in certain configurations: Due to efficiencies in power usage, heat emissions, and data center floor space utilization, when a configuration of other servers exceeds one rack, it is often less expensive over a multi-year period to use a single rack of BladeCenter chassis and multiple BladeCenter JS21. Physical server consolidation: You can use a BladeCenter server as an ideal replacement for many uniprocessor or two-way servers to save space. You can replace 14U of rack space for 14 1U servers with one 7U BladeCenter chassis. Additional rack space that is normally taken for Ethernet, Fibre Channel, and other switches is eliminated by the integrated switches in the BladeCenter chassis.

2

IBM BladeCenter JS21: The POWER of Blade Innovation

High availability: The BladeCenter chassis offers redundant and hot-swap components that can prevent failure of the chassis or blade servers when one of these components fails. The chassis has redundant, hot-swap cooling, power, midplane features, management modules, and I/O switches. It also has a hot-swap media tray, with CD-ROM and diskette, that can be removed and serviced while all blade servers are operating. Integrated switch technology: All service is performed from the front and rear of the BladeCenter. Therefore, there is no need to slide the chassis out of the rack and remove the top cover for service. Also, numerous cables are eliminated, reducing both cabling cost and servicer or administrator time. More integrated systems management features: The BladeCenter chassis includes a management module. This module eliminates the need for individual management adapters, such as the Remote Supervisor Adapter I or Remote Supervisor Adapter II, in each blade server for remote control. It also eliminates the need for RS485 interconnect cabling between the blade servers. SAN optimization: A Fibre Channel switch module installed in a BladeCenter chassis provides storage area network (SAN) access to all blade servers in the chassis without internal cabling.

1.3 BladeCenter versus an IBM rack server
BladeCenter is an ideal solution for certain environments. In other environments, an IBM rack server might be a better fit. You can choose servers in a rack in the following cases: Need for a small number of servers: A BladeCenter chassis is required for one blade. To be cost-effective compared to certain stand-alone servers, you require multiple blades in a chassis. Therefore, a chassis must be full or nearly full. Need for existing Peripheral Component Interconnect (PCI) adapters: BladeCenter does not include adapter slots as shipped. An optional I/O expansion card feature offers one PCI slot per blade, limiting the blade to one integrated development environment (IDE) drive. The expansion feature supports PCI cards designed for BladeCenter. Therefore, a mix of BladeCenter and traditional rack-optimized servers might be appropriate. Need for large internal storage or (non-SAN) external Small Computer System Interface (SCSI) storage: The BladeCenter JS21 supports a maximum of 146 gigabytes (GB) of Serial-Attached SCSI (SAS) storage internally. While there is no provision for external SCSI storage, there is an external Fibre Channel SAN storage option. Using the optional internal SCSI

Chapter 1. Introduction to BladeCenter and JS21 technology

3

storage feature doubles the space requirement of the blade, reducing by half the number of blade servers that can be installed in a chassis. Need for re-installation or re-purposing at end of original: You can distribute stand-alone uniprocessor and 4-core servers individually for use as departmental file/print servers and other low-horsepower uses. Because you cannot use blade servers without a chassis, the entire chassis-and-blades combination has to stay together. The original chassis can be useful if older blade servers are discarded and newer, faster ones take their place in the chassis. Power requirements: The data center is wired for 110 volts (V) - 120 V power. BladeCenter requires 220 V -240 V power. Learn more about IBM BladeCenter and its components in The Cutting Edge: IBM eServer BladeCenter, REDP-3581.

1.4 Introduction to IBM BladeCenter JS21
The IBM BladeCenter JS21 Type 8844 blade server offers significant improvements in performance and reliability over the previous generation JS20 blade server, delivering up to three times better performance by using the following features: The faster IBM PowerPC® 970MP dual-core processors Faster, more reliable double data rate 2 (DDR2) memory options SAS hard disk drives The first IBM blade server optimized for BladeCenter H With built-in support for Advanced Performance Optimization with Enhanced RISC (IBM POWER™) Virtualization, the BladeCenter JS21 offers an ideal blade server solution for high-performance computing (HPC), AIX 5L, and server consolidation. The BladeCenter JS21 supports a maximum of 16 GB 533 MHz error-checking and correction (ECC) IBM Chipkill™ DDR2 memory. Model 31X, with 1 GB memory standard, offers two single active core, 64-bit PowerPC 970MP processors. The 51X, with 2 GB memory standard, offers two dual-core, 64-bit PowerPC 970MP processors. Each processor core includes 32/64 KB L1 (data/instruction) and 1 MB (non-shared) L2 cache. Processor clock rates vary based on the model of the BladeCenter chassis that the BladeCenter JS21 is inserted into. For processor speed details, see Table 2-7 on page 19. Designed with the demands of enterprise and scientific computing in mind, the BladeCenter JS21 is a highly differentiated solution for high-performance Linux clusters, seismic analysis for oil and gas, UNIX® applications for retail and

4

IBM BladeCenter JS21: The POWER of Blade Innovation

finance, Web serving such as with IBM WebSphere®, grid solutions, and any other custom or commercial application able to exploit the performance acceleration of AltiVec technology for data-intensive workloads. The BladeCenter JS21 represents a convergence of leadership technologies and is therefore a critical component of the IBM BladeCenter and System p server portfolio. The BladeCenter JS21 allows customers the ability to better use the value proposition of the BladeCenter design combined with the reliability of AIX 5L and enterprise Linux. The BladeCenter JS21 delivers outstanding deployment flexibility. You can install it in the BladeCenter or BladeCenter H chassis to optimize your current and future investments. As a heterogeneous infrastructure consolidation platform, all BladeCenter chassis also support running the BladeCenter JS21 alongside your Intel®-based HS blades and Advanced Micro Devices (AMD) Opteron-based LS blades in the same chassis with independent monitoring, security, power, and systems management. The BladeCenter JS21 includes power management capabilities to allow the maximum uptime and performance possible for your servers. Built for speed and reliability, the BladeCenter JS21 supports a choice of operating systems for running HPC Linux clusters as well as AltiVec-optimized applications on AIX 5L or Linux. Virtualization support is standard on the BladeCenter JS21. Therefore, server and workload consolidation of multiple independent applications on a single blade can be provided by Advanced POWER Virtualization for AIX 5L and Linux environments by ordering Virtual I/O Server (VIOS) V1.2.1.

Chapter 1. Introduction to BladeCenter and JS21 technology

5

6

IBM BladeCenter JS21: The POWER of Blade Innovation

2

Chapter 2.

BladeCenter JS21: Hardware components
This chapter describes the major hardware components associated with the IBM BladeCenter JS21. It begins by reviewing the core BladeCenter infrastructure. It also provides a detailed description of the BladeCenter JS21 and the range of options that you can order from IBM.

© Copyright IBM Corp. 2006. All rights reserved.

7

2.1 BladeCenter JS21 system specifications
The IBM BladeCenter JS21 Type 8844 blade server offers significant improvements in performance and reliability over the previous generation JS20 blade server, delivering up to three times better performance by using the following features: The faster IBM PowerPC 970MP dual-core processors Faster, more reliable double data rate 2 (DDR2) memory options Serial-Attached SCSI (SAS) hard disk drives The first IBM blade server optimized for BladeCenter H The BladeCenter JS21 supports a maximum of 16 gigabytes (GB) 400 MHz error-checking and correction (ECC) Chipkill DDR2 memory or 8 GB 533 MHz ECC Chipkill DDR2 memory. Model 31X, with 1 GB memory standard, offers two single active core, 64-bit PowerPC 970MP processors. The 51X, with 2 GB memory standard, offers two dual-core, 64-bit PowerPC 970MP processors. Each processor core includes 32/64 KB L1 (data/instruction) and 1 MB (non-shared) L2 cache. Processor clock rates vary based on the model of BladeCenter chassis that the BladeCenter JS21 is inserted into. For processor speed details, see Table 2-7 on page 19. The BladeCenter JS21 includes power management capabilities to allow the maximum uptime and performance possible for your servers. Built for speed and reliability, the BladeCenter JS21 supports a choice of operating systems for running HPC Linux clusters as well as AltiVec-optimized applications on AIX 5L or Linux. Virtualization support is standard on the BladeCenter JS21. Therefore, server and workload consolidation of multiple independent applications on a single blade can be provided by Advanced POWER Virtualization for AIX 5L and Linux environments by ordering Virtual I/O Server (VIOS) V1.2.1.

8

IBM BladeCenter JS21: The POWER of Blade Innovation

Table 2-1 describes the general system specifications of the BladeCenter JS21.
Table 2-1 BladeCenter JS21specifications Description Operating temperature Relative humidity Maximum altitude Operating voltage Operating frequency Maximum power consumption Range 10 to 35 degrees C (50 to 95 degrees F) 10 to 32 degrees C (50 to 90 degrees F) 8% to 80% 2,133 m (7,000 ft) BladeCenter (8677) 200-240 V ac BladeCenter H (8852) 12.2 V dc 50 Hz or 60 Hz 31x: 283 watts 51x: 308 watts

2.2 Physical specifications
Table 2-2 describes the physical specifications of the BladeCenter JS21.
Table 2-2 BladeCenter JS21 physical specifications 8844-31X Processor Number of cores L2 cache Memory (400/533 MHz synchronous dynamic random access memory (SDRAM)) Hard disk drives (HDD) controller Internal capacity Integrated Ethernet controller 2-socket single active core 64-bit PowerPC 970MP 2 1 MB/processor core 1 GB/16 GB 400 MHz 1 GB/8 GB 533 MHz (1 GB/16 GB 533 MHz) Integrated SAS or Redundant Array of Independent Disks (RAID) 36/73 GB - 146 GB maximum Dual 1 gigabit (Gb) 8844-51X 2-socket dual-core 64-bit PowerPC 970MP 4 1 MB/processor core 2 GB/16 GB 400 MHz 2 GB/8 GB 533 MHz (2 GB/16 GB 533 MHz) Integrated SAS or RAID

36/73 GB - 146 GB maximum Dual 1 Gb

The processor operating frequency depends on the BladeCenter chassis, as shown in Table 2-7 on page 19.

Chapter 2. BladeCenter JS21: Hardware components

9

2.3 Physical packaging
Table 2-3 describes the major physical attributes of the BladeCenter JS21.
Table 2-3 BladeCenter JS21 physical packaging Dimension Height Width Depth Weight Minimum configuration 5.4 kg (12 lb) depending on the configuration 245 mm (9.7 inches) 29 mm (1.14 inches) 446 mm (17.6 inches)

Figure 2-1 shows the JS21 with the cover off. You can see the two processors and the memory slots completely full. The two optional SAS disks are installed.

Figure 2-1 View of the BladeCenter JS21 with the cover off

10

IBM BladeCenter JS21: The POWER of Blade Innovation

2.4 BladeCenter JS21 models
There are two models of the BladeCenter JS21. Both models have similar base features with the exception of the microprocessors used in the processor subsystem and memory options available. The JS21 Type 8844-31X blade server with the PowerPC 970MP single-active core microprocessor The JS21 Type 8844-51X blade server with the PowerPC 970MP dual-core microprocessor Table 2-4 is a summary of the new BladeCenter JS21 models.
Table 2-4 Summary of JS21 models High-density blade Family Models JS21 8844-31X Blade description Family of 64-bit PowerPC technology based blade servers, running 64-bit AIX 5L or Linux operating systems 2-core PowerPC 970 Blade 2 x single active core PowerPC 970MP 2 x 512 MB 400 MHz memory dual inline memory modules (DIMMs) Operating frequency depends on the chassis type 4-core PowerPC 970 Blade 2 x dual-core PowerPC 970MP 2 x 1 GB 400 MHz DIMMs Operating frequency depends on the chassis type

8844-51X

2.5 Minimum and optional features
Table 2-5 lists the standard configurations that the BladeCenter JS21 supports.
Table 2-5 Standard BladeCenter JS21 standard configuration Model 8844-31X 8844-51X Processor 2-socket single active core 2.7 GHz/2.6 GHz 2-socket dual-core 2.5 GHz/2.3 GHz L2 cache 1 MB/core 1 MB/core Memory 1 GB (2 x 512 MB) 2 GB (2 x 1 GB) Ethernet Dual Gb Dual Gb HDD Open Open

Chapter 2. BladeCenter JS21: Hardware components

11

Restriction: The processor frequency depends on the type of BladeCenter chassis that the BladeCenter JS21 is installed in, as described in Table 2-7 on page 19. Figure 2-2 shows the memory, SAS disk, and expansion option connectors, which are the same for the 31X and 51X models.

Figure 2-2 BladeCenter JS21 connectors

The following list provides some of the optional features available on the BladeCenter JS21. For a complete list of the supported modules and adapters, see the following IBM ServerProven® Web site: http://www.ibm.com/servers/eserver/serverproven/compat/us/eserver.html Up to 16 GB of system memory Up to 2 internal hard disk drives for up to 146 GB of internal storage Support for small-form-factor ((SFF), 2.5”) 36 GB/73 GB SAS 10,000 revolutions per minute HDD Standard or SFF Gigabit Ethernet expansion cards

12

IBM BladeCenter JS21: The POWER of Blade Innovation

Topspin Host Channel Adapter expansion card QLogic iSmall Computer System Interface (iSCSI) expansion card QLogic 4Gb SFF Fibre Channel expansion card Myrinet Cluster expansion card

2.6 Processor features
This section provides more detailed information about the IBM PowerPC 970MP microprocessor, which is used for the main processors on the BladeCenter JS21. The BladeCenter JS21 leverages the high-performance, low-power 64-bit IBM PowerPC 970MP microprocessor in either single-active core or dual-core configurations. Two processor modules are supported per single-wide BladeCenter JS21. They are directly mounted to the blade planar board providing multi-processing capability. Each processor core includes 32/64 KB L1 (data/instruction) and 1 MB (non-shared) L2 cache. The 4-core configuration comprises two dual-core PowerPC 970MP processors, and the 2-core configuration comprises two single-active core PowerPC 970MP processors. The PowerPC 970MP is the higher frequency, dual-core capable, 90 nm follow-on to the single-core capable PowerPC 970FX 2.2 GHz microprocessor used in the previous generation BladeCenter JS20. The processor operating frequency depends on the BladeCenter chassis, as shown in Table 2-7 on page 19.

AltiVec
The AltiVec is an extension to the IBM PowerPC Architecture™. It defines additional registers and instructions to support single-instruction multiple-data (SIMD) operations that accelerate data-intensive tasks.

A short history of vector processing
The basic concept behind vector processing is to enhance the performance of data-intensive applications by providing hardware support for operations that can manipulate an entire vector (or array) of data in a single operation. The number of data elements operated upon at a time is called the vector length.

Chapter 2. BladeCenter JS21: Hardware components

13

Scalar processors perform operations that manipulate single data elements such as fixed-point or floating-point numbers. For example, scalar processors usually have an instruction that adds two integers to produce a single-integer result. Vector processors perform operations on multiple data elements arranged in groups called vectors (or arrays). For example, a vector add operation to add two vectors performs a pairwise addition of each element of one source vector with the corresponding element of the other source vector. It places the result in the corresponding element of the destination vector. Typically a single vector operation on vectors of length n is equivalent to performing n scalar operations. Figure 2-3 illustrates the difference between scalar and vector operations.

Vector Add Operation Scalar Add Operation
4 + 7 11 7 1 6 3 3 10 5 6 11 4 5 2 12 7 17 7 8 12

+

Figure 2-3 Scalar and vector operations

Processor designers are continually looking for ways to improve application performance. The addition of vector operations to a processor architecture is one method that a processor designer can use to make it easier to improve the peak performance of a processor. However, the actual performance improvements that can be obtained for a specific application depend on how well the application can exploit vector operations. The concept of vector processing has existed since the 1950s. Early implementations of vector processing (known as array processing) were installed in the 1960s. They used special purpose peripherals attached to general purpose computers. An example is the IBM 2938 Array Processor, which could be attached to some models of the IBM System/360™. This was followed by the IBM 3838 Array Processor in later years. By the mid-1970s, vector processing became an integral part of the main processor in large supercomputers manufactured by companies such as Cray Research. By the mid-1980s, vector processing became available as an optional feature on large general-purpose computers such as the IBM 3090™. In the 1990s, developers of microprocessors used in desktop computers adapted the concept of vector processing to enhance the capability of their microprocessors when running desktop multimedia applications. These

14

IBM BladeCenter JS21: The POWER of Blade Innovation

capabilities were usually referred to as SIMD extensions and operated on short vectors. Examples of SIMD extensions in widespread use today include: Intel Multimedia Extensions (MMX™) Intel Streaming SIMD Extensions (SSE) AMD 3DNow! Motorola AltiVec The SIMD extensions in microprocessors that are used in desktop computers operate on short vectors of length 2, 4, 8, or 16. This is in contrast to the classic vector supercomputers that can often exploit long vectors of length 64 or more.

AltiVec extensions to PowerPC Architecture
The AltiVec extensions to PowerPC Architecture add a vector processor (VXU) to the PowerPC logical processing model. This is illustrated in Figure 2-4.
Instructions from Storage

Branch Processing (BPU)

Fixed-Point Instructions

Floating-Point Instructions

Vector Instructions

Fixed-Point Processing (FXU)

FloatingPoint Processing (FPU)

Vector Processing (VXU)

Data to/from Storage

Storage

Figure 2-4 PowerPC with AltiVec logical processing model

The VXU operates on vectors that are a total of 128 bits long. These can be interpreted by the VXU as either: A vector of sixteen 8-bit bytes A vector of eight 16-bit half words A vector of four 32-bit words

Chapter 2. BladeCenter JS21: Hardware components

15

The AltiVec extensions to PowerPC Architecture define 32 vector registers that form the vector register file (VRF). The VRF is architecturally distinct from the standard PowerPC floating-point registers (FPRs) and general purpose registers (GPRs). The AltiVec extensions to PowerPC also define two additional registers: The AltiVec status and control register, which is used to control the operation of the VXU and report the status of some AltiVec operations The VRSAVE register, which can be used to assist the operating system save state across context switches by providing a mechanism for software to indicate what vector registers are in use Figure 2-5 illustrates the additional registers provided by AltiVec.

VRSAVE
0 31

VRSAVE Register

VSCR
0 31

Vector Status and Control Register

Vector Register File (VRF) VR0 VR1 ... VR31
0 127

Figure 2-5 AltiVec registers

The AltiVec extensions to PowerPC Architecture define new instructions that use the VXU to manipulate vectors stored in the VRF. These instructions fall into the following categories: Vector integer arithmetic instructions (on 8-bit, 16-bit, or 32-bit integers) Vector floating-point arithmetic instructions (32-bit only) Vector load and store instructions Vector permutation and formatting instructions Processor control instructions used to read and write from the AltiVec status and control register Memory control instructions used to manage caches

16

IBM BladeCenter JS21: The POWER of Blade Innovation

For additional information about the topics presented in this chapter, we recommend that you read the PowerPC Microprocessor Family: AltiVec Technology Programming Environments Manual. You can find this manual on the Web at: http://www-306.ibm.com/chips/techlib/techlib.nsf/techdocs/FBFA164F82437 0F987256D6A006F424D For technical details about AltiVec, visit one of the following Web sites: Unrolling AltiVec, Part 1: Introducing the PowerPC SIMD unit http://www.ibm.com/developerworks/power/library/pa-unrollav1/ High-Performance Processors: Altivec Technology http://www.freescale.com/files/32bit/doc/fact_sheet/ALTIVECFACT.pdf AltiVec - Wikipedia http://en.wikipedia.org/wiki/AltiVec

2.7 Memory features
The BladeCenter JS21 supports four DIMM slots with two-way interleaving for pairs of 400 MHz or 533 MHz DDR2 SDRAM DIMMs including ECC and Chipkill. The available DIMM sizes range from 512 MB to 4 GB. The minimum memory size on the BladeCenter JS21 is 1 GB and the maximum memory size is 16 GB. With two-way interleaving, memory must be populated two DIMMs at a time, and the minimum requirement is two DIMMS. The 2-core BladeCenter JS21 ships with 1 GB main memory (two PC2-3200 512 MB DIMMs). The 4-core BladeCenter JS21 ships with 2 GB main memory (two PC2-3200 1 GB DIMMs).

Chapter 2. BladeCenter JS21: Hardware components

17

Memory DIMMs
Table 2-6 lists the supported memory on the BladeCenter JS21.
Table 2-6 JS21 support memory DIMM types DIMM size PC2-3200 (400 MHz) IBM PN 512 MB 1 GB 2 GB 4 GB 38L5914 38L5915 38L5916 38L5918 Option PN 73P2865 39M5809 39M5812 41Y2703 FRU PN 73P2869 39M5808 39M5811 41Y2702 PC2-4200 (533 MHz) IBM PN 38L5919 38L5920 38L5921 38L5922 Option PN 41Y2707 41Y2711 41Y2715 41Y2723 FRU PN 41Y2708 41Y2710 41Y2714 41Y2722

2.8 Internal Serial-Attached SCSI disk
The BladeCenter JS21 provides support for up to two internal hard disk drives.

Hard disk drives
The base BladeCenter JS21 does not ship with any hard disk drives. You can order up to two of the 2.5-inch SFF SAS hard disk drive options. The BladeCenter JS21 can have up to two 73 GB SAS hard disk drives for a maximum of 146 GB of internal storage or up to two of the 36 GB SAS hard disk drives. The BladeCenter JS21 also supports Redundant Array of Independent Disks (RAID 0 or 10) mirroring standard. You do not have to install a hard disk drive if you have installed the QLogic 4Gb SFF Fibre Channel expansion card and configured the BladeCenter JS21 to boot from storage area network (SAN). Restriction: Certain input/output (I/O) expansion cards that are installed might preclude the attachment of the second SAS hard disk drive unless the I/O expansion card is an SFF card. In this case, both the SFF I/O expansion card and the second SAS HDD can be installed at the same time.

18

IBM BladeCenter JS21: The POWER of Blade Innovation

2.9 BladeCenter chassis
The core component of the BladeCenter infrastructure is the BladeCenter chassis. Each BladeCenter chassis occupies seven rack units. With an additional two rack units to accommodate high-speed switches, the BladeCenter H chassis occupies nine rack units. You can install up to six BladeCenter or up to four BladeCenter H chassis in a single 42U rack. The blade server processor operating frequency depends on the BladeCenter chassis on which it is installed, as shown in Table 2-7.
Table 2-7 Processor operating frequency based on the BladeCenter Chassis and machine type BladeCenter (8677) BladeCenter H (8852) BladeCenter T (8720) JS21 8844-31X 2.6 GHz 2.7 GHz 2.6 GHz JS21 8844-51X 2.3 GHz 2.5 GHz 2.3 GHz

BladeCenter chassis (8677)
This BladeCenter is a 7U rack mountable chassis that contains bays for up to 14 blade servers, four power modules, two switch modules, and two management modules. Figure 2-6 shows the front view of the BladeCenter.

Figure 2-6 BladeCenter front view

Chapter 2. BladeCenter JS21: Hardware components

19

Figure 2-7 shows a rear view of the BladeCenter.

Figure 2-7 BladeCenter rear view

This BladeCenter supports the following power-module options: IBM BladeCenter 1200 watt Power Supply Module (part number 48P7052) IBM BladeCenter 1200 watt to 1400 watt Power Supply Upgrade Kit (part number 90P0197) IBM BladeCenter 1800 watt Power Supply Module (part number 13N0570) IBM BladeCenter 2000 watt Power Supply Module (part number 26K4816) If the existing power modules are replaced with the 2000 watt power modules, you must upgrade the management module firmware. If two management modules are installed in the BladeCenter chassis, upgrade both management modules to the same level of firmware.

20

IBM BladeCenter JS21: The POWER of Blade Innovation

If a BladeCenter JS21 is being installed in the 8677-1xx/2xx BladeCenter, the power module upgrade guidelines contain a table identifying the power load factor of all possible blades. You can find the guidelines at the following Web address: http://www.ibm.com/pc/support/site.wss/document.do?sitestyle=ibm&lndoci d=MIGR-53353

BladeCenter H (8852)
This BladeCenter H is a 9U rack mountable chassis that contains bays for up to 14 blade servers, four power modules, four switch modules, four high-speed switch modules, four high-speed bridge modules, and two management modules. The BladeCenter H requires 2900 watt hot-swap redundant power supply modules. Figure 2-8 shows the front view of this chassis.

Figure 2-8 Front view of BladeCenter H

Chapter 2. BladeCenter JS21: Hardware components

21

Figure 2-9 shows the rear view of this chassis.

Figure 2-9 Rear view of BladeCenter H

22

IBM BladeCenter JS21: The POWER of Blade Innovation

Figure 2-10 shows a rear view drawing of BladeCenter H.

Figure 2-10 Rear view drawing of BladeCenter H

BladeCenter chassis T (8720)
This BladeCenter chassis is an 8U rack mountable chassis that meets the special requirements of the telecommunications industry and is tested for Network Equipment Building Standards (NEBS) compliance. With its low-profile handles, the BladeCenter JS21 is intended to be the first blade to be supported in all three chassis in the BladeCenter family. The BladeCenter T chassis requires 1300 watt hot-swap redundant power supply modules.

2.9.1 BladeCenter power modules
There are multiple versions of power supplies available. The BladeCenter JS21 supports the power/thermal management architecture of BladeCenter, power oversubscription is an option. There is no strict dependency of BladeCenter JS21 on the size or capacity of a particular power supply.

Chapter 2. BladeCenter JS21: Hardware components

23

The standard redundant power supplies are installed in power bays one and two of the BladeCenter. They provide power to the first six blade server bays. To install blade servers in the remaining bays, seven through 14, you must install an additional pair of redundant power supply modules in power bays three and four. The standard redundant power supplies are installed in power bays one and three of the BladeCenter H. They provide power to the first seven blade server bays. To install blade servers in the remaining bays, eight through 14, you must install an additional pair of redundant power supply modules in power bays two and four.

2.9.2 BladeCenter management modules
The BladeCenter chassis contains two (redundant) management modules that provide the manageability interface for the BladeCenter chassis. The management module communicates with the BladeCenter JS21 within the BladeCenter using an RS-485 SAS HDD inter-management network. The BladeCenter JS21 supports both the standard IBM BladeCenter management module and the advanced management module.

2.9.3 Media features
The BladeCenter chassis contains a media bay that contains one compact-disc read-only memory (CD-ROM) drive or digital video-disc read-only memory (DVD-ROM), a diskette drive, and a Universal Serial Bus (USB) port that you can dynamically assign to any single BladeCenter JS21 in the BladeCenter chassis. If your BladeCenter chassis was shipped before June 2003, an update to the interface card on the media tray might be required for proper CD-ROM operation with the BladeCenter JS21. To determine the part number of your existing media tray, from the management module Web interface, under the heading Monitors in the left column, select Hardware vital product data (VPD), and then look at the module name media tray. If the field-replaceable units (FRU) number of the media tray is 59P6629, call your hardware support center and request a free replacement media tray. Restriction: Diskette drives are not supported as boot devices on the BladeCenter JS21.

24

IBM BladeCenter JS21: The POWER of Blade Innovation

3

Chapter 3.

BladeCenter JS21: Software environment
This chapter discusses the software environments that are available for the IBM BladeCenter JS21. It covers the operating systems and Virtual Input/Output Server (VIOS) supported on the BladeCenter JS21, as well as the system management tools.

© Copyright IBM Corp. 2006. All rights reserved.

25

3.1 Operating system support
The BladeCenter JS21 is supported by the AIX 5L, Red Hat Enterprise Linux, and SUSE Linux Enterprise Server (SLES) operating systems. Specific version details are listed in the following sections. Both AIX 5L and Linux operating systems work with a 64-bit kernel and 64-bit/32-bit programs. Some of the included programs were compiled for 64-bit when the code fully exploited the 64-bit address space.

3.1.1 AIX 5L
AIX has evolved from its beginnings on the IBM RT to become the operating system of choice for the largest UNIX servers of IBM. AIX is an enterprise operating system that scales from workstations all the way up to massively parallel supercomputers. Certified as C2 security-compliant by the U.S. government, AIX also supports industry-standard security features such as Pluggable Authentication Modules (PAM), authentication by x.509 certificates, OpenSSH, Kerberos, and Lightweight Directory Access Protocol (LDAP). AIX fully exploits the features of the JS21 with its 64-bit kernel and support for over 16 terabytes (TB) of disk space. With AIX 5L installed, the JS21 can run thousands of software titles that were written for the AIX 5L platform, taking full advantage of the rich capabilities of AIX 5L. At the time of writing of this book, AIX 5L 5.2M and AIX 5L 5.3E support the BladeCenter JS21.

3.1.2 Red Hat Enterprise Linux
Red Hat has made its distribution of Linux available since 1994. Originally, Red Hat offered its distribution for free download, and sold support contracts. In 2002, Red Hat began marketing Red Hat Enterprise Linux. Unlike the original Red Hat distribution, Red Hat Enterprise Linux was available only with a maintenance contract from Red Hat. Red Hat Enterprise Linux runs on x86_64, i386™, ia64, ppc/64, s390, and s390x platforms. By early 2003, Red Hat stopped developing its non-enterprise distribution and focused its efforts on Red Hat Enterprise Linux. Red Hat elected to turn the development of its free software over to the community. This free community-supported distribution became known as Fedora. Red Hat Enterprise Linux AS 4 - Update 3 and later is supported on the JS21.

26

IBM BladeCenter JS21: The POWER of Blade Innovation

3.1.3 SUSE Linux Enterprise Server
SUSE has been a Linux distributor since 1992 and was acquired by Novell in early 2004. Originally, SUSE provided a free downloadable distribution, for which support contracts could be purchased as required. Eventually, their product line was split. Novell now markets SUSE Linux Desktop, as well as SUSE Linux Enterprise Server (SLES). SUSE Linux Desktop is free to download, but SLES is available only with the purchase of a maintenance contract. The desktop product is also available only for x86_64 and IA32, but SLES supports IA32, ia64, ppc/64, s390, s390x, and x86_64. SLES Version 9 Update 2 and Update 3 are supported on the BladeCenter JS21. SLES9, the latest version at the time of writing, ships with the 2.6 Linux kernel. This brings significant gains to performance and scalability. SLES is AltiVec savvy and has versions of GNU C Compiler (gcc) that allow manual exploitation of the vector engine.

3.2 Virtual input/output Server
The Integrated Virtualization Manager (IVM) is a component included with the VIOS Version 1.2, which is part of the Advanced Performance Optimization with Enhanced RISC (POWER) Virtualization hardware feature. It enables companies to consolidate multiple partitions onto a single server in a cost-effective way. With its intuitive, browser-based interface, the IVM is easy to use. It significantly reduces the time and effort required to manage virtual devices and partitions. Restriction: The BladeCenter JS21 requires version 1.2.1 or later of the VIOS for logical partitions (LPARs).

Integrated Virtualization Manager
To ease virtualization technology adoption in a server environment, IBM has developed IVM, a simplified hardware management solution that inherits most of the Hardware Management Console (HMC) features. It is limited to manage a single server, avoiding the need for an independent personal computer. It is designed to provide a solution that enables the administrator to reduce system setup time and to make hardware management even easier, at a lower cost. IVM provides a simple management model for a single system such as a JS21. Although it does not provide all of the HMC capabilities, it enables the exploitation of the IBM Virtualization Engine™ technology.

Chapter 3. BladeCenter JS21: Software environment

27

IVM is an enhancement of the VIOS, the product that enables I/O virtualization in IBM POWER5™ processor-based systems. It enables management of VIOS functions and uses a Web-based graphical interface that enables the administrator to remotely manage the server with a browser. The Hypertext Transfer Protocol Secure (HTTPS) protocol and server login with password authentication provide the security required in an enterprise. Because one of the goals of IVM is simplification of management, some implicit rules apply to the configuration and setup: When a system is designated to be managed by IVM, do not partition it. The first operating system that you install must be the VIOS. The VIOS is automatically configured to own all of the I/O resources and it can be configured to provide service to other LPARs through its virtualization capabilities. Therefore, all other LPARs cannot own any physical adapters. They must access disk, network, and optical devices only through the VIOS as virtual devices. Otherwise, the LPARs operate as they have previously with respect to processor and memory resources. Figure 3-1 shows a sample configuration using IVM. The VIOS owns all the physical adapters, and the other two partitions are configured to use only virtual devices. The administrator can use a browser to connect to IVM to set up the system configuration.

LPAR #2 Virtual adapters

Administrator’s browser

LPAR #1

VIOS + IVM

Physical adapters

Corporate LAN

Figure 3-1 Integrated Virtualization Manager configuration

28

IBM BladeCenter JS21: The POWER of Blade Innovation

A specific device called the Virtual Management Channel (VMC) has been developed on the VIOS to enable a direct Hypervisor configuration without requiring additional network connections. This device is activated, by default, when the VIOS is installed as the first partition. The VMC allows IVM to provide basic logical partitioning functions: Logical partitioning configuration Boot, start, and stop actions for individual partitions Display of partition status Management of virtual Ethernet Management of virtual storage Basic system management Because IVM runs on an LPAR, it has limited service-based functions, and the BladeCenter management module must be used. For example, a system power on must be performed by physically pushing the system power on button or remotely accessing the BladeCenter management module, because IVM does not run while the power is off. The BladeCenter management module and IVM together provide a simple but effective solution for a single partitioned server. LPAR management using IVM is through a common Web interface developed for basic administration tasks. Being integrated within the VIOS code, IVM also handles all virtualization tasks that normally require VIOS commands to be run. A major difference in IVM LPAR handling compared to HMC is dynamic partitioning. The IVM does not use the RMC protocol that enables hardware management to interact with the operating system’s resource allocation. Although IVM is capable of performing a dynamic LPAR operation on itself (the VIOS), changes on other LPARs are not applied until the LPAR is restarted.

Integrated Virtualization Manager design
All IBM System p5™ and IBM eServer™ p5 systems have the capability of being partitioned because they are all pre-loaded with all the necessary firmware support for a partitioned environment. On some systems, IBM Micro-Partitioning™ is a priced option, but on high-end systems, such as the IBM System p5 590 and p5-595, it is a standard element. It is also a standard element on the IBM BladeCenter JS21.

Chapter 3. BladeCenter JS21: Software environment

29

Because the partitioning schema is designed by the client, every system is set up by manufacturing in the same manufacturing default configuration that can be changed or reset to when required. When configured using the manufacturing default configuration, the system has the following setup from a partitioning point of view: There is a single predefined partition. All hardware resources are assigned to the single partition. The partition has system service authority, therefore, it can update the firmware. The partition is auto-started at power on. Standard operating system installation methods apply for the partition (network based or media based). The system’s physical control panel is mapped to the partition, displaying its operating system messages and error codes. Base platform management functions, such as power control, are provided through integrated system control functions (for example, BladeCenter management module and control panel). The manufacturing default configuration allows the system to be used immediately as a stand-alone server with all resources allocated to a single LPAR. If the VIOS is the first LPAR installed, the system configuration can be changed to make the Hypervisor manage multiple LPARs.

Integrated Virtualization Manager architecture
The IVM has been developed to provide a simple environment where a single control program has the ownership of the physical hardware and other logical partitions use it to access resources. The VIOS has most of the required features because it can provide virtual SCSI and virtual networking capability. Starting with Version 1.2, the VIOS has been enhanced to provide management features using the IVM. To set up logical partitions, the IVM requires management access to the Hypervisor. It has no service processor connection used by the HMC and it relies on a new virtual I/O device type called the VMC. This device is only activated when the VIOS installation detects that the environment has to be managed by IVM.

30

IBM BladeCenter JS21: The POWER of Blade Innovation

On a VIOS partition with IVM activated, a new ibmvmc0 virtual device is present and a management Web server is started listening to Hypertext Transfer Protocol (HTTP) port 80 and to HTTPS port 443. The presence of the virtual device can be detected using the lsdev -virtual command, as shown in Example 3-1.
Example 3-1 Virtual Management Channel device

$ lsdev -virtual | grep ibmvmc0 ibmvmc0 Available Virtual Management Channel Because IVM relies on VMC to set up logical partitioning, it can manage only the system on which it is installed. For each IVM managed system, the administrator must open an independent Web browser session. Figure 3-2 provides the schema of the IVM architecture. The primary user interface is a Web browser that connects to port 80 of the VIOS. The Web server provides a simple graphical user interface (GUI) and runs commands using the same command-line interface (CLI) that can be used to log on to the VIOS. One set of commands provides LPAR management through the VMC, and a second set controls VIOS virtualization capabilities.
Terminal emulator Web browser

VIOS
Web server Command line CGI/HTML

LPAR 1

LPAR 2

IVM LPAR CLI Virtual I/O CLI

I/O subsystem VMC Hypervisor
Figure 3-2 IVM high-level design

Service processor

Chapter 3. BladeCenter JS21: Software environment

31

LPARs in an IVM managed system are isolated exactly as before and cannot interact except using the virtual devices. Only the IVM has been enabled to perform limited actions on the other LPARs such as: Activate and deactivate Send a power off (EPOW) signal to the operating system Create and delete View and change configuration

3.3 System management tools
IBM has devised a strategy for systems management called the Universal Manageability Initiative. There are several technologies that have been incorporated under this umbrella. They all work toward the same goal: simple, effective solutions for managing heterogeneous environments. One of the main building blocks in this strategy is the use of simple yet powerful software to oversee the management of many different systems. This section investigates some of the possible options that are available for managing such systems. It reviews the BladeCenter Web interfaces, IBM Director, and Cluster Systems Management (CSM).

3.3.1 BladeCenter Web interfaces
The BladeCenter Web interface allows system administrators to easily and effectively manage up to 14 blades from an integrated interface. From trivial tasks such as powering blades on or off, to more complex tasks such as firmware management, the Web interface allows powerful control over all blades and I/O modules that are attached to the BladeCenter chassis. Management of other BladeCenter resources, such as I/O modules, is also possible from here, as well as the retrieval of system health information. BladeCenter-specific features are also configured from here such as the Serial over LAN (SoL). For more information, see IBM eServer BladeCenter Systems Management, REDP-3582.

32

IBM BladeCenter JS21: The POWER of Blade Innovation

3.3.2 IBM Director
IBM Director assists with the remote management of many IBM and non-IBM machines including the BladeCenter JS21. The IBM Director console allows system administrators to manage multiple BladeCenter chassis from a common interface. It is the ideal solution for administering BladeCenter chassis in heterogeneous environments or environments where a Director infrastructure exists. From the IBM Director console, system administrators can monitor resource utilization. You can use this key feature for performance and capacity planning, and to alert support staff of critical errors such as hardware failure. IBM Director also allows the remote management of software. You can remotely reconcile and install software from the console interface. This can be useful in large environments for resource-intensive tasks such as patch management. By using the Software Distribution Premium Edition, you can remotely install patches to several servers at the same time with the click of a button. You can learn more about IBM Director in 10.2, “IBM Director” on page 346.

3.3.3 IBM Cluster Systems Management
IBM CSM provides many useful functions to manage a cluster from a single point of control. These include resource monitoring, automated monitoring and operation, remote hardware control, remote command execution, security, configuration file management, parallel network installation, and diagnostics. By consolidating these capabilities, CSM helps to increase efficiency of administrators’ time and reduce their expenses. It helps administrators install their clusters rapidly by automating many configuration tasks and by leveraging existing open source products. CSM also provides efficient monitoring of cluster resources without overwhelming network bandwidth. The automated error detection that CSM provides helps catch problems before they impact the environment, and assists with rapid resolution and recovery of problems that occur. CSM’s architecture and modular construction maximizes flexibility so that cluster solutions can evolve and grow as requirements change. CSM is a collection of components that have been integrated to provide the basis to construct and manage a cluster. Each of these components provides specific capabilities related to the management of the cluster. This component-based architecture provides flexibility for future expansion of the capabilities provided by CSM. Each of the CSM components can be easily personalized to help meet specific needs. For example, a cluster administrator can set up monitoring of

Chapter 3. BladeCenter JS21: Software environment

33

application processes and take action if these processes end or stop abnormally. In short, CSM has been designed for large-scale cluster environments that require simple control over multiple homonymous servers. We provide more details about CSM in 10.3, “Cluster Systems Management” on page 362.

3.3.4 IBM middleware
The IBM middleware portfolio is enabled to run on the JS21 with either AIX 5L or Linux as the operating system. For more information, refer to the following Web site: http://www.ibm.com/software/os/linux/software

34

IBM BladeCenter JS21: The POWER of Blade Innovation

4

Chapter 4.

BladeCenter JS21: Virtualization
Logical partitions (LPARs) and virtualization increase the utilization of system resources. This chapter provides details and configuration specifications about this topic. The virtualization discussion includes virtualization enabling technologies that are standard on the system, such as the Performance Optimization with Enhanced RISC (POWER) Hypervisor, and optional ones, such as the Virtual Input/Output Server (VIOS) feature. The BladeCenter JS21 has native virtualization features that are enabled by ordering and installing the no-charge VIOS. Without the VIOS installed, the BladeCenter JS21 is a 2-core or 4-core symmetric multiprocessor (SMP) server.

© Copyright IBM Corp. 2006. All rights reserved.

35

4.1 Virtualization
With the introduction of more advanced POWER processors, partitioning technology moved from a dedicated resource allocation model to a virtualized shared resource model. This partitioning technology called Advanced POWER Virtualization, the same virtualization capability supported on the System p5 servers, is built into the PowerPC 970MP processor, making the JS21 the first IBM blade server with native virtualization. As an added incentive, a client can obtain a license for VIOS to activate all the virtualization features of the JS21 at no additional cost more than the cost of the blade (mandatory software maintenance agreement not included). This section briefly discusses the key components of virtualization on BladeCenter JS21 servers. For more information about virtualization, see the following Web site: http://www.ibm.com/servers/eserver/about/virtualization/

POWER Hypervisor
Combined with features designed into the PowerPC 970MP, the IBM POWER Hypervisor™ delivers functions that enable other system technologies, including Micro-Partitioning, virtualized processors Institute of Electrical and Electronics Engineers (IEEE) virtual local area network (VLAN), compatible virtual switch, virtual Small Computer System Interface (SCSI) adapters, and virtual consoles. The POWER Hypervisor is a component of system firmware that is always active, regardless of the system configuration. Therefore, it requires no separate license apart from the VIOS for setup and usage. The POWER Hypervisor provides the following functions: Provides an abstraction layer between the physical hardware resources and the logical partitions using them Enforces partition integrity by providing a security layer between logical partitions Controls the dispatch of virtual processors to physical processors Saves and restores all processor state information during logical processor context switch Controls hardware I/O interrupt management facilities for logical partitions Provides VLAN channels between logical partitions, which removes the need for physical Ethernet adapters for inter-partition communication

36

IBM BladeCenter JS21: The POWER of Blade Innovation

The POWER Hypervisor supports the following three types of virtual I/O adapters.

Virtual Small Computer System Interface
The BladeCenter JS21 blade server uses SCSI as the mechanism for virtual storage devices. This is accomplished using two paired adapters: a virtual SCSI server adapter and a virtual SCSI client adapter. Virtual SCSI is available with the Advanced POWER Virtualization and optional VIOS feature, which we describe in 4.1.1, “Advanced POWER Virtualization” on page 38.

Virtual Ethernet
The POWER Hypervisor provides a virtual Ethernet switch function that allows partitions on the same server as a means for fast and secure communication. Virtual Ethernet working on LAN technology allows a transmission speed in the range of 1 GBps to 3 GBps depending on the maximum transmission unit (MTU) size. Virtual Ethernet requires a BladeCenter JS21 running either AIX 5L Version 5.3 or the level of Linux supporting virtual Ethernet devices. Virtual Ethernet is part of the base system configuration. Virtual Ethernet has the following features: A partition supports two virtual Ethernet connections. Each virtual Ethernet connection can be connected to one of four VLANs. The VLAN provides connectivity to other virtual Ethernet connections on client LPARs or the VIOS. Each partition operating system detects the VLAN switch as an Ethernet adapter, without the physical link properties and asynchronous data transmit operations. Layer-2 bridging to a physical Ethernet adapter is also included in the virtual Ethernet features on the VIOS LPAR. Note: Virtual Ethernet is based on the IEEE 802.1Q VLAN standard. No physical I/O adapter is required when creating a VLAN connection between partitions, and no access to an outside network is required. The Integrated Virtualization Manager (IVM) simplifies the virtual Ethernet configuration by controlling the VLAN numbers. Simplifying this for the user results in restrictions where you do not have access to other IEEE 802.1Q capabilities such as assigning one virtual Ethernet interface to multiple VLANs.

Chapter 4. BladeCenter JS21: Virtualization

37

Virtual teletypewriter console
Each partition requires access to a system console. Tasks such as operating system installation, network setup, and some problem analysis activities require a dedicated system console. The POWER Hypervisor provides the virtual console using a virtual teletypewriter (TTY) or serial adapter and a set of Hypervisor calls to operate on them. The operating system console is provided by a console session with the IVM. Note: The POWER Hypervisor is active when the server is running in partition and non-partition mode. Consider the Hypervisor memory requirements when planning the amount of system memory required. In AIX 5L V5.3, use the lshwres command to view the memory usage. lshwres -r mem --level sys -F sys_firmware_mem You can also determine this using the console of the IVM: View/Modify Partitions → System Overview → Reserved Firmware Memory.

4.1.1 Advanced POWER Virtualization
The VIOS Version 1.2.1 feature is an optional feature available at no additional cost over the price of the base blade. Although included in the base price, the client must still obtain a VIOS license for each JS21. This feature enables the implementation of logical partitions on BladeCenter JS21 servers. The virtual feature includes installation image for the VIOS software that supports the following features: Ethernet adapter sharing Virtual SCSI Server VIO software ships on a DVD Software support of: – AIX 5L V5.3 – SUSE Linux Enterprise Server 9 (SLES9) for POWER (Service Pack 3 or later) – Red Hat Enterprise Linux AS 4 for POWER Partition management using IVM (VIOS V1.2.1 or later) For details about Advanced POWER Virtualization and virtualization in general, visit the following Web site: http://www.ibm.com/servers/eserver/pseries/ondemand/ve/resources.html

38

IBM BladeCenter JS21: The POWER of Blade Innovation

Micro-Partitioning technology
The concept of Micro-Partitioning allows the resource definition of a partition to allocate fractions of processors to the partition. Micro-Partitioning is only available with POWER5 systems and the BladeCenter JS21. From an operating system perspective, a virtual processor is indistinguishable from a physical processor unless the operating system is enhanced to be aware of the difference. The firmware on the BladeCenter JS21 (host firmware) virtualizes the physical central processing units (CPUs). The host firmware layer presents logical CPU numbers (0 - 3) to the operating systems. A partition can be defined with a processor capacity as small as 10 processor units. This represents one-tenth of a physical processor. The shared processor partitions are dispatched and time-sliced on the physical processors under control of the POWER Hypervisor. The shared processor partitions are created and managed by the Integrated Virtualization Management (included with VIOS Version software 1.2.1 or later). Dedicated and micro-partitioned processors can coexist on the same BladeCenter JS21 blade server if they are available. Table 4-1 lists processor partitioning information related to the BladeCenter JS21.
Table 4-1 Processor partitioning overview of the BladeCenter JS21 Partitioning implementation Cores (maximum configuration) Dedicated processor partitions (maximum configuration) Shared processor partitions (maximum configuration) JS21 4 4 40

The maximum values stated in Table 4-1 are supported by the hardware. However, the practical limits based on production workload demands and application utilization might be significantly lower.

Virtual I/O Server
The VIOS is a special purpose partition that provides virtual I/O resources to client partitions. The VIOS owns the real resources that are shared with the other LPARs. The virtual I/O technology allows one or more partitions to share a physical adapter assigned to a partition. This enables you to minimize the number of physical adapters. The VIOS eliminates the requirement that every partition must own a dedicated network adapter, disk adapter, and disk drive.

Chapter 4. BladeCenter JS21: Virtualization

39

Figure 4-1 shows an organization view of Micro-Partitioning, including the VIOS. It also includes virtual SCSI and Ethernet connections and mixed operating system partitions.

Figure 4-1 Virtual partition: Organization view

Because the VIOS is an AIX 5L V5.3 operating system-based appliance, you can provide redundancy for physical devices attached to the VIOS by using capabilities such as Multipath I/O and IEEE 802.3ad Link Aggregation. Install the VIOS partition from a special bootable DVD that is provided when you order the VIOS. This dedicated software is only for the VIOS operations, therefore, the VIOS software is only supported in VIOS partitions. You can install the VIOS in the following ways: Media (assigning the DVD-ROM drive to the partition and booting from the media) Using the Network Installation Manager (NIM)

40

IBM BladeCenter JS21: The POWER of Blade Innovation

Note: The VIOS supports logical mirroring and Redundant Array of Independent Disks (RAID) configurations. Logical volumes created on RAID or Just a Bunch of Disks (JBOD) configurations are bootable. Additionally, the number of logical volumes is limited to the amount of storage available and the architectural limits of the Logical Volume Manager (LVM). Two major functions are provided with the VIOS: a shared Ethernet adapter and Virtual SCSI. The following sections discuss these functions.

Shared Ethernet Adapter
A Shared Ethernet Adapter is a new service that acts as a layer 2 network switch to route network traffic from a virtual Ethernet to a real network adapter. The shared Ethernet adapter must be assigned to the VIOS partition.

Virtual Small Computer System Interface
Access to real storage devices is implemented through the Virtual SCSI services, a part of the VIOS partition. This is accomplished using a pair of virtual adapters: a virtual SCSI server adapter and a virtual SCSI client adapter. The virtual SCSI server (target) adapter is responsible for issuing any SCSI commands it receives. It is owned by the VIOS partition. The virtual SCSI client adapter allows the client partition to access standard SCSI devices and logical unit numbers (LUNs) that are assigned to the client partition. All current storage device types, such as storage area network (SAN), SCSI, and RAID are supported. iSCSI and Serial Storage Architecture (SSA) are not supported. For more information about the specific storage devices that are supported, visit the following Web site: http://techsupport.services.ibm.com/server/vios/home.html

Integrated Virtualization Manager
To ease the virtualization technology adoption in the BladeCenter JS21 environment, IBM developed IVM, a simplified hardware management solution that inherits most Hardware Management Console (HMC) features, but is limited to managing a single server to avoid the need for a dedicated control workstation. The goal is to provide a simple solution that enables the administrator to reduce system setup time and to make hardware management even simpler. The IVM provides a simple management model for a single system such as a BladeCenter JS21. Although it does not provide the full flexibility of an HMC, it enables the exploitation of the IBM Virtualization Engine technology. The BladeCenter JS21 is ideally suited for management using the IVM. The HMC,

Chapter 4. BladeCenter JS21: Virtualization

41

which is designed to manage IBM System p servers with POWER5 processors, does not support the BladeCenter JS21 with the PowerPC 970MP processor. IVM is an enhancement of VIOS offered as part of VIOS V1.2.1, which is the product that enables I/O virtualization in certain IBM Power Architecture™ systems. It provides the same VIOS features plus a Web-based graphical interface that enables the administrator to remotely manage the BladeCenter JS21 server with an Internet browser. The IVM provides the following basic logical partitioning functions: Logical partitioning configuration Boot, start, and stop actions for individual partitions Display of partition status Management of virtual Ethernet Management of virtual storage Basic system management The requirements for an IVM managed server are as follows: Install the IVM (with VIOS) as the first operating system. An IVM partition requires a minimum of one virtual processor and 512 MB of random access memory (RAM). The major differences in IVM in comparison to an HMC-managed system are as follows: All adapters are owned by IVM, and LPARs use virtual devices only. No dynamic resource changes are allowed. There are four virtual Ethernet networks available inside the system. Each LPAR can have a maximum of one virtual SCSI adapter assigned. It is not possible to have redundant VIOSs because the IVM manages all I/O. Despite these limitations, IVM provides advanced virtualization functionality without the need for an extra-cost workstation. For more information about IVM functionality and best practices, see Virtual I/O Server Integrated Virtualization Manager, REDP-4061.

42

IBM BladeCenter JS21: The POWER of Blade Innovation

4.1.2 Dynamic logical partitioning
The BladeCenter JS21 does not support dynamic logical partitioning (DLPAR) on the client partitions. The BladeCenter JS21 uses the IVM of the VIOS to manage the LPAR configuration. IVM does not support dynamic movement of system resources across client partitions that are operational. You can reconfigure resources on client LPARs without recycling the whole server. You can use IVM to reconfigure resources across client LPARs if the LPARs are stopped. This dynamic movement of resources does not affect other LPARs that are running.

Chapter 4. BladeCenter JS21: Virtualization

43

44

IBM BladeCenter JS21: The POWER of Blade Innovation

5

Chapter 5.

Planning considerations
This chapter provides the planning elements that are important for a successful implementation of the BladeCenter JS21 in various environments. At the time of publication, there are new technology levels for AIX Version 5.2 and Version 5.3 and new Linux distributions that support the JS21 BladeCenter. In addition, there are new updates for the IBM Director and Cluster Systems Management (CSM) that support the BladeCenter JS21. The details about versions and releases are provided in this chapter and in the following chapters.

© Copyright IBM Corp. 2006. All rights reserved.

45

5.1 Documentation and planning resources
For the IBM Account team: If the IBM BladeCenter is new to your client, you might want to review the IBM BladeCenter Solution Assurance Planning Review Guide, Document SA749, before the ordering and delivery of the client’s system. This document is not specific to the JS21 BladeCenter, but does provide valuable guidance on what you have to consider for the BladeCenter chassis, blade server installation, or both these installations, software installation, networking, storage and site planning. For clients: The BladeCenter JS21 ships with a documentation CD that provides multiple installation, user and setup guides that are integral to your efficient implementation of the BladeCenter. Current updates to these documents are available online for your review and planning considerations. Visit the following Web site: http://www.ibm.com/pc/support/site.wss From this site, select Servers as your product followed by BladeCenter or BladeCenter H as the family.

5.2 Network planning
For a successful installation of the BladeCenter JS21, you must have a clear plan on how you plan to use the various networking capabilities of the BladeCenter infrastructure. This plan must address the following questions: What network connectivity is required for the blade servers to support the applications installed on them? What network connectivity is necessary to manage the BladeCenter, input/output (I/O) modules, and blade servers? What virtual local area networks (VLANs) are required for each local area network (LAN) switch I/O module to provide the network connectivity established previously? What Internet Protocol (IP) subnet is used for each VLAN and how are IP addresses allocated to each device on the VLAN? Are the IP addresses assigned statically or dynamically using Dynamic Host Configuration Protocol (DHCP)? What host names are used for each network interface?

46

IBM BladeCenter JS21: The POWER of Blade Innovation

How are host names resolved to IP addresses? Are there requirements for a high-performance, low-latency interconnection network? Where multiple BladeCenter chassis are installed, how are they be interconnected? The following sections explore common network planning requirements that we expect to arise in the installation of BladeCenter JS21s.

5.2.1 Minimal network requirements
At a minimum, most BladeCenter JS21 environments have the following network requirements: A dedicated hardware management subnet: It is used to access both the management module and management interfaces of I/O modules that are installed in each BladeCenter chassis. A Serial over LAN (SoL) subnet internal to each BladeCenter chassis that supports the SoL remote text console function: This is always implemented using a LAN switch I/O module installed in I/O module bay 1. A subnet connected to each BladeCenter JS21: It is used to install and manage the operating system on the blade server. Where you run AIX 5L on some blade servers, and Linux on other blade servers, you might have to use multiple subnets for this purpose. It is possible to configure the installation servers on the same subnet if you use directed Bootstrap Protocol (BOOTP) with the Network Installation Manager (NIM) server. One or more subnets connected to each BladeCenter JS21: They are used by applications installed on the blade server to communicate with other systems.

Chapter 5. Planning considerations

47

Figure 5-1 illustrates how these requirements can be provided in a logical network view.

Hardware Management Subnet

I/O I/O Modules I/O Module I/O Module Module

Management Module

SOL Subnet

Console

Console

Console

Console

Console

Console

JS20 Blade Server
eth0 eth1

JS20 Blade Server
eth0 eth1

JS20 Blade Server
eth0 eth1

JS20 Blade Server
eth0 eth1

JS20 Blade Server
eth0 eth1

JS20 Blade Server
eth0 eth1

Application Subnet

OS Management Subnet

Figure 5-1 Network logical view

The following sections describe each of the logical networks illustrated in Figure 5-1.

48

IBM BladeCenter JS21: The POWER of Blade Innovation

Hardware management subnet
We recommend that you establish a dedicated hardware management subnet. The 10/100BaseT Ethernet interface on the management modules installed in each BladeCenter chassis provides the gateway to this subnet for each BladeCenter chassis. You have to install external LAN switches or hubs to interconnect the 10/100BaseT Ethernet interface on the management modules of each BladeCenter chassis with external management systems. You can use this subnet to access the management module Web interface and command-line interface (CLI). You can also use this subnet to access the Web interface and CLI of I/O modules. System management applications such as the IBM Director or CSM also use this subnet to communicate with the hardware management functions of the BladeCenter infrastructure. Restrict access to this subnet to those management systems, system administrators, and operators who have responsibility for managing the BladeCenter infrastructure. You have to allocate multiple IP addresses to each BladeCenter chassis on the hardware management subnet, including: One IP address for the external interface of the management module in each BladeCenter chassis One IP address for the internal interface of the management module in each BladeCenter chassis One IP address for the management interface of each I/O module in each BladeCenter chassis Note: Although the logical network view (illustrated in Figure 5-1 on page 48) shows the I/O management module interfaces connecting directly to the hardware management subnet, they are physically connected using the management module, which acts as gateway to these interfaces. The management module performs a proxy Address Resolution Protocol (ARP) function to make it appear as though the I/O module management interfaces are attached to the hardware management subnet. It is possible to use a different subnet for the management module internal network interface and I/O module management interfaces. However, we do not recommend this configuration. Learn how to configure these IP addresses in 6.2, “BladeCenter internal network” on page 66.

Chapter 5. Planning considerations

49

Serial over LAN subnet
The SoL remote text console function requires a subnet and underlying VLAN that is implemented by a LAN switch I/O module installed in I/O module bay 1 of the BladeCenter chassis. See Figure 2-10 on page 23. This subnet and VLAN are entirely internal to each BladeCenter chassis and must not be externally accessible. If you use the 4-Port Gigabit Ethernet switch module or the Nortel Networks Layer 2-7 Gigabit Ethernet switch module, the VLAN uses VLAN ID 4095. If you use the Cisco Systems Intelligent Gigabit Ethernet switch module, you must configure the VLAN ID to be 4095. You have to assign a unique range of IP addresses to this subnet for use by the SoL remote text console function.You have to specify only the starting IP address within the range of IP addresses that you assign to the management module. The management module then automatically assigns consecutive IP addresses from the starting address that you provide to each blade server that you have installed. Important: One IP address is required for each blade. You can learn how to configure the SoL subnet and VLAN in 6.7.1, “Configuring Serial over LAN” on page 124

Operating system management subnet
We expect most environments that use the BladeCenter JS21 to rely on the network installation procedure to install the operating systems. We discuss this further in Chapter 7, “Installing and managing the Virtual I/O Server” on page 193; Chapter 8, “Installing AIX” on page 259; and Chapter 9, “Installing Linux” on page 281. The operating system management subnet is used to support both the initial installation and subsequent management of the operating systems installed on BladeCenter JS21s. This subnet is implemented using a VLAN provided by the Ethernet Switch I/O modules installed in I/O module bay 2 of each BladeCenter chassis. You might want to install both AIX and Linux operating systems on different BladeCenter JS21s in the same environment. In this case, you might have to set up multiple operating system management subnets and underlying VLANs, one for blade servers running AIX and the other for blade servers running Linux if you are performing network installations that use broadcast packets.

50

IBM BladeCenter JS21: The POWER of Blade Innovation

Application subnet
The primary reason you install a BladeCenter JS21 is to support applications. Many applications have requirements to communicate with other systems. Use one or more application subnets for this purpose. The primary application subnet is implemented using a VLAN provided by the LAN switch I/O modules installed in I/O module bay 2 of each BladeCenter chassis. The LAN switch I/O module installed in I/O module bay 1 is used to support the SoL VLAN. Where different BladeCenter JS21s participate in different applications and there is a requirement to separate application traffic, you might have to define multiple application subnets and VLANs. Each BladeCenter JS21 is connected to the appropriate application subnet based on the applications that are installed on the blade server. If application communication requirements with other systems are complex, you can install an additional pair of Gigabit Ethernet interfaces on each BladeCenter JS21. Do this using the Gigabit Ethernet expansion card in conjunction with compatible I/O modules installed in I/O module bays 3 and 4.

5.2.2 High-performance, low-latency network requirements
Distributed memory parallel applications might require the installation of a high-performance, low-latency interconnection network between BladeCenter JS21s. This requirement can be supported through the use of an optional Myrinet or Infiniband network. To install a Myrinet network, you must install a Myrinet expansion card on each BladeCenter JS21 that requires connectivity to the high-performance, low-latency interconnection network. You must also install an optical pass-thru I/O module in I/O module bay 4 of each BladeCenter chassis that contains blade servers equipped with Myrinet expansion cards. Then connect the optical pass-thru I/O module to external Myrinet switches to complete the Myrinet network infrastructure.

Chapter 5. Planning considerations

51

Figure 5-2 illustrates this procedure.

Blade Server
Myrinet Expansion Card

Blade Server
Myrinet Expansion Card

Blade Server
Myrinet Expansion Card

...

Blade Server
Myrinet Expansion Card

Blade Server
Myrinet Expansion Card

Blade Server
Myrinet Expansion Card

Optical Pass-Thru Module I/O Module Bay 4

Break-out Cables

External Myrinet Switch

Figure 5-2 Myrinet network infrastructure

You can use the Myrinet network infrastructure to support application programming interfaces such as the message passing interface (MPI) that are often used by distributed memory parallel applications. You can define IP addresses for each Myrinet network interface. You can also use the Myrinet network infrastructure to support any application communication based on IP protocols. For example, you can use this capability to support a clustered file system such as IBM General Parallel File System (GPFS). You can define a dedicated IP subnet for use with the Myrinet network infrastructure that is distinct from the other IP subnets that are identified in 5.2.1, “Minimal network requirements” on page 47.

52

IBM BladeCenter JS21: The POWER of Blade Innovation

5.2.3 Multiple BladeCenter environments
Large configurations of BladeCenter JS21s require the installation of multiple BladeCenter chassis. To support the network requirements identified previously, you have to install external LAN switches to interconnect the BladeCenter chassis with each other.

5.3 Operating system support and installation
At the time of publication, the operating systems discussed in the following sections support installation on the JS21.

AIX 5L
The following AIX 5L technology levels support installation on the JS21. AIX 5L for Advanced Performance Optimization with Enhanced RISC V5.2 (POWER V5.2) with the 5200-08 technology level (APAR IY77270), plus APAR IY80493 or a later technology level AIX 5L for POWER V5.3 with the 5300-04 technology level (APAR IY77273), plus APAR IY80499 or a later technology level Advanced POWER Virtualization for AIX 5L V5.3 and Linux environments require Virtual I/O Server (VIOS) V1.2.1 (5765-G34)

Linux
The following Linux distributions support installation on the JS21. SUSE Linux Enterprise Server 9 for POWER Service Pack 3 (SP3), or later Red Hat Enterprise Linux AS 4 for POWER update 3, or later For AltiVec-optimized application development Note: The BladeCenter JS21 is not shipped with an operating system pre-installed. System administrators must acquaint themselves with the installation instructions for their respective operating system. There are several methods to install the JS21 in a BladeCenter. The first is an attended method, using CD-ROM media. The second is a remote method through a network installation. You can accomplish network installation using NIM, Linux installation server, IBM Director, or CSM depending on the operating systems involved. NIM and Linux installation servers are discussed in Chapter 8, “Installing AIX” on page 259 and Chapter 9, “Installing Linux” on page 281.

Chapter 5. Planning considerations

53

Using the CD installation media is similar to other System p systems. You assign the CD-ROM to a blade, follow the installation steps answering any prompts, and then repeat the process on the next blade. With network installation, you can perform several installations at the same time. The method is designed to reduce the installation time required when a large number of blades require operating system installation. We choose this method as the focus for the following sections.

Network installation planning
You can use two approaches to set up a network installation environment for the BladeCenter JS21: Establish one or more network installation servers and manually initiate network installation tasks. Use a systems management tool, such as the IBM CSM or xCAT, which you can use to automate much of the setup of network installation servers and initiation of network installation tasks. The remainder of this section focuses on the planning considerations for the first approach. You can find planning considerations for the second approach in Chapter 7 of Cluster Systems Management for AIX and Linux V1.5 Planning and Installation Guide, SA23-1344. You can find this guide at: http://publib.boulder.ibm.com/infocenter/clresctr/vxrx/index.jsp?topic= /com.ibm.cluster.csm.doc/clusterbooks.html

Required network services
Network installation depends on several different network services: A BOOTP server or DHCP server A Trivial File Transfer Protocol (TFTP) server A Network File System (NFS) server You can use one physical server to provide all three of these network services. If you want to install AIX on BladeCenter JS21s, this server must run AIX. If you want to install Linux on the BladeCenter JS21s, we recommend that you use the same Linux distribution in the network installation server that you are planning to install on the BladeCenter JS21s. In some situations, you might want to install AIX on some BladeCenter JS21s and Linux on other BladeCenter JS21s. Although it is possible to use a single AIX server to do this, we recommend that you establish two network installation servers, one for installing AIX and the other for installing your chosen Linux distribution.

54

IBM BladeCenter JS21: The POWER of Blade Innovation

Important: A potential problem might arise when you have multiple servers acting as BOOTP or DHCP servers. Be careful when planning your network so that they do not interfere with one another. One approach is to place the BladeCenter JS21s running AIX and the AIX network installation server on a different VLAN from the BladeCenter JS21s running Linux and the Linux installation server. If you use this approach, you must also disable the relay of BOOTP or DHCP requests in your network routers. To learn how to set up AIX network installation servers, see 8.3.3, “Configuring the NIM master” on page 263. For installation instructions for specific Linux distributions, see 9.2.1, “Installing Linux using the network: General remarks” on page 284. For instructions about setting up fundamental network installation parameters for Linux, see 9.2, “Basic preparations for a Linux network installation” on page 283.

Preparing for network installation
The BladeCenter JS21 firmware has the capability to boot an operating system over a network using the BOOTP. Use this capability to initiate network installation of the BladeCenter JS21. The BOOTP protocol is a client-server protocol. When initiating a network installation on a BladeCenter JS21, it behaves as a BOOTP client. Therefore, you have to set up a BOOTP server to support initiating a network installation. When you instruct the BladeCenter JS21 firmware to boot over a network using BOOTP, it sends a request to the BOOTP server. The BOOTP server must generate a response that contains the following information: The IP address, subnet mask, and default gateway that the BladeCenter JS21 must use for the network interface that sent the BOOTP request The IP address of a TFTP server that the BladeCenter JS21 must contact, and the name of the file on that server which the BladeCenter JS21 must request, to get the operating system installation boot image The BOOTP protocol has been around since the mid1980s. In many environments, BOOTP has now been replaced by the newer DHCP protocol. DHCP was designed to interoperate with BOOTP, and most DHCP servers can serve BOOTP clients. When you perform network installations of AIX, use the AIX BOOTP server. When you perform network installations of Linux, use a DHCP server that is configured to support BOOTP clients.

Chapter 5. Planning considerations

55

When the BladeCenter JS21 firmware has received a response from a BOOTP server, it contacts the TFTP server specified in the BOOTP response to load the operating system’s installation boot image. The operating system’s installation boot image then starts running on the BladeCenter JS21. Eventually it contacts the NFS server to obtain the files that it requires to perform the operating system installation.

Selecting network interface
The BladeCenter JS21 has two standard Gigabit Ethernet interfaces. The first is connected to I/O module bay 1, and the second is connected to I/O module bay 2. See Figure 2-10 on page 23. Restriction: The BladeCenter JS21 does not support the keyboard, video, and mouse (KVM) console supported by other blade servers at this time. Instead the BladeCenter JS21 supports SoL remote text consoles through the BladeCenter management module. The SoL remote text console function uses the first Gigabit Ethernet interface on each BladeCenter JS21 for communication between the management module and the service processor found in each BladeCenter JS21. This interface is known as eth0 under Linux and en0 under AIX. Under normal conditions, using the Gigabit Ethernet interface by the SoL remote text console function is entirely transparent. It does not impact usage of the Ethernet interface by the operating system running on the BladeCenter JS21. Unfortunately, the initiation of network installation using the BOOTP protocol from the BladeCenter JS21 firmware temporarily disrupts the operation of the SoL remote text console when it occurs on the same physical Gigabit Ethernet interface. This disruption makes it difficult to diagnose problems with the network installation process. Therefore, we recommend that you use the second Gigabit Ethernet interface on each BladeCenter JS21 during network installation of the operating system. The second interface is known as eth1 under Linux and en1 under AIX. To use the second interface, you must have a LAN switch I/O module, or pass-thru I/O module connected to an external LAN switch, installed in I/O module bay 2. You can learn more about why you must use the second Gigabit Ethernet interface for network installation in the IBM RETAIN® tip H181655 on the Web at: http://www.ibm.com/pc/support/site.wss/document.do?lndocid=MIGR-55282

56

IBM BladeCenter JS21: The POWER of Blade Innovation

5.4 Systems management
The BladeCenter JS21 supports a rich set of systems management capabilities. This section discusses planning considerations associated with exploiting some of these systems management capabilities.

5.4.1 BladeCenter Web interface
The BladeCenter Web interface allows the following functions: System administrators can easily and effectively manage up to 14 blade servers from an integrated interface. You can power the BladeCenter JS21 on or off. You can perform firmware management. You can have powerful control over all blade servers and I/O modules that are attached to the BladeCenter chassis even with a mixed environment. System Management Services (SMS) menus are supported on the BladeCenter JS21. IBM only supports the Linux systems of clients with a SupportLine contract covering Linux. Otherwise, contact the Linux distributor for support. You can manage other BladeCenter resources such as I/O modules and retrieval of system health information. You can configure SoL for the JS21 blade server.

5.4.2 IBM Director
IBM Director is a systems management tool. It was originally developed to provide a comprehensive management solution for servers and workstations based on Intel microprocessors. Recently its capabilities have been expanded to support management of other platforms such as the BladeCenter JS21. IBM Director enables you to remotely manage many IBM and non-IBM servers including the BladeCenter JS21. The IBM Director console allows system administrators to manage multiple BladeCenter chassis in a heterogeneous environment or environments where a Director infrastructure exists. IBM Director V5.1 supports the following functions on the BladeCenter JS21: Events Resource monitoring Inventory (limited) Remote session Software distribution

Chapter 5. Planning considerations

57

File transfer Process management Management Processor Assistant (MPA) Visit the following Web site to download IBM Director 5.1: http://www.ibm.com/servers/eserver/xseries/systems_management/ibm_direc tor/ This section reviews the major planning considerations associated with the installation of IBM Director. For a comprehensive treatment of planning for IBM Director, refer to Implementing IBM Director 5.10, SG24-6188. IBM Director has several components. We address three of the primary components in this section and in 10.2, “IBM Director” on page 346. IBM Director Server: This is the core of the IBM Director system management solution. IBM Director Console: This provides the user interface that system administrators and operators use to interact with IBM Director. IBM Director Agents: These provide the mechanism by which the IBM Director can monitor and control a specific server and operating system environment.

IBM Director Server for the BladeCenter JS21
Support for IBM Director Server on the BladeCenter JS21 was announced in IBM Director Version 5.10. For installation instructions for the IBM Director Server, see 10.2.1, “Installing the components for IBM Director” on page 347.

IBM Director Agent for the BladeCenter JS21
Support for the BladeCenter JS21 was introduced in IBM Director Agent Version 4.12 and is enhanced in Version 5.10. You can learn how to install the IBM Director Agent in 10.2.1, “Installing the components for IBM Director” on page 347 For maximum functionality, the IBM Director Server requires IP connectivity to the management module in each BladeCenter chassis and to every blade server. This connectivity can be provided by connecting the IBM Director Server to both the hardware management subnet and operating system management subnets described in 6.3.6, “Management module: TCP/IP ports” on page 83. If your environment primarily comprises servers running Linux, consider installation of the IBM Director Server under Linux.

58

IBM BladeCenter JS21: The POWER of Blade Innovation

5.4.3 IBM Cluster Systems Management
IBM CSM is a systems management tool. It simplifies the management of clusters of servers running the AIX and Linux operating systems. This section focuses on the planning considerations that are associated with using CSM to manage BladeCenter JS21s. For more planning information, refer to Cluster Systems Management for AIX and Linux V1.5 Planning and Installation Guide, SA23-1344.

Supported CSM and operating system releases
Preliminary support for the BladeCenter JS21 was introduced in CSM for AIX V1.5.1.1, and CSM for Linux V1.5.1.1. There is also a Red Hat Package Manager (RPM) for CSM for Linux update V1.5.1.1 and a program temporary fix (PTF) for CSM for AIX V1.5.1.1. CSM requires specific support for each operating system version that is running on the servers that it manages. Currently (at the time of publication) CSM Version 1.5.1 supports the following operating systems on the BladeCenter JS21: AIX 5L Version 5.2 with APAR IY77440 AIX 5L Version 5.3 with APAR IY7740 The default base versions of SUSE Linux Enterprise Server 9 (SLES9) for POWER systems The default base versions of Red Hat Enterprise Linux (AS) 4 for POWER Certain CSM for Linux on POWER functions require non-IBM software. The following non-IBM software is required. You can obtain it from the listed sources. AutoUpdate V4.3.4,or later levels You must use this if you want to perform the software maintenance installation and upgrade of non-CSM RPMs on your Linux-managed nodes from the management server. Download the software from: http://freshmeat.net/projects/autoupdate sg3_utils-1.06-1.ppc64 http://people.redhat.com/pknirsch

Chapter 5. Planning considerations

59

For your convenience, the following required IBM and non-IBM software is included on the CSM CD-ROM: tftp-HPA 0.34 fping-2.4b2-5 conserver 8.1 IBMJava2-JRE 1.4.2 Planning elements for CSM for AIX and Linux is covered in detail in Chapter 3 of IBM Cluster Systems Management for AIX V5L and Linux Planning and Installation Guide V1.50, SA23-1344. To check for updates to this document, refer to: http://publib.boulder.ibm.com/infocenter/clresctr/vxrx/index.jsp?topic= /com.ibm.cluster.csm.doc/clusterbooks.html Important: Follow the instructions regarding software installation and prerequisites closely. This is especially important if you are upgrading from CSM Version 1.5.1.0 using PTFs.

Management node selection
Each CSM cluster requires a management node, which provides the central point of administration and control for the entire cluster. CSM has complex rules concerning the compatibility of management nodes, non-management nodes (cluster nodes), and the operating system they run. We attempt to simplify these rules by focusing exclusively on the requirements for a management node that only manages BladeCenter JS21s. If your environment is more complex, refer to the CSM planning documents. You can use a CSM management node to both install and manage BladeCenter JS21s, or you can use a management node to manage BladeCenter JS21s that are installed using other mechanisms. Most environments use CSM to both install and manage BladeCenter JS21s; therefore, we focus on this scenario. If you use a management node to both install and manage BladeCenter JS21s running Linux, you must use the same type of Linux distribution on both the management node and the BladeCenter JS21s. However, the management node does not require the same processor architecture that is used in the BladeCenter JS21. For example, if you plan to install SLES on BladeCenter JS21s, you can use the following types of management node and operating system combinations to both install and manage the BladeCenter JS21s: A supported IBM eServer xSeries® server running SLES A supported IBM eServer pSeries® server running SLES An IBM HS20 Blade Server running SLES An IBM BladeCenter JS21 running SLES

60

IBM BladeCenter JS21: The POWER of Blade Innovation

We performed our testing of CSM using a BladeCenter JS20 as the management server, another BladeCenter JS20 node with AIX, and an AIX LPAR on a JS21 and a Linux LPAR on a JS21 as two managed cluster nodes.

Network considerations and summary
IBM CSM is designed to work in the type of network environment that is outlined in 5.2, “Network planning” on page 46, with multiple subnets that segregate different types of network traffic. The CSM management node is normally equipped with multiple Ethernet network interfaces that are connected as follows: One interface connects to the hardware management subnet to support CSM’s hardware management functions. One interface connects to the operating system management subnet to support CSM’s capabilities to install operating systems, maintain software and configuration files, and provide ongoing monitoring of cluster nodes. One interface connects to an external network to enable remote access to the management node by system administrators and operators.

Chapter 5. Planning considerations

61

62

IBM BladeCenter JS21: The POWER of Blade Innovation

6

Chapter 6.

Hardware setup
This chapter explains the various activities that you have to perform before you can install an operating system on the BladeCenter JS21. You have to understand the components, their relationships, and how to configure them. This chapter covers the following topics: 6.1, “BladeCenter chassis” on page 64 6.2, “BladeCenter internal network” on page 66 6.3, “Management module configuration” on page 71 6.4, “I/O module configuration” on page 85 6.5, “Blade server configuration” on page 90 6.6, “Firmware” on page 94 6.7, “Providing a console for the BladeCenter JS21” on page 121 6.8, “System Management Services interface” on page 133 6.9, “Open Firmware interface” on page 152 6.10, “Blade server Ethernet controller enumeration” on page 162 6.11, “Blade server I/O module routing” on page 169 6.12, “SAS hardware RAID configuration” on page 170

© Copyright IBM Corp. 2006. All rights reserved.

63

6.1 BladeCenter chassis
Each BladeCenter chassis can accept the same blades, but uses different switches and power modules. Figure 2-7 on page 20 shows the rear view of BladeCenter, Figure 2-10 on page 23 shows the rear view of BladeCenter H, and Figure 6-2 on page 65 shows the rear view of the BladeCenter T as an orientation for locating the management and input/output (I/O) modules. For more information, see Installation and User's Guide - IBM BladeCenter (Type 8677), Installation and User's Guide - IBM BladeCenter H (Type 8852), or Planning and Installation Guide - IBM BladeCenter (Type 8677).

BladeCenter product documentation
You can find these manuals on the following IBM Web site: http://www-03.ibm.com/servers/eserver/support/bladecenter/allproducts/i nstalling.html For each product, select Hardware only → Product documentation. Figure 6-1 shows the front view of the BladeCenter T.

Figure 6-1 Front view of the BladeCenter T

64

IBM BladeCenter JS21: The POWER of Blade Innovation

Figure 6-2 shows the rear view drawing of the BladeCenter T.

Figure 6-2 Rear view of the BladeCenter T

All BladeCenter chassis run with one or two management modules (MM) or advanced management modules (AMM). BladeCenter-H supports the AMM only. The first management module must be located in management module bay 1 and the second in management module bay 2. The Ethernet switch modules must be located in I/O module bays 1 and 2. Additional modules can be added in the remaining I/O module bays depending on the type of the chassis. Because the interfaces are always paired and the pair associated to I/O module bays 1 and 2 is always Ethernet, there must be Ethernet switches in these bays. However, the type of Ethernet switches might be different in bay 1 compared to bay 2.

Chapter 6. Hardware setup

65

6.2 BladeCenter internal network
This section explains the BladeCenter’s internal network. Understanding the internal network paths is essential for a correct setup and integration of a BladeCenter into a network environment. It is also important for planning and implementing the management network besides the production network end optional using virtual LANs (VLANs). Figure 6-3 on page 67 shows a schematics abstract of the BladeCenter’s internal network. Note: This schematic does not exactly match the real electric design. It is a simplified abstract to explain network paths for planning and configuring the BladeCenter’s network setup. You can find a more detailed schematic in BladeCenter chassis management: http://www.research.ibm.com/journal/rd/496/brey.pdf

66

IBM BladeCenter JS21: The POWER of Blade Innovation

BladeCenter Chassis

Management Module (MM) (redundant MM not shown)
INT (eth1) Proxy ARP Management Interface HTTP/HTTPS/Telnet/ SSH/DOT/SNMP/LDAP EXT (eth0)

Management Workstation

MANAGEMENT MODULE BAY 1 Ethernet Switch Module
Management Interface MGT 1 MGT 2 INT 1 EXT 1 EXT 2 EXT 3 EXT 4

Management Network

. .

INT 14

I/O MODULE BAY 1 Ethernet Switch Module
Management Interface MGT 1 MGT 2 INT 1 EXT 1 EXT 2 EXT 3 EXT 4

. .

INT 14

I/O MODULE BAY 2 Production Network

Port 1 (eth0)

Port 2 (eth1)

Port 1 (eth0)

Port 2 (eth1)

Network (LAN)

SoL Data BSMP Build in Ethernet Controller

SoL Data

...

BSMP

Build in Ethernet Controller

Optional Network connection Production VLANs SoL VLAN default ID = 4095 Management VLAN (untagged)

BladeCenter JS21 BAY 1

...

BladeCenter JS21 BAY 14

External management over all ports (enable/disable)

The AMM has no internal interface eth1

Figure 6-3 BladeCenter internal network

Chapter 6. Hardware setup

67

Within the BladeCenter there are two types of physical networks: The internal management network and the internal production network. The internal management network connects the management module to the management port of the I/O modules (for simplification, we do not show the optional redundant management module in Figure 6-3 on page 67). The internal production network is a separate (individual) connection from each blade to the I/O module bay. Each blade, by default, has two Ethernet ports. Port 1 (eth0) is connected to I/O module bay 1 and port 2 (eth1) to I/O module bay 2. If an expansion card (daughter card) is installed in a blade, the two ports of the expansion card are connected to I/O module bays 3 and 4. Note: BladeCenter HS20-8678 blade servers map the switch modules in the opposite way: The switch module in module bay 1 accesses the blade server’s eth1 interface, and the switch module in module bay 2 accesses the blade server's eth0 interface. To use HS20-8678 blade servers in the same chassis with other HS20 blade servers, you must upgrade the HS20-8678 blade server Basic I/O system (BIOS) to level 1.05 or later. Then modify the BIOS settings for each HS20-8678 blade server. When you turn on the server, press F1 to enter BIOS setup. In the BIOS setup screen, select BIOS setup → Advanced Setup → Core Chipset Control → Swap the numbering of onboard NICs [Yes]. In Figure 6-3 on page 67, we show two Ethernet switch modules (ESM) in I/O module bays 1 and 2. The ESMs have two internal management ports going to MM1 and MM2 (MM2 is optional and not shown), 14 internal ports going to the blades and four to six external (uplink) ports to connect to the external production network. To connect the management module’s external interface (eth0) to the external management network, you must use the management module’s remote management and console RJ45 Ethernet connector. Note: The management module provides the only access to the internal management network. This means that you cannot access the internal management network through the Ethernet switch modules from the production network, from outside the BladeCenter (EXTx ports), or from a blade server itself (INTx ports). The management module performs a proxy Address Resolution Protocol (ARP) function for the Internet Protocol (IP) traffic on the internal management network and then passes it along to its external (eth0) interface. Therefore, you can directly reach the Ethernet switch module’s management interface through the management module from the external management network. The management

68

IBM BladeCenter JS21: The POWER of Blade Innovation

module only responds to ARP requests that are either addressed to the IP address of the ESM, or originate from the IP address of the ESM. Note: Due to the proxy ARP function of the management module, you see all of the IP addresses of the internal management network, that is, the management module’s internal port eth1 (the advanced management module has no internal port eth1) and the management ports of the ESMs (if accessed through the management module and not using the external uplink ports) associated with the hardware Medium Access Control (MAC) address of the management module’s eth0 interface in the outside network. In the I/O module bays there are connections to both networks, the internal management network and the internal production network. Note: Although the internal management network and the internal production network are connected through the ESM, there are methods implemented to separate the management and production traffic. BladeCenter ESMs have certain hard-coded filters that prevent any traffic that enters any of the upstream ports from exiting out of the management module facing ports and vice versa. This also prevents any unexpected spanning-tree loops. The only way to control the management module is through the external RJ45 Ethernet connector of the management module. It is also not possible to pass substantive data between switch modules across the midplane using the MGTx ports. The ESM does not forward data between the MGTx ports and any of the internal (INTx) or external (EXTx) uplink ports. If you want to pass data from one switch module to another, then the modules must either be cabled directly to each other or connected by way of an external switch or router. This is especially important if you have enabled “External management over all ports” (see 6.4.2, “Setting external management over all ports” on page 87) for the ESM (represented by a closed “External management over all ports” switch in Figure 6-3 on page 67). In this case, you can reach the ESM’s management interface (HTTP, Telnet, Secure Shell (SSH), Simple Network Management Protocol (SNMP), and so on) through two paths. You can reach it from the external management network through the management module that supports pass-through function to the internal management port of the ESM on the one side, and from the external production network through the ESM’s external ports on the other side. Even in this scenario the ESM’s management interface does not pass through data from one path to the other.

Chapter 6. Hardware setup

69

Note: The BladeCenter Ethernet switch modules are designed for secure management. Accessing the management interfaces of the Ethernet switch modules using the management network is enabled by the management module interface performing a proxy ARP. This enables the secure management subnet to access the management interfaces of the I/O modules through a single physical Ethernet port on the management module. If you do not have a separate subnet for management available to you, then you can also attach the ESM external ports to this subnet. To keep duplicate IP addresses from being reported, the default setting is off for “External management over all ports” (see 6.4.2, “Setting external management over all ports” on page 87). If you want to turn on the “External management over all ports” setting, then keep the management module Ethernet interface in a separate network because this is the proper configuration. You must carefully consider when you connect the external management network and the external production network (see the dotted Ethernet link in Figure 6-3 on page 67) or leave them separated. When you connect both, use different VLANs to separate the networks. Another method of separating the network paths is used for Serial over LAN (SoL). SoL traffic goes from the management module’s management interface through the ESM in I/O module bay 1 to the blade servers’s blade system management processor (BSMP). Note: The BSMP is also called integrated systems management processor (ISMP) or baseboard management controller (BMC). To do the separation, SoL uses the VLAN ID (VID) 4095. The default VID for the production network is 1 (you can define additional VLANs with an ID less than (<) 4095 in the ESM for the production network, and the VID 4095 is blocked for this purpose). Therefore, the SoL traffic does not interact with the internal production network. However, it partly uses the same physical network path. For more information, see the following IBM Redpapers: Nortel Networks L2/3 Ethernet Switch Module for IBM eServer BladeCenter, REDP-3586 Cisco Systems Intelligent Gigabit Ethernet Switch Module for the IBM eServer BladeCenter, REDP-3869 IBM eServer BladeCenter Layer 2-7 Network Switching, REDP-3755

70

IBM BladeCenter JS21: The POWER of Blade Innovation

For more information, see the following documents: BladeCenter chassis management http://www.research.ibm.com/journal/rd/496/brey.pdf BladeCenter networking http://www.research.ibm.com/journal/rd/496/hunter.pdf BladeCenter processor blades, I/O expansion adapters, and units http://www.research.ibm.com/journal/rd/496/hughes.pdf

6.3 Management module configuration
This section describes the steps to set up the BladeCenter management module so that you can work with BladeCenter JS21s and other blade servers. You can find further information about the configuration of the management module in Management Module Installation Guide - IBM BladeCenter and Advanced Management Module Installation Guide - IBM BladeCenter, BladeCenter H. Refer to the procedure described in “BladeCenter product documentation” on page 64. Note: You can access the management module’s management interface (HTTP, Telnet, SSH, and so on) only by using TCP/IP over the module’s remote management and console RJ45 Ethernet connector or the serial port of the advanced management module. You cannot use the keyboard, video, mouse (KVM) connectors for this purpose. The primary setup task is to assign IP addresses, which are necessary to communicate with the management module using TCP/IP. To learn more about selecting these IP addresses, see Chapter 5, “Planning considerations” on page 45. The management module has two Ethernet interfaces, an external interface (eth0) and an internal interface (eth1) as seen in Figure 6-3 on page 67.

Chapter 6. Hardware setup

71

6.3.1 Connecting to the management module
All management module types support connection through the remote management and console RJ45 Ethernet connector. Figure 6-4 shows the connectors of the management module. Note: The advanced management module also supports command-line Interface only (CLI-only) connection through the external serial port of the advanced management module. For information about accessing the management module’s CLI, see the Management Module Command Line Interface Reference Guide - IBM BladeCenter and BladeCenter T. Refer to the procedure described in “BladeCenter product documentation” on page 64.

Figure 6-4 Management module: Output connectors

72

IBM BladeCenter JS21: The POWER of Blade Innovation

Figure 6-5 shows the connectors of the advanced management module.

Figure 6-5 Advanced management module: Output connectors

You can access the management interface through a network or through a computer that is connected directly to the management module. To establish a remote console to the management module, you require the following equipment and information: A computer with Internet browser capability. To facilitate connections at multiple locations, you can use a notebook computer. The management module’s MAC address (listed on the label on the management module).

Chapter 6. Hardware setup

73

For a network connection to the management module, the following equipment: – A standard Ethernet cable – A local Ethernet network port (facility connection) An Ethernet crossover cable for direct connection of a computer to the management module’s remote management and console Ethernet connector Note: The advanced management module can use either a standard Ethernet cable or an Ethernet crossover cable to make this connection. The external interface (eth0) is accessible using the 10/100BaseT RJ45 connector on the management module. The internal interface (eth1) is connected to the management interfaces of all the installed I/O modules that support such interfaces (Figure 6-3 on page 67). This includes all switch I/O modules. Tip: Note that: The function of a new management module is to request an IP address for the external interface using Dynamic Host Configuration Protocol (DHCP). If the management module does not receive a valid response from a DHCP server within two minutes of powering on, it uses the static IP address 192.168.70.125 with the subnet mask 255.255.255.0. The default host name is MMxxxxxxxxxxxx, where xxxxxxxxxxxx is the burned-in MAC address. The default IP addresses with a subnet mask of 255.255.255.0 are: Management module: – External port: 192.168.70.125 – Internal port: 192.168.70.126 Devices in I/O module bays: – – – – Module bay 1: 192.168.70.127 Module bay 2: 192.168.70.128 Module bay 3: 192.168.70.129 Module bay 4: 192.168.70.130

You can reset the IP addresses of a management module that was previously configured back to the factory defaults by using the IP reset button (shown in Figure 6-4 on page 72) on the management module. You can find the procedure for doing this in 6.3.7, “Resetting the management module” on page 84.

74

IBM BladeCenter JS21: The POWER of Blade Innovation

6.3.2 Configuring the management module
Perform the following procedure to set the IP addresses for the management module’s external and internal interfaces when the management module is not connected to a DHCP network. 1. Connect the management module to an isolated private Ethernet network. Also connect a workstation that has a Web browser to the same isolated private Ethernet network. You can also connect the workstation directly to the management module using an appropriate cable (crossover for any management module that is not an advanced management module). 2. Configure a static IP address for the workstation Ethernet interface that is in the same subnet as the management module default IP addresses. For example 192.168.70.100 with a subnet mask of 255.255.255.0 for the workstation Ethernet interface. Do not use addresses in the range of 192.168.70.125 through 192.168.70.130. These addresses conflict with the default addresses assigned by the management module. 3. Connect to the management module Web interface by pointing your Web browser on the workstation to: http://192.168.70.125 Note: Alternatively, you can connect the management module to a normal network that has a DHCP server running. Either define the MAC address of the management module in the DHCP server before connecting. Or check which address has been given to the management module after connecting, and use this address instead of 192.168.70.125. This keeps you from having to change your computer’s IP address. This method is only possible if DHCP is not disabled on the management module. 4. Enter a valid user ID and password. The factory default configuration of a management module defines a user ID named USERID with a password of PASSW0RD. The “0” in the password is a zero. In production environments, consider changing these defaults. 5. From the management module Web interface, select MM Control → Network Interfaces, as shown in Figure 6-6. Enter the external and internal IP addresses that you want (standard management module only), subnet masks, and default gateway for the management module. We also recommend that you switch off DHCP by setting it to Disabled - Use static IP configuration. Click Save to store the new IP addresses.

Chapter 6. Hardware setup

75

Figure 6-6 Management module: IP address configuration

76

IBM BladeCenter JS21: The POWER of Blade Innovation

Figure 6-7 shows the interface for the advanced management module. Note: The management of the I/O modules by the management module is not based on the IP interfaces and the internal management network at all. It uses a different hardware link (I2C). Therefore, you can configure the I/O modules by the management module even when the IP addresses are not set up correctly.

Figure 6-7 Advanced management module: IP address configuration

Attention: The advanced management module has no internal Ethernet interface eth1. For the management module, the IP address for eth1 must be on the same subnet as eth0 on the management module as well as the management interfaces of the I/O modules. Otherwise, the I/O module’s management interfaces cannot be reached through the management module. The data rate, duplex mode, and maximum transmission unit are read-only fields for eth1.

Chapter 6. Hardware setup

77

6. Restart the management module. In the management module Web interface, select MM Control → Restart MM. Click the Restart button on the displayed page. You are prompted to confirm the restart before it occurs. 7. Remove the management module from the isolated private Ethernet network and connect it to the network that you will use to manage the BladeCenter. You can now connect to the management module Web and command line interfaces using the IP address that you assigned to the management module external network interface. Now consider performing other management module setup tasks, such as: Changing the factory set default user ID and password for security reasons Setting the management module date and time so that log entries have useful time stamps Defining user IDs and passwords for system administrators and operators who manage the BladeCenter and disable the factory set default user ID Alternatively, you can configure the management module to use a Lightweight Directory Access Protocol (LDAP) directory for this purpose. Configuring the management module to send alerts to management systems using SNMP, or system administrators using e-mail using Simple Mail Transfer Protocol (SMTP) Enabling the use of Secure Sockets Layer (SSL) to securely access the management module Web interface Enabling the use of SSH to securely access the management module CLI Note: The management module’s remote disk feature is currently not available for blade servers JS20 and JS21. For additional information about how to perform these tasks, refer to the Advanced Management Module and Management Module User's Guide - IBM BladeCenter, BladeCenter T, BladeCenter H and the Management Module Command Line Interface Reference Guide - IBM BladeCenter and BladeCenter T. You can find these publications using the procedure described in “BladeCenter product documentation” on page 64.

78

IBM BladeCenter JS21: The POWER of Blade Innovation

6.3.3 Advanced management module: Management interface changes
The Web interface of the advanced management module is mostly identical with the Web interface of the standard management module. The new items are Serial Port, Configuration Mgmt (which was Restore Defaults), and the Service Tools menu, as shown in Figure 6-8.

Figure 6-8 Advanced management module: Web interface system status

Chapter 6. Hardware setup

79

The configuration management menu has new functions to back up and restore configuration, as shown in Figure 6-9.

Figure 6-9 Advanced management module: Configuration management

80

IBM BladeCenter JS21: The POWER of Blade Innovation

6.3.4 Management module: Hardware vital product data
To get an overview of the hardware that is installed in the BladeCenter, select Monitors → Hardware VPD. Figure 6-10 shows an example of the displayed data. Tip: You can see the BladeCenter JS21 MAC addresses in the hardware vital product data (VPD).

Figure 6-10 Management module: Hardware VPD

Chapter 6. Hardware setup

81

6.3.5 Management module redundancy
Each BladeCenter unit comes with at least one management module. Some BladeCenter units support installation of a second, redundant management module. Only one of the management modules in a BladeCenter unit can be active, and this functions as the primary management module. If a redundant management module is installed, it remains inactive until it is switched to act as primary, either manually or automatically, if the primary management module fails. You can configure the failover behavior using the uplink command described in the Management Module Command Line Interface Reference Guide - IBM BladeCenter and BladeCenter T. To locate this document, use the procedure described in “BladeCenter product documentation” on page 64. Note: During failover, the ARP table is flushed and automatically rebuilt over a time period of some minutes. When it is rebuilt, you will be able to re-establish a network connection to the management module. This ARP table rebuilding does not impact the network attached to the Ethernet switch modules. To check which management module is currently active, in the management module’s Web interface, select Monitors → System Status → Management Module, as shown in Figure 6-11.

Figure 6-11 Displaying the active and backup management module

If two management modules are installed in a BladeCenter unit, both must always have the same level of firmware and the same IP address. The firmware must support redundant management module function to enable changeover of control from the primary (active) management module to the redundant management module. The configuration settings of the primary management module are automatically transferred to the second management module. This transfer can take up to 45 minutes.

82

IBM BladeCenter JS21: The POWER of Blade Innovation

Note: Configuration data and the firmware itself are automatically transferred from the primary management module to the redundant management module. Therefore, you have to apply firmware updates only on the primary (active) management module. Sometimes the firmware transfer does not work on the back-level firmware. (See 6.6, “Firmware” on page 94 for more information.) In this case, you have to update both the management modules separately. To do this, you must activate the backup management module by deactivating the primary management module. Note: To switch the active management module, from the Web interface, select MM control → Restart MM → Switch Over.

6.3.6 Management module: TCP/IP ports
The BladeCenter management module uses a variety of TCP/IP ports for communication. In this section, we show which ones the administrators can change and which ones they cannot change. You have to ensure that your network (firewalls and routers) allows communications through these ports for the adapter and management module to function properly. The ports in Table 6-1 are user configurable. The default port numbers used are indicated in column 2.
Table 6-1 User configurable TCP/IP ports Port name HTTP HTTPS Telnet SSH SNMP agent SNMP traps Default port number 80 443 23 22 161 162 Description Web server HTTP connection - TCP SSL connection - TCP Telnet command-line interface connection TCP SSH command-line interface - TCP SNMP get/set commands - User Datagram Protocol (UDP) SNMP traps - UDP

Chapter 6. Hardware setup

83

The TCP/IP ports in Table 6-2 are fixed and cannot be changed.
Table 6-2 Fixed TCP/IP ports Port number 25 53 68 427 1044 1045 5900 6090 13991 Description E-mail alerts - TCP Domain Name System (DNS) resolver - UDP DHCP client connection - UD Service Location Protocol (SLP) connection - UDP Remote disk function - TCP Persistent remote disk (disk on card) - TCP Remote console video redirect - TCP IBM Director commands - TCP IBM Director alerts - UDP

6.3.7 Resetting the management module
This section shows how to reset the BladeCenter management module in a scenario where the passwords are lost or the IP addresses are unknown. The management module reset button is shown in Figure 6-4 on page 72. Use a straightened paper clip or a similar item to access it. The reset button has two uses: Resetting the network configuration. To do this, press and hold the reset button for 3 seconds or less. Resetting the entire management module (including user IDs and passwords) to the factory defaults. This requires a sequence of keeping the button pressed and releasing it: a. Press and hold the reset button for 5 seconds. b. Release the button and wait for 5 seconds. c. Press and hold the button for another 10 seconds. Note: This sequence must be as precise as possible to ensure success: Keep the button pressed for 5 seconds, release it and wait for 5 seconds, and again keep it pressed for 10 seconds. After you do a full factory defaults reset in this way, the default user ID and password will be in effect: USERID (all caps) and PASSW0RD (with a zero, not the letter O).

84

IBM BladeCenter JS21: The POWER of Blade Innovation

After you reset the network configuration, you can access the Web interface to reconfigure it. To do this, you have to know the management module's IP address as described in 6.3.2, “Configuring the management module” on page 75. Attention: Because it is possible to reset the passwords using the reset button, you must ensure that your BladeCenter chassis is physically secure so that only authorized personnel can physically access the reset button.

6.4 I/O module configuration
This section explains the basic setup of the I/O modules so that you can work with BladeCenter JS21s. An advanced configuration of the I/O modules is beyond the scope of this book. The primary setup tasks for I/O modules are: Assigning IP addresses to the I/O module management interfaces Enabling the I/O module external ports Optionally enabling or disabling “External management over all ports” You can learn about the selection of IP addresses for the I/O module management interfaces in “Hardware management subnet” on page 49. When you first install a new I/O module, the management module assigns a default IP address to the management interface of the I/O module. The default IP address is chosen based on the I/O module bay where the I/O module is installed. The I/O module installed in I/O module bay 1 is assigned the IP address 192.168.70.127, the I/O module installed in I/O module bay 2 is assigned 192.168.70.128, the I/O module installed in I/O module bay 3 is assigned 192.168.70.129, and the I/O module installed in I/O module bay 4 is assigned 192.168.70.130.

6.4.1 Setting the IP address of an I/O module
Set the IP address of each I/O module management interface and enable the external ports. 1. Connect to the management module using the Web browser interface as explained in 6.3.2, “Configuring the management module” on page 75. 2. From the management module Web interface, select I/O Module Tasks → Configuration.

Chapter 6. Hardware setup

85

3. In the window shown in Figure 6-12, scroll down to the entry for the I/O module that you want to configure. Enter the IP address, subnet mask, and gateway that you want to assign to the I/O module management interface. Click Save to activate the new IP address. You are prompted to confirm that you want to change the IP address.

Figure 6-12 I/O module IP address

86

IBM BladeCenter JS21: The POWER of Blade Innovation

6.4.2 Setting external management over all ports
The setting shown in Figure 6-13 defines whether you can access the management interface (HTTP, Telnet, SSH, SNMP, and so on) of the I/O module over the internal management port only (see the “External management over all ports” switch in Figure 6-3 on page 67) or using the INTx and EXTx ports too.

Figure 6-13 I/O module external management over all ports

Chapter 6. Hardware setup

87

Note: Although “External management over all ports” is an option of the I/O module, you can only control it (switch it on and off) using the management module. You can do this for each I/O module separately. 1. Select the Advanced Configuration link for the I/O module. 2. Scroll down to the Advanced Setup item of the displayed page, as shown in Figure 6-13. In the “External management over all ports” field, select either Enabled or Disabled from the drop-down list. Click the Save button. Note: It depends on your network topology whether you want to enable or disable the external management of the I/O module over all ports. Several scenarios are described in Cisco Systems Intelligent Gigabit Ethernet Switch Module for the IBM eServer BladeCenter, REDP-3869.

6.4.3 Enabling the external I/O module ports
To enable the external I/O module ports: 1. From the management module Web interface, select I/O Module Tasks → Admin/Power/Restart. 2. In the I/O Module Advanced Setup section, from the Select a module menu, select the required I/O module. Set the external ports to Enabled, as shown in Figure 6-14. 3. Click the Save button. Note: Depending on the firmware level of the management module, this task can be listed under Configuration instead of Admin/Power/Restart.

88

IBM BladeCenter JS21: The POWER of Blade Innovation

Figure 6-14 Enabling or disabling the I/O module external ports

Attention: You have to select the I/O module that you want using the Select a module menu, and not by using the check boxes in the table shown in Figure 6-14. At this point, the I/O module management interface has an IP address. Also, the external ports on the I/O module are enabled so that they can be used to communicate with blade servers. The SoL remote text console function of the management module depends on a VLAN provided by an Ethernet switch I/O module installed in I/O module bay 1. This VLAN is automatically provided by the 4-port Gigabit Ethernet switch module and the Nortel Networks Layer 2-7 Gigabit Ethernet switch module using VLAN 4095.

Chapter 6. Hardware setup

89

For the Cisco Systems Intelligent Gigabit Ethernet switch module for the IBM BladeCenter, the SoL VLAN ID is set as follows: For Cisco I/O supervisor (IOS) releases earlier than 12.1(22)EA6a, you can set the SoL VLAN ID to any value between 3 and 1001: it cannot be set to the default value of 4095. For Cisco IOS release 12.1(22)EA6a and later, you can set the SoL VLAN ID to a default value of 4095. If necessary, you can still set your own defined VLAN ID. For further information, see BladeCenter Serial over LAN Setup Guide or the Cisco Systems Intelligent Gigabit Ethernet Switch Module for the IBM eServer BladeCenter, REDP-3869. The Cisco guide describes how to manually set up the VLAN that is necessary to support the SoL remote text console function. You can access these guides on the Internet by following the instructions in “BladeCenter product documentation” on page 64.

6.5 Blade server configuration
Minimal configuration is required for each BladeCenter JS21 before installing an operating system. The two main configuration tasks are: Assigning a name to the blade server Setting the blade server boot sequence The easiest way to perform these tasks is through the management module Web interface.

6.5.1 Assigning names to blade servers
Set the name of each blade server in a BladeCenter chassis by performing the following steps: 1. From the management module Web interface, select Blade Tasks → Configuration, as shown in Figure 6-15. 2. In the Blade Information section, type the name that you want to assign to each blade server. 3. Scroll down the page and select the Save button to save the names that you have assigned to each blade server.

90

IBM BladeCenter JS21: The POWER of Blade Innovation

Figure 6-15 Blade server naming

6.5.2 Setting the boot sequence
Set the boot sequence of each blade server by using this procedure: 1. From the management module Web interface, select Blade Tasks → Configuration. 2. Scroll down the page to the Boot Sequence section to see the current boot sequence for all the blade servers. If a blade server does not have the correct boot sequence, you can change the boot sequence by selecting the blade server name.

Chapter 6. Hardware setup

91

3. This displays the window shown in Figure 6-16. Set the necessary boot sequence using the drop-down lists. Click the Save button to set the new boot sequence.

Figure 6-16 Boot sequence of blades

The correct boot sequence depends on the method that you plan to use to install an operating system on the blade server. Note: You can also change the boot sequence using the System Management Services (SMS) (see “Configuring boot device order” on page 141) or the Open Firmware interface (see boot-device variable in 6.9.1, “Activating the Open Firmware interface” on page 152). Independent of which of the three methods you use to change the boot sequence, it will be stored in the same location in the nonvolatile random access memory (NVRAM). This means that when you use one method to change the boot sequence, the changes are also reflected in the other interfaces with at least one exception: If you select network - BOOTP in the management module’s management interface (as shown in Figure 6-16), it does not give you an option to specify the individual Ethernet ports. Using the network - BOOTP option results in two entries in the boot device order where the secondary adapter (location code -T8) shows up first as recommended for use with SoL (6.7, “Providing a console for the BladeCenter JS21” on page 121). It also does not allow you to specify IP addresses for remote IPL, which is possible in SMS and Open Firmware.

92

IBM BladeCenter JS21: The POWER of Blade Innovation

You can see that the boot sequence changes that are made by the management module’s interface (shown in Figure 6-16 on page 92) are also reflected in the Open Firmware interface shown in Example 6-1.
Example 6-1 Open Firmware boot sequence

boot-device network1:speed=auto,duplex=auto,0.0.0.0,,0.0.0.0,0.0.0.0 network:speed=auto,duplex=auto,0.0.0.0,,0.0.0.0,0.0.0.0 cdrom hdd0 hdd1 The Open Firmware aliases network and network1 definitions are shown in Example 6-2, which shows that the secondary Ethernet port comes first.
Example 6-2 Open Firmware device aliases created by MM

network network1

/pci@8000000f8000000/pci@2/ethernet@4 /pci@8000000f8000000/pci@2/ethernet@4,1

The boot sequence changes are also reflected in the SMS interface shown in Figure 6-17. PowerPC Firmware Version MB240_470_013 SMS 1.6 (c) Copyright IBM Corp. 2000,2005 All rights reserved. --------------------------------------------------------------------------Current Boot Sequence 1. Ethernet ( loc=U788D.001.23A1137-P1-T8 ) 2. Ethernet ( loc=U788D.001.23A1137-P1-T7 ) 3. USB CD-ROM ( loc=U788D.001.23A1137-P1-T1-L1-L3 ) 4. SCSI 36401 MB Harddisk, part=1 () ( loc=U788D.001.23A1137-P1-T10-L1-L0 ) 5. SCSI 36401 MB Harddisk, part=1 () ( loc=U788D.001.23A1137-P1-T11-L1-L0 ) --------------------------------------------------------------------------Navigation keys: M = return to Main Menu ESC key = return to previous screen X = eXit System Management Services --------------------------------------------------------------------------Type menu item number and press Enter or select Navigation key:
Figure 6-17 SMS Configure Boot Device Order: Displaying current setting

Chapter 6. Hardware setup

93

6.5.3 Assigning the media tray
To assign the media tray to a specific blade server: 1. Select Blade Tasks → Remote Control and select Start Remote Control. 2. A new window opens as shown in Figure 6-18. In the upper right, scroll down the Change media tray owner menu and select a blade server. You can assign the media tray to one specific blade server.

Figure 6-18 Blade server management: Remote control

Note: KVM, Remote Disk, and Remote Console are currently not supported for the blade server JS21.

6.6 Firmware
This section deals with the firmware management of the BladeCenter JS21 and the various hardware components used in combination with it. Important: To provide a stable environment for production and to avoid hard-to-detect problems, spend some time planning and implementing the firmware management. You must determine if it is a good choice to upgrade to the latest firmware or to synchronize the firmware versions to an earlier but compatible level.

94

IBM BladeCenter JS21: The POWER of Blade Innovation

6.6.1 Getting the latest firmware, tools, and support
IBM provides a great variety of support pages to address different client requirements. The following sections help you to find firmware and additional utilities or information.

General support
To stay informed about the BladeCenter JS21 and to get the latest firmware for BladeCenter JS21 components: 1. Visit the following IBM Web site: http://www.ibm.com/pc/support Note: It might happen that there is an automatic redirect to another URL, but you can always identify the page by looking for Personal computing support. On this service Web page, in the Browse by product section, enter the product or model number of the hardware. In the case of the BladeCenter JS21, you have to enter the Type number 8844 and click the GO button. 2. In the new window that opens, you can optionally choose a model number, an operating system, or choose both. Click the Continue button. 3. The BladeCenter JS21 service page opens. On this page, use the Downloads and drivers link to check the firmware that is available.

IBM BladeCenter: Software and device drivers
To visit the Software and device drivers - IBM BladeCenter Web page, use the following link: http://www.ibm.com/pc/support/site.wss/document.do?lndocid=MIGR-63017 You can choose BladeCenter JS21 or choose the Link for BladeCenter or BladeCenter H to get the latest firmware especially for BladeCenter switches.

Finding AIX support for BladeCenter JS21
To find the AIX support Web page for BladeCenter JS21, perform the following steps: 1. Start at the following Web site: http://www.ibm.com On the IBM home page, select Support & Downloads from the main menu at the top of the page.

Chapter 6. Hardware setup

95

2. On the Support & downloads page, in the Support by product section, select Servers. 3. On the Support for IBM Systems page, select the Blade servers link. 4. On the Support for BladeCenter page, in the Select your product section, select BladeCenter JS21 as Hardware and AIX environment as Software. Click Go. This opens the Support for BladeCenter JS21page.

Finding Linux support for the BladeCenter JS21 in the Internet
To find the Linux support Web page for BladeCenter JS21, perform the following steps: 1. Start at the following Web site: http://www.ibm.com On the IBM homepage, select Support & Downloads, from the main menu at the top of the page. 2. On the Support & downloads page, in the Support by product section, select Servers. 3. On the Support for IBM System page, select the Blade servers link. 4. On the Support for BladeCenter page, in the Select your product section, select BladeCenter JS21 as Hardware, and Linux on POWER environment as Software. Click Go. 5. On the Support for BladeCenter JS21 page, in the Tools and utilities section, select the Diagnostics for Linux on POWER link to get the tools for different Linux distributions. Tip: For hardware acquired from another vendor, it might be helpful to also consult the Web page of the manufacturer.

6.6.2 Identifying and updating the latest firmware in general
If you are planning to use already existing hardware with a BladeCenter JS21, check the related firmware readme file. It includes a release note about which version of the BladeCenter JS21 is supported. You can use the BladeCenter management module to identify many firmware levels of installed hardware. See Table 6-3 on page 97, which lists the available update methods and the possible impact on different features. In the management module, you can access the information by using the menu on the left. Navigate to Monitors → Firmware VPD. On the command-line interface of the management module, type the info command to get this information.

96

IBM BladeCenter JS21: The POWER of Blade Innovation

Tip: To learn more about the BladeCenter management module command-line interface, see the Management Module Command Line Interface Reference Guide - IBM BladeCenter and BladeCenter T. To locate these manuals, refer to the procedure described in “BladeCenter product documentation” on page 64. In Table 6-3, the column Update method comprises different approaches to update the firmware: Diag-CD OS Web-MM Com-MM Web-SM Com-SM Stand-alone diagnostics CD V5.3.0.40 or later, based on AIX and diag Operating system Web interface of the BladeCenter management module Command-line interface of the BladeCenter management module using Trivial File Transfer Protocol (TFTP) Web interface of BladeCenter I/O modules Command-line interface of BladeCenter I/O modules using File Transfer Protocol (FTP) or TFTP

Table 6-3 lists the hardware components and the firmware levels used during the creation of this book.
Table 6-3 Firmware of common BladeCenter JS21 related hardware compounds BladeCenter firmware components BladeCenter JS21system firmware (BIOS) BSMP of a BladeCenter JS21 Update methoda Version (build ID) used for this bookb MB240_470_014 Version information available using MM Yes Impact

Diag-CD, OS

All BladeCenter JS21 related functions must always be synchronized with the BSMP firmware. System management interface using management module and BladeCenter JS21 General Ethernet communication, VLAN, network boot, SoL

Diag-CD, OS, Web-MM, Com-MM Diag-CD, OS

BYBT07A

Yes

JS21 onboard Ethernet adapter

Linux: 3.18 AIX: 03180608

No

Chapter 6. Hardware setup

97

BladeCenter firmware components Serial-Attached SCSI (SAS) Controller SAS drives non-hot swappable Management module

Update methoda

Version (build ID) used for this bookb 06160020

Version information available using MM No

Impact

Diag-CD, OS Diag-CD, OS Web-MM, Com-MM

SAS interface and onboard Redundant Array of Independent Disks (RAID) functions Hard drive and interface functions Provided management functions and all system management related network communications including SoL

42353139

No

Main application: 1.21F (BRET85F) Boot ROM: Not important Remote control: 1.21F (BRET85F) V1.1 (BPET15D)

Yes

Advanced management module Cisco Systems Intelligent Gigabit Ethernet switch modulec (Copper and Fiber version) Nortel Networks Layer 2/3 Copper or Fiber Gigabit Ethernet switch module 4-Port Ethernet switch module from IBM (DLink) IBM BladeCenter 2-port Fibre Channel switch module Qlogic 4Gb Fibre Channel smallform-factor (SFF) expansion card

Web-MM, Com-MM Web-SM, Com-SM

Yes

12.1(22)EA6a

Yes

Com-SM

1.2.3.11

Yes

Switch management interface and all networking features including VLAN and SoL

Web-SM, Com-SM COM-SM

1.08 or 00.00.96 (ibmrun.096) v2.0.1.09

Yes

Yes

General Fibre Channel issues, especially storage area network (SAN) boot and interaction with SAN Utility or SANsurfer General Fibre Channel issues, especially important for SAN boot

Diag-CD,
OS

Driver(IBM): 1.14 Firmware version: 4.00.22

No

98

IBM BladeCenter JS21: The POWER of Blade Innovation

BladeCenter firmware components IBM Ethernet expansion card (LFF/SFF) Optical path-thru module Copper path-thru module

Update methoda

Version (build ID) used for this bookb 3.38

Version information available using MM No

Impact

Diag-CD,
OS Web-MM, Com-MM Web-MM, Com-MM

General Ethernet communication, VLAN, network boot

9 7

Yes Yes Interaction with MM, for example, temperature control

a. The explanations for the abbreviations used are described in the previous section. The supported update method might change with other firmware levels b. We recommend that you use at least the mentioned firmware levels to minimize problems and ensure BladeCenter JS21 support for external components c. Switch module is an alternative name for I/O module

There are many firmware update possibilities and different hardware components. Therefore, it is not possible to show all update procedures. The descriptions in the following sections might be helpful even if the actual case is slightly different. For example, the firmware update process of different expansion cards with the stand-alone diagnostics CD is almost identical. In all cases, we recommend that you read the readme file related to the firmware. Tip: To automate all firmware update processes, a script language based on the Tcl named Expect is useful. See the following Web site: http://expect.nist.gov

6.6.3 Updating BladeCenter JS21 firmware using AIX
AIX 5L includes diag to update the BladeCenter JS21 firmware (BIOS). The procedure is very similar to the stand-alone diagnostics CD, hence diag is used by the stand-alone diagnostics CD. The flash procedure with diag is not described in this section, see 6.6.9, “Updating the firmware using the stand-alone diagnostics CD” on page 111 for more information about this topic.

Chapter 6. Hardware setup

99

It is also possible to use the update_flash utility as part of the AIX 5L diagnostics package. The following section illustrates the usage of this utility (you require root privileges): 1. Download the firmware update package from the IBM support Web site. See 6.6.1, “Getting the latest firmware, tools, and support” on page 95. The firmware image itself is delivered using an IMG file. Note: Even if the IMG file size is approximately 24 MB, it is meant to be used as a flash image for the 4 MB flash memory. Typically, the IMG file contains more than the firmware data of a BladeCenter JS21. 2. To use the update_flash utility, the operating system running on the BladeCenter JS21 must have access to the IMG file. If the name of the firmware IMG file is mb-240.470.013_ppc64 and the file is in the actual directory, enter the following command to verify an IMG file: /usr/lpp/diagnostics/bin/update_flash -v -f ibm_fw_bios_mb-240.470.013_anyos_ppc64.img 3. Use the following command to update the firmware: /usr/lpp/diagnostics/bin/update_flash -f ibm_fw_bios_mb-240.470.013_anyos_ppc64.img 4. The update_flash utility asks for a confirmation as shown in Figure 6-19. After confirmation, the update of the temporary system firmware starts. To avoid the confirmation dialog, additionally use the parameter -q. # /usr/lpp/diagnostics/bin/update_flash -f mb-240.470.013_anyos_ppc64.img The image is valid and would update the temporary image to MB240_470_013. The new firmware level for the permanent image would be MB240_470_012. The current permanent system firmware image is MB240_470_012. The current temporary system firmware image is MB240_470_012.

***** WARNING: Continuing will reboot the system! ***** Do you wish to continue? Enter 1=Yes or 2=No
Figure 6-19 Confirmation dialog of the update_flash utility

100

IBM BladeCenter JS21: The POWER of Blade Innovation

5. If there are problems with the new firmware it is possible to use the firmware image stored on the permanent side of the flash memory to reject the new version. Enter the following command to copy the firmware level of the permanent system firmware image to the temporary side: /usr/lpp/diagnostics/bin/update_flash -r Note: The firmware stored on the temporary side of the flash memory is the default firmware used during a BladeCenter JS21 startup. It is possible to boot the firmware stored on the permanent side of the flash memory by using the SMS interface. See 6.8, “System Management Services interface” on page 133. Alternatively, if the new firmware works fine, you can commit the firmware update and copy the image from the temporary side to the permanent side. To do this, use the following command: /usr/lpp/diagnostics/bin/update_flash -c

6.6.4 Updating BladeCenter JS21 firmware (BIOS) using Linux
IBM provides a script for Linux called update_flash that uses the interfaces provided by rtas_flash. It supports the same options and syntax as the update_flash utility under AIX. This tool uses the Run-Time Abstraction Services (RTAS), which enables access to the firmware during the run time of the operating system. The interface functions are included in most recent 2.4 kernels and in all 2.6 kernels. You can obtain the latest version of the update_flash script by installing the diagnostic utilities for Linux on Advanced Performance Optimization with Enhanced RISC (POWER) that IBM distributes from the Web at: https://www14.software.ibm.com/webapp/set2/sas/f/lopdiags/home.html Note: Some Linux distributions provide a version of the update_flash script that is not fully functional. Always obtain the latest version of the update_flash script from IBM before you attempt to update the firmware.

Chapter 6. Hardware setup

101

To use the update_flash script, follow these steps: 1. Download the latest firmware from the IBM support Web site. 2. Transfer it to a disk that is accessible to the operating system running on the BladeCenter JS21. 3. Enter the update_flash command from the directory where the new firmware is located. The update_flash command requires that you have root privileges. The following example illustrates the usage of update_flash: /usr/sbin/update_flash -f ibm_fw_bios_mb-240.470.013_anyos_ppc64.img In this example, ibm_fw_bios_mb-240.470.013_anyos_ppc64.img is the firmware image that was previously downloaded from the IBM support Web site. The update_flash utility copies the new firmware image into the kernel and then reboots the operating system to perform the actual firmware update. Subsequently, if you have a problem with the new firmware and want to revert to the previous firmware level, then use the following command: /usr/sbin/update_flash -r If you are satisfied with the new firmware, commit the firmware upgrade before you install any future firmware by using the following command: /usr/sbin/update_flash -c

6.6.5 Updating BladeCenter JS21 Ethernet adapter using Linux
Get the Broadcom flash diagnostic utility as a Red Hat Package Manager (RPM) file. See “Finding Linux support for the BladeCenter JS21 in the Internet” on page 96. This RPM is part of the Linux on POWER diagnostic package. The software version used in this book is V1.24. With the RPM stored on the JS21 in the /srv/data/tools/ directory, issue the following command: rpm -i /srv/data/tools/bcmflashdiag-js20-1.24-1.ppc.rpm After you install the RPM, you can obtain information about the installed firmware level by issuing the following command: /usr/sbin/bcmflashdiag ethX

102

IBM BladeCenter JS21: The POWER of Blade Innovation

Here x is the Ethernet port in the range of 0 to 3. The output of this command depends on the actual firmware level, one possible output is shown in Figure 6-20. With an installed Ethernet expansion card, the onboard Ethernet adapter ports are in most cases eth2 and eth3. The firmware is always identical for eth0 and eth1 and accordingly for eth2 and eth3 because each interface pair uses one common firmware. Therefore, a firmware update for eth0 also affects eth1 and the same is true for eth2 and eth3. /usr/sbin/bcmflashdiag eth3 Firmware versions: Type --------BootCode ASF CFG ASF CPUB ASF CPUA ASF Init Version ------5780s-v3.18 40 ASFIPMI v6.08 ASFIPMI v6.08 ASFIPMI v6.08

Figure 6-20 Firmware information of onboard Ethernet adapter eth3

To update to a new firmware level, if the firmware is located in /srv/data/firmware/, use the following command: /usr/sbin/bcmflashdiag -n /srv/data/firmware/FIRMWARE.BIN ethX Here FIRMWARE.BIN is a placeholder for the actual name of the binary firmware file. See the command line output produced by this command in Figure 6-21. The flash process does not interrupt the data transfer on the interface, but you have to restart the BladeCenter JS21 to activate the new firmware. Attention: The binary firmware files for BladeCenter JS21 onboard Ethernet and Ethernet expansion cards are not compatible. To get a list with more options, issue /usr/sbin/bcmflashdiag without any parameters.

Chapter 6. Hardware setup

103

/usr/sbin/bcmflashdiag -n js21_318_608_ver2.bin eth3 Firmware versions: Type --------BootCode ASF CFG ASF CPUB ASF CPUA ASF Init Version ------5780s-v3.17 40 ASFIPMI v6.08 ASFIPMI v6.08 ASFIPMI v6.08

Start programming Broadcom firmware... File Name : js21_318_608_ver2.bin File Size : 131072 Offset : 0x0000 Bytes Read : 131072 Restoring original MAC address 00-11-25-C9-0B-A6 Restoring original MAC address 00-11-25-C9-0B-A7 Restoring original Serial Number: 0123456789 Broadcom firmware has been upgraded successfully. Reboot the machine to take effect.
Figure 6-21 Command line output during and after the flash process

6.6.6 Updating the management module and advanced management module firmware using the Web interface
To upgrade the firmware of the BladeCenter management module itself using the Web interface, perform the following steps: 1. Download the firmware update package from the IBM support Web site. See 6.6.1, “Getting the latest firmware, tools, and support” on page 95. This is usually a compressed file that contains one or more files with a PKT extension. Note: The management module requires two PKT files. For the advanced management module, we found the package to contain only one PKT file.

104

IBM BladeCenter JS21: The POWER of Blade Innovation

2. Extract the firmware update package into a directory on the workstation where you are running the Web browser that you use to connect the management module Web interface. Note for X Server users: Make sure that the directory where the PKT file is stored is accessible from the computer system that provides the Web browser. 3. Update each PKT file to the BladeCenter management module as explained in the following steps. a. In the navigation panel of the management module Web interface, select MM Control → Firmware Update, as shown in Figure 6-22. b. In the Update MM Firmware pane, select the Browse button and locate the directory where the PKT file is stored. c. Click the Update button to send and check the PKT file to the management module. This step might take some time to complete. Note: The Web interface is locked during download or update processes, but the command-line interface is still available.

Figure 6-22 Updating the management module firmware

Chapter 6. Hardware setup

105

d. A window opens that shows the progress of the download. Click the Continue button when prompted to confirm the firmware update. This step might also take some time to complete. Another window opens that shows you the progress. 4. After all PKT files are updated, you must restart the management module to activate the new firmware. Select MM Control → Restart MM in the management module Web interface. You can also use the update command on the BladeCenter management module command-line interface in combination with a running TFTP server. See 9.2.3, “Configuring a Trivial File Transfer Protocol service” on page 287.

6.6.7 Updating I/O module firmware using a Web interface
The method to update the firmware of a BladeCenter I/O module depends on the type of the module. As an example, we present the firmware update of a 4-port Gigabit Ethernet switch module from IBM. For information about upgrading firmware of other BladeCenter I/O modules use the search function in the following Web site: http://www.redbooks.ibm.com In the Search field, type BladeCenter AND Module. To update the firmware of a 4-port Gigabit Ethernet switch module from IBM using the switch module Web interface, follow this procedure: 1. Download the firmware update package from the IBM support Web site. See 6.6.1, “Getting the latest firmware, tools, and support” on page 95. This is usually a compressed file that contains the firmware file. In this case it is called ibmrun.096. 2. Extract the firmware update package into a directory on the workstation where you are running the Web browser that you use to connect the management module Web interface. 3. Before the firmware of the switch module can be updated, configure an IP address for the switch module management interface. This enables the capability to directly connect to the switch module Web interface. For details about how to configure the IP address of the switch module management interface, see 6.4, “I/O module configuration” on page 85.

106

IBM BladeCenter JS21: The POWER of Blade Innovation

4. Connect to the switch module Web interface using the management module Web interface using the following procedure: a. From the management module Web interface, select I/O Module Tasks → Management. b. Select the Advanced Management link for the I/O module bay where the switch module is installed. c. Scroll down to the bottom of the page to the Start Telnet/Web Session category. Click the Start Web Session button. d. A new window opens for the switch module Web interface. You are prompted to provide a user ID and a password. The default user ID is USERID, and the default password is PASSW0RD. Here the number 0 (zero) is between the letters W and R in the password. Consider changing the defaults in a production environment. You can also connect directly to the switch module user interface from any Web browser, if you know the IP address of the switch module. 5. Upgrade the firmware using the following procedure: a. From the switch module Web interface, select Maintenance → Using Browser → Upgrade Firmware/Configuration File. b. You see the screen shown in Figure 6-23. Select the Browse button and locate the switch module firmware file in the directory where you saved it earlier. c. Select the Start button to initiate the switch module firmware update process.

Chapter 6. Hardware setup

107

Figure 6-23 Firmware upgrade of 4-port Gb Ethernet switch module

6. A window opens requesting confirmation. After you confirm the update, the main window displays the status of the update until it completes and reboots the switch module. 7. After the restart of the Ethernet switch module, verify that the IP address of the switch module is still set correctly through the management module Web interface.

6.6.8 Updating IBM BladeCenter 2-port Fibre Channel switch module firmware using FTP
A graphical interface to this switch is provided by external tools such as the Blade Center SAN Utility and in most cases, you must update the firmware using these tools. See also Chapter 12, “Storage area network installation and booting” on page 409. In some cases, it might be necessary to perform a direct firmware update using the FTP service running on the switch module. This is described in the following section. We assume that the switch module is already configured using the BladeCenter management module. See 6.4, “I/O module configuration” on page 85.

108

IBM BladeCenter JS21: The POWER of Blade Innovation

Note: In other cases, it might be necessary to have an external FTP or TFTP server running to update firmware. To learn how to set up these services, see 9.2, “Basic preparations for a Linux network installation” on page 283. 1. Store the firmware file in a directory on your workstation, for example, /srv/data/firmware/IBM_2port_fc_switch/firmware_2.0.1.09 2. Connect to the FTP server running in the Fibre Channel switch module to transfer the firmware file. In this case, the module host name is ibm_fcsm_1. Log in as images, use the password images, and send the file to the Fibre Channel switch module. Example 6-3 shows the command line dialog. The commands are highlighted.
Example 6-3 FTP command line dialog

myworkstation:/srv/ftp/firmware # ftp ibm_fcsm_1 Connected to ibm_fcsm_1. 220 Sanbox2 FTP server (Version wu-2.6.1-18) ready. Name (ibm_fcsm_1:root): images 331 Password required for images. Password:****** 230 User images logged in. Access restrictions apply. Remote system type is UNIX. Using binary mode to transfer files. ftp> put firmware_2.0.1.09 local: firmware_2.0.1.09 remote: firmware_2.0.1.09 500 'EPSV': command not understood. 227 Entering Passive Mode (192,168,1,200,26,53) 150 Opening BINARY mode data connection for firmware_2.0.1.09. 100% |*************************************| 9148 KB 2.30 MB/s 00:00 ETA 226 Transfer complete. 9367804 bytes sent in 00:03 (2.29 MB/s) ftp> quit 221-You have transferred 9367804 bytes in 1 files. 221-Total traffic for this session was 9368346 bytes in 1 transfers. 221 Thank you for using the FTP service on Sanbox2.

Chapter 6. Hardware setup

109

3. Connect using Telnet, with the user name USERID and password as PASSW0RD. Issue the highlighted commands as shown in Example 6-4 to finish the firmware activation.
Example 6-4 Telnet command line dialog

linux:/srv/ftp/firmware # telnet ibm_fcsm_1 Trying ibm_fcsm_1... Connected to ibm_fcsm_1. Escape character is '^]'. Linux 2.4.18-xfs (Sanbox2) (01:28 on Friday, 01 January 1988)

Sanbox2 login: USERID Password:******** Establishing connection... Please wait.

***************************************************** * * * Command Line Interface SHell (CLISH) * * * ***************************************************** SystemDescription Switch Module Eth0NetworkAddress FCNetworkAddress MACAddress WorldWideName SymbolicName ActiveSWVersion ActiveTimestamp DiagnosticsStatus SecurityEnabled Alarms history ... -----------------[1][Fri Jan 01 00:01:08.332 1988][A1005.0002][cmon: unable to set gw addr] FCSM: USERID> admin start IBM BladeCenter(TM) 2-port Fibre Channel 9.3.5.23 (use 'set setup system' to update) 0.0.0.0 (use 'set setup system' to update) 00:c0:dd:02:1c:21 10:00:00:c0:dd:02:1c:22 FCSM V2.0.0.15-0 Wed Oct 29 21:18:28 2003 Passed False

110

IBM BladeCenter JS21: The POWER of Blade Innovation

FCSM (admin): USERID> image list firmware_2.0.1.09 FCSM (admin): USERID> image unpack firmware_2.0.1.09 Processing the command ... A notification with the command result will appear on the screen when the command has completed. FCSM (admin): USERID> image unpack command result: Passed. The firmware will become active and the SWImageVersion information will be updated after the next switch reset. reset The switch will be reset. Please confirm (y/n): [n] y

Alarm Msg: [Fri Jan 01 01:30:25.880 1988][A1000.000B][SM][The switch will be reset in 3 seconds] Reset switch in progress... Connection closed by foreign host.

6.6.9 Updating the firmware using the stand-alone diagnostics CD
The stand-alone diagnostics CD contains a bootable mini-AIX and uses diag to flash various firmware as described in Table 6-3 on page 97. You can use a Network Installation Manager (NIM) server to boot the stand-alone diagnostics to avoid the usage of the BladeCenter CD/DVD drive. See Chapter 8, “Installing AIX” on page 259. However, the following procedure uses the CD/DVD drive.

Chapter 6. Hardware setup

111

Tip: If you use a NIM server, you must select diag, as shown in Figure 8-8 on page 269. If you copy the firmware image into usr/lib/microcode on the NIM server's Shared Product Object Tree (SPOT), the client can access this image from /usr/lib/microcode, which is the Network File System (NFS) mounted from the NIM server during the network boot. You do not have to assign the media tray or use a CD or DVD media in this case. To enable the firmware update of onboard Ethernet or expansion cards using diag, it is essential that the firmware file name starts with an uppercase I.xxxxxxxxx and not with lowercase i.xxxxxxxxx. As an example, we describe the update process for the Qlogic 4Gb Fibre Channel expansion card. 1. Assign the CD/DVD drive to the BladeCenter JS21 where the expansion card for the update is installed. 2. Set the CD/DVD drive as first boot device using the management module. 3. Start the BladeCenter JS21 and wait until the boot process is finished.

112

IBM BladeCenter JS21: The POWER of Blade Innovation

4. Type 1 and press Enter to activate the console, as shown in Figure 6-24. Then follow the directions in the subsequent menus. -------------------------------------------------------------------Welcome to AIX. boot image timestamp: 15:58 12/19 The current time and date: 15:08:30 06/06/2006 number of processors: 4 size of memory: 3968MB boot device: /pci@8000000f8000000/pci@7/usb@0/hub@1/hub@1/cdrom@3:\ppc\chrp\bootf ile.exe kernel size: 13421062; 64 bit kernel -------------------------------------------------------------------******* Please define the System Console. ******* Type a 1 and press Enter to use this terminal as the system console. Pour definir ce terminal comme console systeme, appuyez sur 1 puis sur Entree. Taste 1 und anschliessend die Eingabetaste druecken, um diese Datenstation als Systemkonsole zu verwenden. Premere il tasto 1 ed Invio per usare questo terminal come console. Escriba 1 y pulse Intro para utilizar esta terminal como consola del sistema. Escriviu 1 1 i premeu Intro per utilitzar aquest terminal com a consola del sistema. Digite um 1 e pressione Enter para utilizar este terminal como console do sistema.
Figure 6-24 Selecting the system console

Chapter 6. Hardware setup

113

5. When the Function Selection menu opens, type 3 and press Enter, as shown in Figure 6-25. FUNCTION SELECTION 1 Diagnostic Routines This selection will test the machine hardware. Wrap plugs and other advanced functions will not be used. 2 Advanced Diagnostics Routines This selection will test the machine hardware. Wrap plugs and other advanced functions will be used. 3 Task Selection (Diagnostics, Advanced Diagnostics, Service Aids, etc.) This selection will list the tasks supported by these procedures. Once a task is selected, a resource menu may be presented showing all resources supported by the task. 4 Resource Selection This selection will list the resources in the system that are supported by these procedures. Once a resource is selected, a task menu will be presented showing all tasks that can be run on the resource(s). 99 Exit Diagnostics NOTE: The terminal is not properly initialized. You will be prompted to initialize the terminal after selecting one of the above options.

To make a selection, type the number and press Enter.
Figure 6-25 Function Selection menu

[3 ]

114

IBM BladeCenter JS21: The POWER of Blade Innovation

6. In most cases, you can choose the vt340 terminal type in the Define Terminal menu, as shown in Figure 6-26. With this terminal type, most functions keys (F-keys) must work as expected. However, even if the F1 to F4 keys are not working, you can press Esc and then use the regular 1, 2, 3, or 4 key to select F1 to F4 in a menu. DEFINE TERMINAL The terminal is not properly initialized. The following are some of the terminal types that are supported. ibm3101 ibm3151 ibm3161 ibm3162 ibm3163 ibm3164 ibmpc lft NOTE: tvi912 tvi925 tvi920 tvi950 vs100 vt100 vt320 sun vt330 vt340 wyse30 wyse50 wyse60 wyse100 wyse350

If you are using a Graphics Display, such as a 5081 or 6091 display, enter 'lft' as the terminal type. If the next screen is unreadable, press <CTRL> C.

Please enter a terminal type, or press Enter to return.
Figure 6-26 Define Terminal menu

Chapter 6. Hardware setup

115

7. After the Resource Database is built, the Task Selection List menu opens. Use the cursor keys to scroll down the list and choose Microcode Tasks, as shown in Figure 6-27. TASKS SELECTION LIST 801004

From the list below, select a task by moving the cursor to the task and pressing 'Enter'. To list the resources for the task highlighted, press 'List'. [MORE...18] Display or Change Bootlist Format Media Hot Plug Task Identify and Attention Indicators Local Area Network Analyzer Microcode Tasks Process Supplemental Media RAID Array Manager SSA Service Aids This selection provides tools for diagnosing and resolving problems on SSA attached devices. Update and Manage System Flash [BOTTOM] F1=Help F3=Previous Menu F4=List F10=Exit Enter

Figure 6-27 Task Selection List menu with highlighted Microcode Tasks selection

116

IBM BladeCenter JS21: The POWER of Blade Innovation

8. In the Microcode Tasks menu, choose Download Microcode to update the firmware of expansion cards or onboard devices, as shown in Figure 6-28. Microcode Tasks Move cursor to desired item and press Enter. Display Microcode Level Download Microcode Generic Microcode Download Update and Manage System Flash 801004

F1=Help F3=Previous Menu

F4=List

F10=Exit

Enter

Figure 6-28 Microcode Tasks

Chapter 6. Hardware setup

117

9. The Resource Selection List menu opens. The selectable devices depends on the specific BladeCenter JS21 configuration. Move the cursor to the device you require for a firmware update, as shown in Figure 6-29. Press F7 to commit the selection. RESOURCE SELECTION LIST 801006

From the list below, select any number of resources by moving the cursor to the resource and pressing 'Enter'. To cancel the selection, press 'Enter' again. To list the supported tasks for the resource highlighted, press 'List'. Once all selections have been made, press 'Commit'. To avoid selecting a resource, press 'Previous Menu'.

All Resources This selection will select all the resources currently displayed. U788D.001.23A1137sisioa0 P1 PCI-XDDR Dual Channel SAS RAID Adapter + fcs0 P1-C5-T1 FC Adapter fcs1 P1-C5-T2 FC Adapter ent0 P1-T7 Gigabit Ethernet-SX PCI-X Adapter (14101403) ent1 P1-T8 Gigabit Ethernet-SX PCI-X Adapter (14101403) F1=Help F3=Previous Menu F4=List F7=Commit F10=Exit

Figure 6-29 Resource Selection List with fcs0 selected for firmware update

118

IBM BladeCenter JS21: The POWER of Blade Innovation

10.If the device has two ports, for example, fcs0 and fcs1, both the ports will be updated, even if you have selected only one port. See Figure 6-30. INSTALL MICROCODE FC Adapter 802113

fcs0

Please stand by. lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk x x x x x [TOP] x x *** NOTICE *** NOTICE *** NOTICE *** x x x x fcs0 represents a port on a dual-port x x fibre channel adapter. To update microcode on x x this adapter, microcode needs to be installed x x on both fcs0 and fcs1. x x Selecting either fcs0 or fcs1 installs x x microcode on both ports. x x [MORE...11] x x x x F3=Cancel F10=Exit Enter x F3=Cancel mqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqj
Figure 6-30 Notice that both ports will be updated

11.In the next menu, select cd0 and press Enter.

Chapter 6. Hardware setup

119

12.Insert the CD with the microcode file stored in the root directory of the CD and confirm that the medium is in the CD/DVD drive. If the system does not find firmware files in the /etc/microcode directory on the CD-ROM, another message is shown. If you use the stand-alone diagnostics CD, this is always the case. If a valid microcode file is found, it is shown in the menu, as shown in Figure 6-31. Press Enter to start the update process. INSTALL MICROCODE FC Adapter 802116

fcs0

The current microcode level for fcs0 is 0003040010. Available levels to install are listed below. Select the microcode level to be installed. Use Help for explanations of "M", "L", "C" and "P". Make selection, use Enter to continue. M 0117040022

F1=Help

F10=Exit

F3=Previous Menu

Figure 6-31 Diag found a valid microcode on the CD

120

IBM BladeCenter JS21: The POWER of Blade Innovation

After the firmware update, you get the message that the installation is completed successfully, as shown in Figure 6-32. INSTALL MICROCODE fcs0 FC Adapter 802118

Installation of the microcode has completed successfully on fcs0 and fcs1. The current microcode level is 0003040016. Please run diagnostics on the resources fcs0 and fcs1 to ensure that the adapter is functioning properly. Use Enter to continue.

F3=Cancel

F10=Exit

Enter

Figure 6-32 Update successful message

6.7 Providing a console for the BladeCenter JS21
The BladeCenter servers JS20 and JS21 differ from other blade servers that are available for the BladeCenter in that they do not provide an interface to a KVM console. Therefore, you must set up the SoL remote text console function to provide a console (American Standard Code for Information Interchange (ASCII) terminal) for a BladeCenter JS20 and JS21. The use of SoL is optional for other types of blade servers that support KVM consoles. SoL is a virtual serial connection over Ethernet. It overcomes the limitations posed by a blade server with no video controller, and also offers an alternative way of connecting to an operational blade server. Note: You can use SoL to connect to running systems as well. However, we recommend that you use standard network access methods over the production network path if the network connection is working well because it provides a much better performance, and there is less risk of dropping connections. SoL is mostly intended during installation or when a normal network connection fails for some reason.

Chapter 6. Hardware setup

121

SoL provides a connection between the BladeCenter server and the management workstation in the external management network as shown in Figure 6-33 on page 123. Typically, there are two connections: One from the management workstation to the management module using Telnet or SSH, and one from the management module to the BSMP using SoL. The management module acts as a proxy between these two connections. Therefore, you cannot reach the BSMP directly from the management workstation. Note: The SoL ends at the BSMP that emulates the console (ASCII terminal) and not at the BladeCenter server’s IBM Power PC® processor. Therefore, the SoL session can be active even when the BladeCenter server is powered off. In addition, the ESM in switch bay 1 is used to create the path from the management module to the BSMP. However, its function is transparent if it is set up correctly. Note: For SoL to work, it is essential to have a supported and properly setup ESM in I/O module bay 1. SoL will not work over a copper or optical pass-thru module in bay 1 or an ESM in bay 2.

122

IBM BladeCenter JS21: The POWER of Blade Innovation

BladeCenter Chassis

Management Module (MM)
Management Interface Telnet/SSH SoL Proxy

INT (eth1)

EXT (eth0)

MANAGEMENT MODULE BAY 1 Ethernet Switch Module

Management Network

MGT 1 MGT 2 INT 1

EXT 1 EXT 2 EXT 3 EXT 4

. .

INT 14 Port 1 (eth0) Port 2 (eth1)

I/O MODULE BAY 1

SoL Data BSMP Build in Ethernet Controller

Management Workstation

BladeCenter JS21 BAY 1

Network (LAN) SoL VLAN default ID = 4095

Figure 6-33 SoL schematics

Note: Although it is possible to have concurrent SoL and regular Ethernet traffic across ESM 1 and the eth0 port of the blade server (Figure 6-33), there is a tendency for the SoL session to drop because of the traffic volume and resets of the eth0 port caused by the running software. The production network is typically configured to use ESM 2 and the eth1 Ethernet controller of the blade server, which requires at least two ESMs. The SoL remote console function works as follows: 1. Using a Telnet or SSH client, connect to the BladeCenter management module CLI. This is usually using an external management network that is connected to the management module’s 10/100BaseT Ethernet interface. 2. From the management module’s CLI, initiate an SoL remote console session to the blade server that you want.

Chapter 6. Hardware setup

123

3. The management module uses a private VLAN (typically VLAN ID 4095) provided by the Ethernet switch module in I/O module bay 1 (ESM 1), to transport the SoL data stream to the Ethernet interface of the target blade server’s primary Ethernet interface (eth0). 4. The Ethernet controller (network interface controller (NIC)) of the target blade server passes the SoL data stream received from the private VLAN to the BSMP using I2 C, which manages the ASCII text console for the blade server. Note: The management module can handle 20 login sessions in parallel. Therefore, it is possible to have up to 14 SoL sessions (one for each blade server) active at the same time leaving at least 6 additional logins open for other purposes.

6.7.1 Configuring Serial over LAN
Before you attempt to configure the SoL remote text console function, verify that you have all the prerequisites in place: A supported LAN switch module installed in I/O module bay 1 This switch module is used to provide a VLAN (typical VID 4095) that connects the management module to the first Ethernet interface (eth0) on each blade server. The minimum firmware levels described in 6.6, “Firmware” on page 94 This is important if you install a BladeCenter JS21 in an existing BladeCenter chassis that might have earlier firmware in the management module or Ethernet switch modules. A reliable Telnet or SSH client on the management workstation Note: Depending on the operating system where you run the Telnet or SSH client, there might be differences in the mapping of special keys such as function keys, cursor keys, Backspace, Delete, and Insert keys. Especially when you log on to AIX or Linux, you might encounter difficulties in System Management Interface Tool (SMIT) or Yet Another Setup Tool (YaST) menus. For most purposes, we found PUTTY to be a good choice for the client. A network connecting the management workstation to the BladeCenter management module external 10/100BaseT Ethernet interface An identified range of IP addresses that is used by the management module to communicate with the BSMP on each blade server using the private VLAN. This is discussed in Chapter 5, “Planning considerations” on page 45.

124

IBM BladeCenter JS21: The POWER of Blade Innovation

The easiest way to configure the SoL remote text console function is through the management module Web interface as explained in the following steps. Alternatively, you can perform the same task using the management module’s CLI. To configure the SoL remote text console function, use the following procedure: 1. From the management module Web interface, select Blade Tasks → Serial Over LAN, as shown in Figure 6-34. In the right pane, scroll down to the Serial Over LAN Configuration section. Complete the following tasks: a. From the Serial over LAN list, select the Enabled option. b. Leave the value for SoL VLAN ID at the default (4095) if you have either a 4-port Gigabit Ethernet switch module or a Nortel Networks Layer 2-7 Gigabit Ethernet switch module installed in I/O module bay 1. If you have a Cisco Systems Intelligent Gigabit Ethernet switch module installed in I/O module bay 1, set the VLAN ID to the same value that you used when you configured the Cisco Systems Intelligent Gigabit Ethernet switch module as described in 6.4.3, “Enabling the external I/O module ports” on page 88. c. In the BSMP IP address range field, enter the start of the IP address range that will be used by the management module to communicate with the BSMP on each blade server. d. Leave the values for Accumulate timeout, Send threshold, Retry count, and Retry interval at their defaults (5, 250, 3, and 250). e. In the User Defined Keystroke Sequences section, leave the values at their defaults. f. Click the Save button.

Chapter 6. Hardware setup

125

Figure 6-34 Serial over LAN configuration

2. Restart the management module. To do this, in the management module Web interface, select MM Control → Restart MM. Click the Restart button on the displayed page. You are prompted to confirm the restart before it occurs. Important: When you configure the SoL VLAN on the management module, it never asks you to reboot after saving. We recommended that you restart the management module after you configure the SoL VLAN properly, especially if the SoL session status does not become ready. 3. After you restart the management module and reconnect to the management module Web interface from your Web browser, enable the SoL remote text console for each blade server by doing the following steps. a. Select Blade Tasks → Serial Over LAN from the management module Web interface. In the right pane, scroll down until you see the Serial Over LAN Status section (see Figure 6-35 on page 127).

126

IBM BladeCenter JS21: The POWER of Blade Innovation

b. Select the blade servers you want to enable for SoL. You can choose all of them by selecting the check box at the top of the table. c. Click the Enable Serial Over LAN link below the table. You might have to scroll down to see this link. d. After a few seconds, the window refreshes. In the SoL column of the table, verify that each blade server has a status of Enabled and the SoL Session column has a status of Ready. Note: If the blade server does not show the Ready state, use Blade Tasks → Power/Restart → Restart Blade System Mgmt Processor or Blade Tasks → Power/Restart → Restart Blade.

Figure 6-35 Serial over LAN status

The configuration of the SoL remote text console function is now complete.

Chapter 6. Hardware setup

127

6.7.2 Using Serial over LAN
Access to the SoL remote text console function is through the management module CLI. The CLI is documented in the IBM BladeCenter Management Module Command-Line Interface Reference Guide, which is one of the documents provided on the CD supplied with your BladeCenter. You can also find it on the product document site described in “BladeCenter product documentation” on page 64. 1. Connect to the management module CLI using either a Telnet or SSH client. Note: To use SSH, you must first configure the management module using MM Control → Security as described in Advanced Management Module and Management Module User's Guide - IBM BladeCenter, BladeCenter T, BladeCenter H. 2. You are prompted to enter a user ID and password. Note: The default user ID is USERID and the default password is PASSW0RD, where 0 is a zero. All the letters are in uppercase. Consider changing the defaults in a production environment. Figure 6-36 shows an active Telnet session that displays the help command.

128

IBM BladeCenter JS21: The POWER of Blade Innovation

username: USERID password: ******** system> help ?- Display commands alertentries- View/edit remote alert recipients boot- Boot target clear- Clear the config clearlog- Clear the event log console- Start SOL session to a blade dhcpinfo- View DHCP server assigned settings displaylog- Display log entries dns- View/edit DNS config env- Set persistent command target exit- Log off fuelg- Power management health- View system health status help- Display command list history- Display command history identify- Control target location LED ifconfig- View/edit network interface config info- Display identity and config of target list- Display installed targets power- Control target power reset- Reset target shutdown- Shutdown target slp- View/edit SLP parameters smtp- View/edit SMTP config snmp- View/edit SNMP config sol- View SOL status and view/edit SOL config tcpcmdmode- View/edit TCP command mode config telnetcfg- View/edit telnet config update- Update firmware from TFTP server uplink- View/edit failover on network uplink loss config users- View/edit user login profiles Type "<command> -h" [ ] is used < > denotes { } denotes | denotes system> for individual command syntax help. for indexing (by bay number) a variable optional arguments choice

Figure 6-36 Management module’s command-line interface

Chapter 6. Hardware setup

129

The management module CLI is context-sensitive. When you issue CLI commands, you can accept the current context or override the context using the -T option provided on most commands. In our examples, we use the -T option to make it clear on which entity the command is operating. You can change the context for the management module CLI by using the env -T command. There are different commands that you can use to access the SoL remote text console function. Table 6-4 lists the most useful commands.
Table 6-4 Management module commands for SoL Command console console -o boot -c env -T reset -c power -on -c power -cycle -c Function Opens an SoL remote text console for the blade server. This command fails if another SoL remote text console is already open for the blade server. Terminates any existing SoL remote text console for the blade server and opens an SoL remote text console for the blade server. Resets the blade server, and then opens an SoL remote text console for the blade server. One of the built-in commands. Sets the target (management module, BladeCenter, blade server, or switch) for the current session. This is functionally equivalent to the boot -c command when used in a blade server context. Powers on the blade server and then opens an SoL remote text console for the blade server. Powers on the blade server and then opens an SoL remote text console for the blade server. If the blade server is already powered on, powers it off first, and then powers on.

We provide some examples of how to use these commands. To open an SoL remote text console to the blade server in bay 3, use the following command: console -T system:blade[3] Note: You can open many types of consoles to an installed BladeCenter JS21: SoL, SSH, Virtual Network Computing (VNC), and others. You can have only one active SoL remote text console connection to a BladeCenter JS21. When you try to open a second SoL session, you get the message: SOL session is already active. To force a console to be established if it is already active from elsewhere use: console -T system:blade[3] -o This interrupts the already existing SoL session.

130

IBM BladeCenter JS21: The POWER of Blade Innovation

To reset the blade server in bay 2 and then open an SoL remote text console to the blade server, use the following command: boot -c -T system:blade[2] To set the management module in bay 1 as the persistent target for commands during the current session: env -T system:mm[1] Note: To terminate an active SoL remote text console, press the Esc key followed by an open parenthesis “(“ (Shift+9 on U.S. keyboards). You can see active SoL sessions in the Web interface as shown for the blade server in bay 3 in Figure 6-37. Using the CLI, the following command shows you the status for only one blade server at a time: sol -T system:blade[3]

Figure 6-37 Active SoL session for blade server in bay 3

When the SoL remote text console ends, you return to the management module CLI prompt.

Chapter 6. Hardware setup

131

Inactivity timeout
The management module CLI has an aggressive default inactivity timeout. If you do not use the CLI or an SoL remote text console accessed through the CLI for 2 minutes, you are disconnected. You can increase the timeout using the telnetcfg command. For example, the following command increases the inactivity timeout to 10 minutes (600 seconds): telnetcfg -t 600 -T system:mm[1] Note: The command telnetcfg -t 0 means no timeout.

Setting carriage return-line feed
Over the years, there have been many variations on the interpretation of the Enter key in a Telnet session. Some Telnet daemons expect carriage return-line feed (CRLF) and some expect carriage return alone (CR). From our testing, CR seems to be the most compatible. During your SoL session, check whether the key acts as though it is pressed twice and not once when you press the Enter key. If it does, then you have to correct the CRLF option. Under Windows 2000 Telnet, do this as shown in Figure 6-38 before you open a connection to the management module. Microsoft (R) Windows 2000 (TM) Version 5.00 (Build 2195) Welcome to Microsoft Telnet Client Telnet Client Build 5.00.99206.1 Escape Character is 'CTRL+]' Microsoft Telnet> unset crlf Microsoft Telnet> open bcmm
Figure 6-38 Setting CRLF in Windows 2000 Telnet

Refer to the documentation of your client for other Telnet implementations.

132

IBM BladeCenter JS21: The POWER of Blade Innovation

6.8 System Management Services interface
This section explains how to access the SMS interface and how to use some of the functions that it provides. The SMS is the preferred interface to the BladeCenter JS21 firmware (BIOS) where you can configure initial and basic hardware settings such as boot order, network boot parameters, and firmware temporary versus permanent selection. Note: The BladeCenter JS21 supports the SMS and Open Firmware interface, but the BladeCenter JS20 supports only the Open Firmware interface.

6.8.1 Activating the System Management Services interface
You can access the SMS interface whenever you power on a BladeCenter JS21. Before you access the SMS interface, set up the SoL remote text console function as described in 6.7, “Providing a console for the BladeCenter JS21” on page 121. Use the following procedure to access the SMS interface: 1. Using a Telnet or SSH client, connect to the management module external Ethernet interface IP address. 2. When prompted, enter a valid user ID and password. The default management module user ID is USERID, and the default password is PASSW0RD, where the 0 is a zero. Consider changing the defaults in any production environment. 3. Power cycle the blade and start an SoL console by using the power -cycle -c command. For example, to power cycle and start an SoL remote text console with the blade server in the first bay (bay 1), use the command: power -cycle -c -T system:blade[1] To open a console with a blade that is already powered on, use the command: console -T system:blade[1] 4. After approximately 30 seconds, you see a sequence of checkpoint codes displayed on the console. These codes are generated by the Power On Self Test (POST). The meaning of the checkpoint codes is documented in the Problem Determination and Service Guide - IBM BladeCenter JS21 (Type 8844), which is on the CD that is shipped with your system. You can find the latest update of the document on the Web site described in “BladeCenter product documentation” on page 64.

Chapter 6. Hardware setup

133

5. If you get the message To select this console as the active console press 0, then press 0 (zero) to confirm the console. 6. When the POST menu and indicators are displayed as shown in Figure 6-39, type 1 after the word Keyboard is displayed and before the word Speaker is displayed. ... CA00D003 CA00D004 CA00E139 CA00E1FB CA00E100 CA00D008 CA00E1DC

1 = SMS Menu List 8 = Open Firmware Prompt

5 = Default Boot 6 = Stored Boot List

Memory

Keyboard

Network

SCSI

Speaker

Figure 6-39 POST checkpoint codes: SMS

134

IBM BladeCenter JS21: The POWER of Blade Innovation

7. Type 1 and wait until the SMS menu is displayed as shown in Figure 6-40. If you do not type any key when the POST menu and indicators are displayed, the blade server proceeds to boot using the default boot sequence. To learn how to configure the boot sequence, see 6.5, “Blade server configuration” on page 90.
PowerPC Firmware

Version MB240_470_009 SMS 1.6 (c) Copyright IBM Corp. 2000,2005 All rights reserved. -------------------------------------------------------------------Main Menu 1. Select Language 2. Setup Remote IPL (Initial Program Load) 3. Change SCSI Settings 4. Select Console 5. Select Boot Options 6. Firmware Boot Side Options 7. Progress Indicator History -------------------------------------------------------------------Navigation Keys: X = eXit System Management Services -------------------------------------------------------------------Type menu item number and press Enter or select Navigation key:
Figure 6-40 PowerPC Firmware: SMS menu

6.8.2 Setting up remote initial program load
This section describes how to set up a network boot that is necessary for a network installation of an operating system. When you want to boot or install from a local media (such as CD or DVD), perform the procedure as described in “Booting from CD/DVD” on page 145. Ensure that you have the information in the worksheet shown in Table 6-5 for your JS21 before you proceed with the installation.
Table 6-5 Network configuration information worksheet Network attribute Network interface Host name IP address _______.________.________.________ Value

Chapter 6. Hardware setup

135

Network attribute Network mask Name server Domain name Gateway

Value _______.________.________.________ _______.________.________.________

_______.________.________.________

Depending on the operating system you want to install, refer to 8.3, “Preparing AIX network installation using NIM” on page 261; 9.3, “Installing SLES using the network” on page 290; or 9.4, “Installing Red Hat Enterprise Linux AS 4 Update 3 using the network” on page 321.

Broadcast Bootstrap Protocol versus directed Bootstrap Protocol
You can use a broadcast Bootstrap Protocol (BOOTP) or directed BOOTP to install a JS21 blade from a network installation server. Broadcast BOOTP requires that you define the client’s hardware MAC address in the server’s BOOTP or DHCP configuration and additionally that client and server are in the same broadcast domain. However, a directed BOOTP does not require the network installation server to be on the same subnet as the client. This option also does not require that you define the MAC address of the client’s network adapter on the server. To perform a directed BOOTP, you require an SoL connection to the blade so that you can specify the IP parameters to SMS or Open Firmware. Attention: If your server uses DHCP instead of BOOTP, directed BOOTP might not work. NIM on AIX uses BOOTP, although on Linux the preferred method is DHCP. Our experience shows that directed BOOTP does not work with a DHCP server.

136

IBM BladeCenter JS21: The POWER of Blade Innovation

Specifying IP parameters in System Management Services
For a network installation, perform these steps: 1. Select option 2 to set up the remote IPL. 2. The screen shown in Figure 6-41 opens. As described in 6.7, “Providing a console for the BladeCenter JS21” on page 121, use the second port of the blade server’s network interface when using the standard built-in Ethernet interface. In this case, select the adapter with the location code “...-T8” by entering 2. PowerPC Firmware Version MB240_470_009 SMS 1.6 (c) Copyright IBM Corp. 2000,2005 All rights reserved. --------------------------------------------------------------------------NIC Adapters Device Location Code Hardware Address 1. Port 1-IBM 2 PORT 1000 Base-SX U788D.001.23A1137-P1-T7 001125c90ba6 2. Port 2-IBM 2 PORT 1000 Base-SX U788D.001.23A1137-P1-T8 001125c90ba7

--------------------------------------------------------------------------Navigation keys: M = return to Main Menu ESC key = return to previous screen X = eXit System Management Services --------------------------------------------------------------------------Type menu item number and press Enter or select Navigation key:
Figure 6-41 SMS remote IPL: NIC adapters

Chapter 6. Hardware setup

137

3. In the screen shown in Figure 6-42, type 1. PowerPC Firmware Version MB240_470_009 SMS 1.6 (c) Copyright IBM Corp. 2000,2005 All rights reserved. ------------------------------------------------------------------Network Parameters Port 2-IBM 2 PORT 1000 Base-SX PCI-X Adapter: U788D.001.23A1137-P1-T8 1. IP Parameters 2. Adapter Configuration 3. Ping Test 4. Advanced Setup: BOOTP

-------------------------------------------------------------------Navigation keys: M = return to Main Menu ESC key = return to previous screen X = eXit System Management Services ------------------------------------------------------------------Type menu item number and press Enter or select Navigation key:
Figure 6-42 SMS remote IPL: Network parameters

138

IBM BladeCenter JS21: The POWER of Blade Innovation

4. In the screen shown in Figure 6-43, enter the IP parameters. PowerPC Firmware Version MB240_470_009 SMS 1.6 (c) Copyright IBM Corp. 2000,2005 All rights reserved. -------------------------------------------------------------------IP Parameters Port 2-IBM 2 PORT 1000 Base-SX PCI-X Adapter: U788D.001.23A1137-P1-T8 1. Client IP Address [9.3.5.231] 2. Server IP Address [9.3.5.228] 3. Gateway IP Address [0.0.0.0] 4. Subnet Mask [255.255.255.000]

-------------------------------------------------------------------Navigation keys: M = return to Main Menu ESC key = return to previous screen X = eXit System Management Services -------------------------------------------------------------------Type menu item number and press Enter or select Navigation key:
Figure 6-43 SMS remote IPL: IP parameters

If you want, you can also check other menu items such as Adapter Configuration in Figure 6-42 on page 138 and Spanning Tree Enabled, which we recommend that you disable. Although this option was enabled in earlier firmware versions, it is already disabled in the current firmware. You do not have to change the speed and duplex settings for blade servers. Note: You can always get back to the main menu by pressing M (without pressing Enter).

Chapter 6. Hardware setup

139

6.8.3 Selecting boot options
In the main menu, type 5 for Select Boot Options. You can set the default Boot Device Order and the Install/Boot Device as shown in Figure 6-44. PowerPC Firmware Version MB240_470_009 SMS 1.6 (c) Copyright IBM Corp. 2000,2005 All rights reserved. -------------------------------------------------------------------Multiboot 1. Select Install/Boot Device 2. Configure Boot Device Order 3. Multiboot Startup <OFF>

-------------------------------------------------------------------Navigation keys: M = return to Main Menu ESC key = return to previous screen X = eXit System Management Services -------------------------------------------------------------------Type menu item number and press Enter or select Navigation key:
Figure 6-44 SMS Select Boot Options: Multiboot

140

IBM BladeCenter JS21: The POWER of Blade Innovation

Configuring boot device order
To Configure the boot device order: 1. Type 2 to display (option 6) or select a Boot Device (options 1- 5), as shown in Figure 6-45. Type 1 to select the first boot device. PowerPC Firmware Version MB240_470_013 SMS 1.6 (c) Copyright IBM Corp. 2000,2005 All rights reserved. -------------------------------------------------------------------Configure Boot Device Order 1. Select 1st Boot Device 2. Select 2nd Boot Device 3. Select 3rd Boot Device 4. Select 4th Boot Device 5. Select 5th Boot Device 6. Display Current Setting 7. Restore Default Setting

-------------------------------------------------------------------Navigation keys: M = return to Main Menu ESC key = return to previous screen X = eXit System Management Services -------------------------------------------------------------------Type menu item number and press Enter or select Navigation key:
Figure 6-45 SMS Select Boot Options: Configuring boot device order

Chapter 6. Hardware setup

141

2. The screen shown in Figure 6-46 opens. In this screen, specify 5 for hard drive. PowerPC Firmware Version MB240_470_013 SMS 1.6 (c) Copyright IBM Corp. 2000,2005 All rights reserved. -------------------------------------------------------------------Select Device Type 1. Diskette 2. Tape 3. CD/DVD 4. IDE 5. Hard Drive 6. Network 7. None 8. List All Devices

-------------------------------------------------------------------Navigation keys: M = return to Main Menu ESC key = return to previous screen X = eXit System Management Services -------------------------------------------------------------------Type menu item number and press Enter or select Navigation key:
Figure 6-46 SMS Select Boot Options: Selecting device type

142

IBM BladeCenter JS21: The POWER of Blade Innovation

3. In the screen shown in Figure 6-47, select the first adapter with the location code ending -T10. PowerPC Firmware Version MB240_470_013 SMS 1.6 (c) Copyright IBM Corp. 2000,2005 All rights reserved. -------------------------------------------------------------------Select Media Adapter 1. U788D.001.23A1137-P1-T10 /pci@8000000f8000000/pci@1/pci1014,028C@1/scsi@0 2. U788D.001.23A1137-P1-T11 /pci@8000000f8000000/pci@1/pci1014,028C@1/scsi@1 3. U788D.001.23A1137-P1-T12-T1 /pci@8000000f8000000/pci@1/pci1014,028C@1/scsi@ff 4. None 5. List all devices

-------------------------------------------------------------------Navigation keys: M = return to Main Menu ESC key = return to previous screen X = eXit System Management Services -------------------------------------------------------------------Type menu item number and press Enter or select Navigation key:
Figure 6-47 SMS Select Boot Options: Selecting media adapter

Chapter 6. Hardware setup

143

4. In the screen shown in Figure 6-48, select the device that you want. check /pci@8000000f8000000/pci@1/pci1014,028C@1/scsi@0/sd@1,0 PowerPC Firmware Version MB240_470_013 SMS 1.6 (c) Copyright IBM Corp. 2000,2005 All rights reserved. -------------------------------------------------------------------Select Device Device Current Device Number Position Name 1. SCSI 36401 MB Harddisk, part=1 () ( loc=U788D.001.23A1137-P1-T10-L1-L0 ) 2. None

-------------------------------------------------------------------Navigation keys: M = return to Main Menu ESC key = return to previous screen X = eXit System Management Services -------------------------------------------------------------------Type menu item number and press Enter or select Navigation key:
Figure 6-48 SMS Select Boot Options: Selecting device hard drive SCSI

After you select the necessary disk, you can see the current boot device order and go back to the main menu. Attention: If the firmware does not detect the disk as bootable (that is, having an operating system installed), it will not display the disks here. In particular, new drives or drives that have been reformatted with the RAID functions as described in 6.12, “SAS hardware RAID configuration” on page 170 cannot be seen here. They first show up in the installation dialog of the corresponding operating system or diagnostics.

144

IBM BladeCenter JS21: The POWER of Blade Innovation

Selecting install/boot device
This section shows you how to select the install/boot device.

Booting from CD/DVD
To boot from CD/DVD, perform these steps: 1. Select Select Boot Options → Select Install/Boot Device. 2. The menu shown in Figure 6-49 opens. Select 7. List all Devices. PowerPC Firmware Version MB240_470_013 SMS 1.6 (c) Copyright IBM Corp. 2000,2005 All rights reserved. -------------------------------------------------------------------Select Device Type 1. Diskette 2. Tape 3. CD/DVD 4. IDE 5. Hard Drive 6. Network 7. List all Devices

-------------------------------------------------------------------Navigation keys: M = return to Main Menu ESC key = return to previous screen X = eXit System Management Services -------------------------------------------------------------------Type menu item number and press Enter or select Navigation key:
Figure 6-49 SMS Select Install/Boot Device: Selecting device type

Attention: To select CD/DVD in a BladeCenter environment, you must select item 7. List all Devices and not items 3. CD/DVD or 4. IDE. This is because the CD-ROM in the BladeCenter is a Universal Serial Bus (USB) CD-ROM.

Chapter 6. Hardware setup

145

3. The screen in Figure 6-50 opens. This screen might look different on your blade server depending on the hardware configuration. We can see two interesting items in this screen: – Because a Virtual I/O Server (VIOS) is installed on this JS21, the virtual Ethernet adapters are also shown. However, the VIOS is not currently running because we are in the SMS. – More items can be displayed at once. Therefore, consider the option N = Next page of list within the navigation keys. In this screen, type N. PowerPC Firmware Version MB240_470_013 SMS 1.6 (c) Copyright IBM Corp. 2000,2005 All rights reserved. -------------------------------------------------------------------Select Device Device Current Device Number Position Name 1. - Virtual Ethernet ( loc=U8844.51X.23A0248-V1-C3-T1 ) 2. Virtual Ethernet ( loc=U8844.51X.23A0248-V1-C4-T1 ) 3. Virtual Ethernet ( loc=U8844.51X.23A0248-V1-C5-T1 ) 4. Virtual Ethernet ( loc=U8844.51X.23A0248-V1-C6-T1 ) 5. Ethernet ( loc=U788D.001.23A0248-P1-T7 ) 6. Ethernet ( loc=U788D.001.23A0248-P1-T8 ) -------------------------------------------------------------------Navigation keys: M = return to Main Menu N = Next page of list ESC key = return to previous screen X = eXit System Management Services -------------------------------------------------------------------Type menu item number and press Enter or select Navigation key:
Figure 6-50 SMS Select Install/Boot Device: Listing all devices

146

IBM BladeCenter JS21: The POWER of Blade Innovation

4. In the screen in Figure 6-51, select USB CD-ROM. PowerPC Firmware Version MB240_470_013 SMS 1.6 (c) Copyright IBM Corp. 2000,2005 All rights reserved. -------------------------------------------------------------------Select Device 7. - USB CD-ROM ( loc=U788D.001.23A0248-P1-T1-L1-L3 ) 8. 1 SCSI 36401 MB Harddisk, part=2 (AIX 5.3.0) ( loc=U788D.001.23A0248-P1-T10-L1-L0 )

-------------------------------------------------------------------Navigation keys: M = return to Main Menu P = Previous page of list ESC key = return to previous screen X = eXit System Management Services -------------------------------------------------------------------Type menu item number and press Enter or select Navigation key:
Figure 6-51 SMS Select Install/Boot Device: USB CD-ROM

Chapter 6. Hardware setup

147

5. In the next screen (Figure 6-52), select Normal Mode Boot. PowerPC Firmware Version MB240_470_013 SMS 1.6 (c) Copyright IBM Corp. 2000,2005 All rights reserved. -------------------------------------------------------------------Select Task USB CD-ROM ( loc=U788D.001.23A0248-P1-T1-L1-L3 ) 1. 2. 3. Information Normal Mode Boot Service Mode Boot

-------------------------------------------------------------------Navigation keys: M = return to Main Menu ESC key = return to previous screen X = eXit System Management Services ------------------------------------------------------------------Type menu item number and press Enter or select Navigation key:2
Figure 6-52 SMS Select Install/Boot Device: CD-ROM Select Task

Proceed with “Initiating boot” on page 151.

148

IBM BladeCenter JS21: The POWER of Blade Innovation

Booting from network
To select the install/boot device, we provide an example of how to use a network boot. 1. From the main menu, select Select Boot Options → Select Install/Boot Device → Network. 2. The screen in Figure 6-53 opens. In this screen, you do not see any virtual Ethernet interfaces, only physical ones. In other cases, you might see only virtual Ethernet interfaces if this is a logical partition (LPAR). Select the adapter with the location code ending with -T8 because the first adapter is used for SoL. PowerPC Firmware Version MB240_470_013 SMS 1.6 (c) Copyright IBM Corp. 2000,2005 All rights reserved. -------------------------------------------------------------------Select Device Device Current Device Number Position Name 1. Ethernet ( loc=U788D.001.23A1137-P1-T7 ) 2. Ethernet ( loc=U788D.001.23A1137-P1-T8 ) 3. None

-------------------------------------------------------------------Navigation keys: M = return to Main Menu ESC key = return to previous screen X = eXit System Management Services -------------------------------------------------------------------Type menu item number and press Enter or select Navigation key:
Figure 6-53 SMS Select Install/Boot Device: Network

Chapter 6. Hardware setup

149

3. In Figure 6-54, choose Normal Mode Boot. PowerPC Firmware Version MB240_470_009 SMS 1.6 (c) Copyright IBM Corp. 2000,2005 All rights reserved. -------------------------------------------------------------------Select Task Ethernet ( loc=U788D.001.23A1137-P1-T8 ) 1. 2. 3. Information Normal Mode Boot Service Mode Boot

-------------------------------------------------------------------Navigation keys: M = return to Main Menu ESC key = return to previous screen X = eXit System Management Services -------------------------------------------------------------------Type menu item number and press Enter or select Navigation key:
Figure 6-54 SMS Select Install/Boot Device: Selecting the task

150

IBM BladeCenter JS21: The POWER of Blade Innovation

Initiating boot
To boot from your selected device, you must exit the SMS menu. To do this, select 1 in Figure 6-55. PowerPC Firmware Version MB240_470_013 SMS 1.6 (c) Copyright IBM Corp. 2000,2005 All rights reserved. -------------------------------------------------------------------Are you sure you want to exit System Management Services? 1. Yes 2. No

-------------------------------------------------------------------Navigation Keys: X = eXit System Management Services -------------------------------------------------------------------Type menu item number and press Enter or select Navigation key:
Figure 6-55 SMS Select Boot Options: Exiting SMS

The system tries to boot from the selected media. In the case of a network boot, you see a screen similar to Figure 6-56. However, this requires that the network boot is prepared as described in Chapter 8, “Installing AIX” on page 259 or Chapter 9, “Installing Linux” on page 281. STARTING SOFTWARE PLEASE WAIT...

BOOTP: BOOTP: BOOTP: BOOTP: BOOTP: BOOTP: BOOTP: BOOTP:

chosen-network-type = ethernet,auto,none,auto server IP = 9.3.5.228 requested filename = client IP = 9.3.5.231 client HW addr = 0 11 25 c9 b a7 gateway IP = 0.0.0.0 device /pci@8000000f8000000/pci@2/ethernet@4,1 loc-code U788D.001.23A1137-P1-T8

Elapsed time since release of system processors: 0 mins 48 secs
Figure 6-56 Starting software BOOTP

Chapter 6. Hardware setup

151

Other SMS options
For other SMS options such as Select Language, Change SCSI Settings, Select Console, Firmware Boot Side Options, and Progress Indicator History, see Installation and User's Guide - IBM BladeCenter JS21 (Type 8844). To access these documents, follow the procedure described in “BladeCenter product documentation” on page 64.

6.9 Open Firmware interface
In some situations, you might want to access the Open Firmware interface to the BladeCenter JS21 firmware, either for debugging or because you are already familiar with Open Firmware. This section explains how to access the Open Firmware interface and how to use some of the functions they provide. Because SMS is the recommended interface to use, we show only a few examples. You can find further information about Open Firmware technology on the following Web site: http://www.firmworks.com There is no guarantee that everything you find in this site will exactly match the JS21’s Open Firmware implementation.

6.9.1 Activating the Open Firmware interface
You can access the Open Firmware interface whenever you power on a BladeCenter JS21. Before you access the Open Firmware interface, set up the SoL remote text console function as described in 6.7, “Providing a console for the BladeCenter JS21” on page 121. Use the following procedure to access the Open Firmware interface: 1. Using a Telnet or SSH client, connect to the management module external Ethernet interface IP address. 2. When prompted, enter a valid user ID and password. The default management module user ID is USERID, and the default password is PASSW0RD, where 0 is a zero. Consider changing the defaults in any production environment. 3. Power cycle the blade and start an SoL console by using the power -cycle -c command. For example, to power cycle and start a SoL remote text console with the blade server in the first bay (bay 1), use the command: power -cycle -c -T system:blade[1]

152

IBM BladeCenter JS21: The POWER of Blade Innovation

4. After approximately 30 seconds, you see a sequence of checkpoint codes displayed on the console. These codes are generated by the Power On Self Test (POST). The meaning of the checkpoint codes is documented in the Problem Determination and Service Guide - IBM BladeCenter JS21 (Type 8844), which is on the CD that is shipped with your system. You can also find the latest update of the document on the Web site described in “BladeCenter product documentation” on page 64. 5. If you get the message To select this console as the active console press 0, then press 0 (zero) to confirm the console. 6. When the POST menu and indicators are displayed as shown in Figure 6-57, type 8 after the word Keyboard is displayed and before the word Speaker is displayed. ... CA00D003 CA00D004 CA00E139 CA00E1FB CA00E100 CA00D008 CA00E1DC

1 = SMS Menu 5 = Default Boot List 8 = Open Firmware Prompt

6 = Stored Boot List

Memory

Keyboard

Network

SCSI

Speaker

Figure 6-57 POST checkpoint codes: Open Firmware

7. Type 8 and wait until the Open Firmware prompt is shown. If you do not type any key when the POST menu and indicators are displayed, the blade server proceeds to boot using the default boot sequence. To learn how to configure the boot sequence, see 6.5, “Blade server configuration” on page 90. In the following sections, we provide some examples of how to use the Open Firmware interface.

Chapter 6. Hardware setup

153

Printenv command
The printenv [variable] command shows the values of variables. If you do not specify any variable name, it shows all the configuration variables and their current and default values. Figure 6-58 and Figure 6-59 on page 155 show the output of the printenv command on a completely new BladeCenter JS21. 0 > printenv ---------- Partition: of-config -------- Signature: 0x50 ---------ibm,fw-dc-select 100 0 ibm,fw-default-mac-address? false false ibm,fw-forced-boot ibm,fw-n-bc 255.255.255.255 255.255.255.255 ibm,fw-n-bretry 00 00 ibm,fw-n-tretry 00 00 ibm,fw-n-dbfp 00000000 00000000 ibm,fw-n-dafp 00000000 00000000 ibm,fw-n-rc A A ibm,fw-n-ru Y Y ---------- Partition: common -------- Signature: 0x70 -------------little-endian? false false real-mode? true true auto-boot? true true diag-switch? false false fcode-debug? false false oem-banner? false false oem-logo? false false use-nvramrc? true false ibm,fw-tty-language 1 1 ibm,fw-new-mem-def false false ibm,fw-prev-boot-vpd 00000000: 08 07 07 0c 09 ff 07 07 07 07 07 07 07 07 07 07 |................| ibm,fw-keyboard real-base virt-base real-size virt-size load-base screen-#columns screen-#rows selftest-#megs 1 c00000 ffffffff 1000000 ffffffff 4000 64 28 0 1 c00000 ffffffff 1000000 ffffffff 4000 64 28 0

Figure 6-58 Open Firmware printenv output: Page 1 of 2

154

IBM BladeCenter JS21: The POWER of Blade Innovation

boot-device boot-file diag-device diag-file output-device input-device oem-banner oem-logo nvramrc boot-command reboot-command menu? ibm,fw-find-tape-alias ibm,fw-find-cdrom-alias ibm,dasd-spin-interval security-#badlogins security-mode security-password pw-status ok 0 >

hdd0 network1 network cdrom hdd1 hdd0 network1 network cdrom hdd1 diag diag /vdevice/vty@30000000 com1 /vdevice/vty@30000000 com1

Defined : use NVEDIT related words to view boot boot false false false 5 0 none false true true 5

Figure 6-59 Open Firmware printenv output: Page 2 of 2

After you set the parameters for remote IPL and install AIX, you can see additional entries as shown in Figure 6-60. Note: When you have already configured a JS21 blade, you might find additional entries in the output of the printenv command as shown in Figure 6-60.

Chapter 6. Hardware setup

155

... pw-status bootinfo-aix /pci@8000000f8000000/pci@1/pci1014,028C@1/scsi@0/s d@1,0:1 nu-001125c9113c auto,auto,000.000.000.000,000.000.000.000,000.000.000.000,255.255.25 5.000,Standard,No,,5,512,5 ibm,fw-menu-001125c9113d auto,auto,000.000.000.000,000.000.000.000,000.000.000.000,255.255.25 5.000,Standard,No,,5,512,5 ok 0 >
Figure 6-60 Open Firmware: Printenv configured output

Devalias command
In Open Firmware you can define aliases to specify devices in a more convenient way instead of using full path names. Use the following command to set aliases for the current session: devalias alias device-path Alternatively, use the following command to set them as persistent in the NVRAM. nvalias alias device-path

156

IBM BladeCenter JS21: The POWER of Blade Innovation

Figure 6-61 shows the aliases that we found on JS21 blade using the firmware as stated in 6.6, “Firmware” on page 94. Attention: Be aware that the aliases found in a completely new JS21 might not be set up correctly already. You might not yet find the aliases net and cdrom, and the aliases for hdd* and network* might not point to the correct Peripheral Component Interconnect (PCI) adapter. Correct this if you define the boot sequence as described in 6.5.2, “Setting the boot sequence” on page 91.

0 > devalias ibm,sp hdd1 hdd0 disk network net network1 scsi cdrom nvram rtc screen ok 0 >

/vdevice/IBM,sp@4000 /pci@8000000f8000000/pci@1/pci1014,028C@1/scsi@1/sd@1,0 /pci@8000000f8000000/pci@1/pci1014,028C@1/scsi@0/sd@1,0 /pci@8000000f8000000/pci@1/pci1014,028C@1/scsi@0/sd@1,0 /pci@8000000f8000000/pci@2/ethernet@4 /pci@8000000f8000000/pci@2/ethernet@4 /pci@8000000f8000000/pci@2/ethernet@4,1 /pci@8000000f8000000/pci@1/pci1014,028C@1/scsi@0 /pci@8000000f8000000/pci@7/usb@0/hub@1/hub@1/cdrom@3 /vdevice/nvram@4002 /vdevice/rtc@4001 /vdevice/vty@30000000

Figure 6-61 Open Firmware: Devalias configured output

Ls command
Use the ls command to display the names of the current node’s children as shown in Figure 6-62 on page 158 and Figure 6-63 on page 159. The output might be useful if you do not have an alias already and want to specify a device or define an alias. It is also useful to check whether the aliases that are already defined (see “Devalias command” on page 156) are correct. Be aware that the output is actually a tree example. Therefore, the second Ethernet adapter is: /pci@8000000f8000000/pci@2/ethernet@4,1 This is the adapter referred to by the network1 alias in Figure 6-61.

Chapter 6. Hardware setup

157

0 > ls 000000c887d0: 000000c89658: 000000c89898: 000000c89990: 000000c8fd68: 000000cd0288: 000000cd0a48: 000000cd60f0: 000000cd7190: 000000cdbc10: 000000cdbef8: 000000ceab88: 000000cf0cd8: 000000cf14f8: 000000cf3870: 000000cfd0f8: 000000d16518: 000000d1c060: 000000d1d420: 000000d31578: 000000d31758: 000000d32008: 000000d357f0: 000000d36568: 000000d36660: 000000de6ba0: 000000e02340: 000000e61338: 000000ca25a0: 000000ca6a90: 000000ca7af8: 000000ca81b0: 000000ca8dd8: 000000ca99c8: 000000ca9df8: 000000caa9e8: 000000caae18: 000000cb0788: 000000cb0968: 000000cb0b48: 000000cb0d28:

/ibm,serial /chosen /packages /disassembler /assembler /dev-tree /lpevents /deblocker /disk-label /tape-label /obp-tftp /ip /prep-boot /fat-files /iso-13346-files /utilities /net /iso-9660-files /boot-mgr /chrp-loader /pe-loader /elf-loader /nls-support /terminal-emulator /dynamic-reconfig /gui /iscsi /post /cpus /PowerPC,POWER@0 /l2-cache@2000 /PowerPC,POWER@1 /PowerPC,POWER@2 /l2-cache@2001 /PowerPC,POWER@3 /l2-cache@2002 /memory@0 /memory@8000000 /memory@9000000 /memory@a000000 /memory@b000000 ...

Figure 6-62 Open Firmware ls: Page 1 of 2

158

IBM BladeCenter JS21: The POWER of Blade Innovation

000000ccc5c8: 000000ccc7a8: 000000cccdc8: 000000ccdd08: 000000d36b28: 000000d38998: 000000d3a4c8: 000000d3a630: 000000d3be60: 000000d3dc08: 000000d43ef0: 000000d46f38: 000000d47ea8: 000000d49188: 000000d4a220: 000000d4ab10: 000000d4b148: 000000d4d5f0: 000000d55088: 000000d6e6d0: 000000d785e0: 000000d7ac40: 000000d7c660: 000000d7e1a8: 000000d80808: 000000d82228: 000000d83d70: 000000d863d0: 000000d87df0: 000000d5bd30: 000000d89a10: 000000d990b0: 000000d629b8: 000000da8750: 000000dc4c68: 000000dc5f58: 000000de2470: 000000d6c6c8: ok 0 >

/memory@f6000000 /memory@f7000000 /options /aliases /openprom /event-sources /epow-events /interrupt-controller@0 /interrupt-controller@f8000400 /rtas /vdevice /vty@30000000 /vty@30000001 /IBM,sp@4000 /rtc@4001 /nvram@4002 /tpm@100f4003000 /pci@8000000f8000000 /pci@1 /pci1014,028C@1 /scsi@0 /sd /st /scsi@1 /sd /st /scsi@ff /sd /st /pci@2 /ethernet@4 /ethernet@4,1 /pci@7 /usb@0 /hub@1 /usb@0,1 /hub@1 /tnk-memory-map

Figure 6-63 Open Firmware ls: Page 2 of 2

Chapter 6. Hardware setup

159

6.9.2 Specifying IP parameters to Open Firmware
You have to specify the IP parameters only if you want to use a directed BOOTP as explained in “Broadcast Bootstrap Protocol versus directed Bootstrap Protocol” on page 136.

Specifying a directed BOOTP from the JS21
Typically, you can initiate a directed BOOTP in Open Firmware by typing: boot net:bootp,server_ip,,client_ip,gateway_ip Here server_ip, client_ip and gateway_ip are the IP addresses of the network installation server, the client, and an optional gateway (note the double comma). If there is no gateway, use 0.0.0.0 for the gateway_ip address. Because SoL uses ent0, use ent1 as explained in 6.7, “Providing a console for the BladeCenter JS21” on page 121. First you have to check whether an alias net is defined using the devalias command, as shown in “Devalias command” on page 156. Figure 6-61 on page 157 shows that the alias net points to the first Ethernet port which is ent0. Therefore, you can use the ls command (as shown in “Ls command” on page 157) to determine the full path name of the second Ethernet port and run a command similar to: boot /pci@8000000f8000000/pci@2/ethernet@4,1:bootp,9.3.4.9,,9.3.5.3,9.3.5.1 Alternatively, you can use the alias network1 (as shown in Figure 6-61 on page 157). This alias refers to /pci@8000000f8000000/pci@2/ethernet@4,1: boot network1:bootp,9.3.4.9,,9.3.5.3,9.3.5.1

160

IBM BladeCenter JS21: The POWER of Blade Innovation

Note: You must specify the full device path name with this command. To determine the full path to your device, list the device tree by running the ls command at the Open Firmware prompt. This command displays an output similar to Figure 6-64. The highlighted items are the path to the second Ethernet adapter. Pass this information to the boot command to initiate a network boot from the second Ethernet adapter.

0 > ls 000000c887d0: 000000c89658: 000000c89898: ... 000000d4b148: 000000d4d5f0: 000000d55088: 000000d6e6d0: 000000d785e0: 000000d7ac40: 000000d7c660: 000000d7e1a8: 000000d80808: 000000d82228: 000000d83d70: 000000d863d0: 000000d87df0: 000000d5bd30: 000000d89a10: 000000d990b0: 000000d629b8:

/ibm,serial /chosen /packages /tpm@100f4003000 /pci@8000000f8000000 /pci@1 /pci1014,028C@1 /scsi@0 /sd /st /scsi@1 /sd /st /scsi@ff /sd /st /pci@2 /ethernet@4 /ethernet@4,1 /pci@7

Figure 6-64 ls command output

After you run the boot command, the network installation begins and you see an output similar to Figure 6-56 on page 151.

Chapter 6. Hardware setup

161

6.10 Blade server Ethernet controller enumeration
The Standard (73P9030) and SFF (26K4842) Gigabit Ethernet expansion cards increase the number of Gigabit Ethernet network interfaces on the BladeCenter JS21 from two to four. The enumeration of the Ethernet controllers in a blade server is operating system dependent. You must number the two interfaces (ports) consecutively on the controller itself with a lower number for the primary and a higher number for the secondary interface. Because of the changes in the hardware layout, the enumeration of the Ethernet controllers on the JS21 is different from the JS20. Attention: If you install an Ethernet expansion card (daughter card) in a JS21 blade server, the expansion card might be configured first, with the logical names eth0 and eth1, and the integrated Ethernet interfaces get the names eth2 and eth3. On AIX, this happens if you have already installed the Ethernet expansion card before installing AIX. However, when you install the Ethernet expansion after installing AIX, the names eth0 and eth1 remain on the integrated Ethernet interfaces because AIX has saved this information in the Object Data Manager (ODM). If this is an issue, you must first install the operating system to allow the onboard ports to be recognized and configured before the ports on the expansion card. If you install the Ethernet expansion card before you install the operating system, be aware that the expansion card ports are assigned before the onboard ports.

Changing Ethernet controller enumeration on AIX
For AIX, there are other ways to change the Ethernet controller enumeration. Example 6-5 on page 163 shows one of these methods. This method changes the ODM. Example 6-6 on page 167 shows a method that deletes the device first and recreates it with the name that you want.

Example: Changing the Object Data Manager
Example 6-5 shows you how to change the name of an Ethernet interface in the ODM. The main functions of the code are: Changing devices to defined state because later we do not want to add a drive into the ODM, which is not already there but has the available state Retrieving all related configuration data from the customized ODM and saving it in a text file. For Ethernet interfaces, we also require enX and etX besides the entX.

162

IBM BladeCenter JS21: The POWER of Blade Innovation

Changing the name in the text file using standard string manipulation (we use sed) Removing the old device Adding the changed text file’s data back to the ODM Configuring the new device to the available state
Example 6-5 Changing Ethernet enumeration using ODM

#!/bin/ksh Abort () { echo "Error: $*" 1>&2 ; exit 1; } if [ $# -ne 2 ] ; then echo "Usage: $0 entX entY ; where entX & entY are old & new adapter names" exit 1 fi if [ "`id -un`" != "root" ] ; then Abort "$0 must be run as root" fi case "$1" in ent[0-9]*) entX="$1" ;; *) Abort "ERROR: entX format is incorrect" ;; esac case "$2" in ent[0-9]*) entY="$2";; *) Abort "ERROR: entY format is incorrect" ;; esac # remove "ent" to get number only idX=${entX##ent} idY=${entY##ent} # get correspondin en and et device name enX=en$idX etX=et$idX enY=en$idY etY=et$idY

Chapter 6. Hardware setup

163

# check if entX is present lsattr -El $entX >/dev/null 2>&1 if [ $? -ne 0 ] ; then Abort "\"$entX\" does not exist" fi # while entY must not exist yet lsattr -El $entY >/dev/null 2>&1 if [ $? -eq 0 ] ; then Abort "\"$entY\" exists already" fi echo "Changing $entX $en and $et to Defined status" for i in $enX $etX $entX do rmdev -l $i if [ $? -ne 0 ] ; then Abort "\"rmdev -l $i\" FAILED" fi done echo "Gathering ODM data for $entX" odmget -q name="$entX" CuDv >/tmp/$entX if [ $? -ne 0 ] ; then Abort "\"odmget -q name=$entX CuDv\" FAILED" fi odmget -q name="$entX" CuAt >>/tmp/$entX if [ $? -ne 0 ] ; then Abort "\"odmget -q name=$entX CuAt\" FAILED" fi odmget -q value3="$entX" CuDvDr >>/tmp/$entX if [ $? -ne 0 ] ; then Abort "\"odmget -q value3=$entX CuDvDr\" FAILED" fi odmget -q name="$entX" CuVPD >>/tmp/$entX if [ $? -ne 0 ] ; then Abort "\"odmget -q name=$entX CuVPD\" FAILED" fi odmget -q name="$enX" CuDv >>/tmp/$entX 2>/dev/null if [ $? -ne 0 ] ; then

164

IBM BladeCenter JS21: The POWER of Blade Innovation

Abort "\"odmget -q name=$enX CuDv\" FAILED" fi odmget -q name="$enX" CuAt >>/tmp/$entX 2>/dev/null if [ $? -ne 0 ] ; then Abort "\"odmget -q name=$enX CuAt\" FAILED" fi odmget -q name="$etX" CuDv >>/tmp/$entX 2>/dev/null if [ $? -ne 0 ] ; then Abort "\"odmget -q name=$et CuDv\" FAILED" fi odmget -q name="$et" CuAt >>/tmp/$entX 2>/dev/null if [ $? -ne 0 ] ; then Abort "\"odmget -q name=$et CuAt\" FAILED" fi echo "Preparing ODM directives to change $entX to $entY" sed "s/= \"$entX\"/= \"$entY\"/ s/= \"$enX\"/= \"$enY\"/ s/= \"$etX\"/= \"$etY\"/g" /tmp/$entX >/tmp/$entY if [ $? -ne 0 ] ; then Abort "\"sed\" of ODM directives FAILED" fi echo "Removing $entX and deleting definition" for i in $enX $etX $entX do rmdev -dl $i if [ $? -ne 0 ] ; then Abort "\"rmdev -dl $i\" FAILED" fi done echo "Critical point - Modifying ODM to add $entY" odmadd /tmp/$entY if [ $? -ne 0 ] ; then Abort "\"odmadd /tmp/$entY\" FAILED" fi

Chapter 6. Hardware setup

165

echo "Remaking device $entY" mkdev -l $entY if [ $? -ne 0 ] ; then Abort "\"mkdev -l $entY\" FAILED" fi echo "Done"

Example: Using the device method
The second way to change the Ethernet device name is to use the device method. The advantage is that you do not have to directly manipulate the ODM. The disadvantage is that the device information is deleted and all the settings such as IP address are lost. But this does not cause any harm because we typically perform this operation on a newly installed machine. Use the following command to manually create an Ethernet interface and specify the name that you want: /usr/lib/methods/define_rspc -c adapter -s pci -t e414a816 -p pci1 -w 8 -d -L '01-08' -l ent5 In this example, we use ent5 as the required name. Note: This is not an officially supported AIX command. Therefore, it is not documented and its availability or function might change in the future. The main problem is to get the necessary parameters, which depend on the hardware of the blade server and the type of expansion card. Use the following command to extract the information from the ODM, when the interface has to be configured. odmget -q name=ent0 CuDv

166

IBM BladeCenter JS21: The POWER of Blade Innovation

Figure 6-65 shows that the class (-c flag), subclass (-s), and the type (-t) come from the PdDvLn entry. The parent (-p), location (-L), and connection (connwhere, -w) have separate entries. CuDv: name = "ent0" status = 1 chgstatus = 0 ddins = "pci/bentdd" location = "01-08" parent = "pci1" connwhere = "8" PdDvLn = "adapter/pci/e414a816"
Figure 6-65 Ethernet controller ODM data

The actual task shown in Example 6-6 performs the following functions: Extract ODM data Remove the enX, etX, and entX interface Create the new entY interface with the name that you want Run cfgmgr to change the entY from defined to available state and to create the corresponding interfaces enY and etY.
Example 6-6 Changing Ethernet enumeration using the device method

#!/bin/ksh Abort () { echo "Error: $*" 1>&2 ; exit 1; } if [ $# -ne 2 ] ; then echo "Usage: $0 entX entY ; where entX & entY are old & new adapter names" exit 1 fi if [ "`id -un`" != "root" ] ; then Abort "$0 must be run as root" fi case "$1" in ent[0-9]*) entX="$1" ;; *) Abort "ERROR: entX format is incorrect" ;; esac

Chapter 6. Hardware setup

167

case "$2" in ent[0-9]*) entY="$2";; *) Abort "ERROR: entY format is incorrect" ;; esac idX=${entX##ent} idY=${entY##ent} lsattr -El $entX >/dev/null 2>&1 if [ $? -ne 0 ] ; then Abort "\"$entX\" does not exist" fi lsattr -El $entY >/dev/null 2>&1 if [ $? -eq 0 ] ; then Abort "\"$entY\" already not exist" fi odmget -q name=$entX CuDv | awk -v IdX=$idX -v IdY=$idY ' { gsub("\"","") } $1 $1 $1 $1 == == == == "PdDvLn" { split($3,Pd,"/") } "location" { Loc=$3 } "connwhere" { Conn=$3 } "parent" { Parent=$3 } Parent

END { args="-c " Pd[1] " -s " Pd[2] " -t " Pd[3] " -p " args=args " -w " Conn " -d -L " Loc " -l ent" IdY system ("rmdev -dl en" IdX ) system ("rmdev -dl et" IdX ) system ("rmdev -dl ent" IdX ) system ("/usr/lib/methods/define_rspc " args) print "" system ("cfgmgr") } '

168

IBM BladeCenter JS21: The POWER of Blade Innovation

6.11 Blade server I/O module routing
The routing of an Ethernet controller to a particular I/O module bay depends on the type of blade server and the type of the expansion card. For the JS21, the routing is as shown in Table 6-6.
Table 6-6 BladeCenter I/O routing Controller Integrated Ethernet Interface Interface 1 Interface 2 Expansion card Interface 1 Interface 2 Interface 1 Interface 2 I/O module bay 1 2 3 4 4 3

(except 73P9030)
Standard (73P9030) Gigabit Ethernet expansion card

You can verify which Ethernet controller is routed to which I/O module bay by using the following test: 1. Install only one Ethernet switch module or pass-thru module in I/O module bay 1. 2. Ensure that the ports on the switch module or pass-thru module are enabled (click I/O Module Tasks → Admin/Power/Restart in the management module Web interface). 3. Enable only one of the Ethernet controllers on the blade server. Note the designation that the blade server operating system has for the controller. 4. Use ping in an external computer in the network that is connected to the switch module or pass-thru module. If you can use ping in the external computer, the Ethernet controller that you enabled is associated with the switch module or pass-thru module in I/O-module bay 1. The other Ethernet controller in the blade server is associated with the switch module or pass-thru module in I/O-module bay 2. If you have installed an I/O expansion card in the blade server, communications from the expansion card are routed to I/O module bay 3 and module bay 4, if these bays are supported by your BladeCenter unit. You can verify which controller on the card is routed to which I/O module bay by performing the same test, and using a controller on the expansion card and a compatible switch module or pass-thru module in I/O module bay 3 or module bay 4.

Chapter 6. Hardware setup

169

6.12 SAS hardware RAID configuration
The storage subsystem on the BladeCenter JS21 is based on the 2-port SAS controller attached to the PCI-X bus running at 100 MHz. The subsystem provides RAID 0 or RAID 1 mirroring functionality. As RAID is considered an option, the disks are shipped blank, ready for use as individual disk drives. If you want to configure RAID for any onboard drives, run the RAID configuration tools first to prepare the disks to be used in a RAID.

6.12.1 RAID configuration methods
The RAID configuration occurs within the AIX 5L or Linux operating system. However, you cannot perform this function from an operating system running on these disks. Because not more than two internal drives are available in a JS21, boot the operating system from CD/DVD, network, or an external disk. A typical method to configure a RAID array is to boot the AIX stand-alone diagnostics from the CD/DVD or network (use the NIM operation diag = enable a machine to boot a diagnostic image), and configure the RAID from there. This function is also available when booting into rescue mode on the SUSE Linux Enterprise Server 9 (SLES9) Service Pack 3 (SP3) installation CD. Before you can create a RAID array, you must reformat the hard disk drives so that the sector size of the drives changes from 512 bytes to 522 bytes. Later if you decide to remove the hard disk drives, delete the RAID array before you remove the drives. If you decide to delete the RAID array and reuse the hard disk drives, you must reformat the drives so that the sector size of the drives changes from 522 bytes to 512 bytes.

170

IBM BladeCenter JS21: The POWER of Blade Innovation

6.12.2 Configuring RAID using the AIX stand-alone diagnostics
This section provides an example of how to configure a RAID using the AIX stand-alone diagnostics. 1. After diagnostics starts, select 3 Task Selection in the Function Selection menu, as shown in Figure 6-66. FUNCTION SELECTION 1 Diagnostic Routines This selection will test the machine hardware. Wrap plugs and other advanced functions will not be used. 2 Advanced Diagnostics Routines This selection will test the machine hardware. Wrap plugs and other advanced functions will be used. 3 Task Selection (Diagnostics, Advanced Diagnostics, Service Aids, etc.) This selection will list the tasks supported by these procedures. Once a task is selected, a resource menu may be presented showing all resources supported by the task. 4 Resource Selection This selection will list the resources in the system that are supported by these procedures. Once a resource is selected, a task menu will be presented showing all tasks that can be run on the resource(s). 99 Exit Diagnostics NOTE: The terminal is not properly initialized. You will be prompted to initialize the terminal after selecting one of the above options.

To make a selection, type the number and press Enter.
Figure 6-66 RAID diag: Selecting the function

[1 ]

Chapter 6. Hardware setup

171

2. Define the terminal type and scroll down the Tasks Selection menu (Figure 6-67) and select Raid Array Manager. TASKS SELECTION LIST 801004

From the list below, select a task by moving the cursor to the task and pressing 'Enter'. To list the resources for the task highlighted, press 'List'. [MORE...17] Display USB Devices Display or Change Bootlist Format Media Hot Plug Task Identify and Attention Indicators Local Area Network Analyzer Microcode Tasks RAID Array Manager SSA Service Aids This selection provides tools for diagnosing and resolving problems on SSA attached devices. Update and Manage System Flash [BOTTOM] F1=Help F3=Previous Menu F4=List F10=Exit Enter

Figure 6-67 RAID diag: Selecting the task

3. Select PCI-X SCSI Disk Array Manager, as shown in Figure 6-68. RAID Array Manager Move cursor to desired item and press Enter. PCI SCSI Disk Array Manager PCI-X SCSI Disk Array Manager
Figure 6-68 Selecting the RAID Array Manager

801004

172

IBM BladeCenter JS21: The POWER of Blade Innovation

4. From the Disk Array Manager screen (Figure 6-69), select List PCI-X SCSI Disk Array Configuration. PCI-X SCSI Disk Array Manager Move cursor to desired item and press Enter. List PCI-X SCSI Disk Array Configuration Create an Array Candidate pdisk and Format to 522 Byte Sectors Create a PCI-X SCSI Disk Array Delete a PCI-X SCSI Disk Array Add Disks to an Existing PCI-X SCSI Disk Array Configure a Defined PCI-X SCSI Disk Array Change/Show Characteristics of a PCI-X SCSI Disk Array Reconstruct a PCI-X SCSI Disk Array Change/Show PCI-X SCSI pdisk Status Diagnostics and Recovery Options

F1=Help F9=Shell

F2=Refresh F10=Exit

F3=Cancel Enter=Do

F8=Image

Figure 6-69 Disk Array Manager: Main dialog

Chapter 6. Hardware setup

173

If you have not yet configured a RAID, the output looks similar to Figure 6-70. COMMAND STATUS Command: OK stdout: yes stderr: no

Before command completion, additional instructions may appear below. ---------------------------------------------------------------------------Name Location State Description Size ---------------------------------------------------------------------------sisioa0 01-08 Available PCI-XDDR Dual Channel SAS RAID Adapter

hdisk0 hdisk1

01-08-00-1,0 01-08-01-1,0

Available Available

SCSI Disk Drive SCSI Disk Drive

36.4GB 36.4GB

F1=Help F8=Image n=Find Next

F2=Refresh F9=Shell

F3=Cancel F10=Exit

F6=Command /=Find

Figure 6-70 List PCI-X SCSI disk array configuration: No array candidates

174

IBM BladeCenter JS21: The POWER of Blade Innovation

5. Create array candidates from the native hdisks. Select the adapter as shown in Figure 6-71. PCI-X SCSI Disk Array Manager Move cursor to desired item and press Enter. List PCI-X SCSI Disk Array Configuration Create an Array Candidate pdisk and Format to 522 Byte Sectors Create a PCI-X SCSI Disk Array Delete a PCI-X SCSI Disk Array Add Disks to an Existing PCI-X SCSI Disk Array Configure a Defined PCI-X SCSI Disk Array Change/Show Characteristics of a PCI-X SCSI Disk Array ???????????????????????????????????????????????????????????????????????? ? Available Controllers ? ? ? ? Move cursor to desired item and press F7. ? ? ONE OR MORE items can be selected. ? ? Press Enter AFTER making all selections. ? ? ? ? sisioa0 Available 01-08 PCI-XDDR Dual Channel SAS RAID Adapter ? ? ? ? F1=Help F2=Refresh F3=Cancel ? ? F7=Select F8=Image F10=Exit ? F1? Enter=Do /=Find n=Find Next ? F9????????????????????????????????????????????????????????????????????????
Figure 6-71 RAID diag: Available controllers

Chapter 6. Hardware setup

175

6. Select the disks as shown in Figure 6-72. PCI-X SCSI Disk Array Manager Move cursor to desired item and press Enter. List PCI-X SCSI Disk Array Configuration Create an Array Candidate pdisk and Format to 522 Byte Sectors Create a PCI-X SCSI Disk Array Delete a PCI-X SCSI Disk Array Add Disks to an Existing PCI-X SCSI Disk Array Configure a Defined PCI-X SCSI Disk Array ???????????????????????????????????????????????????????????????????????? ? Create an Array Candidate pdisk and Format to 522 Byte Sectors ? ? ? ? Move cursor to desired item and press F7. Use arrow keys to scroll. ? ? ONE OR MORE items can be selected. ? ? Press Enter AFTER making all selections. ? ? ? ? > hdisk0 01-08-00-1,0 Available SCSI Disk Drive 36.4GB ? ? > hdisk1 01-08-01-1,0 Available SCSI Disk Drive 36.4GB ? ? ? ? F1=Help F2=Refresh F3=Cancel ? ? F7=Select F8=Image F10=Exit ? F1? Enter=Do /=Find n=Find Next ? F9????????????????????????????????????????????????????????????????????????
Figure 6-72 Creating an array candidate pdisk and formatting to 522-byte sectors

The reformat to 522-byte sectors starts as shown in Figure 6-73. This takes some time to complete. Format in progress |#################################################-| | 98% hdisk1 deleted |##################################################| / 99% hdisk0 deleted |##################################################| - 100% Formats complete. Press the Enter key to continue ...
Figure 6-73 RAID diag format in progress

176

IBM BladeCenter JS21: The POWER of Blade Innovation

7. Now when you perform a List PCI-X SCSI Disk Array Configuration in the main screen (Figure 6-69 on page 173), you get the output shown in Figure 6-74. Compared to Figure 6-70 on page 174, the disk names have changed from hdisks to pdisks and the description is now Array Candidate. COMMAND STATUS Command: OK stdout: yes stderr: no

Before command completion, additional instructions may appear below. -------------------------------------------------------------------Name Location State Description Size --------------------------------------------------------------------sisioa0 01-08 Available PCI-XDDR Dual Channel SAS RAID Adapter pdisk1 pdisk0 01-08-01-1,0 01-08-00-1,0 Active Active Array Candidate Array Candidate 34.8GB Zeroed 34.8GB Zeroed

F1=Help F8=Image n=Find Next

F2=Refresh F9=Shell

F3=Cancel F10=Exit

F6=Command /=Find

Figure 6-74 List PCI-X SCSI disk array configuration: With array candidates (pdisks)

Chapter 6. Hardware setup

177

8. You can create the array by selecting Create a PCI-X SCSI Disk Array in the main screen. Figure 6-75 shows that first you have to select the RAID level. In this example, we use RAID level 0. PCI-X SCSI Disk Array Manager Move cursor to desired item and press Enter. List PCI-X SCSI Disk Array Configuration Create an Array Candidate pdisk and Format to 522 Byte Sectors Create a PCI-X SCSI Disk Array Delete a PCI-X SCSI Disk Array Add Disks to an Existing PCI-X SCSI Disk Array Configure a Defined PCI-X SCSI Disk Array Change/Show Characteristics of a PCI-X SCSI Disk Array ???????????????????????????????????????????????????????????????????????? ? Select a Raid Level ? ? ? ? Move cursor to desired item and press Enter. ? ? ? ? 0 ? ? 5 ? ? 10 ? ? ? ? F1=Help F2=Refresh F3=Cancel ? ? F8=Image F10=Exit Enter=Do ? F1? /=Find n=Find Next ? F9????????????????????????????????????????????????????????????????????????
Figure 6-75 Creating the array: Selecting the RAID level

Restriction: Figure 6-75 shows that the Disk Array Manager supports RAID levels 0, 5, and 10. The JS21 has only two physical drives. RAID level 5 requires a minimum of three drives. RAID level 10 requires a minimum of three drives. RAID level 0 is striping, which you can do with two disks. When you select RAID level 10 with two disks, we suspect that the system is performing RAID 1, which is mirroring with two disks.

178

IBM BladeCenter JS21: The POWER of Blade Innovation

9. Select the Stripe Size as shown in Figure 6-76. PCI-X SCSI Disk Array Manager Move cursor to desired item and press Enter. List PCI-X SCSI Disk Array Configuration Create an Array Candidate pdisk and Format to 522 Byte Sectors Create a PCI-X SCSI Disk Array Delete a PCI-X SCSI Disk Array Add Disks to an Existing PCI-X SCSI Disk Array Configure a Defined PCI-X SCSI Disk Array Change/Show Characteristics of a PCI-X SCSI Disk Array ???????????????????????????????????????????????????????????????????????? ? Select a Stripe Size (in Kb) ? ? ? ? Move cursor to desired item and press Enter. ? ? ? ? 16 Kb ? ? 64 Kb (recommended) ? ? 256 Kb ? ? ? ? F1=Help F2=Refresh F3=Cancel ? ? F8=Image F10=Exit Enter=Do ? F1? /=Find n=Find Next ? F9????????????????????????????????????????????????????????????????????????
Figure 6-76 Creating the array: Selecting the stripe size

Chapter 6. Hardware setup

179

10.Select the Array Candidates (pdisks) that you want to use, as shown in Figure 6-77. PCI-X SCSI Disk Array Manager Move cursor to desired item and press Enter. List PCI-X SCSI Disk Array Configuration Create an Array Candidate pdisk and Format to 522 Byte Sectors Create a PCI-X SCSI Disk Array ???????????????????????????????????????????????????????????????????????? ? Select Disks to Use in the Array ? ? ? ? Move cursor to desired item and press F7. Use arrow keys to scroll. ? ? ONE OR MORE items can be selected. ? ? Press Enter AFTER making all selections. ? ? ? ? # RAID 0 supports a minimum of 1 and a maximum of 18 drives. ? ? ? ? ? ? > pdisk0 01-08-00-1,0 Active Array Candidate 34.8GB ? ? > pdisk1 01-08-01-1,0 Active Array Candidate 34.8GB ? ? ? ? F1=Help F2=Refresh F3=Cancel ? ? F7=Select F8=Image F10=Exit ? F1? Enter=Do /=Find n=Find Next ? F9????????????????????????????????????????????????????????????????????????
Figure 6-77 Creating the array: Selecting the pdisks

180

IBM BladeCenter JS21: The POWER of Blade Innovation

The system shows a final dialog with the chosen settings (Figure 6-78). Create a PCI-X SCSI Disk Array Type or select values in entry fields. Press Enter AFTER making all desired changes. [Entry Fields] sisioa0 0 64 pdisk0 pdisk1

Controller RAID Level Stripe Size in KB Selected Disks

F1=Help Esc+5=Reset F9=Shell

F2=Refresh F6=Command F10=Exit

F3=Cancel F7=Edit Enter=Do

F4=List F8=Image

Figure 6-78 Creating the array: Final dialog

Figure 6-79 shows the successful creation of the array (). COMMAND STATUS Command: OK stdout: yes stderr: no

Before command completion, additional instructions may appear below.

hdisk0 Available

F1=Help F8=Image n=Find Next

F2=Refresh F9=Shell

F3=Cancel F10=Exit

F6=Command /=Find

Figure 6-79 Creating the array: hdisk created

Chapter 6. Hardware setup

181

11.When you select the List PCI-X SCSI Disk Array Configuration again in the main dialog, you get an output similar to Figure 6-80. You can see the RAID array (hdisk0) and also the corresponding array members (pdisks). COMMAND STATUS Command: OK stdout: yes stderr: no

Before command completion, additional instructions may appear below. -----------------------------------------------------------------------Name Location State Description Size -----------------------------------------------------------------------sisioa0 01-08 Available PCI-XDDR Dual Channel SAS RAID Adapter hdisk0 pdisk0 pdisk1 01-08-ff-0,0 01-08-00-1,0 01-08-01-1,0 Optimal Active Active RAID 0 Array Array Member Array Member 69.7GB 34.8GB 34.8GB

F1=Help F8=Image n=Find Next

F2=Refresh F9=Shell

F3=Cancel F10=Exit

F6=Command /=Find

Figure 6-80 List PCI-X SCSI disk array configuration: RAID 0 array

Example 6-7 shows how the RAID array disk is displayed by different disk-related commands in AIX 5L.
Example 6-7 Commands output for RAID disk

# lsdev -Ccdisk hdisk0 Available 01-08-ff-0,0 SCSI RAID 0 Disk Array # lscfg -vp hdisk0

U788D.001.23A1137-P1-T12-T1-L0-L0

SCSI RAID 0 Disk Array

# lsattr -El hdisk0 pvid 000004718f5fc27f0000000000000000 queue_depth 8 raid_level 0 serial_number B5FA7003

Physical volume identifier False Queue DEPTH False RAID Level False Serial Number False

182

IBM BladeCenter JS21: The POWER of Blade Innovation

sis_unique_id 1D08B5FA700303IBM0885CB1410scsi size_in_mb 69797 stripe_size 64

Unique device identifier Size in Megabytes Stripe Size in KB

False False False

Figure 6-81 shows how the RAID array disk shows up in the SMS. PowerPC Firmware Version MB240_470_013 SMS 1.6 (c) Copyright IBM Corp. 2000,2005 All rights reserved. ----------------------------------------------------------------Select Device Device Current Device Number Position Name 1. 1 SCSI 69793 MB Harddisk, part=2 (AIX 5.3.0) ( loc=U788D.001.23A1137-P1-T12-T1-L0-L0 )
Figure 6-81 RAID array in SMS

6.12.3 Configuring RAID using iprconfig on Linux
On Linux, you can use the iprconfig tool to configure the RAID. In this example, we use SLES9 from a CD-ROM drive. Attention: The drivers for the Small Computer System Interface (SCSI) adapter are on the SLES SP3 CD-ROM. You must boot from CD 1 of the SP CD-ROMs or a network location of SLES where SP3 is installed. If you boot from the SLES9 CD 1, you will get a message that no hard drives are found. 1. Insert CD-ROM 1 of the SLES9 SP3 into the CD-ROM drive. 2. Ensure that the BladeCenter media tray is connected to the JS21 that you will configure. You can do this in the Web interface of the management module or by pressing the appropriate button in front of the blade. 3. Within the management module, change the boot order so that the CD-ROM is the first bootable device. 4. Start a Telnet session to the management module. 5. Start an SoL session using the following command (in this example, we use blade 3) power -cycle -c -T system:blade[3]

Chapter 6. Hardware setup

183

6. When the CD-ROM is ready to load the kernel, you see the screen shown in Figure 6-82. Type the word install to start the process. Welcome to SuSE Linux (SLES9)! Use Use "install" "install32" to boot the pSeries 64bit kernel to boot the 32bit RS/6000 kernel

You can pass the option "noinitrd" to skip the installer. Example: install noinitrd root=/dev/sda4 Welcome to yaboot version 1.3.11.SuSE Enter "help" to get some basic usage information boot: install
Figure 6-82 Booting SLES9 SP3 CD-ROM 1

7. Wait for the screen shown in Figure 6-83. Type 1 and press Enter to continue. >>> SUSE LINUX Enterprise Server 9 installation program v1.6.53 (c) 1996-2004 SUSE LINUX AG <<< Starting hardware detection... Activating usb devices... done Searching for info file... Driver Update: Service Pack 3 Make sure that CD number 1 is in your drive. 1) OK 2) Back > 1
Figure 6-83 SLES9 SP3 booting

184

IBM BladeCenter JS21: The POWER of Blade Innovation

8. The screen shown in Figure 6-84 opens. Select the language in which you want to see the screens. Select the language. 1) 2) 3) 4) 5) 6) 7) 8) 9) 10) 11) 12) 13) 14) 15) 16) 17) 18) > 4
Figure 6-84 Choosing the installation language

Bosnia Cestina Deutsch English Español Français Hellenic Italiano Japanese Magyar Nederlands Polski Português Português Brasileiro Russian Slovencina Simplified Chinese Traditional Chinese

Chapter 6. Hardware setup

185

9. You are now ready to start the process as shown in Figure 6-85. Type 4 to start the installation. >>> Linuxrc v1.6 (Kernel 2.6.5-7.244-pseries64) (c) 1996-2004 SUSE LINUX AG <<< Main Menu 1) 2) 3) 4) 5) 6) 7) > 4
Figure 6-85 Starting the installation process

Settings System Information Kernel Modules (Hardware Drivers) Start Installation or System Eject CD Exit or Reboot Power off

10.You go into the rescue mode as shown in Figure 6-86. Type 3 to start the rescue system. Start Installation or System 1) Start Installation or Update 2) Boot Installed System 3) Start Rescue System > 3
Figure 6-86 Starting the rescue mode

186

IBM BladeCenter JS21: The POWER of Blade Innovation

11.You are prompted for the source medium as shown in Figure 6-87. Choose 1 to start the rescue system from the CD-ROM. Choose the source medium. 1) 2) 3) 4) > 1
Figure 6-87 Choosing the source of the SUSE installation

CD-ROM Network Hard Disk Floppy

12.Log in as root and run the iprconfig command, as shown in Figure 6-88. INIT: Entering runlevel: 3 Boot logging started on /dev/hvc0(/dev/console) at Wed Jun 21 19:42:01 2006 Master Resource Control: previous runlevel: N, switching to runlevel:3 Initializing random number generator done Starting syslog services done Starting RPC portmap daemon done Importing Net File System (NFS) unused Master Resource Control: runlevel 3 has been reached Skipped services in runlevel 3: nfsboot nfs Rescue login: root iprconfig
Figure 6-88 Logging in as root

Chapter 6. Hardware setup

187

13.The first iprconfig screen is shown in Figure 6-89. Specify 1 to verify the hardware status. IBM Power RAID Configuration Utility Select one of the following: 1. 2. 3. 4. 5. 6. 7. 8. Display hardware status Work with disk arrays Work with disk unit recovery Work with SCSI bus configuration Work with driver configuration Work with disk configuration Download microcode Analyze log

Selection: 1
Figure 6-89 iprconfig main screen

The output is shown in Figure 6-90. Display Hardware Status Type option, press Enter. 1=Display hardware resource information details OPT Name PCI/SCSI Location --- ------ -------------------------0000:01:01.0/2: sda 0000:01:01.0/2:0:1:0 sdb 0000:01:01.0/2:1:1:0 e=Exit q=Quit r=Refresh t=Toggle
Figure 6-90 Output of iprconfig: Hardware status display

Description ------------------------PCI-X SAS RAID Adapter Physical Disk Physical Disk

Status ---------------Operational Active Active

Tip: If you get a message that there are no hard drives, then check the CD-ROM that you booted from. If you booted from the SLES9 CD-ROM 1, you will get this message. You must boot from the SLES9 SP 3 CD-ROM 1 for this to work.

188

IBM BladeCenter JS21: The POWER of Blade Innovation

14.Type q to return to the main menu shown in Figure 6-89 on page 188. 15.Type 2 to work with disk arrays. 16.Type 5 to format a device for a disk array as shown in Figure 6-91. Work with Disk Arrays Select one of the following: 1. 2. 3. 4. 5. 6. 7. 8. 9. Display disk array status Create a disk array Delete a disk array Add a device to a disk array Format device for RAID function Format device for JBOD function (512) Create a hot spare Delete a hot spare Force RAID Consistency Check

Selection: 5
Figure 6-91 iprconfig: Working with disk arrays

17.. You see a list of disks that you can format as shown in Figure 6-92. Type 1 beside each hard drive that you want to format for RAID. We put a 1 beside both the hard drives. Select Disks to format for RAID Function Type option, press Enter. 1=Select OPT --1 1 Name -----sda sdb PCI/SCSI Location -------------------------0000:01:01.0/2:0:1:0 0000:01:01.0/2:0:1:0 Vendor -------IBM-ESXS IBM-ESXS Product ID -----------ST936701SS ST936701SS Status -------Active Active

e=Exit q=Cancel t=Toggle
Figure 6-92 iprconfig: Selecting the disks to format

Chapter 6. Hardware setup

189

18.You see a confirmation screen as shown in Figure 6-93. Type c to confirm the formatting of the disks. Confirm Initialize and Format Disks ATTENTION! System crash may occur if selected device is in use. Data loss will occur on selected device. Proceed with caution. Press 'c' to confirm your choice for 1=Initialize and format. q=Return to change your choice. OPT --1 1 Name -----sda sdb PCI/SCSI Location -------------------------0000:01:01.0/2:0:1:0 0000:01:01.0/2:1:1:0 Vendor -------IBM-ESXS IBM-ESXS Product ID ------------ST936701SS ST936701SS Status -------Active Active

c=Confirm q=Cancel t=Toggle
Figure 6-93 iprconfig: Formatting disks confirmation

19.It takes a few minutes to format the disks. When they are finished, you see the disk array screen shown in Figure 6-91 on page 189. Type 2 to create a disk array. 20.This opens the screen shown in Figure 6-94. Type 1 beside the adapter that owns the hard drives that you just formatted. Create a Disk Array Select the adapter. Type choice, press Enter. 1=create a disk array OPT Name PCI/SCSI Location Vendor Product ID Status --- ------ -------------------------- -------- ------------ --------0000:01:01.0/2: IBM 572E001 Operational e=Exit q=Cancel t=Toggle
Figure 6-94 Selecting the adapter

190

IBM BladeCenter JS21: The POWER of Blade Innovation

21.You see the list of formatted disks as shown in Figure 6-95. Type 1 beside both the hard drives to add them both to the array. Select Disk Units for Disk Array Type option, press Enter. 1=Select OPT Name PCI/SCSI Location --- ------ -------------------------0000:01:01.0/2:0:1:0 0000:01:01.0/2:1:1:0
Figure 6-95 List of formatted disks

Vendor -------IBM-ESXS IBM-ESXS

Product ID ------------ST936701SS ST936701SS

Status ------Zeroed Zeroed

22.When you press Enter you see the screen to select the protection level as shown in Figure 6-96. If you select c to change the setting, you will see an option of either RAID 0 or RAID 10. RAID 0 is just stripping. RAID 10 is normally stripping and mirroring, but because we only have two drives internally in the JS21, it becomes just mirroring. This can be helpful to guard against a disk crash. When you press Enter on the screen it creates the array and takes you back to the Work with Disk Arrays screen shown in Figure 6-91 on page 189. Select Protection Level and Stripe Size Default array configurations are shown. To change setting hit "c" for options menu. Highlight desired option then hit Enter c=Change Setting Protection Level . . . . . . . . . . . . : Raid 10 Stripe Size. . . . . . . . . . . . . . . : 64 K Queue Depth. . . . . . . . . . . . . . . : 8
Figure 6-96 Selecting the protection level

23.Reboot the JS21. You have finished creating the disk array. If you are going to install SLES9 from a CD-DROM, you have to boot from CD-ROM 1 of the SLES9 SP3. After you choose to start the installation, you are prompted for CD-ROM 1 of the SLES9 installation media. Insert this CD-ROM and follow the instructions in the screen menus.

Chapter 6. Hardware setup

191

192

IBM BladeCenter JS21: The POWER of Blade Innovation

7

Chapter 7.

Installing and managing the Virtual I/O Server
The Integrated Virtualization Manager (IVM) is a feature of the Virtual Input/Output (I/O) Server (VIOS). This chapter describes how to install and manage the VIOS on a BladeCenter JS21. It further describes the two different interfaces to the IVM: The graphic interface and the command line based interface. It also provides examples of how to create and modify partitions, and configure storage and network.

© Copyright IBM Corp. 2006. All rights reserved.

193

7.1 VIOS installation in a JS21
This chapter describes how to install the VIOS on a BladeCenter JS21using either a DVD or the network.

7.1.1 VIOS image installation from a DVD
VIOS Version 1.2.1 is shipped as a single DVD media that contains a bootable image of the software. It also contains the IVM component. CD media is not supported for the VIOS. Attention: When using the JS21 in a BladeCenter chassis that does not have a DVD drive in the media tray, you can install the VIOS through the network from a Network Installation Manager (NIM) server or a Linux server. When you install from a DVD, assign the media tray to the blade that you want (see 6.5.3, “Assigning the media tray” on page 94) and mount the VIOS installation media in the DVD drive of the media tray. The remaining steps are similar to a normal AIX installation as described in 8.2, “Preparing AIX installation from CD/DVD” on page 260.

7.1.2 VIOS image installation from a NIM server
You can install the VIOS from a NIM server, especially when the BladeCenter chassis has no DVD drive. Note: The method described in this section is different from the option where you type smitty nim_power5 from an AIX command-line, and then select Virtual I/O Server and Integrated Virtualization Manager Installation Tasks (/usr/sbin/installios command). This option requires a Hardware Management Console (HMC). Because we have no HMC in an IVM environment, we require another method in our case. To install the VIOS using NIM, perform the following steps: 1. Install or define an existing server running AIX that you can configure as a NIM server. 2. If your NIM server does not have a DVD drive, get access to a computer with a DVD drive and a network connection. This computer can run an operating system other than AIX, for example, Linux or Windows.

194

IBM BladeCenter JS21: The POWER of Blade Innovation

Note: The AIX version must be at least 5300, technology level 04. To verify this, use one of the following commands: /usr/bin/oslevel -rf instfix -i | grep AIX_ML Use the following command to see the service pack level: /usr/bin/oslevel -s 3. Configure the NIM server as described in 8.3, “Preparing AIX network installation using NIM” on page 261. 4. Mount the VIOS installation DVD in the computer and transfer the files mksysb and bosinst.data from the /nimol/ioserver_res directory on the DVD to the NIM server: # mount –oro –vcdrfs /dev/cd0 /mnt # cp /mnt/nimol/ioserver_res/mksysb /export/vios # cp /mnt/nimol/ioserver_res/bosinst.data /export/vios Note: If your operating system is not capable of resolving links, then you might have to copy the file mksysb from the directory /usr/sys/inst.images instead. The file mksysb must have a size of approximately 600 MB.

Chapter 7. Installing and managing the Virtual I/O Server

195

Configuring NIM using System Management Interface Tool
To configure NIM using System Management Interface Tool (SMIT): 1. Go to the NIM server and define a mksysb resource from the mksysb file you copied from the DVD. To do this, type smitty nim_mkres, select mksysb and enter the details as shown in Figure 7-1. Define a Resource Type or select values in entry fields. Press Enter AFTER making all desired changes. [Entry Fields] * Resource Name * Resource Type * Server of Resource [master] + * Location of Resource [/export/vios/mksysb] [MORE...9] F1=Help F5=Reset F9=Shell F2=Refresh F6=Command F10=Exit F3=Cancel F7=Edit Enter=Do [vios_mksysb] mksysb

F4=List F8=Image

Figure 7-1 NIM: Defining VIOS mksysb resource

196

IBM BladeCenter JS21: The POWER of Blade Innovation

2. Define a bosinst_data resource in the smitty nim_mkres fast path by selecting bosinst_data and enter the details as shown in Figure 7-2. Define a Resource Type or select values in entry fields. Press Enter AFTER making all desired changes. [Entry Fields] * Resource Name * Resource Type * Server of Resource + * Location of Resource /export/vios/bosinst.data] Comments Source for Replication F1=Help F5=Reset F9=Shell F2=Refresh F6=Command F10=Exit F3=Cancel F7=Edit Enter=Do [] [] + F4=List F8=Image [vios_bid] bosinst_data [master]

Figure 7-2 NIM: Defining VIOS bosinst_data resource

Note: By default this bosinst.data file performs a non-prompted installation. This means that it selects the target hard disk automatically. If you do not want this, then change PROMPT = no to PROMPT = yes in the bosinst.data file. You can do this even if the resource is already defined and is active immediately.

Chapter 7. Installing and managing the Virtual I/O Server

197

3. Define a Shared Product Object Tree (SPOT) resource from the mksysb resource defined in Figure 7-1 on page 196. From the smitty nim_mkres fast path, select spot and enter the details as shown in Figure 7-3. Define a Resource Type or select values in entry fields. Press Enter AFTER making all desired changes. [Entry Fields] [vios_spot] spot [master] [vios_mksysb] [/export/vios] yes []

* * * * *

Resource Name Resource Type Server of Resource Source of Install Images Location of Resource Expand file systems if space needed? Comments installp Flags COMMIT software updates? SAVE replaced files? AUTOMATICALLY install requisite software? OVERWRITE same or newer versions? VERIFY install and check file sizes?

+ + / +

no yes yes no no

+ + + + +

F1=Help F5=Reset F9=Shell

F2=Refresh F6=Command F10=Exit

F3=Cancel F7=Edit Enter=Do

F4=List F8=Image

Figure 7-3 NIM: Defining VIOS SPOT resource

Important: It is essential to define the SPOT from the just created mksysb resource and not from the lpp_source of the NIM server as highlighted in Figure 7-3. This guarantees that the SPOT and the corresponding network boot image match the operating system version of the VIOS to be installed.

198

IBM BladeCenter JS21: The POWER of Blade Innovation

4. Define a client machine as described in 8.3.4, “Defining the NIM client” on page 266. Allocate the mksysb (vios_mksysb), bosinst_data (vios_bid), and spot (vios_spot) resources as described in 8.3.5, “Allocating resources to a client” on page 268. Figure 7-4 shows the resources that you must allocate to the client. Manage Network Install Resource Allocation Mo???????????????????????????????????????????????????????????????????????? ? Available Network Install Resources ? ? ? ? Move cursor to desired item and press F7. ? ? ONE OR MORE items can be selected. ? ? Press Enter AFTER making all selections. ? ? ? ? [MORE...10] ? ? > vios_bid bosinst_data ? ? > vios_spot spot ? ? > vios_mksysb mksysb ? ? [BOTTOM] ? ? ? ? F1=Help F2=Refresh F3=Cancel ? ? F7=Select F8=Image F10=Exit ? F1? Enter=Do /=Find n=Find Next ? F9????????????????????????????????????????????????????????????????????????
Figure 7-4 NIM: Allocating VIOS resources

Chapter 7. Installing and managing the Virtual I/O Server

199

5. Activate the installation as described in 8.3.6, “Activating the client installation” on page 268. Here we select Source for BOS Runtime Files = mksysb as shown in Figure 7-5. Perform a Network Install Type or select values in entry fields. Press Enter AFTER making all desired changes. [Entry Fields] Target Name Source for BOS Runtime Files installp Flags Fileset Names Remain NIM client after install? Initiate Boot Operation on Client? Set Boot List if Boot not Initiated on Client? Force Unattended Installation Enablement? ACCEPT new license agreements? F1=Help F5=Reset F9=Shell F2=Refresh F6=Command F10=Exit F3=Cancel F7=Edit Enter=Do js21_vios mksysb [-agX] [] no no no no [no]

+

+ + + + +

F4=List F8=Image

Figure 7-5 NIM: Activating VIOS installation from mksysb

200

IBM BladeCenter JS21: The POWER of Blade Innovation

If you get the warning as shown in Figure 7-6, you can ignore it. COMMAND STATUS Command: OK stdout: yes stderr: no

Before command completion, additional instructions may appear below. warning: 0042-360 m_bos_inst: The SPOT level is older than the mksysb level. Therefore, the BOS installation may encounter problems. Update the SPOT to match the mksysb level or create a new SPOT that has the same level. F1=Help F8=Image n=Find Next F2=Refresh F9=Shell F3=Cancel F10=Exit F6=Command /=Find

Figure 7-6 NIM: Activating VIOS installation warning message

The rest of the installation is similar to 8.4, “Installing AIX on the client” on page 270.

Configuring NIM using the command line
You can also configure NIM (described in the previous section) using the command line.

NIM resources
The following commands define NIM resources. bosinst_data resource: # nim –o define –t bosinst_data –a server=master –a location=/export/vios/bosinst.data vios_bid mksysb resource: # nim –o define –t mksysb –a server=master –a SPOT resource (/usr file system): # nim –o define –t spot –a server=master –a source=vio_mksysb –a location=/export/vios vios_spot

Chapter 7. Installing and managing the Virtual I/O Server

201

Client definition of Virtual I/O Server
For the client definition of VIOS, perform the following steps: 1. Define the client: # nim –o define –t standalone –a if1=“network1 js21_vios.itsc.austin.ibm.com 0” –a cable_type1=tp js21_vios 2. Initiate a pull installation from the NIM master: # nim –o bos_inst –a source=mksysb –a mksysb=vios_mksysb -a bosinst_data=vios_bid –a spot=vios_spot –a no_client_boot=yes js21_vios This allocates all the necessary resources, but waits for the VIOS to initiate the installation itself. To finish the installation, proceed with the steps described in 8.4, “Installing AIX on the client” on page 270.

7.1.3 VIOS image installation from a Linux server
We describe a method that uses a Linux server for the network installation in the case where there is no AIX server available. Therefore, you cannot use the nimol Red Hat Package Manager (RPM) because it ships with the AIX license CD. This section describes how to set up the Linux server for a VIOS network installation. The following description is based on SUSE Linux Enterprise Server 9, Service Pack 3 (SLES9 SP3).

General notes
We use the following names and values in the configuration file examples of this section. You might have to change these based on your environment: Subnet: 9.153.99.0 Subnet mask: 255.255.255.0 Gateway: 9.153.99.1 Table 7-1 lists the VIOS installation network settings.
Table 7-1 VIOS installation network settings Host name Client: bch-6s Server: bc-4s Internet Protocol (IP) address 9.153.99.65 9.153.99.49 Medium Access Control (MAC) address 00:11:25:c9:30:b7

202

IBM BladeCenter JS21: The POWER of Blade Innovation

Note: The server must be able to resolve the client's host name, by either the Domain Name System (DNS), /etc/hosts, or another method. Depending on the setup, you might have to use the fully qualified domain name (FQDN).

Installing RPM packages
You have to install the following packages using rpm -i, if they are not already there: tftp-0.36-44.4.ppc.rpm dhcp-3.0.1rc13-28.18.ppc.rpm dhcp-server-3.0.1rc13-28.20.ppc.rpm

Preparing Linux server resources
The network installation requires some resources on the server that we create now.

Directories
Create the directories for storing the VIOS installation source: mkdir /tftpboot mkdir -p /export/vios

VIOS resources from the DVD
Mount the VIOS DVD and copy the resources booti.chrp.mp.ent.Z, bosinst.data, ispot.tar.Z, and mksysb from the directory /nimol/ioserver_res on the DVD to the local directory /export/vios: cp /mnt/nimol/ioserver_res/* /export/vios

If you do not want an unattended installation later, you have to edit bosinst.data and change PROMPT=no to PROMPT=yes. Later this opens the installation dialog and offers you the possibility to select the target hard disk.

Unpacking the SPOT
You have to unpack the so-called SPOT from the compressed image. This creates the directory SPOT under the current directory /export/vios: cd /export/vios tar -xzf ispot.tar.Z

Chapter 7. Installing and managing the Virtual I/O Server

203

Preparing the boot image and information file
To prepare the boot image and information file: 1. Uncompress the boot image from the DVD so that you can use it for the network installation. cd /tftpboot gunzip < /export/vios/booti.chrp.mp.ent.Z > bch-6s 2. Create a file named bch-6s.info with the content shown in Example 7-1.
Example 7-1 Syntax of the information for file installation

#------------------ Network Install Manager --------------export NIM_SERVER_TYPE=linux export NIM_SYSLOG_PORT=514 export NIM_SYSLOG_FACILITY=local2 export NIM_NAME=bch-6s export NIM_HOSTNAME=bch-6s export NIM_CONFIGURATION=standalone export NIM_MASTER_HOSTNAME=bc-4s export REMAIN_NIM_CLIENT=no export RC_CONFIG=rc.bos_inst export NIM_BOSINST_ENV="/../SPOT/usr/lpp/bos.sysmgt/nim/methods/c_bosinst_env" export NIM_BOSINST_RECOVER="/../SPOT/usr/lpp/bos.sysmgt/nim/methods/c_bosinst_ env -a hostname=bch-6s" export NIM_BOSINST_DATA=/NIM_BOSINST_DATA export SPOT=bc-4s:/export/vios/SPOT/usr export NIM_BOS_IMAGE=/NIM_BOS_IMAGE export NIM_BOS_FORMAT=mksysb export NIM_HOSTS=" 9.153.99.65:bch-6s 9.153.99.49:bc-4s " export NIM_MOUNTS=" bc-4s:/export/vios/bosinst.data:/NIM_BOSINST_DATA:file bc-4s:/export/vios/mksysb:/NIM_BOS_IMAGE:file " export ROUTES=" default:0:9.153.99.1 "

204

IBM BladeCenter JS21: The POWER of Blade Innovation

Network File System exports and server daemons
To configure the Network File System (NFS) service, perform the following steps: 1. Add the following entries to /etc/exports as shown in Example 7-2.
Example 7-2 /etc/exports

/export/vios/mksysb /export/vios/SPOT/usr /export/vios/bosinst.data

*(ro,insecure,no_root_squash) *(ro,insecure,no_root_squash) *(ro,insecure,no_root_squash)

2. Check if portmap is already running on SLES9 SP3. This was true with the default installation on our system: /etc/init.d/portmap status If not, start portmap: /etc/init.d/portmap start 3. Start the Network File System (NFS) server: /etc/init.d/nfsserver start

Preparing Trivial File Transfer Protocol service
To prepare Trivial File Transfer Protocol (TFTP) service: 1. Change the setting for disable to no in /etc/xinetd.d/tftp, as shown in Example 7-3.
Example 7-3 /etc/xinetd.d/tftp

service tftp { socket_type protocol wait user server server_args disable }

= = = = = = =

dgram udp yes root /usr/sbin/in.tftpd -s /tftpboot no

2. Restart the xinited daemon to activate your change: /etc/init.d/xinetd restart

Chapter 7. Installing and managing the Virtual I/O Server

205

Configuring syslog
To configure syslog: 1. Edit the file /etc/sysconfig/syslog and add the -r option to SYSLOGD_PARAMS. Depending on the Linux distribution, this variable might also be named SYSLOGD_OPTIONS: SYSLOGD_PARAMS="-r " 2. Edit the file /etc/syslog.conf and make the changes as shown in Example 7-4.
Example 7-4 /etc/syslog.conf

#local2,local3.* local3.* local4,local5.* local6,local7.* local2.* 3. Restart the syslogd daemon: /etc/init.d/syslog restart

-/var/log/localmessages -/var/log/localmessages -/var/log/localmessages -/var/log/localmessages /var/log/nimol.log

Configuring the firewall
If the firewall is active, you have to allow the client to update the server's syslog. Run the following command. On the default SLES9 system, this is not necessary. iptables -I INPUT 1 -s bch-6s -j ACCEPT

Setting up Dynamic Host Configuration Protocol
To set up Dynamic Host Configuration Protocol (DHCP): 1. Edit the file /etc/dhcpd.conf and add the lines shown in Example 7-5.
Example 7-5 /etc/dhcpd.conf

always-reply-rfc1048 true; allow bootp; deny unknown-clients; not authoritative; default-lease-time 600; max-lease-time 7200; ddns-update-style none;

206

IBM BladeCenter JS21: The POWER of Blade Innovation

subnet 9.153.99.0 netmask 255.255.255.0 { host bch-6s { fixed-address 9.153.99.65; hardware ethernet 00:11:25:c9:30:b7; next-server 9.153.99.49; filename "bch-6s"; } } Note: The value for the file name does not necessarily have to be identical with the host name, but it must match with the file name that we use in /tftpboot. 2. Make the following changes in /etc/sysconfig/dhcpd to allow your server to handle DHCP requests on Ethernet port eth1: DHCPD_INTERFACE="eth1" 3. Restart your dhcpd server to activate the changed configuration: /etc/init.d/dhcpd restart

Installing on the client
This section describes the actual installation process on the client. Important: Linux is configured to boot efficiently by only starting the required daemons. Depending on the setup on the Linux server, you might have to start the services xinetd, dhcp, portmap, and NFS server manually after a reboot. You can also update the configuration so that they are automatically started at the next reboot. Start a Serial over LAN (SoL) session to the client and power on the JS21. Enter the System Management Services (SMS) menu and start the network installation as described in 6.7, “Providing a console for the BladeCenter JS21” on page 121; 6.8.1, “Activating the System Management Services interface” on page 133; and 8.4, “Installing AIX on the client” on page 270.

Chapter 7. Installing and managing the Virtual I/O Server

207

Important: Directed Bootstrap Protocol (BOOTP) does not work with the BOOTP function provided by DHCP. Therefore, the IP address of the client and the server must be 0.0.0.0 in the SMS IP parameters as shown in Example 7-6. Specify the MAC address of the client in the server's /etc/dhcpd.conf file as shown in Example 7-5 on page 206 to use broadcast BOOTP. However, broadcast BOOTP works only if the client and server are in the same subnet. Directed BOOTP with the BOOTPD implementation on Linux was not verified during the writing of this book.
Example 7-6 SMS IP parameters

PowerPC Firmware Version MB240_470_014 SMS 1.6 (c) Copyright IBM Corp. 2000,2005 All rights reserved. ----------------------------------------------------------------------IP Parameters Port 2-IBM 2 PORT 1000 Base-SX PCI-X Adapter: U788D.001.99DWL3F-P1-T8 1. Client IP Address [0.0.0.0] 2. Server IP Address [0.0.0.0] 3. Gateway IP Address [000.000.000.000] 4. Subnet Mask [255.255.255.000] You can monitor the progress of the client installation on the server by monitoring the syslog file that the installation creates: tail -f /var/log/nimol.log Tip: If your installation environment is temporary, then you must consider removing access to the installation server and stopping the installation services after a successful installation (NFS, DHCP daemon (DHCPD), TFTP).

7.2 Initial VIOS configuration
The new VIOS requires a simple configuration setup using the command-line interface (CLI), as described in 7.3.2, “IVM command-line interface” on page 213. Later, you can use the Web interface besides the CLI, as described in 7.3.1, “IVM graphical user interface” on page 211. 1. Log on to the VIOS using the user padmin and the default password padmin. When prompted, change the login password to a secure one. 2. Accept the VIOS licence by issuing the license -accept command.

208

IBM BladeCenter JS21: The POWER of Blade Innovation

7.2.1 Virtualization setup
First we check whether the four virtual Ethernet interfaces that VIOS manages are already created during the VIOS installation. Perform the lsdev command shown in Example 7-7 on page 209 to verify whether the four virtual I/O Ethernet adapters are already available. If they are not yet available, the administrator must issue the mkgencfg command to create them with the following syntax: mkgencfg -o init [-i "configuration data"] For example: mkgencfg -o init -i "mac_prefix=06ABC0" Note: If the virtual Ethernet adapters are already available after first booting the VIOS (which is possible if a VIOS is already installed), you might have to run: lpcfgop -o clear Restart the system and then run mkgencfg. The help function in the IVM’s CLI does not display the mkgencf and the lpcfgop command. You can get the description by using: man mkgencf You can use the optional configuration data to define the prefix of the MAC address of all four virtual Ethernet adapters of the VIOS and to define the maximum number of partitions supported by the VIOS after the next restart. Example 7-7 shows the output of the mkgencfg command.
Example 7-7 Creating virtual Ethernet adapters

$ lsdev | grep ^ent ent0 Available ent1 Available $ mkgencfg -o init $ lsdev | grep ^ent ent0 Available ent1 Available ent2 Available ent3 Available ent4 Available ent5 Available

Gigabit Ethernet-SX PCI-X Adapter (14101403) Gigabit Ethernet-SX PCI-X Adapter (14101403)

Gigabit Gigabit Virtual Virtual Virtual Virtual

Ethernet-SX PCI-X Adapter (14101403) Ethernet-SX PCI-X Adapter (14101403) I/O Ethernet Adapter (l-lan) I/O Ethernet Adapter (l-lan) I/O Ethernet Adapter (l-lan) I/O Ethernet Adapter (l-lan)

Chapter 7. Installing and managing the Virtual I/O Server

209

Attention: Although you can change the maximum number of partitions later using the IVM Web graphical user interface (GUI), you can no longer modify the MAC address easily. Here is a method that worked in our environment. You will not get a network connection after the reboot and therefore have to use SoL. $ bkprofdata -o backup -f profile.bak $ lpcfgop -o clear Restart the system and then run: $ $ # # $ mkgencfg -o init -i "mac_prefix=06ABC0" oem_setup_env mkdev -l inet0 exit rstprofdata -l 1 -f profile.bak --ignoremac # # # # # change mac prefix change to root user fix IP routes back to padmin restore VIOS profile # backup VIOS profile # clear VIOS profile

7.2.2 Setting the date and time
Use the chdate command to set the VIOS date and time, using the following syntax: chdate [-year YYyy] [-month mm] [-day dd] [-hour HH] [-minute MM] [-timezone TZ] chdate mmddHHMM[YYyy | yy] [-timezone TZ]

7.2.3 Initial network setup
The IVM Web interface requires a valid network configuration to work. Configure the IP address by choosing a physical network adapter and issuing the mktcpip command from the command line, using the following syntax: mktcpip -hostname HostName -inetaddr Address -interface Interface [-start] [-netmask SubnetMask] [-cabletype CableType] [-gateway Gateway] [-nsrvaddr NameServerAddress [-nsrvdomain Domain]] Example 7-8 shows how to set the host name, address, and IP address for the VIOS.
Example 7-8 VIOS network setup at the command line

$ mktcpip -hostname js21_vios -inetaddr 9.3.5.229 -interface en0 -netmask 255.255.255.0 -gateway 9.3.5.41 -nsrvaddr 9.3.4.2 -nsrvdomain itsc.austin.ibm.com

210

IBM BladeCenter JS21: The POWER of Blade Innovation

Important: The IVM, like all Web servers, requires a valid name resolution to work correctly. If DNS is involved, check that both the name and the IP address resolution of the IVM host name are correct. After the IVM Web server has access to the network, it is possible to use the Web GUI with the Hypertext Transfer Protocol (HTTP) or the Hypertext Transfer Protocol-Secure (HTTPS) protocol pointing to the IP address of the IVM server application. Authentication requires the use of the padmin user, unless other users have been created.

7.3 VIOS management using IVM
The IVM provides a unique environment to administer logical partition capable servers. It provides two ways to configure and manage logical partitions (LPARs): A GUI designed to be as simple and intuitive as possible, incorporating partition management, storage management, serviceability, and monitoring capabilities (see 7.3.1, “IVM graphical user interface”) A CLI to enable scripting capabilities (see 7.3.2, “IVM command-line interface” on page 213) You can use either of the two interfaces to create, delete, and update the logical partitions and perform non-dynamic operations on LPARs including the partition of the VIOS itself. As described in 4.1.2, “Dynamic logical partitioning” on page 43, the JS21does not support dynamic logical partitioning (DLPAR) on the client partitions. This means that you can change some options such as memory and processor assignment only when the corresponding LPAR is stopped. However, you can assign and remove storage and optical media when the LPAR is running. The following examples use both methods, GUI and CLI.

7.3.1 IVM graphical user interface
The GUI is a Hypertext Markup Language (HTML)-based interface. It enables you to create LPARs on a single managed system, manage the virtual storage and virtual Ethernet on the managed system, perform maintenance, and view service information related to the managed system.

Chapter 7. Installing and managing the Virtual I/O Server

211

Connecting to the IVM
Open a Web browser window and connect using HTTP or HTTPS to the IP address that you assigned to the VIOS during the installation process (see 7.2.3, “Initial network setup” on page 210). As a result, a Welcome window that contains the login and the password prompts opens, as shown in Figure 7-7. The default user ID is padmin, and the password is the one you defined during the VIOS installation. The default password is padmin until you change it.

Figure 7-7 IVM login page

After the authentication process, the default IVM console window opens, as shown in Figure 7-8 on page 213. The IVM GUI consists of several elements. The following elements are the most important: Navigation area Work area The navigation area displays the tasks that you can access in the work area. The work area contains information related to the management tasks that you perform using the IVM and related to the objects on which you can perform management tasks. The task area lists the tasks that you can perform for items displayed in the work area. The tasks listed in the task area can change depending on the page that is displayed in the work area, or even depending on the tab that is selected in the work area.

Task area

212

IBM BladeCenter JS21: The POWER of Blade Innovation

Work Area

Navigation area
Figure 7-8 IVM console

Task area

7.3.2 IVM command-line interface
The text-based console with the CLI is accessible through an SoL terminal connection or through network connectivity using Telnet. The IP address is the same as the one used to connect to the GUI, and which you defined during the installation process. The CLI requires more experience to master than the GUI, but it offers more possibilities to tune the partition’s definitions and can be automated using scripts.

Chapter 7. Installing and managing the Virtual I/O Server

213

Use Telnet to connect to the VIOS login as padmin as shown in Example 7-9.
Example 7-9 CLI login to the VIOS

telnet (js21a2) IBM Virtual I/O Server login: padmin padmin's Password: $ Enter help to show an overview of the available commands. Example 7-10 shows the output of the help command.
Example 7-10 IVM CLI global help

$ help Install Commands ioslevel license lssw oem_platform_level oem_setup_env remote_management updateios LAN Commands cfglnagg cfgnamesrv entstat hostmap hostname lsnetsvc lstcpip mktcpip netstat optimizenet ping rmtcpip startnetsvc stopnetsvc traceroute Device Commands

Security Commands lsfailedlogin lsgcl UserID Commands chuser lsuser mkuser passwd rmuser Maintenance Commands backupios bootlist cattracerpt chdate chlang diagmenu errlog fsck invscout ldfware loginmsg lsfware lslparinfo motd mount

214

IBM BladeCenter JS21: The POWER of Blade Innovation

chdev chpath cfgdev lsdev lsmap lspath mkpath mkvdev mkvt rmdev rmpath rmvdev Physical Volume Commands lspv migratepv Logical Volume Commands chlv cplv extendlv lslv mklv mklvcopy rmlv rmlvcopy Volume Group Commands activatevg chvg deactivatevg exportvg extendvg importvg lsvg mirrorios mkvg redefvg reducevg syncvg unmirrorios Storage Pool Commands chsp lssp

pdump restorevgstruct savevgstruct showmount shutdown snap startsysdump starttrace stoptrace sysstat topas unmount viostat Shell Commands awk cat chmod clear cp date ftp grep head ls man mkdir more mv rm sed stty tail tee vi wall wc who

Chapter 7. Installing and managing the Virtual I/O Server

215

mkbdsp mksp rmbdsp $ Help relating to an individual command is available with the -h flag. Example 7-11 shows the help for the mkvt command.
Example 7-11 IVM CLI mkvt help

$ mkvt -h Usage: mkvt

{-id PartitionID}

Opens a virtual terminal session for an AIX or virtual I/O server partition in a managed system. -id The ID of the partition for which to open the virtual terminal session.

7.4 Storage management
The VIOS uses the following storage management concepts: Physical volume: A physical disk or a logical unit number (LUN) on a storage area network (SAN). They are all owned by the VIOS. You can assign a physical volume not belonging to any storage pool whole to a single partition as a virtual device. Storage pool: A set of physical volumes treated as a single entity. There can be multiple pools and they cannot share physical disks. One pool is defined as the default storage pool. Logical volume: A portion of a storage pool, which can belong to more than one physical volume Virtual disk: Logical volume that the VIOS assigns to a single partition as a virtual device You can assign both physical volumes and virtual disks to an LPAR to provide disk space. Each of them is represented by the LPAR operating system as a single disk. For example, if you assign a 73.4 GB physical disk and a 3 GB virtual disk to an LPAR running AIX 5L, then the operating system creates two hdisk devices.

216

IBM BladeCenter JS21: The POWER of Blade Innovation

Note: In practice, there is no difference for the user of the IVM between virtual disk and logical volume, because a virtual disk is stored in a logical volume that acts as a container for the virtual disk. However, in the GUI, you find differences in the menu options. For example, creating a virtual disk is done in the default storage pool and creating a logical volume gives you the possibility to select the storage pool (see “Creating the logical volume” on page 222). At installation time of the VIOS, there is only one storage pool named rootvg, typically containing only one physical volume. All the remaining physical volumes are available but not assigned to any pool. Note: Typically a storage pool is the same as a volume group in AIX. While the VIOS is running on an underlying AIX, it uses volume groups as storage pools, which you can especially see with the name rootvg. The different name is related to the possibility of implementing VIOS on Linux platforms too. The rootvg storage pool is used for the VIOS. We recommend that you do not use it to provide disk space to logical partitions. Because it is the only pool available at installation time, it is also defined as the default pool. Create another pool and set it as the default before creating other partitions. Important: Create at least one additional storage pool so that the rootvg pool is not the default storage pool. However, if you have only one disk, which might be the case if you have configured all the disks as a RAID array as described in 6.12, “SAS hardware RAID configuration” on page 170, your only choice is to use rootvg. In this case, you must be very careful not to overwrite rootvg, for example, when reinstalling the VIOS. Be sure to back up all essential data. From any storage pool, logical volumes can be defined and configured as virtual disks. They can be created in several ways, depending on the IVM menu that is in use: During LPAR creation: A logical volume is created in the default storage pool and assigned to the partition. Using the Create Devices link: A logical volume is not assigned to any partition and it is created in the default storage pool. There is an Advanced tab that enables the storage pool selection. We discuss basic storage management in the following section, and the advanced setup in 7.7, “Advanced storage management” on page 244.

Chapter 7. Installing and managing the Virtual I/O Server

217

Storage pool disk management
During the installation of the VIOS, a default storage pool is created and named rootvg. During the creation of the logical partition, the IVM automatically creates virtual disks in the default storage pool. We recommend that you create another storage pool and add virtual disks to it for the logical partitions. For advanced configuration of the storage pool, refer to 7.7, “Advanced storage management” on page 244.

Creating the storage pool
A storage pool consists of a set of physical disks that can be different types and sizes. You can create multiple storage pools. However, a disk can only be a member of a single storage pool. Important: All the data of a physical volume (hard disk) is erased when you add this volume to a storage pool. To create a storage pool, perform the following steps: 1. From the Storage Management menu in the navigation area, click the Create Devices link. 2. Select the Advanced Create Devices tab in the work area. Click Create Storage Pool in the work area, as shown in Figure 7-9.

Figure 7-9 Creating the storage pool

218

IBM BladeCenter JS21: The POWER of Blade Innovation

3. In the new window that opens: a. Enter a name in the Storage pool name field, as shown in Figure 7-10. b. Select the required disks. c. Click OK to create the storage pool.

Figure 7-10 Storage pool name

Default storage pool
The default storage pool that is created during the VIOS installation is rootvg. This is because rootvg is the only volume group created at that time. Important: Create at least one additional storage pool. Because the rootvf storage pool is the default storage pool, this causes VIOS data and user data to be merged. Therefore, do not use the rootvg storage pool for virtual disk storage. Because the VIOS is installed in rootvg, when VIOS is reinstalled, the rootvg storage pool is overwritten. Change the default storage pool to another one to avoid creating virtual disks within the rootvg by default. This prevents the loss of user data during a VIOS update. To change the default storage pool, perform the following steps: 1. From the Storage Management menu in the navigation area, click Advanced View/Modify Devices.

Chapter 7. Installing and managing the Virtual I/O Server

219

2. Select the storage pool you want as the default. Click Assign as default storage pool in the task area, as shown in Figure 7-11.

Figure 7-11 Storage pools list

3. A summary with the current and the next default storage pool opens, as shown in Figure 7-12. Click OK to validate the change.

Figure 7-12 Assigning default storage pool

220

IBM BladeCenter JS21: The POWER of Blade Innovation

7.5 VIOS partition configuration
The first panel that opens after the login process is the partition configuration, as shown in Figure 7-13.

Figure 7-13 Initial partition configuration

After the initial installation of the VIOS, there is only one VIOS partition on the system with the following characteristics: The ID is 1 The name is equal to the system’s serial number The state is Running The allocated memory is the maximum value between 512 MB and one-eighth of the installed system memory The number of allocated processors is equal to the number of physical processors, and the processing units is equal to 0.1 times the number of allocated processors The default configuration for the partition is designed to be appropriate for most VIOS installations. If the administrators want to change memory or processing unit allocation of the VIOS partition, they can perform a re-configuration using either the Web GUI or the command line, as described in 7.5.8, “Logical partition configuration changes” on page 235.

Chapter 7. Installing and managing the Virtual I/O Server

221

7.5.1 Creating the logical volume
Logical volumes belong to a storage pool and are also known as virtual disks. Logical volumes are used to provide disk space to logical partitions, but they are not assigned to logical partitions when you create them. They can be created in several ways, depending on the menu that is in use: During LPAR creation: A logical volume is created in the default storage pool and assigned to the partition. After or before LPAR creation: A logical volume is not assigned to any partition and is created in the default storage pool. There is an Advanced tab that enables the storage pool selection. To create a new logical volume, perform the following steps: 1. From the Storage Management menu in the navigation area, click Create Devices. 2. Select the Advanced Create Devices tab in the work area. Click Create Logical Volume in the work area. 3. In the new window that opens: a. b. c. d. Enter a name for the logical volume, as shown in Figure 7-14. Select a storage pool name from the drop-down list. Enter a size for the logical volume. Click OK to create the logical volume.

Figure 7-14 Logical volume name and size

222

IBM BladeCenter JS21: The POWER of Blade Innovation

To view your new logical volume and use it, select the View/Modify Devices link in the Storage Management menu in the navigation area. The list of available logical volumes is shown in the work area.

7.5.2 Creating logical partitions
A logical partition is a set of resources: Processors, memory, and I/O devices. Each resource assigned to a logical partition is allocated regardless of whether the logical partition is running or not. The VIOS does not allow the overcommitment of resources. To create a logical partition, perform the following steps: 1. From the Partition Management menu in the navigation area, click Create Partitions. Click the Start Wizard button in the work area. 2. In the new window that opens, enter a name for the new partition, as shown in Figure 7-15. Click Next.

Figure 7-15 Creating LPAR: Partition name

Chapter 7. Installing and managing the Virtual I/O Server

223

3. In the new window, enter the amount of memory necessary, as shown in Figure 7-16. Click Next.

Figure 7-16 Creating LPAR: Assigned memory

224

IBM BladeCenter JS21: The POWER of Blade Innovation

4. In the new window, select the number of processors necessary and choose a processing mode, as shown in Figure 7-17. In shared mode, each virtual processor uses 0.1 processing units. Click Next.

Figure 7-17 Creating LPAR: Selecting processors and processing mode

Chapter 7. Installing and managing the Virtual I/O Server

225

5. Each partition has two virtual Ethernet adapters that you can configure to one of the four available virtual Ethernet networks. In Figure 7-18, adapter one uses the virtual Ethernet ID 1. The Virtual Ethernet Bridge Overview section of the panel shows the physical network interface on which every virtual network is bridged. In Figure 7-18, no bridge is created. Perform the procedure described in “Ethernet bridging” on page 241 later to enable the partition to connect to the physical network. Click Next.

Figure 7-18 Creating LPAR: Virtual Ethernet

226

IBM BladeCenter JS21: The POWER of Blade Innovation

6. In the new window, select Assign existing virtual disks and physical volumes, as shown in Figure 7-19. You can also let the IVM create the virtual disk for you by selecting Create virtual disk, but be aware that the virtual disk is created in the default storage pool. To create storage pool and virtual disks or change the default storage pool, refer to , “Storage pool disk management” on page 218. Click Next.

Figure 7-19 Creating LPAR: Storage type

Chapter 7. Installing and managing the Virtual I/O Server

227

7. In the new window, select the required virtual disks from the list, as shown in Figure 7-20. Click Next.

Figure 7-20 Creating LPAR: Assigning storage

228

IBM BladeCenter JS21: The POWER of Blade Innovation

8. In the new window, select the required optical devices, as shown in Figure 7-21. If there is no optical device available, see 7.5.7, “Optical device sharing” on page 233 for further information. Click Next.

Figure 7-21 Creating LPAR: Assigning optical devices

Chapter 7. Installing and managing the Virtual I/O Server

229

9. A summary of the partition to be created is shown in Figure 7-22. Click Finish to create the logical partition.

Figure 7-22 Creating LPAR: Summary

To view the new logical partition and use it, from the Partition Management menu in the navigation area, click the View/Modify Partitions link. A list opens in the work area.

230

IBM BladeCenter JS21: The POWER of Blade Innovation

7.5.3 Updating the logical partition profile
Example 7-12 shows how to change the name of a logical partition with the chsyscfg command using the CLI.
Example 7-12 Updating the profile

$ lssyscfg -r prof --filter "lpar_names=LPAR2" -F lpar_name LPAR2 $ chsyscfg -r prof -i "lpar_name=LPAR2,new_name=LPAR2_new_name" $ lssyscfg -r prof --filter "lpar_names=LPAR2_new_name" -F lpar_name LPAR2_new_name

7.5.4 Powering on a logical partition
Example 7-13 shows how to start a logical partition using the chsysstate command. To follow the boot process, use the lsrefcode command.
Example 7-13 Powering on a partition

$ chsysstate -o on -r lpar -n LPAR2 $ lsrefcode -r lpar --filter "lpar_names=LPAR2" -F refcode CA00E1F1 $ lsrefcode -r lpar --filter "lpar_names=LPAR2" -F refcode CA00E14D

7.5.5 Opening a virtual terminal to a logical partition
Sometimes you have to access a logical partition using a virtual terminal, for example, during installation or when there is no network connection to the logical partition. To open a virtual terminal, perform the following steps: 1. Log on to the VIOS partition. 2. Open a virtual terminal for the logical partition to be installed with the mkvt command. You have to specify the ID of the logical partition, as shown in Example 7-14. Note: When connecting a virtual terminal to an already running partition, press Enter again after you issue mkvt to interact with the running system. If you get the message Virtual terminal is already connected, you can use the rmvt command to free up the session.

Chapter 7. Installing and managing the Virtual I/O Server

231

Example 7-14 Opening a virtual terminal

$ mkvt -id 3 AIX Version 5 (C) Copyrights by IBM and by others 1982, 2005. Console login:

7.5.6 Installing an operating system on a logical partition
The operating system installation process is similar to the process for stand-alone systems. The main steps are: 1. Log on to the VIOS partition. 2. Open a virtual terminal for the logical partition to be installed with the mkvt command as shown in Example 7-14. 3. Start the logical partition in SMS mode. To do so, you can change the boot mode in the properties of the partition’s profile before starting it. Alternatively, enter 1 on the virtual terminal at the start of the boot process, as shown in Example 7-15.
Example 7-15 Boot display

IBM IBM IBM IBM IBM IBM IBM IBM

IBM IBM IBM IBM IBM IBM IBM IBM

IBM IBM IBM IBM IBM IBM IBM IBM

IBM IBM IBM IBM IBM IBM IBM IBM

IBM IBM IBM IBM IBM IBM IBM IBM

IBM IBM IBM IBM IBM IBM IBM IBM

IBM IBM IBM IBM IBM IBM IBM IBM

IBM IBM IBM IBM IBM IBM IBM IBM

IBM IBM IBM IBM IBM IBM IBM IBM

IBM IBM IBM IBM IBM IBM IBM IBM

IBM IBM IBM IBM IBM IBM IBM IBM

IBM IBM IBM IBM IBM IBM IBM IBM

IBM IBM IBM IBM IBM IBM IBM IBM

IBM IBM IBM IBM IBM IBM IBM IBM

IBM IBM IBM IBM IBM IBM IBM IBM

IBM IBM IBM IBM IBM IBM IBM IBM

IBM IBM IBM IBM IBM IBM IBM IBM

IBM IBM IBM IBM IBM IBM IBM IBM

IBM IBM IBM IBM IBM IBM IBM IBM

1 = SMS Menu 8 = Open Firmware Prompt

5 = Default Boot List 6 = Stored Boot List

Memory

Keyboard

Network

SCSI

Speaker

4. To install an operating system, proceed as described in Chapter 8, “Installing AIX” on page 259, or Chapter 9, “Installing Linux” on page 281.

232

IBM BladeCenter JS21: The POWER of Blade Innovation

7.5.7 Optical device sharing
You can dynamically add, move, or remove optical devices from or to any logical partition, regardless of whether the logical partition is running or not. Attention: In a BladeCenter environment, there are two levels of device sharing. Within the BladeCenter by the media tray assignment as described in 6.5.3, “Assigning the media tray” on page 94, and the device sharing provided by the VIOS. The following description requires that the media tray is assigned to the blade server running the VIOS. When the media tray is assigned to the VIOS after it is booted, you must run cfgdev within the CLI of the IVM to make the optical drive Available. Use the lsdev command to see the presence and the status of the required devices. To change the assignment of an optical device, perform the following steps: 1. From the Storage Management menu in the navigation area, click View/Modify Devices. 2. Select the Optical Devices tab in the work area. Select the optical device you want to modify. Click Modify partition assignment in the tasks area, as shown in Figure 7-23.

Figure 7-23 Selecting the optical devices

Chapter 7. Installing and managing the Virtual I/O Server

233

3. In the new window, select the name of the logical partition to which you want to assign the optical device, as shown in Figure 7-24. You can also remove the optical device from the current logical partition by selecting None. Click OK.

Figure 7-24 Assigning the optical device partition

4. If you move or remove an optical device from a running logical partition, you are prompted to confirm the forced removal before the optical device is removed. Because the optical device becomes unavailable, log on to the logical partition and unmount the optical device before going further. Click the Eject button. If the drawer opens, this is an indication that the device is not mounted. Click OK. Note: We strongly recommend that you unmount the optical device’s file systems within the operating system before unassigning the device to avoid endless loop conditions. 5. The new list of optical devices is shown with the changes you made. Log on to the related logical partition and use the appropriate command to discover the new optical device. On AIX 5L, use the cfgmgr command.

234

IBM BladeCenter JS21: The POWER of Blade Innovation

7.5.8 Logical partition configuration changes
You might want to modify the properties of the VIOS or logical partitions, if necessary.

Logical partition operations on a VIOS partition
You can allocate or release resources such as processors and memory on the VIOS partition. You can run these operations either on the GUI or the CLI. Although changing the processing units has immediate effect, changing processors and memory remains as pending until the next reboot of the VIOS partition. The GUI displays a yellow triangle with an exclamation mark inside for pending updates as shown in Figure 7-27 on page 237. To reboot the VIOS, you must stop all other client partitions (LPARs) before.

Logical partition operation on memory using the GUI
The following steps describe how to increase memory size dynamically for the VIOS partition: 1. From the Partition Management menu in the navigation area, click View/Modify Partitions. 2. Select the VIOS partition (see Figure 7-25). Click Properties in the task area.

Figure 7-25 LPAR memory operation: Selecting logical partition

Chapter 7. Installing and managing the Virtual I/O Server

235

3. In the new window, modify the pending values as necessary. In Figure 7-26, the assigned memory is increased to 1 GB. Click OK.

Figure 7-26 LPAR memory operation: Increasing the memory size

236

IBM BladeCenter JS21: The POWER of Blade Innovation

Memory is not added or removed immediately. The update is pending until the next reboot of the VIOS partition. The pending update is shown by a yellow triangle with an exclamation mark inside, as shown in Figure 7-27.

Figure 7-27 LPAR change memory pending

Logical partition operation on virtual processors using the CLI
Log on to the VIOS using the CLI and run your LPAR operation. Example 7-16 shows how to add a 0.1 processing unit dynamically to the VIOS using the chsyscfg command.
Example 7-16 LPAR virtual processor operation

$ lshwres -r proc --level lpar --filter "lpar_names=VIOS" -F curr_proc_units 0.20 $ chsyscfg -r prof -i lpar_name=VIOS,desired_proc_units=0.3 $ lshwres -r proc --level lpar --filter "lpar_names=VIOS" -F curr_proc_units 0.30

Client logical partition resources management
The IVM does not allow dynamic operations on resources such as the processor, memory, and virtual Ethernet on a client logical partition. But dynamic operations on the disks, optical devices, partition name, and boot mode are allowed.

Chapter 7. Installing and managing the Virtual I/O Server

237

Logical partition operation on virtual disks using the GUI
The following steps describe how to assign logical volumes to a partition using the GUI: 1. From the Storage Management menu in the navigation area, click Advanced View/Modify Devices. 2. Select the Logical Volumes tab in the work area. Select the required logical volumes, as shown in Figure 7-28. Click Modify partition assignment in the task area.

Figure 7-28 Selecting the logical volumes

238

IBM BladeCenter JS21: The POWER of Blade Innovation

3. In the new window, select the partition name that you want to assign the logical volumes, as shown in Figure 7-29. Click OK to validate the logical volume partition assignment.

Figure 7-29 Assigning logical volume partition

4. Log on to the related logical partition and discover the new disks. On AIX 5L, use the cfgmgr command. Example 7-17 shows how the partition discovers the two new virtual disks on AIX 5L.
Example 7-17 Virtual disk discovery

# lsdev -Ccdisk hdisk0 Available # cfgmgr # lsdev -Ccdisk hdisk0 Available hdisk1 Available hdisk2 Available

Virtual SCSI Disk Drive

Virtual SCSI Disk Drive Virtual SCSI Disk Drive Virtual SCSI Disk Drive

You can also assign virtual disks by selecting Partition Management → View/Modify Partitions → Properties (Tasks) → Storage (Tab).

Chapter 7. Installing and managing the Virtual I/O Server

239

Operation on partition definitions using the CLI
You can perform the same operations with the CLI as with the GUI. The CLI also enables some non-dynamic updates on the partition’s definitions. Example 7-18 shows how to decrease processing units for a logical partition using the chsyscfg command.
Example 7-18 Decreasing the processing units

$ lssyscfg -r prof --filter "lpar_names=AppServer" -F desired_proc_units 0.40 $ chsyscfg -r prof -i "lpar_name=AppServer,desired_proc_units=0.3" $ lssyscfg -r prof --filter "lpar_names=AppServer" -F desired_proc_units 0.30 These updates become pending and are taken in account only at the next reboot of the related logical partition. A triangle with an exclamation mark inside it is displayed in the View/Modify Partitions screen if the current and pending values are not synchronized. This also shows that changes made by the CLI are immediately reflected by the GUI.

7.6 Network management
When installed, the VIOS configures one network device for each physical Ethernet interface present on the system and creates four virtual Ethernet adapters, each belonging to a separate virtual network. All physical Ethernet adapters installed in the system are managed by the VIOS. Any client logical partition can be created with its own virtual adapters connected to any of the four available virtual networks. Client logical partitions can only have up to two virtual Ethernet adapters, each connected to one of the four virtual networks present in the system. No bridging is provided with physical adapters at installation time.The VIOS enables any virtual network to be bridged with any physical adapter, provided that the same physical adapter is not used to bridge more than one virtual network. To allow partitions to access an external corporate network, every virtual network can be bridged to a physical adapter interface. For each network, a separate adapter interface is required. IVM provides a Web interface to configure bridging.

240

IBM BladeCenter JS21: The POWER of Blade Innovation

Ethernet bridging
To configure Ethernet bridging: 1. From the Virtual Ethernet Management menu in the navigation area, click View/Modify Virtual Ethernet. 2. In the work area, the virtual Ethernet panel shows the partitions that are connected to the four available networks. Select the Virtual Ethernet Bridge tab to configure bridging, as shown in Figure 7-30. For each virtual Ethernet, you can select one physical device. Use the drop-down menu to select the physical Ethernet. Click Apply to create the bridging device. Note: If the physical Ethernet that you selected for bridging is already configured with an IP address using the CLI, all connections to that address are reset. This might drop an SoL session when configuring the interface associated with I/O module 1 as described in 6.7, “Providing a console for the BladeCenter JS21” on page 121.

Figure 7-30 Creating the virtual Ethernet bridge

Chapter 7. Installing and managing the Virtual I/O Server

241

The Web GUI hides the details of the VIOS network configuration. Example 7-19 describes the VIOS configuration before the creation of the bridge. For each physical and virtual network adapter, an Ethernet device is configured. The VIOS is connected to a physical network using the en0 interface and four virtual network adapters are available (ent1 and the corresponding en1 interface are currently not used).
Example 7-19 VIOS Ethernet adapters with no bridging

$ lsdev | grep ^en en0 Available en1 Defined en2 Defined en3 Defined en4 Defined en5 Defined ent0 Available ent1 Available ent2 Available ent3 Available ent4 Available ent5 Available $ lstcpip Name Mtu en0 1500 en0 1500 lo0 16896 lo0 16896 lo0 16896

Standard Ethernet Network Interface Standard Ethernet Network Interface Standard Ethernet Network Interface Standard Ethernet Network Interface Standard Ethernet Network Interface Standard Ethernet Network Interface Gigabit Ethernet-SX PCI-X Adapter (14101403) Gigabit Ethernet-SX PCI-X Adapter (14101403) Virtual I/O Ethernet Adapter (l-lan) Virtual I/O Ethernet Adapter (l-lan) Virtual I/O Ethernet Adapter (l-lan) Virtual I/O Ethernet Adapter (l-lan)

Network link#2 9.3.5 link#1 127 ::1

Address 0.11.25.c9.11.3d js21_vios.itsc.au localhost

Ipkts Ierrs 1379465 1379465 82 82 82

0 0 0 0 0

Opkts Oerrs 44040 44040 295 295 295

Coll 0 0 0 0 0 0 0 0 0 0

When a virtual Ethernet bridge is created, a new shared Ethernet adapter (SEA) is defined, binding the physical device with the virtual device. If a network interface is configured on the physical adapter, the IP address is migrated to the new SEA.

242

IBM BladeCenter JS21: The POWER of Blade Innovation

Example 7-20 shows the result of bridging virtual network 1 with the physical adapter ent0 when the VIOS uses the network interface en0. A new ent6 SEA device is created, and the IP address of the en0 interface is migrated to the en6 interface. Because of the migration, all active network connections on en0 are reset.
Example 7-20 Shared Ethernet adapter configuration

$ lsdev | grep ^en en0 Defined en1 Defined en2 Defined en3 Defined en4 Defined en5 Defined en6 Available ent0 Available ent1 Available ent2 Available ent3 Available ent4 Available ent5 Available ent6 Available $ lstcpip Name Mtu en6 1500 en6 1500 lo0 16896 lo0 16896 lo0 16896

Standard Ethernet Network Interface Standard Ethernet Network Interface Standard Ethernet Network Interface Standard Ethernet Network Interface Standard Ethernet Network Interface Standard Ethernet Network Interface Standard Ethernet Network Interface Gigabit Ethernet-SX PCI-X Adapter (14101403) Gigabit Ethernet-SX PCI-X Adapter (14101403) Virtual I/O Ethernet Adapter (l-lan) Virtual I/O Ethernet Adapter (l-lan) Virtual I/O Ethernet Adapter (l-lan) Virtual I/O Ethernet Adapter (l-lan) Shared Ethernet Adapter

Network link#2 9.3.5 link#1 127 ::1

Address 0.11.25.c9.11.3d js21_vios.itsc.au localhost

Ipkts Ierrs 1380345 1380345 82 82 82

0 0 0 0 0

Opkts Oerrs 44078 44078 295 295 295

Coll 0 0 0 0 0 0 0 0 0 0

Use the following command to display the associated SEA devices, as shown in Example 7-21. lsmap -net -all
Example 7-21 SEA lsmap output

$ lsmap -net -all SVEA Physloc ------ -------------------------------------------ent2 U8844.5CZ.23A1137-V1-C3-T1 SEA Backing device ent6 ent0

Chapter 7. Installing and managing the Virtual I/O Server

243

Physloc

U788D.001.23A1137-P1-T7

SVEA Physloc ------ -------------------------------------------ent3 U8844.5CZ.23A1137-V1-C4-T1 SEA NO SHARED ETHERNET ADAPTER FOUND

SVEA Physloc ------ -------------------------------------------ent4 U8844.5CZ.23A1137-V1-C5-T1 SEA NO SHARED ETHERNET ADAPTER FOUND

SVEA Physloc ------ -------------------------------------------ent5 U8844.5CZ.23A1137-V1-C6-T1 SEA NO SHARED ETHERNET ADAPTER FOUND

7.7 Advanced storage management
You can assign virtual disks and physical volumes to any logical partition, one at a time. You can change storage allocation over time, and the content of the virtual storage is kept. When you create a virtual disk using a logical volume, you can also increase its size. Data protection against single disk failure is available using software mirroring: On the VIOS to protect the VIOS itself but not the LPAR data Using two virtual disks for each of the managed system’s LPAR to protect the LPAR data

7.7.1 Virtual storage assignment to a partition
Unassigned virtual disks and physical volumes can be associated to a running partition. After the operation is complete, the LPAR’s operating system must issue its device discovery procedure to detect the newly added disk. In an AIX 5L environment, do this by issuing the cfgmgr command. Before removing a physical disk or a virtual disk from a running partition, the operating system must remove the corresponding disk device because it becomes unavailable. In an AIX 5L environment, do this by using the rmdev command.

244

IBM BladeCenter JS21: The POWER of Blade Innovation

On the Web GUI, it is possible to remove a virtual disk or a physical volume from a running LPAR, but a warning sign always appears requiring an additional confirmation. Figure 7-31 shows an example of this message.

Figure 7-31 Forced removal of a physical volume

7.7.2 Virtual disk extension
Several options are available to provide additional disk space to a logical partition. The primary solution is to create a new virtual disk or select an entire physical disk and dynamically assign it to a partition. Because you can do this operation when the partition is running, it is the preferred one. After the partition's operating system issues its own device reconfiguration process, a new virtual Small Computer System Interface (SCSI) disk is available for use. You can use this disk to extend existing data structures when using Linux with a logical volume manager or AIX 5L.

Chapter 7. Installing and managing the Virtual I/O Server

245

When disk space is provided to a partition using a virtual disk, a secondary solution is to extend it using the IVM. You can perform this operation when the partition is running, but you must take the virtual disk offline to activate the change. Schedule disk outages carefully so that they do not impact the overall application availability. Consider using this solution to increase an existing operating system's volume in size and when you cannot add a new virtual SCSI disk for this purpose, that is, when using Linux without a logical volume manager. Important: We recommend that you do not perform virtual disk extension when using AIX 5L, because the same result is achieved by adding a new virtual disk. If the virtual disk is used by a rootvg volume group, you cannot extend it and you must create a new virtual disk. The following steps describe how to extend a virtual disk: 1. On the operating system, stop any activity on the disk to be extended. If this is not possible, shut down the partition. On AIX 5L, issue the varyoff command on the volume group to which the disk belongs. 2. From the Storage Management menu in the IVM navigation area, click View/Modify Devices. 3. From the work area, select the virtual disk and click Extend. Enter the disk space to be added and click OK.

246

IBM BladeCenter JS21: The POWER of Blade Innovation

4. If the virtual disk is owned by a running partition, a warning message opens, as shown in Figure 7-32. You must select the “Force extend on running partition” check box to force the expansion. The additional disk space is allocated to the virtual disk, but it is not available to the operating system.

Figure 7-32 Forced expansion of a virtual disk

5. From the Storage Management menu in the IVM navigation area, click View/Modify Devices. 6. From the work area, select the virtual disk. Click Modify partition assignment.

Chapter 7. Installing and managing the Virtual I/O Server

247

7. Unassign the virtual disk by selecting None in the New partition field. If the disk is owned by a running partition, a warning message opens, as shown in Figure 7-33. You must select the “Force device removal from running operations” check box to force the expansion.

Figure 7-33 Forced unassignment of a virtual disk

8. Perform the same action as in step 5, but assign the virtual disk back to the partition. 9. On the operating system, issue the appropriate procedure to recognize the new disk size. On AIX 5L, issue the varyonvg command on the volume group to which the disk belongs and, as suggested by a warning message, issue the chvg -g command on the volume group to recompute the volume group size.

248

IBM BladeCenter JS21: The POWER of Blade Innovation

7.7.3 VIOS system disk mirroring
To prevent a VIOS outage due to system disk failure, make the rootvg storage pool of the VIOS redundant. The default installation of VIOS uses only one physical disk. Disk mirroring on the VIOS is an advanced feature that, at the time of writing, is not available on the Web GUI. You can configure it by using VIOS capabilities on the CLI, but only system logical volumes can be mirrored. The following steps describe how to provide a mirrored configuration for the rootvg storage pool. Important: Mirrored logical volumes are not supported as virtual disks. This procedure mirrors all logical volumes defined in rootvg and must not be run if rootvg contains virtual disks. 1. Use the IVM to add a second disk to rootvg. From the Storage Management menu in the navigation area, click Advanced View/Modify Devices. 2. Select the Physical Volumes tab. Select a disk not assigned to any storage pool. Click Add to storage pool, as shown in Figure 7-34.

Figure 7-34 Adding a second disk to rootvg

Chapter 7. Installing and managing the Virtual I/O Server

249

3. In the Storage Pool field, select rootvg and click OK. 4. The actual mirroring is done using the IVM command line. Log in as the padmin user ID and issue the mirrorios command, as shown in Example 7-22. The command asks for confirmation and causes a VIOS reboot to activate the configuration after performing data mirroring.
Example 7-22 rootvg mirroring at command line

$ mirrorios This command causes a reboot. Continue [y|n]? y SHUTDOWN PROGRAM Mon Aug 15 10:20:20 CDT 2005 Wait for 'Rebooting...' before stopping.

AIX 5L mirroring on the managed system LPARs
The AIX 5L logical volume manager is capable of data mirroring. You can also use this feature when the partition is provided twice the number of virtual disks. A VIOS administrator must create virtual storage that is used by AIX 5L for mirroring purposes with careful consideration of data placement. The virtual storage must not have any physical disks in common to avoid a disk failure that affects both the mirror copies. On the VIOS, virtual disks are created out of storage pools. They are created using the minimum number of physical disks in the pool. If there is no sufficient space on a single disk, they can span multiple disks. If the virtual disks are expanded, the same allocation algorithm is applied. To guarantee mirror copy separation, we recommend that you create two storage pools and create one virtual disk from each of them. After virtual storage is created and made available as an hdisk to AIX 5L, it is important to correctly map it. On the IVM, the CLI is required. On the IVM, the lsmap command provides all the mapping between each physical and virtual device. For each partition, there is a separate stanza, as shown in Example 7-23. Each logical or physical volume displayed in the IVM GUI is defined as a backing device, and the command provides the virtual storage’s assigned LUN value.

250

IBM BladeCenter JS21: The POWER of Blade Innovation

Example 7-23 IVM command line mapping of virtual storage

$ lsmap -all ... SVSA Physloc Client Partition ID --------------- ------------------------------------ ----------------vhost1 U9113.550.105E9DE-V1-C13 0x00000003 VTD LUN Backing device Physloc VTD LUN Backing device Physloc VTD LUN Backing device Physloc VTD LUN Backing device Physloc ... vtscsi1 0x8100000000000000 aixboot1

vtscsi2 0x8200000000000000 extlv

vtscsi3 0x8300000000000000 hdisk6 U787B.001.DNW108F-P1-T14-L5-L0 vtscsi4 0x8400000000000000 hdisk7 U787B.001.DNW108F-P1-T14-L8-L0

On AIX 5L, use the lscfg command to identify the hdisk using the same LUN used by the IVM. Example 7-24 shows the command output with the 12-digit hexadecimal number representing the virtual disk’s LUN number.
Example 7-24 Identification of AIX 5L virtual SCSI disk’s logical unit number

# lscfg -vpl hdisk0 hdisk0 U9113.550.105E9DE-V3-C2-T1-L810000000000 SCSI Disk Drive PLATFORM SPECIFIC Name: disk Node: disk Device Type:

Virtual

block

Chapter 7. Installing and managing the Virtual I/O Server

251

SCSI RAID adapter use
On a system equipped with an SCSI RAID adapter, you can protect data using the adapter’s capabilities, avoiding any software mirroring. Use all physical disks managed by each adapter’s SCSI chain to create a single RAID 5 array, and install VIOS on it. Configure the adapter to create the array before installing the VIOS. To do this operation, boot the system with the stand-alone diagnostic CD and enter the adapter’s setup menu as described in 6.12, “SAS hardware RAID configuration” on page 170. After you create the array and finish formatting, install the VIOS. During the installation, the VIOS partition’s rootvg is created on the array. Provide disk space for logical partitions using the logical volumes created on the rootvg storage pool. Perform adapter maintenance using the IVM command line with the diagmenu command to access diagnostic routines. Figure 7-35 shows the menu related to the SCSI RAID adapter. It enables you to modify the array configuration and to handle events such as the replacement of a failing physical disk. PCI-X SCSI Disk Array Manager Move cursor to desired item and press Enter. List PCI-X SCSI Disk Array Configuration Create an Array Candidate pdisk and Format to 522 Byte Sectors Create a PCI-X SCSI Disk Array Delete a PCI-X SCSI Disk Array Add Disks to an Existing PCI-X SCSI Disk Array Configure a Defined PCI-X SCSI Disk Array Change/Show Characteristics of a PCI-X SCSI Disk Array Reconstruct a PCI-X SCSI Disk Array Change/Show PCI-X SCSI pdisk Status Diagnostics and Recovery Options

F1=Help F9=Shell

F2=Refresh F10=Exit

F3=Cancel Enter=Do

F8=Image

Figure 7-35 The diagmenu menu for SCSI RAID adapter

252

IBM BladeCenter JS21: The POWER of Blade Innovation

7.8 Maintenance
This section provides information about maintenance operations on the IVM. It discusses the following topics: VIOS backup and restore Logical partition backup and restore VIOS upgrade Managed system firmware update VIOS migration Command logging

7.8.1 VIOS maintenance
You can perform operations such as backup, restore, or upgrade using the IVM. Some operations are available using the GUI or using the CLI, depending on the type of operation.

Backing up and restoring the logical partition definitions
You can back up logical partition configuration information to a file. You can use this file to restore information if required, and you can also export it to another system. To back up the logical partition configuration: 1. From the Service Management menu in the navigation area, click Backup/Restore. 2. Select Generate Backup in the work area, as shown in Figure 7-36. A file named profile.bak is generated and stored under the user’s home directory. There is only one unique backup file on the VIOS at a time, and a new backup file replaces an existing one. In the work area, you can select this file name and save it on your workstation, where the browser is running. This allows you to specify a file name and save several copies.

Chapter 7. Installing and managing the Virtual I/O Server

253

Figure 7-36 Backing up the partition configuration

The backup file contains the logical partition’s configuration such as processors, memory, and network and virtual disk assignment. The content of the virtual disks is not included in the backup file. To perform a restore operation, the system must not have any logical partition configuration defined. If necessary, use the following command from the CLI before the restore. lpcfgop -o clear

254

IBM BladeCenter JS21: The POWER of Blade Innovation

In Figure 7-36, select the Restore Partition Configuration button to restore the last backed up file. If you want to restore a backup file stored on your disk, do the following steps: 1. Click Browse and locate the file. 2. Click Upload Backup File. The uploaded file replaces the existing backup file. 3. Click Restore Partition Configuration to restore the uploaded backup file. You can also back up and restore logical partition configuration information from the CLI. Use the bkprofdata command to back up the configuration information and the rstprofdata command to restore it. See the VIO Server and PLM command descriptions in the Information Center at the following Web page for more information: http://publib.boulder.ibm.com/infocenter/eserver/v1r3s/index.jsp?topic= /iphb1/iphb1_vios_commandslist.htm

Backing up and restoring the VIOS operating system
The only way to back up the VIOS operating system is with the backupios command. No operating system backup operation is available within the GUI. This command creates a bootable image that includes the VIOS partition’s rootvg and can also contain the storage pool structure depending on the flags used. Important: You can use AIX or Linux commands to back up and restore the virtual disks or physical volumes assigned to the logical partitions. To create a backup that you can use for the NIM installation described in 7.1, “VIOS installation in a JS21” on page 194, create a directory, for example, backupios -file /home/padmin/ and use the following syntax: backupios -file /home/padmin/backup_loc When this is completed, a file nim_resources.tar is available in this directory. Look at the file using the following command. Look for a file name such as xxxx_mksysb: tar -tf /home/padmin/backup_loc/nim_resources.tar With VIOS 1.2.1, we find file 5300-04_mksysb in the nim_resources.tar file. Extract this file and the file bosinst.data. Use these two files for a NIM installation. Tip: The directory used for the backup can also be a NFS mounted directory, for example, from the NIM server.

Chapter 7. Installing and managing the Virtual I/O Server

255

For further options of the backupios command, see Advanced POWER Virtualization on IBM System p5, SG24-7940.

VIOS updates
To update VIOS to the latest fix pack, use the updateios command. Before it installs the update, the updateios command makes a preview installation and displays the results. You are prompted to continue or exit at the end of the preview. Do not install the updates if the preview installation fails for any reason. Assuming that you have located the update images in /home/padmin/update, the command syntax is: updateios -dev /home/padmin/update The log file /var/adm/ras/install_all_updates.log is created. Important: Ensure that you have the right level of firmware before updating the VIOS.

7.8.2 Logical partition maintenance
Each logical partition hosted by the VIOS works like a stand-alone system. For example, you can back up and restore, boot in maintenance mode, and perform an operating system update or a migration.

Backing up the operating system
There are many ways to back up logical partitions hosted by the VIOS, depending on the operating system installed. The main options for the AIX operating system are: The mksysb command creates a bootable image of the rootvg volume group either in a file or onto a tape. The mkcd command creates a system backup image (mksysb) to CD-Recordable (CD-R) or DVD-Recordable (DVD-RAM) media from the system rootvg or from a previously created mksysb image. NIM creates a system backup image from a logical partition rootvg using the network.

256

IBM BladeCenter JS21: The POWER of Blade Innovation

Restoring the operating system
The restoration process is exactly the same as on stand-alone systems. The main steps are: 1. Log on to the VIOS. 2. Open a virtual terminal for the logical partition to be installed with the mkvt command, providing the ID of the logical partition to be restored. 3. Start the logical partition in SMS mode. 4. Select the boot device that you used for the backup such as CD, DVD-RAM, or network. 5. Boot the logical partition. 6. Follow the specific operating system’s restore procedures.

7.8.3 Command logs
All IVM actions are logged on the system. The log contains all the commands that the IVM Web GUI runs and all IVM-specific commands issued by an administrator on the command line. The log contains the following information for each action: User name Date and time The command including all the parameters The following steps describe how to access the log: 1. From the Service Management menu in the navigation area, click Application Logs. 2. In the work area, use the provided filters to restrict the log search. Click Apply.

Chapter 7. Installing and managing the Virtual I/O Server

257

This generates the selected log entries, as shown in Figure 7-37.

Figure 7-37 Command logs

258

IBM BladeCenter JS21: The POWER of Blade Innovation

8

Chapter 8.

Installing AIX
This chapter describes the installation of AIX on an IBM BladeCenter JS21. The BladeCenter JS21 supports the installation of AIX 5L Version 5.3 Technology Level 5300-04 and later. We discuss the following basic topics: General topics AIX installation from CD Preparing a Network Installation Manager (NIM) master AIX network installation using NIM

© Copyright IBM Corp. 2006. All rights reserved.

259

8.1 General topics
The simplest method to install AIX is installing from the CD/DVD. However, we recommend the network installation procedure. Typical blade server environments and logical partitioning require many operating system installations. Installing over a network saves time and can be fully automated without the need to mount physical media. AIX offers an easy to configure network installation server called NIM, which you can configure on one or more machines in the network. The so-called NIM master can either be outside the BladeCenter environment or even an IBM BladeCenter JS20 or JS21. It must be a machine running AIX 5L. If the NIM master is the first machine running AIX 5L in the given environment, you have to install it from CD, DVD, or tape. Note: Before you install any operating system, we recommend that you upgrade all firmware to the latest available level. You can apply the Flash BIOS Update of the JS21 itself and the Broadcom Firmware Update for the integrated Ethernet adapter only by using an already installed operating system (AIX or Linux) or with the stand-alone diagnostics booted from CD or network (NIM). See 6.6, “Firmware”, for further information.

8.2 Preparing AIX installation from CD/DVD
To prepare an AIX installation from CD/DVD, assign the media tray to the blade server that you want, as described in 6.5.3, “Assigning the media tray” on page 94. Mount the first volume of the AIX 5L installation CDs/DVDs in the media tray’s optical drive. Continue with the procedure as described in “Booting from CD/DVD” on page 145, and in 8.4, “Installing AIX on the client” on page 270.

260

IBM BladeCenter JS21: The POWER of Blade Innovation

8.3 Preparing AIX network installation using NIM
In this section, we describe how to set up the NIM master on a machine that is already running AIX. This can be a machine outside the BladeCenter environment or JS20 or JS21. Note: If a NIM master already exists, define your client installation on the existing NIM master and proceed with 8.4, “Installing AIX on the client” on page 270. Be aware that the NIM master’s AIX version and technology level (maintenance level) must be at least the same level as the level of the AIX it offers to the client for installation. You can check this using one of the following commands: oslevel -rf instfix -i | grep AIX_ML Use the following command to see the service pack level: /usr/bin/oslevel -s

8.3.1 Configuring AIX environment
We recommend that you create at least one extra file system for the NIM data. By default NIM uses the /export directory. You can use the following command or smitty manfs to create this file system. # crfs -v jfs2 -g rootvg -a size=2G -m /export -A yes Note: You can change rootvg and size, if necessary. If you want to create a separate file system for the different NIM resources, refer to 8.3.3, “Configuring the NIM master” on page 263. The NIM setup automatically creates these file systems, if necessary. See the options Create new filesystem for LPP_SOURCE and Create new filesystem for Shared Product Object Tree (SPOT) in Figure 8-6 on page 267.

Chapter 8. Installing AIX

261

8.3.2 Installing the NIM filesets
While the fileset bos.sysmgt.nim.client is automatically installed with AIX, you have to install the following filesets: bos.sysmgt.nim.spot bos.sysmgt.nim.master You can do this by using smitty install_latest and selecting the highlighted filesets in Figure 8-1. bos.sysmgt 5.3.0.40 Filesystem Quota Commands 5.3.0.0 License Management 5.3.0.40 Network Install Manager - Client Tools + 5.3.0.40 Network Install Manager - Master Tools + 5.3.0.30 Network Install Manager - SPOT 5.3.0.40 Software Error Logging and Dump Service Aids 5.3.0.40 Software Trace Service Aids ALL

@ @ @ > > @ @

Figure 8-1 Selecting the NIM fileset

262

IBM BladeCenter JS21: The POWER of Blade Innovation

8.3.3 Configuring the NIM master
There are several ways to configure the NIM master. The examples in this section show the text-based System Management Interface Tool (SMIT) interface. Refer to The IBM eServer BladeCenter JS20, SG24-6342, for information about how to use Web-based Systems Manager (WebSM). Note: You can also perform the commands that we issue using SMIT from the command line. Press the F8 key in SMIT to display the used command. However, most of the NIM functions are performed by a more complex script in SMIT. In this example, it is more convenient to copy the appropriate entry from $HOME/smit.script to another file, for example, /tmp/smit.test. The entry might start with: # # [May 30 2006, 20:51:47] # mkres() At the end, you can find the following entry: nim -o ... exit $? } mkres ... Insert an echo statement in the beginning of the line with the nim -o ... command: echo nim -o ... Save the file and issue the script: ksh /tmp/smit.test This shows the used command.

Chapter 8. Installing AIX

263

You can access all of the tasks using smitty nim. Figure 8-2 shows the SMIT screen. Network Installation Management Move cursor to desired item and press Enter. Configure the NIM Environment Perform NIM Software Installation and Maintenance Tasks Perform NIM Administration Tasks Create IPL ROM Emulation Media NIM POWER5 Tools

F1=Help F9=Shell

F2=Refresh F10=Exit

F3=Cancel Enter=Do

F8=Image

Figure 8-2 NIM: Main SMIT menu

Select the option Configure the NIM Environment. This opens the Configure a Basic NIM Environment (Easy Startup) screen shown in Figure 8-3.

264

IBM BladeCenter JS21: The POWER of Blade Innovation

Configure a Basic NIM Environment (Easy Startup) Type or select values in entry fields. Press Enter AFTER making all desired changes. [TOP] Initialize the NIM Master: * Primary Network Interface for the NIM Master Basic Installation Resources: * Input device for installation images * LPP_SOURCE Name * LPP_SOURCE Directory Create new filesystem for LPP_SOURCE? Filesystem SIZE (MB) VOLUME GROUP for new filesystem * SPOT Name * SPOT Directory Create new filesystem for SPOT? Filesystem SIZE (MB) VOLUME GROUP for new filesystem Create Diskless/Dataless Machine Resources? Specify Resource Name to Define: ROOT (required for diskless and dataless) DUMP (required for diskless and dataless) PAGING (required for diskless) HOME (optional) SHARED_HOME (optional) TMP (optional) Diskless/Dataless resource directory Create new filesystem for resources? Filesystem SIZE (MB) VOLUME GROUP for new filesystem Define NIM System Bundles? Define NIM bosinst_data? Prepend level to resource name Add Machines from a Definition File? Specify Filename * Remove all newly added NIM definitions and filesystems if any part of this operation fails? [BOTTOM] Figure 8-3 NIM: Easy startup screen [Entry Fields] [en0] +

[cd0] + [lpp_source_aix_534] [/export/lpp_source] + [no] + [650] # [rootvg] + [spot_aix_534] [/export/spot] + [no] + [350] # [rootvg] + [no] + [root1] [dump1] [paging1] [home1] [shared_home1] [tmp1] [/export/dd_resource] [yes] + [150] # [rootvg] + [yes] + [yes] + [no] + [no] + [] [no] +

Chapter 8. Installing AIX

265

8.3.4 Defining the NIM client
From the NIM main SMIT menu (Figure 8-2 on page 264), select Perform NIM Administration Tasks → Manage Machines → Define a Machine. This opens the screen shown in Figure 8-4. On this screen, you have to enter the host name of the client machine. Define a Machine Type or select a value for the entry field. Press Enter AFTER making all desired changes. [Entry Fields] * Host Name of Machine (Primary Network Install Interface)
Figure 8-4 NIM: Selecting the client’s host name

[js21client]

Attention: The NIM master must be able to resolve the client’s Internet Protocol (IP) address from the host name. In our example, the host name is js21client. This can be from a name server, /etc/hosts, or another supported resolver method. When you use /etc/hosts, list the short name in addition to the fully qualified host name: 10.1.1.10 js21client.ibm.com js21client

When you press Enter, and if the NIM master cannot resolve the js21client name, you see the screen shown in Figure 8-5 in the SMIT. Type of Network Attached to Primary Network Install Interface Move cursor to desired item and press Enter. tok ent fddi generic atm = = = = = token ring network ethernet network FDDI network generic network (no network boot capability) ATM network

Figure 8-5 NIM SMIT screen when the client’s host name cannot be resolved

266

IBM BladeCenter JS21: The POWER of Blade Innovation

If the host name resolution is successful, you see the screen shown in Figure 8-6. Define a Machine Type or select values in entry fields. Press Enter AFTER making all desired changes. [Entry Fields] * NIM Machine Name * Machine Type * Hardware Platform Type Kernel to use for Network Boot Communication Protocol used by client Primary Network Install Interface * Cable Type Network Speed Setting Network Duplex Setting * NIM Network * Host Name Network Adapter Hardware Address Network Adapter Logical Device Name IPL ROM Emulation Device CPU Id Machine Group Comments
Figure 8-6 NIM: Defining a machine

[js21client] [standalone] + [chrp] + [mp] + [] + tp + [] + [] + network1 js21nim [001125c9113c] [] [] +/ [] [] + []

Tip: It is optional to set the “Network Adapter Hardware Address”. Leaving it as “0” requires that you enter the client’s IP address and the NIM master’s IP address at the client’s firmware boot menu (directed Bootstrap Protocol (BOOTP)). When the client’s hardware Medium Access Control (MAC) address is specified in this screen, no input is necessary at the client (broadcast BOOTP). See 6.9.1, “Activating the Open Firmware interface” for further information.

Chapter 8. Installing AIX

267

8.3.5 Allocating resources to a client
After you define the client, you have to allocate the NIM resources to prepare the installation. From the NIM main SMIT menu (Figure 8-2 on page 264), select Perform NIM Administration Tasks → Manage Machines → Manage Network Install Resource Allocation → Allocate Network Install Resources. This opens the screen shown in Figure 8-7. Available Network Install Resources Move cursor to desired item and press F7. ONE OR MORE items can be selected. Press Enter AFTER making all selections. [TOP] > lpp_source_aix_534 > spot_aix_534
Figure 8-7 NIM: Selecting the resources

lpp_source spot

Tip: For a basic installation, you have to select at least a SPOT and an lpp_resource.

8.3.6 Activating the client installation
Now you have to activate the client installation at the NIM master. From the NIM main SMIT menu (Figure 8-2 on page 264), select Perform NIM Administration Tasks → Manage Machines → Perform Operations on Machines. This opens the screen shown in Figure 8-8. On this screen, select perform a BOS Installation.

268

IBM BladeCenter JS21: The POWER of Blade Innovation

Operation to Perform Move cursor to desired item and press Enter. = enable a machine to boot a diagnostic image cust = perform software customization bos_inst = perform a BOS installation maint = perform software maintenance reset = reset an object's NIM state fix_query = perform queries on installed fixes check = check the status of a NIM object reboot = reboot specified machines maint_boot = enable a machine to boot in maintenance mode showlog = display a log in the NIM environment lppchk = verify installed filesets restvg = perform a restvg operation update_all = update all currently installed filesets
Figure 8-8 NIM: Selecting the client operation

diag

This opens the screen shown in Figure 8-9. Perform a Network Install Type or select values in entry fields. Press Enter AFTER making all desired changes. [Entry Fields] Target Name Source for BOS Runtime Files installp Flags Fileset Names Remain NIM client after install? Initiate Boot Operation on Client? Set Boot List if Boot not Initiated on Client? Force Unattended Installation Enablement? ACCEPT new license agreements?
Figure 8-9 NIM: Performing the client installation operation

js21client rte [-agX] [] yes no no no [no]

Chapter 8. Installing AIX

269

The NIM master is now configured and waiting for the client’s BOOTP request. Attention: If an operation is already defined at the client, you see the following error: 0042-302 m_bos_inst: the state of "js21a2" prevents this operation In this case, use the reset operation as shown in Figure 8-8 on page 269.

8.4 Installing AIX on the client
You have to boot the BladeCenter JS21 from an installation media as described in 6.8.2, “Setting up remote initial program load” on page 135, because this is independent of a specific operating system. When the boot is successful, you see a message similar to Figure 8-10. Tip: The following description shows the manual setup of the client’s installation dialog. To avoid manual input on every client, NIM offers a non-prompted installation method also. See the following Web site: http://publib.boulder.ibm.com/infocenter/pseries Select AIX documentation → Installation and migration → Installing with Network Installation Management → Configuring NIM and other basic operations → Performing a nonprompted BOS installation. Alternatively, refer to NIM: From A to Z in AIX 4.3, SG24-5524.

--------------------------------------------------------------------------Welcome to AIX. boot image timestamp: 03:00 05/15 The current time and date: 16:07:33 05/16/2006 number of processors: 4 size of memory: 3968MB boot device: /pci@8000000f8000000/pci@2/ethernet@4,1:9.3.5.228,,9.3.5.231,0.0.0.0,00,00 kernel size: 11000928; 32 bit kernel --------------------------------------------------------------------------Figure 8-10 AIX welcome screen

270

IBM BladeCenter JS21: The POWER of Blade Innovation

1. After a short time, the installation dialog opens as shown in Figure 8-11. Select the console by entering 1 followed by the Enter key. ******* Please define the System Console. ******* Type a 1 and press Enter to use this terminal as the system console. Pour definir ce terminal comme console systeme, appuyez sur 1 puis sur Entree. Taste 1 und anschliessend die Eingabetaste druecken, um diese Datenstation als Systemkonsole zu verwenden. Premere il tasto 1 ed Invio per usare questo terminal come console. Escriba 1 y pulse Intro para utilizar esta terminal como consola del sistema. Escriviu 1 1 i premeu Intro per utilitzar aquest terminal com a consola del sistema. Digite um 1 e pressione Enter para utilizar este terminal como console do sistema.
Figure 8-11 Installation language panel

2. On the second panel, select the necessary language, as shown in Figure 8-12. >>> 1 Type 1 and press Enter to have English during install. 88 Help ?

>>> Choice [1]:
Figure 8-12 Selecting the language

Tip: If the correct number for the selection that you want is already displayed between the square brackets (default), press Enter without entering the number.

Chapter 8. Installing AIX

271

3. In the panel shown in Figure 8-13, select option 2, review the default settings, and make the necessary changes. Welcome to Base Operating System Installation and Maintenance Type the number of your choice and press Enter. Choice is indicated by >>>. >>> 1 Start Install Now with Default Settings 2 Change/Show Installation Settings and Install 3 Start Maintenance Mode for System Recovery

88 Help ? 99 Previous Menu

>>> Choice [1]: 2
Figure 8-13 Selecting the installation and maintenance

272

IBM BladeCenter JS21: The POWER of Blade Innovation

4. In the Installation and Settings window shown in Figure 8-14, it is important to check the Method of Installation and the Disk Where You Want to Install (option 1). If there is an AIX system on the target hard disk already, the default Method of Installation might be Preservation instead of New and Complete Overwrite. Be sure to make the appropriate selection. Installation and Settings Either type 0 and press Enter to install with current settings, or type the number of the setting you want to change and press Enter. 1 System Settings: Method of Installation.............New and Complete Overwrite Disk Where You Want to Install.....hdisk0 Primary Language Environment Settings (AFTER Cultural Convention................English Language...........................English Keyboard...........................English Keyboard Type......................Default More Options Install): (United States) (United States) (United States)

2

3 >>> 0

(Desktop, Security, Kernel, Software, ...)

Install with the settings listed above. +----------------------------------------------------| WARNING: Base Operating System Installation will | destroy or impair recovery of ALL data on the | destination disk hdisk0.

88 Help ? 99 Previous Menu >>> Choice [0]:1

Figure 8-14 Installation and language options

Attention: If the target hard disk already contains valuable data, this data is overwritten or modified.

Chapter 8. Installing AIX

273

5. After you select 1 to choose the System Settings in the previous menu, the screen shown in Figure 8-15 opens. You can select the installation method that you want. Change Method of Installation Type the number of the installation method and press Enter. >>> 1 New and Complete Overwrite Overwrites EVERYTHING on the disk selected for installation. Warning: Only use this method if the disk is totally empty or if there is nothing on the disk you want to preserve. 2 Preservation Install Preserves SOME of the existing data on the disk selected for installation. Warning: This method overwrites the usr (/usr), variable (/var), temporary (/tmp), and root (/) file systems. Other product (applications) files and configuration data will be destroyed. 88 Help ? 99 Previous Menu >>> Choice [1]:
Figure 8-15 Changing the method of installation

274

IBM BladeCenter JS21: The POWER of Blade Innovation

6. In Figure 8-16, you can select the target hard disk. If you enter the number of a disk, this toggles the selection state. You can enter more than one number at a time separated by blanks. Change Disk(s) Where You Want to Install Type one or more numbers for the disk(s) to be used for installation and press Enter. To cancel a choice, type the corresponding number and Press Enter. At least one bootable disk must be selected. The current choice is indicated by >>>. Name >>> 1 2 hdisk0 hdisk1 Location Code 01-08-00-1,0 01-08-01-1,0 Size(MB) 34715 34715 VG Status rootvg none Bootable Yes Yes No No

>>>

0 Continue with choices indicated above 55 More Disk Options 66 Disks not known to Base Operating System Installation 77 Display Alternative Disk Attributes 88 Help ? 99 Previous Menu

>>> Choice [0]:
Figure 8-16 Changing the target hard disks

Important: You have to verify the actual location code (for example, 01-08-00-1,0) of the hard disk. You have to do this because the logical name for the hard disks (for example, hdisk0) that is displayed in this menu can be different from the logical name for the same hard disk that is listed within the AIX operating system (for example, from the lspv command) that runs on the same machine. This can happen when disks are added after AIX is installed.

Chapter 8. Installing AIX

275

8.5 Changing client from rsh to nimsh
If you decide that the client is manageable by the NIM server even after the successful installation is complete and AIX is running, choose the option Remain NIM client after install? = yes when activating the client installation on the NIM server as shown in Figure 8-9 on page 269. When you define the client, you can select the Communication Protocol used by client. Figure 8-17 shows the corresponding SMIT screen and the selection box that you get when you press F4 on this item. You can select shell, which uses the insecure remote shell (rsh) protocol, and nimsh, which uses the more secure Secure Sockets Layer (SSL) protocol. Define a Machine Type or select values in entry fields. Press Enter AFTER making all desired changes. [Entry Fields] * NIM Machine Name [js21client] * Machine Type [standalone] + * Hardware Platform Type [chrp] + Kernel to use for Network Boot [mp] + Communication Protocol used by client [] + Primary Network Install Interface * ???????????????????????????????????????????????????????????????????? ? Communication Protocol used by client ? ? ? ? Move cursor to desired item and press Enter. ? ? ? ? shell = RSH Protocol is used by client ? ? nimsh = NIM Service Handler is used by client ? ? ? ? F1=Help F2=Refresh F3=Cancel ? F1? F8=Image F10=Exit Enter=Do ? F5? /=Find n=Find Next ? F9????????????????????????????????????????????????????????????????????
Figure 8-17 NIM communication protocol used by the client

When you choose nimsh, a daemon named nimsh runs on the client. The communication with the NIM server uses SSL. For this, you must install SSL on the server and the client. To do this, use: # rpm -i openssl-0.9.7g-1.aix5.1.ppc.rpm

276

IBM BladeCenter JS21: The POWER of Blade Innovation

The OpenSSL Red Hat Package Manager (RPM) package exists on the AIX 5L Linux Toolbox CD-ROM. It might have a different version than our example. You can download the RPM from the following Web site: http://www-1.ibm.com/servers/aix/products/aixos/linux/download.html You have to activate SSL on the NIM server. To do this: 1. Select smitty nim → Perform NIM Administration Tasks → Configure NIM Environment Options → Enable Cryptographic Authentication. 2. Change the option Enable Cryptographic Authentication for client communication to enable as highlighted in Figure 8-18. Enable Cryptographic Authentication Type or select values in entry fields. Press Enter AFTER making all desired changes. [Entry Fields] * Enable Cryptographic Authentication for client communication? Install Secure Socket Layer Software (SSLv3)? Absolute path location for RPM package -ORlpp_source which contains RPM package * DISPLAY verbose output? [enable]

[no] [/dev/cd0] [] [no]

F1=Help F5=Reset F9=Shell

F2=Refresh F6=Command F10=Exit

F3=Cancel F7=Edit Enter=Do

F4=List F8=Image

Figure 8-18 NIM: Enabling cryptographic authentication on the server

3. On the client, select smitty nim → Configure Client Communication Services → Configure Client Communication Services.

Chapter 8. Installing AIX

277

4. Change Communication Protocol used by client to nimsh and Enable Cryptographic Authentication for client communication to enable as shown by the highlighted values in Figure 8-19. Configure Client Communication Services Type or select values in entry fields. Press Enter AFTER making all desired changes. [Entry Fields] * Communication Protocol used by client NIM Service Handler Options Enable Cryptographic Authentication for client communication? Install Secure Socket Layer Software (SSLv3)? Absolute path location for RPM package -ORlpp_source which contains RPM package Alternate Port Range for Secondary Connections (reserved values will be used if left blank) Secondary Port Number Port Increment Range [nimsh]

*

[enable]

[no] [/dev/cd0] [] +

[] []

F1=Help
F5=Reset

F2=Refresh
F6=Command

F3=Cancel
F7=Edit

F4=List F8=Image

F9=Shell

F10=Exit

Enter=Do

Figure 8-19 NIM: Enabling cryptographic authentication on the client

278

IBM BladeCenter JS21: The POWER of Blade Innovation

5. To verify that it works, log in to the NIM server and issue the lsnim command as shown in Example 8-1.
Example 8-1 lsnim command

# lsnim –l js21client js21client: ... connect = nimsh (secure) ... The line containing connect = nimsh must have the word secure at the end. Attention: It is possible that on a JS21 the CPUID is displayed with all zeroes. You can test this with the following command: uname -m In this case, you have to update the BIOS of the JS21 to at least Version 240.470.014. If you get the following error from NIM on the client, this might be the reason. 0042-172 NIMkid: This machine's CPU ID does not match the CPU ID stored in the NIM database.

Chapter 8. Installing AIX

279

280

IBM BladeCenter JS21: The POWER of Blade Innovation

9

Chapter 9.

Installing Linux
This chapter describes the procedures to install Linux on a BladeCenter JS21. The focus is on network installation, but we also provide some hints to install Linux using a CD/DVD. One part of this chapter covers basic generic descriptions about how to set up the infrastructure for a network installation. This information is independent of the operating system and the hardware architecture. In general, it is much easier to install and maintain computer systems if an appropriate network installation infrastructure is configured. This is especially true for BladeCenter JS21, because BladeCenter JS21 is designed to be used in scenarios with a high server density and different operating systems. The BladeCenter JS21 and the related distributions are prepared to meet these concerns. AIX uses similar services and protocols, and the Network Installation Manager (NIM) utility provided by any AIX installation uses these services and protocols to enable AIX installations using a network. Hence it might be possible to use a running AIX server to install Linux using the network. However, this book does not cover this topic.

© Copyright IBM Corp. 2006. All rights reserved.

281

One part covers the network installation of the SUSE Linux Enterprise Server 9 (SLES9) Service Pack 3 (SP3). This chapter also provides a description of how to perform a network installation of a Red Hat Enterprise Linux (RHEL) AS 4 Update 3 (U3). Important: Before installing any operating system, update all firmware to the latest level. See 6.6, “Firmware” on page 94 for instructions about how to complete this task.

Note regarding VIOS: All procedures described in this chapter are also true for Linux installations on logical partitions (LPARs) on BladeCenter JS21, but there might be some minor changes in some actions or procedures. Especially the soft reboot function does not always work if Linux is running on an LPAR. In all cases, use the System Management Services (SMS) menu instead of the BladeCenter management module. For more information about VIOS and LPARs, see Chapter 7, “Installing and managing the Virtual I/O Server” on page 193.

9.1 Installing Linux from a CD/DVD
For testing or as a single application server, it might be adequate to use the CD/DVD device of a BladeCenter to load the operating system. The general preparation steps and instructions about using the Serial over LAN (SoL) console to install an operation system using CD/DVD on a BladeCenter JS21 are described in 6.7, “Providing a console for the BladeCenter JS21” on page 121. You can transfer some of the information provided in the following sections to a CD/DVD installation, but most information such as preparing an infrastructure or configuring the boot image are only helpful for a network installation.

282

IBM BladeCenter JS21: The POWER of Blade Innovation

Important for Red Hat Enterprise Linux installation: At the time of publication, the installation of the Red Hat Enterprise Linux4 U3 using local CD/DVD device on both the BladeCenter JS20 and JS21 requires that you insert a non-bootable diskette in the diskette drive of the media tray. This is posted on the Red Hat Knowledge base, which is available on the Web at: http://www.redhat.com/search/ui.jsp Also note that during the initial startup of the installation, Red Hat provides the opportunity to run a media check on the CD media before using the CD for the installation. If you select this option, it runs the check and then ejects the media at the completion of the check. You have to reinsert the CD tray before starting the installation.

Tip for Microsoft Windows Telnet users: The Serial over LAN Setup Guide provides instructions about how to use the Telnet command to begin your console sessions. You can find this guide at the following Web site: http://www-307.ibm.com/pc/support/site.wss/document.do?sitestyle=ibm &lndocid=MIGR-54666&velxr-layout=print An alternative to using the Telnet command is the putty.exe, which you can download from multiple Internet sources. Keyboard usage is much easier to follow and alter than with Windows Telnet, and monitoring the screen output is easier.

9.2 Basic preparations for a Linux network installation
This section provides all the basic information to set up services for a Linux network installation. In principle, this is not bound to the operation system or distribution, which runs on the infrastructure server to provide the necessary services. Nevertheless, all descriptions in this section are based on general Linux services, commands, and parameters. We presume that the required files for all the services are already installed and that all the commands are issued with superuser rights.

Chapter 9. Installing Linux

283

9.2.1 Installing Linux using the network: General remarks
You always require the following services to perform a network installation: A running Bootstrap Protocol (BOOTP) service or a Dynamic Host Configuration Protocol (DHCP) service that includes BOOTP support to configure the network interface of a BladeCenter JS21 A running Trivial File Transfer Protocol (TFTP) service to serve the boot image to a BladeCenter JS21 It is necessary to set up one the following services to provide the installation packages for a network installation after the boot image is loaded: – File Transfer Protocol (FTP): This is the protocol that causes the fewest problems during the installation. It has good performance and is supported by all operating systems. If you use this protocol as a read-only source, the lack of security features might not be a problem. – Hypertext Markup Language (HTML): If a Web server is already running, this protocol might even be better than FTP. It is beyond the scope of this book to cover the special features of an HTML installation. – Network File System (NFS): Even with its special features, this protocol is not a good choice, because of some incompatibilities between different versions of NFS servers and clients. – Server Message Block (SMB)/Common Internet File System (CIFS): If Microsoft Windows is already running or the Samba Server is used as a file server, you can use this server to serve the necessary data to a BladeCenter JS21. But this protocol generates a lot of overhead and supports starts with SLES10. Note: If a firewall is running on your installation server, ensure that you update the settings to allow traffic for your installation protocol.

284

IBM BladeCenter JS21: The POWER of Blade Innovation

9.2.2 Configuring a BOOTP or DHCP service
DHCP is an extension to the original BOOTP specification. As a result, you can use DHCP to provide the BOOTP information for booting using the network. The standard DHCP daemon is called dhcpd, but there are other DHCP daemons. Note: The directory you use for the configuration files depends on the distribution. The following directories are possible examples: /etc/ /etc/sysconfig/ /etc/default/ /etc/xinet.d/ (eXtended InterNET daemon configuration files) The examples in this chapter use the most common directories. In general, the name of a configuration or script file is related to the name of the installed package. For example, if a DHCP daemon is called dhcpd3-server, you can find the configuration in /etc/dhcpd3-server.conf and /etc/sysconfig/dhcpd3-server, and the start/stop script is in /etc/init.d/dhcp3-server. The standard DHCP daemon is configured through two files: The /etc/sysconfig/dhcpd file, which stores the basic configuration, and the /etc/dhcpd.conf file, which contains the configuration information for each registered client. For a running service, the actual used configuration in most cases is copied in a subdirectory of /var/. See Example 9-1 for a simple client configuration stored in dhcpd.conf. Attention: Keep in mind that Example 9-1 contains environmental-specific Internet Protocol (IP) and Media Access Control (MAC) address information. One way to learn the MAC address of a BladeCenter JS21 is to use the BladeCenter management module. Select Monitors → Hardware VPD and scroll down to the bottom of the Web page (see Figure 6-10 on page 81). For more information about how to customize dhcpd.conf, see the man pages of the dhcpd service and use man dhcpd on the command prompt.
Example 9-1 dhcpd.conf

always-reply-rfc1048 true; allow bootp; deny unknown-clients; not authoritative; default-lease-time 600;

Chapter 9. Installing Linux

285

max-lease-time 7200; subnet 192.168.1.0 netmask 255.255.255.0 { group { next-server 192.168.1.254; host js21-eth0 { fixed-address 192.168.1.100; hardware ethernet 00:11:25:C9:0B:A6; } host js21-eth1 { fixed-address 192.168.1.101; hardware ethernet 00:11:25:C9:0B:A7; filename "/tftpboot/sles9_sp3_basic"; } } } You can find the start and stop scripts of Linux services in the /etc/init.d/ directory. To start the standard DHCP daemon, use the /etc/init.d/dhcpd start command. To restart the DHCP daemon, use the /etc/init.d/dhcpd restart command. Tip for Linux beginners: The following tasks help you to double-check or troubleshoot a configuration in general. To trace messages of running services, type tail -f -n 10 /var/log/messages to get the last 10 messages and auto update if there are new messages. Connect to a running service with a local client, remote client, or both these clients and try to receive the data that you want. Make sure a changed configuration is activated by restarting a service

directly after editing, for example:
a. vi /etc/dhcpd.conf b. /etc/init.d/dhcpd restart

286

IBM BladeCenter JS21: The POWER of Blade Innovation

9.2.3 Configuring a Trivial File Transfer Protocol service
You can use the TFTP to provide a bootable image during a network installation. There are several implementations of TFTP daemons available. The standard TFTP daemon is called tftpd. In general, the xinetd or inetd super daemon is used to create a TFTP daemon. You can also run a TFTP daemon without one of the super daemons. A TFTP daemon configuration for inetd is stored in inetd.conf or /etc/default/tftpd. An example is shown in Example 9-2.
Example 9-2 Configuring a TFTP daemon in the /etc/inetd.conf file

tftp dgram udp wait nobody /usr/sbin/tcpd /usr/sbin/in.tftpd --tftpd-timeout 300 --retry-timeout 5 --mcast-port 1758 --mcast-addr 239.239.239.0-255 --mcast-ttl 1 --maxthread 100 --verbose=5 /srv/tftp Example 9-3 shows a TFTP daemon configuration for xinetd stored in /etc/xinet.d/tftpd.
Example 9-3 Configuring a TFTP daemon in the /etc/xinet.d/tftp file

service tftp { protocol = udp port = 69 socket_type = dgram wait = yes user = nobody group = nobody server = /usr/sbin/in.tftpd server_args = -vvv -s /srv/tftp only_from = 192.168.1.0 disable = no } Independent of the used operating system or TFTP daemon, it is important to configure the public TFTP directory containing the bootable image. In Example 9-2 and Example 9-3, the setting of the public TFTP directory is marked in bold. If you start the TFTP daemon without one of the super daemons, the location of the configuration file depends on the distribution.

Chapter 9. Installing Linux

287

Red Hat Enterprise Linux and SLES create TFTP from the xinetd daemon. To obtain a list of services started by xinet run the chkconfig --list command. By default, TFTP is disabled on both distributions. To enable both xinetd and TFTP, enter: 1. chkconfig tftp on 2. /etc/init.d/xinetd restart Note: The chkconfig command changes the disable parameter in the configuration file of a service. The configuration file is stored in the /etc/xinet.d/ directory. To check the status of a service controlled by xinetd, issue chkconfig without any parameters. In the /srv/tftp/tftpboot directory, we copy the bootable installation kernel later in this chapter. The bootable kernel must match the Linux distribution that you install. It must also match the file name listed in the dhcpd.conf, as shown in Example 9-1 on page 285.

9.2.4 Configuring a File Transfer Protocol service
In this book, we use FTP to serve installation data to the BladeCenter JS21. SLES includes a simple and basic FTP service setup if it is configured as an installation server using Yet Another Setup Tool (YaST1). See 9.3.2, “Preparing the installation source directory and service” on page 293. If you prefer to configure the service manually, follow the presented instruction. The standard FTP daemon is called ftpd, but there are other FTP daemons available. In this book, we use vsftpd. Similar to the TFTP daemon, there are several ways to start and configure an FTP service. In this case, we use the xinetd super daemon to start the FTP daemon. Therefore, the network configuration is stored in /etc/xinet.d/vsftpd. See Example 9-4.
Example 9-4 Configuring an FTP daemon in the /etc/xinet.d/vsftpd file

# description: # The vsftpd FTP server serves FTP connections. It uses # normal, unencrypted usernames and passwords for authentication. # vsftpd is designed to be secure. service ftp { # server_args = # log_on_success += DURATION USERID # log_on_failure += USERID # nice = 10
1

YaST provides an interface to install and configure a SUSE Linux Enterprise Server

288

IBM BladeCenter JS21: The POWER of Blade Innovation

socket_type protocol wait user server

= = = = =

stream tcp no root /usr/sbin/vsftpd

The main configuration file is /etc/vsftpd.conf or /etc/vsftpd/vsftpd.conf. A cutout is presented in Example 9-5. In most cases, for security reasons, an FTP service must provide data read-only for anonymous users. The most important customization in the public ftp directory2, which later stores the installation source files or other public accessible data, is defined by the following setting: anon_root=/srv/ftp/ This is highlighted in bold in Example 9-5. To enable the FTP service and restart xinetd use these commands: 1. chkconfig vsftpd on 2. /etc/init.d/xinetd restart
Example 9-5 Configuring an FTP daemon in the /etc/vsftpd.conf file

. . # Anonymous FTP user Settings # # Allow anonymous FTP? # anonymous_enable=YES # # Anonymous users will only be allowed to download files which are # world readable. # anon_root=/srv/ftp/ # # Public directory for anonymous user # anon_world_readable_only=YES #

2

This is the directory for anonymous login

Chapter 9. Installing Linux

289

# Uncomment this to allow the anonymous FTP user to upload files. This only # has an effect if the above global write enable is activated. Also, you will # obviously need to create a directory writable by the FTP user. # . . Tip: To test if an FTP or TFTP service is running, use the following command on the FTP or TFTP server and look for LISTEN connections: netstat -a |grep ftp If you can connect to the FTP server with an FTP client or even download data, but later the installation stops right after booting from the boot image, in most cases, the installation directory is not set up correctly. See 9.4.2, “Preparing the installation source for Red Hat Enterprise Linux” on page 324. The next step is the preparation of the installation source directory and the corresponding service. Here the preparation depends on the distribution to be installed. Therefore, we document this in separate sections: 9.3, “Installing SLES using the network”, and 9.4, “Installing Red Hat Enterprise Linux AS 4 Update 3 using the network” on page 321.

9.3 Installing SLES using the network
This section focuses on the SLES9 SP3. We emphasize special issues regarding SLES10. The prerequisites to perform an SLES installation using network are: A configured Linux installation infrastructure (see 9.2, “Basic preparations for a Linux network installation” on page 283) Configuring the boot sequence and the boot options of a BladeCenter JS21, especially the Ethernet interface configuration, appropriately (see Chapter 6, “Hardware setup” on page 63) Note: SLES switches the boot sequence of the boot devices automatically from Network - BOOTP to Hard drive X after the installation. Here X can be in the range from 0 to 3.

290

IBM BladeCenter JS21: The POWER of Blade Innovation

9.3.1 Overview of SLES network installation
The basic preparations for the SLES network installation are mentioned in 9.3, “Installing SLES using the network” on page 290. Define the installation server as described in 9.3.2, “Preparing the installation source directory and service” on page 293. After this final step, the basic preparation for the network installation is complete. There are different ways (shown as decision symbols in Figure 9-1 on page 292) to install SLES: Decide whether you want to define the boot parameter such as installation server, installation source directory, or transfer protocol using the Open Firmware prompt, also called Open Firmware interface. To learn more about the Open Firmware prompt in general, see 6.9, “Open Firmware interface” on page 152. Information is also available in 9.3.5, “Unattended installation with SLES” on page 311. In most cases, the Open Firmware method is not used to install SLES. Therefore, for a more sophisticated method, see the following section. You can predefine the installation server, installation source directory, and transfer protocol. You can do this if the installation is attended or unattended. To predefine the parameters by saving the necessary information in a boot image, see 9.3.4, “Configuring the boot image file with mkzimage_cmdline” on page 306. This basic installation procedure is described in 9.3.3, “Basic attended SLES network installation” on page 299. The tasks to enable a fully unattended installation are explained in 9.3.5, “Unattended installation with SLES” on page 311.

Chapter 9. Installing Linux

291

Basic setup of services to prepare the network install infrastructure

Set Network – BOOTP as first boot device via MM or AMM Power on BladeCenter JS21

No

Define Install Server, protocol and source path via Open firmware prompt?

Yes Open firmware prompt, set Install Server, protocol and source path BOOTP request send by BladeCenter JS21 Receive IPAddress and boot image path from BOOTP server Load boot image and start install process

BOOTP request send by BladeCenter JS21 Receive IPAddress and boot image path from BOOTP server Load boot image and start install process

No

Install Server, protocol and source path predefinied?

Yes

Initial installation menu, set Install Server, protocol and source Attended Attended or Unattended install? Unattended

Main installation menu, configure provided options

Data transfer/ install from Install Server to BladeCentre JS21

Figure 9-1 SLES network installation overview

292

IBM BladeCenter JS21: The POWER of Blade Innovation

9.3.2 Preparing the installation source directory and service
In this book, we use the FTP protocol as a service to serve the installation data of a distribution after the network boot. If the running operating system on the server is SLES, it is possible to configure the system with YaST. See “Preparing SLES as an installation server using YaST” on page 294.

Creating and working with CD/DVD ISO images
You can use the following ways to name ISO3 files: SLES CD no. X: SLES-9-ppc-SP3-CDX.iso SLES DVD no. X: SLES-10-ppc-RC1-DVDX.iso As an example, the following command creates a file in the /srv/data/iso-images/ directory directly from the second SLES CD using dd. You might want to have all the Linux distribution CD-ROMs saved on your hard disk as ISO files. dd if=/dev/cdrom of=/srv/data/iso-images/SLES-9-ppc-SP3-CD2.iso bs=32M Repeat this step for all the CDs/DVDs, changing the file name as appropriate. You can also increase the bs parameter as appropriate. This parameter controls block size. The larger the block size is, the more RAM is taken for the dd process, but the faster the process works. Important: Ensure that the CD/DVD is not mounted before beginning the dd process. Also ensure that the destination of the ISO has enough space to store all the data. One CD ISO image file typically requires 650 MB and a single layer DVD ISO image file requires up to 4.7 GB of hard drive space.

Note: We created our ISO files on a remote server and then transferred them to our installation server within the BladeCenter to take advantage of its fast network interfaces. Though working with downloaded CD ISO images makes no difference, a DVD ISO image is sometimes split up in different parts. For example, you can merge the following two parts: SLES-10-ppc-RC1-DVDX.iso#a SLES-10-ppc-RC1-DVDX.iso#b To fuse these parts, use the following command on a Linux operating system: cat SLES-10-ppc-RC1-DVDX.iso#[ab] > SLES-10-ppc-RC1-DVDX.iso
3

An ISO image is a full CD or DVD image of an ISO 9660 file system

Chapter 9. Installing Linux

293

Preparing SLES as an installation server using YaST
Within YaST, there is an applet called Installation Server to configure installation sources on the server. Use this applet to enable a service to serve the data and to prepare the installation source. 1. Access this applet from the Misc section in the left pane, as shown in Figure 9-2.

Figure 9-2 YaST installation server configuration (SLES9)

Important: The SLES version that you use as an installation server must be equal to or a later version than the SLES version which is prepared as an installation source to avoid problems.

294

IBM BladeCenter JS21: The POWER of Blade Innovation

2. The applet opens the window shown in Figure 9-3. At this point, there are no configured sources. Select Settings to do the initial setup of the installation server.

Figure 9-3 Initially no configured installation sources (SLES9) seen

Chapter 9. Installing Linux

295

3. In the Initial Setup -- Servers menu, choose a service. If you decide not to configure any service at all, just set the destination directory for the installation source directory tree. As an example, in Figure 9-4, we enable FTP and define /srv/ftp as the destination directory for the installation sources and as the public FTP directory. To save the settings, click Next.

Figure 9-4 Initial Setup -- Servers menu (SLES9)

4. In the Source Configuration pane, click Configure.

296

IBM BladeCenter JS21: The POWER of Blade Innovation

5. In the Source Configuration pane shown in Figure 9-5, choose a significant Source Name. This name is used as the directory name under the previously defined installation source directory. Use ISO images instead of CDs/DVDs to create the installation server quicker. You can also do the preparation remotely without changing medias. For information about how to create an ISO image, see “Creating and working with CD/DVD ISO images” on page 293. Proceed past this window to build the source directory tree, for example, in /srv/ftp/sles9_sp3/. If you are not building from a DVD ISO image, you are prompted for additional CD-ROMs. You can see the number of the requested CD-ROM in the title of the window.

Figure 9-5 Source configuration pane (SLES9)

Chapter 9. Installing Linux

297

6. Figure 9-6 shows SLES9 in the Configured Sources section. The Source to Configure section shows all installation sources listed that are not active. The active sources are listed in the Configured Sources section. To install the service packs, click the Change button.

Figure 9-6 SLES9 configured as an installation source

7. Choose the configured installation source and proceed with Edit. a. Select Prompt for additional CDs. b. Click Next to select the additional ISO images that make up the service pack. Now the SLES installation server is ready. 8. Copy the bootable installation kernel to the TFTP directory. With SLES, you can program this installation kernel with parameters that make installations easier and more automated. A bootable image contains both the kernel and initial RAM disk configuration in a single file. The actual file name is only important for the configuration of the DHCP daemon and TFTP daemon configuration. For SUSE, this file is called install and you can find it in the root directory of the first CD/DVD.

298

IBM BladeCenter JS21: The POWER of Blade Innovation

Note: Adjust some of the boot parameter of the bootable image with the mkzimage_cmdline tool to reduce the effort during a network installation. For more details about this tool, see 9.3.4, “Configuring the boot image file with mkzimage_cmdline” on page 306. Copy and rename the bootable image to the public TFTP boot directory of the TFTP service: cp /mnt/loop/sles9_sp3/SUSE-SLES-Version-9/CD1/install /srv/tftp/tftpboot/sles9_sp3_basic Note: In general, use the bootable image that is shipped with the distribution. This supports many different hardware drivers or software versions. You can also build a new bootable image. The explicit location during the boot process of the boot image file, /srv/tftp/tftpboot/sles9_sp3_basic is defined by two entries: – The setting of the public TFTP boot directory (for example, /srv/tftp/) defined in the TFTP server configuration file (for example, /etc/xinet.d/tftpd). See the bold line in Figure 9-2 on page 287 or Figure 9-3 on page 287. – The file name parameter in the /etc/dhcpd.conf. See the bold line in Figure 9-1 on page 285. If there are any changes in the configuration files of a service, restart the services to activate the changes (see 9.2, “Basic preparations for a Linux network installation” on page 283).

9.3.3 Basic attended SLES network installation
At this point, you must have prepared all the network services. If this is not the case, see 9.2.1, “Installing Linux using the network: General remarks” on page 284, and 9.3.2, “Preparing the installation source directory and service” on page 293. If you have fulfilled the prerequisites, the integrated TFTP client of the BladeCenter JS21 requests a bootable image4 during the boot process. Tip: You can avoid the manual setup shown in this part by using the mkzimage_cmdline tool from SUSE (see 9.3.4, “Configuring the boot image file with mkzimage_cmdline” on page 306).

4

This bootable image is also called zimage or bzimage

Chapter 9. Installing Linux

299

If you have finished all the preparations, use an SoL session to follow the boot process. If everything is set up correctly, the output shown in Figure 9-7 opens after the initial boot phase. Note: Ensure that the boot sequence is set to enable network boot using Network - BOOTP. Additionally, do not set the BOOTP configuration of the BladeCenter JS21 to a directed BOOTP request. Set the server IP and client IP to 0.0.0.0, as shown in Figure 9-7.

. . BOOTP: BOOTP: BOOTP: BOOTP: BOOTP: BOOTP: BOOTP: BOOTP:

chosen-network-type = ethernet,auto,none,auto server IP = 0.0.0.0 requested filename = client IP = 0.0.0.0 client HW addr = 0 11 25 c9 b a7 gateway IP = 0.0.0.0 device /pci@8000000f8000000/pci@2/ethernet@4,1 loc-code U788D.001.23A1137-P1-T8

BOOTP R = 1 BOOTP S = 2 FILE: /tftpboot/sles9_sp3_basic FINAL Packet Count = 10401 FINAL File Size = 5324986 bytes. load-base=0x4000 real-base=0xc00000 Elapsed time since release of system processors: 0 mins 42 secs zImage starting: loaded at 0x400000 initial ramdisk moving 0x3d7d000 <- 0x680000 (282925 bytes) trying: 0x01400000 trying: 0x01500000 . .
Figure 9-7 SoL of the SUSE network installation during startup using BOOTP of the BladeCenter JS21

300

IBM BladeCenter JS21: The POWER of Blade Innovation

Figure 9-8 shows the screen from SoL after SLES9 has booted from the original bootable image. The following message is seen because in this case the installation source is not defined yet: *** Could not find the SUSE Linux Enterprise Server 9 Installation CD. We did not use mkzimage_cmdline in this example to program in an installation source parameter into the installation kernel. You can still do the installation by choosing it manually. 1. Select the language that you want to determine the language for the following menus. We type 4 to choose English. Press Enter.
. . >>> SUSE Linux Enterprise Server 9 installation program v1.6.36 (c) 1996-2004 SUSE Linux AG <<< Starting hardware detection... Activating usb devices... done Searching for info file... *** Could not find the SUSE Linux Enterprise Server 9 Installation CD. Activating manual setup program.

Select the language. 1) 2) 3) 4) 5) 6) 7) 8) 9) 10) 11) 12) 13) 14) 15) 16) > _ Figure 9-8 SoL after operating system booted from the original SLES9 boot image Bosnia Cestina Deutsch English Español Français Hellenic Italiano Japanese Magyar Nederlands Polski Português Português Brasileiro Russian Slovencina

Chapter 9. Installing Linux

301

1. After you select the language, the main menu opens with a number of options as shown in Figure 9-9. Note: If you use hardware that is available during the installation process but is not supported by the SLES install kernel version, you can load additional kernel modules by typing 3 before starting the installation. In this example, type 4 to choose Start Installation or System and proceed with the installation process.
.

>>> Linuxrc v1.6 (Kernel 2.6.5-7.69-pseries64) (c) 1996-2004 SUSE Linux AG <<< Main Menu 1) 2) 3) 4) 5) 6) 7) > _
Figure 9-9 SLES9 main menu

Settings System Information Kernel Modules (Hardware Drivers) Start Installation or System Eject CD Exit or Reboot Power off

2. The screen shown in Figure 9-10 opens. Type 1 to start the installation or update. Start Installation or System 1) Start Installation or Update 2) Boot Installed System 3) Start Rescue System > _
Figure 9-10 SLES9 installation menu

302

IBM BladeCenter JS21: The POWER of Blade Innovation

3. You are prompted for the source of the installation medium as shown in Figure 9-11. To perform a network installation, type 2 to choose Network as the installation source. Choose the source medium. 1) CD-ROM 2) Network 3) Hard Disk > _
Figure 9-11 SLES9 source menu

4. You are prompted for the network protocol as shown in Figure 9-12. In our lab, we set up our installation source to use FTP. In this screen, we specify option 1. Choose the network protocol. 1) 2) 3) 4) > _
Figure 9-12 SLES9 protocol menu

FTP HTTP NFS TFTP

5. You are now prompted for the network options as shown in Figure 9-13. a. The menu provides options to choose an Ethernet device, for example, eth1. (We choose eth1 because eth0 is used by SoL.) b. If a DHCP server is running in the subnet, see 9.2, “Basic preparations for a Linux network installation” on page 283. You can type 1 to configure using DHCP. In our case anonymous login is suitable, therefore, we type 2. c. If no DHCP proxy is running, again type 2. d. If the DHCP service serves data about the installation server and directory, it is presented as default, but you can also choose a different configuration.

Chapter 9. Installing Linux

303

Choose the network device. 1) eth0 2) eth1 > 2 Automatic configuration via DHCP? 1) Yes 2) No > 1 Sending DHCP request... Enter the IP address of the FTP server [192.168.1.254]> Usually an FTP installation is performed via anonymous FTP. It is possible to specify a user name and password for the FTP server access. Use a user name and password? 1) Yes 2) No > 2 Use a HTTP proxy? 1) Yes 2) No > 2 Trying to connect to the FTP server... Enter the directory on the server [/sles9]>
Figure 9-13 SLES9 initial network setup for the installation source

: ethernet network card : ethernet network card

304

IBM BladeCenter JS21: The POWER of Blade Innovation

At this point, the initial setup of the SLES installation is complete and you can install SLES as usual. But there are some particularities for a network installation on a BladeCenter JS21: Package evaluation: This process takes several minutes Partitioning: Remember that SLES configures no logical volumes like Red Hat Enterprise Linux as default. A simple configuration tested with single-path storage area network (SAN) boot is shown in Example 9-6 and Example 9-7.
Example 9-6 SLES9 partition table with logical volumes

Disk /dev/sda: 146.7 GB, 146772852736 bytes 255 heads, 63 sectors/track, 17844 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot /dev/sda1 * /dev/sda2 /dev/sda3 Start 1 4 135 End 3 134 17843 Blocks 24066 1052257+ 142247542+ Id 6 82 8e System FAT16 Linux swap Linux LVM

Example 9-7 SLES9 logical volume configuration

--- Logical volume --LV Name VG Name LV UUID LV Write Access LV Status # open LV Size Current LE Segments Allocation Read ahead sectors Block device

/dev/system/all system Fhbu32-LwuK-z9MQ-zqut-MYMp-rzW0-Kio7KT read/write available 2 135.60 GB 34714 1 inherit 0 253:0

You can avoid the manual setup shown in this section by using the mkzimage_cmdline tool from SUSE (see 9.3.4, “Configuring the boot image file with mkzimage_cmdline” on page 306).

Chapter 9. Installing Linux

305

9.3.4 Configuring the boot image file with mkzimage_cmdline
Use the mkzimage_cmdline tool to adjust the bootable image. You can find it in the /ppc/netboot/ directory on the first CD/DVD of SLES. Note for experts: You can create the mkzimage_cmdline binary for other architectures using a C or C++ Compiler and the source code file called mkzimage_cmdline.c. The binary that is on the SLES9 installation media only runs on a Performance Optimization With Enhanced RISC (POWER) system running Linux. This binary does not ship on SLES9 installation CD-ROMs for x86 architecture. The following examples demonstrate some possibilities and outputs of the mkzimage_cmdline tool. Example 9-8 shows an example command to get the boot parameter information stored in the bootable image file called install. The output indicates that no command line options are stored in the image.
Example 9-8 Using mkzimage_cmdline to see boot parameters in boot image

# ./mkzimage_cmdline /srv/tftp/tftpboot/install cmd_line size:512 cmd_line: active: 0 To add new options, use -s “STRING”, where STRING is a variable which can contain several options as shown in Table 9-1. To activate the options saved in the boot image file use -a 1, and to deactivate use -a 0. ./mkzimage_cmdline -a 1 -s “STRING” /srv/tftp/tftpboot/install
Table 9-1 Some options to adjust a bootable image Optiona insmod=MODULE dhcp=0 netdevice=ethX Function Adds special device driver to the kernel, where MODULE is a placeholder for the filename of the module Disables IP address configuration using DHCP. The default is dhcp=1. Defines the physical network interface, which is used as boot device. X is the number of the device, where X ≥ 0. For a basic blade installation without Ethernet daughter cards, we recommend that you use eth1. Defines the IP address of the Ethernet boot device to X.X.X.X Defines the subnet mask of the Ethernet boot device to Y.Y.Y.Y

hostip=X.X.X.X netmask=Y.Y.Y.Y

306

IBM BladeCenter JS21: The POWER of Blade Innovation

Optiona gateway=G.G.G.G nameserver=N.N.N.N install=ftp://192.168.1.254/sles9_sp3

Function Defines the IP address of the gateway Defines the IP address of the name server Defines the protocol (ftp), the IP address of the server (192.168.1.254) and the path (/sles9_sp3) where the installation source of the operating system is stored. The supported protocols are NFS, Hypertext Transfer Protocol (HTTP), FTP, and TFTP. Defines the protocol (ftp), the IP address of the server (192.168.1.254) and the file (/conf/sles9_sp3_basic.xml) where the AutoYaST configuration for an unattended installation is stored Enables a Virtual Network Computing (VNC) service on the BladeCenter JS21 to get an insecure but graphical user interface during the installation Enables a Secure Shell (SSH) service on the BladeCenter JS21 to get a secure user interface during the installation. If an X Server is running on the client connected to the BladeCenter JS21, use a graphical interface in combination with a secure data communication.

autoyast=ftp://192.168.1.254/conf/ sles9_sp3_basic.xml

vnc=1 vncpassword=asyoulike

usessh=1 sshpassword=asyoulike

a. We did not verify all the options.

Note: When you install multiple blades, avoid reconfiguring the hostip network parameters using parameters within a bootable image. This action overwrites the configuration using a DHCP server and causes problems. When you install multiple blades, we recommend that you use the DHCP server to assign IP addresses. The following examples enable attended installation without using SoL. SoL can be helpful when starting the installation. When you use VNC as part of the installation, the SoL screen shows you the IP address to connect to.

Chapter 9. Installing Linux

307

The command without a line break shown in Example 9-9 produces a configuration that enables SSH connections to the system during the installation process.
Example 9-9 Configuring the boot image to enable installation using SSH through eth1

./mkzimage_cmdline -a 1 -s “install=ftp://192.168.1.254/sles9_sp3 usessh=1 sshpassword=mysshpassword netdevice=eth1” /srv/tftp/tftpboot/install The command shown in Example 9-10 is similar, but enables VNC connections to the system during the installation process.
Example 9-10 Configuring the boot image to enable installation using VNC through eth1

./mkzimage_cmdline -a 1 -s “install=ftp://192.168.1.254/sles9_sp3 vnc=1 vncpassword=myvncpassword netdevice=eth1” /srv/tftp/tftpboot/install Note: It is not mandatory to use the netdevice=eth1 parameter, but in general, eth1 is used for a Linux installation on a BladeCenter JS21 because eth0 is used for SoL. Keep in mind that this setting is independent from the boot sequencer definition. The defined interface is only used to install the operating system. To check whether a boot image is configured as required, use the following command: ./mkzimage_cmdline /srv/tftp/tftpboot/install Example 9-11 shows an example of this command usage after preparing the boot image (as shown in Example 9-9).
Example 9-11 mkzimage_cmdline

# ./mkzimage_cmdline /srv/tftp/tftpboot/install cmd_line size:512 cmd_line: install=ftp://192.168.1.254/sles9_sp3 usessh=1 sshpassword=mypassword netdevice=eth1 active: 1 Start the BladeCenter JS21 and connect using an SSH client or VNC client to complete the installation. After a short time, the boot process is finished. You can use an SSH client or VNC client to log on to the installation screen of YaST without SoL. To confirm if a BladeCenter JS21 is ready for login, use ping or follow the process by tracking the log file on the installation server. If the connection is established, follow the instructions on the screen. For the

308

IBM BladeCenter JS21: The POWER of Blade Innovation

SSH-enabled installation, this means type yast and press Enter to start the installation. For the VNC-enabled installation, you see the installation screen when you connect. The installation process is split up in two parts. The client connection is terminated after completion of the first part. It is necessary to reconnect. Figure 9-14 shows the last lines of the SoL output of the first installation part. Starting SSH daemon ...

/sbin/ifconfig eth0 eth1 eth1 Link encap:Ethernet HWaddr 00:11:25:C9:0B:A7 inet addr:192.168.1.101 Bcast:192.168.1.255 Mask:255.255.255.0 *** sshd has been started ***

*** ***

login using 'ssh -X root@192.168.1.101' *** run 'yast' to start the installation ***

Figure 9-14 Last SoL output using SSH during the first part of a SLES9 installation

Figure 9-15 shows the last lines of the SoL output of the second installation part. Starting SSH daemon done *** sshd has been started ***

you can login now and proceed with the installation run the command '/usr/lib/YaST2/bin/YaST2.sshinstall' active interfaces: lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0

NET: Registered protocol family 10 IPv6 over IPv4 tunneling driver
Figure 9-15 Last SoL output using SSH during the second part of a SLES9 installation

Chapter 9. Installing Linux

309

For the second part, the BladeCenter JS21 already boots from the hard drive. To continue the installation, run the following command: /usr/lib/YaST2/bin/YaST2.sshinstall When the second part is finished, the installation is complete. It might happen that an error message is shown during the second connection attempt as shown in Figure 9-16. To resolve this problem, delete the corresponding entry in ~/.ssh/known_hosts5. @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY! Someone could be eavesdropping on you right now (man-in-the-middle attack)! It is also possible that the RSA host key has just been changed. The fingerprint for the RSA key sent by the remote host is 43:9e:49:6e:84:b8:e8:cf:8c:e1:e2:8e:52:64:4f:79. Please contact your system administrator. Add correct host key in /root/.ssh/known_hosts to get rid of this message. Offending key in /root/.ssh/known_hosts:2 RSA host key for 192.168.1.101 has changed and you have requested strict checking. Host key verification failed.
Figure 9-16 SSH error message during the second connection attempt

5

“~” is a placeholder for the actual home directory and will be automatically resolved by Linux

310

IBM BladeCenter JS21: The POWER of Blade Innovation

Figure 9-17 shows another SoL example with VNC and SLES10. starting VNC server... A log file will be written to: /tmp/vncserver.log ... *** *** You can connect to 192.168.1.101, display :1 now with vncviewer *** Or use a Java capable browser on http://192.168.1.101:5801/ *** (When YaST2 is finished, close your VNC viewer and return to this window.) *** Starting YaST2 *** Xlib: extension "XInputExtension" missing on display ":0.0".
Figure 9-17 Last SoL output using VNC during SLES10 installation

To learn more about how to set up a bootable image file to prepare an unattended installation of SLES, see “Preparing an unattended installation using AutoYaST”.

9.3.5 Unattended installation with SLES
To run an unattended installation, there are two main tasks: 1. All the configuration options that are usually chosen by a human during the installation must be predefined. Therefore, SLES provides a tool called AutoYaST to create an Extensible Markup Language (XML) file, which holds all the predefined configurations options (see the following section). 2. After you create the XML file, you have to perform other steps to enable an unattended installation.

Preparing an unattended installation using AutoYaST
There are two steps to create a working configuration file, for example, sles9_sp3_basic.xml: 1. Use the AutoYaST configuration tool provided by the SUSE administration tool called YaST to create a basic XML file. It provides a graphical-based or text-based interface. 2. Adjust the entries in the XML file manually.

Chapter 9. Installing Linux

311

In this book, we use the AutoYaST configuration tool with the graphical interface to demonstrate how to create a basic XML file. There are other options available. However, it is not possible to describe all the features of the tool. Apart from general explanations, this section describes how to build the XML file shown in Appendix A, “SUSE Linux Enterprise Server 9 AutoYaST XML file” on page 419. There are a lot of optional settings, but some are mandatory settings or dependencies. For example, if you configure the firewall for eth0 and eth1, it is also necessary to specify the Ethernet interfaces eth0 and eth1. 1. Start the YaST application, which opens a window as shown in Figure 9-18. Launch the Autoinstallation applet from the Misc section of YaST.

Figure 9-18 AutoYaST module within YaST (SLES9)

312

IBM BladeCenter JS21: The POWER of Blade Innovation

2. After the selection, the main AutoYaST configuration window opens as shown in Figure 9-19.

Figure 9-19 Main AutoYaST menu (SLES9)

There are five menu options: File, View, Classes, Tools, Preferences, and three ways to create an AutoYaST file apart from loading an existing XML file using File → Open. Restriction: It is not possible to use a once created XML file from a different SLES version, because the XML format differs. You can import configuration from Red Hat or Alice6 by selecting File → Import. We do not use this option in this book.

6

A predecessor to AutoYaST

Chapter 9. Installing Linux

313

Clone the configuration of the installation server by selecting Tools → Create Reference Profile, as shown in Figure 9-20.

Figure 9-20 Creating AutoYaST file from the installation server configuration

314

IBM BladeCenter JS21: The POWER of Blade Innovation

3. A second window opens as shown in Figure 9-21. In this window, you can clone parts of the installation server configuration to create the AutoYaST file. In this case, select all three options in the Basic Resources section and click Create to generate a minimal XML file.

Figure 9-21 Selecting the basic resources

After this step, continue with some of the changes using the AutoYaST tool as described in the following section. Note: In other cases, it might be helpful to import more settings. There might be error messages, but you can correct potential errors during the manual adjustment of the XML file later.

Chapter 9. Installing Linux

315

4. After you create the create a basic configuration, adjust some of the options provided in the following sections in the menu and configure the provided options: – Software Select the packages to install, and configure the automatic update client. Figure 9-22 shows the window that opens when you select Software → Package Selection. In this case, we choose the minimum software configuration to save time during test installations.

Figure 9-22 Manual configuration of the AutoYaST file using YaST

– Hardware Configure Partitioning, Audio, Printing, and Graphics Card and Monitor, if necessary. Only the Partitioning settings are critical, but you have to change them manually later. Therefore, leave all options as they are or use an alternative partitioning configuration as a placeholder. – System Set the general system information such as language configuration, time zone, other locale-related settings, logging, and run-level information in this option. The most important configuration is the Boot Loader

316

IBM BladeCenter JS21: The POWER of Blade Innovation

Configuration. This configuration is cloned from the /etc/lilo.conf file on
the installation server. You can add a parameter, for example, max_loop=255. – Network Devices Set the network adapter information. You can set kernel module information and IP details here but avoid MAC-related configurations. In the beginning, no adapter is configured. i. To open the menu, click Configure. ii. The Network cards configuration main menu opens, select Configure to add interfaces. iii. Some selections are already configured, such as Device Type: Ethernet. Type bcm5700 as module name for the adapter and click Next. iv. In the Host name and name server section, choose DHCP for the Hostname and Domain Name (Global) and also choose DHCP for Name servers and domain search list. v. Click OK and click Next. Interface eth0 is ready now. To create interface eth1, repeat the steps. However, the DHCP settings are automatically taken from the eth0 configuration and the interface name automatically changes to eth1. – Network Services Configure network clients and daemons using this option. There are more than 15 daemons to choose from. The settings are all optional. For this example, do not change anything. – Security and Users You can create users, and configure security policies and virtual private network (VPN). The firewall configuration is straightforward, but avoid MAC-related configurations: i. In the External Interface field, type eth0. ii. In the Internal Interface field, type eth1. iii. In the next menu, select Secure Shell to enable Port 22 connections through the firewall. iv. In the following menu, disable Protect from Internal Network. v. Adapt the Security settings. The default is seven characters for the maximum password length. To log in as a root user using SSH, it is mandatory to define the root user password. This password is saved encrypted in the XML file.

Chapter 9. Installing Linux

317

Select the Edit and create users menu. Change the Filter to System Users by clicking the Set Filter button. Edit the root user and set the password. – Misc This option allows you to add complete configuration files, or to add special scripts to run before and after the installation. 5. To save the configuration, select File → Save. For example, save it as sles9_sp3_basic.xml and store the file in the /srv/ftp/conf/ directory on the installation server. 6. Manually configure the XML file. The most important changes apply to the partitioning. Example 9-12 shows a possible configuration of the partitioning section of the XML file that is not working immediately after the creation.
Example 9-12 A partitioning configuration not working in the newly created XML file

. <partitioning config:type="list"> <drive> <device>/dev/sda</device> <partitions config:type="list"> <partition> <partition_id config:type="integer">65</partition_id> <partition_nr config:type="integer">1</partition_nr> <region config:type="list"> <region_entry config:type="integer">0</region_entry> <region_entry config:type="integer">0</region_entry> </region> <size>-2097151</size> </partition> <partition> <filesystem config:type="symbol">swap</filesystem> <format config:type="boolean">true</format> <mount>swap</mount> <partition_id config:type="integer">130</partition_id> <partition_nr config:type="integer">2</partition_nr> <region config:type="list"> <region_entry config:type="integer">8</region_entry> <region_entry config:type="integer">512</region_entry> </region> <size>1071644673</size> </partition> <partition> <filesystem config:type="symbol">reiser</filesystem> <format config:type="boolean">true</format>

318

IBM BladeCenter JS21: The POWER of Blade Innovation

<mount>/</mount> <partition_id config:type="integer">131</partition_id> <partition_nr config:type="integer">3</partition_nr> <region config:type="list"> <region_entry config:type="integer">520</region_entry> <region_entry config:type="integer">16837</region_entry> </region> <size>35307651073</size> </partition> </partitions> <use>all</use> </drive> <drive> <device>/dev/sdb</device> <partitions config:type="list"/> </drive> </partitioning> Example 9-13 shows a changed configuration with two drives that are working fine. A complete functional XML file including partition information is shown in Appendix A, “SUSE Linux Enterprise Server 9 AutoYaST XML file” on page 419. After the changes in the partitioning section, the XML file is ready to use.
Example 9-13 A simple working partitioning configuration in the XML file

. <partitioning config:type="list"> <drive> <device>/dev/sda</device> <partitions config:type="list"/> <use>all</use> </drive> <drive> <device>/dev/sdb</device> <partitions config:type="list"/> </drive> </partitioning> .

Chapter 9. Installing Linux

319

Tip: For information about AutoYaST, see the guide: Automatic Linux Installation and Configuration with YaST2. You can find this guide on the Web at: http://www.suse.com/~ug/autoyast_doc/ Keep in mind that some of the options depend on the architecture or SLES version.

Performing an unattended SLES installation
You can perform an unattended SLES installation by making an AutoYaST file available when you boot the boot image. The AutoYaST file is an XML file that can exist on a network server, such as an FTP server. One example is to add the following line to your boot parameters of a boot image using mkzimage_cmdline: ftp://192.168.1.254/conf/sles9_sp3_basic.xml With the mkzimage_cmdline tool, prepare a fully unattended installation of BladeCenter JS21. Configure the bootable image file with the mkzimage_cmdline tool. To learn more about the usage of this tool, refer to 9.3.4, “Configuring the boot image file with mkzimage_cmdline” on page 306. Issue the command shown in Example 9-14 without new lines to write the configuration to the bootable image sles9_sp3_basic_auto. The file /srv/tftp/tftpboot/sles9_sp3_basic_auto must exist to perform the operation.
Example 9-14 Configuring boot image for unattended SLES installation

./mkzimage_cmdline -a 1 -s “install=ftp://192.168.1.254/sles9_sp3 autoyast=ftp://192.168.1.254/conf/sles9_sp3_basic.xml netdevice=eth1” /srv/tftp/tftpboot/sles9_sp3_basic_auto Important: The unattended installation requires an empty partition table on the target hard drive to avoid problems during the partitioning process.

Restriction: You cannot use SSH or VNC in combination with an AutoYaST installation to follow the progress. Even if it is configured as an option, it does not work. To follow the progress, use SoL or use standard tools such as ping or logging.

320

IBM BladeCenter JS21: The POWER of Blade Innovation

Note: Use the Open Firmware prompt and set all the necessary options instead of using the mkzimage_cmdline tool. Issue the following command at the open firmware prompt: boot net install=ftp://192.168.1.254/srv/ftp/sles9_sp3/ autoyast=ftp://192.168.1.254/conf/sles9_sp3_basic.xml netdevice=eth1 See “Performing an unattended Red Hat Enterprise Linux installation” on page 343, for more information about this type of boot parameter definition. After the installation, which might take some time, the newly installed server must be reachable using SSH at the IP address configured using the DHCP server.

9.4 Installing Red Hat Enterprise Linux AS 4 Update 3 using the network
This section focuses on the Red Hat Enterprise Linux AS 4 (U3). The prerequisites to perform a Red Hat Enterprise Linux installation using the network are: A configured Linux installation infrastructure (see 9.2, “Basic preparations for a Linux network installation” on page 283) The boot device sequence and the boot options of a BladeCenter JS21, especially the Ethernet interface configuration, which you must configure appropriately (see 6.1, “BladeCenter chassis” on page 64) Attention: Red Hat Enterprise Linux does not switch the boot sequence of boot devices after the installation. Perform this using the BladeCenter management module, as described in 6.5, “Blade server configuration” on page 90.

Chapter 9. Installing Linux

321

9.4.1 Overview of Red Hat Enterprise Linux network installation
Figure 9-23 on page 323 shows the steps during the network installation process. Some necessary and basic preparations are already mentioned in the preface of 9.4, “Installing Red Hat Enterprise Linux AS 4 Update 3 using the network” on page 321. However, we describe some more tasks in 9.4.2, “Preparing the installation source for Red Hat Enterprise Linux” on page 324 After this final step, the basic preparation of the network installation is complete, but there are different ways, shown as decision symbols in Figure 9-23 on page 323. One possibility is installation without a defined installation server, installation source directory, or transfer protocol, which in the most cases also means attended installation. This basic installation procedure is described in 9.4.3, “Basic attended Red Hat Enterprise Linux network installation” on page 326. The definition of boot parameters such as installation server, installation source directory, or transfer protocol using the Open Firmware prompt is used in most cases in combination with an unattended installation. To learn more about the Open Firmware prompt in general, see 6.9, “Open Firmware interface” on page 152. More specific informations about unattended installation are provided in 9.4.4, “Unattended installation with Red Hat Enterprise Linux” on page 336. Restriction: You cannot use the mkzimage_cmdline tool provided by SUSE to configure a bootable image shipped with the Red Hat Enterprise Linux.

322

IBM BladeCenter JS21: The POWER of Blade Innovation

Basic setup of services to prepare the network install infrastructure

Set Network – BOOTP as first boot device via MM or AMM Power on BladeCenter JS21

No

Define Install Server, protocol and source path via Open firmware prompt?

Yes

Open firmware prompt, set Install Server, protocol and source path

BOOTP request send by BladeCenter JS21 Receive IPAddress and boot image path from BOOTP server Load boot image and start install process RHEL initial installation menu, set Install Server, protocol and source path

BOOTP request send by BladeCenter JS21 Receive IPAddress and boot image path from BOOTP server Load boot image and start install process

Attended

Attended or Unattended install?

Unattended

Main installation menu, configure provided options

Data transfer/ install from Install Server to BladeCentre JS21

Figure 9-23 Red Hat Enterprise Linux network installation overview

Chapter 9. Installing Linux

323

9.4.2 Preparing the installation source for Red Hat Enterprise Linux
Two steps are necessary to prepare the installation server: 1. It is necessary to have a service that provides installation data after the boot image is loaded. In this part, as an example, we use FTP for this function. Therefore, the first step is the preparation of an FTP service on the installation server to enable access to the installation sources. The general approach is already described in 9.2.4, “Configuring a File Transfer Protocol service” on page 288. 2. When the FTP server is running, copy the Red Hat Enterprise Linux sources from CDs or ISO images to the public FTP directory of the previously prepared FTP server. To build an installation source directory for Red Hat Enterprise Linux AS 4, follow the given instructions in this section. Because it is much easier to work with ISO images, the first step is to create ISO images. You can name the ISO files as follows: Red Hat Enterprise Linux CD no. X: RHEL4-U3-ppc-AS-CDX.iso As an example, the following command creates a file in the /srv/data/iso-images/ directory directly from the third Red Hat Enterprise Linux CD using dd: dd if=/dev/cdrom of=/srv/data/iso-images/RHEL4-U3-ppc-AS-CD3.iso bs=32M Repeat the previous step for all the CDs/DVDs, changing the file name as appropriate. You can also increase the bs parameter as appropriate. This parameter controls the block size. The larger the block size is, the more RAM is taken for the dd process, but the faster the process works. Important: Ensure that you do not mount the CD/DVD before beginning the dd process. Also ensure that the destination of the ISO has enough space to store all the data. One CD ISO image file typically requires 650 MB and a single layer DVD ISO image file requires up to 4.7 GB of hard drive space.

Note: We created our ISO files on a remote server and then transferred them to our installation server within the BladeCenter to take advantage of its fast network interfaces. Now copy the installation data to the FTP directory. Because we are handling five CD-ROMs, we show how to process all the five CD-ROMs with single commands.

324

IBM BladeCenter JS21: The POWER of Blade Innovation

1. Create some directories to mount the ISOs: for X in ‘seq 1 5‘; do mkdir /mnt/loop/rhel4/CD$X -p; done 2. Mount the ISO files using a loop device. You can write the following command in one command line: for X in ‘seq 1 5‘; do mount -t iso9660 -o loop /srv/data/iso-images/RHEL4-U3-ppc-AS-CD$X.iso /mnt/loop/rhel4/CD$X; done 3. Create a new directory in the public FTP directory: mkdir /srv/ftp/rhel4 4. Copy the now accessible files to this directory including hidden files. The slash (“/”) at the end of the source directory is important for the following command: rsync -auv /mnt/loop/rhel4/CD[12345]/ /srv/ftp/rhel4 5. Copy the installation bootable image into the TFTP server. A bootable image contains both the kernel and initial RAM disk configuration in a single file. The actual file name is only important for the configuration of the DHCP-daemon and TFTP-daemon configuration. For the Red Hat Enterprise Linux, this file is called netboot.img and is in the directory /images/pseries/ of the first CD. Copy and rename the bootable image to the public TFTP boot directory of the TFTP service: cp /mnt/loop/rhel4/CD1/images/pseries/netboot.img /srv/tftp/tftpboot/rhel_as_4_u3.img Note: In general, use the bootable image that is shipped with the distribution to avoid trouble with different hardware drivers or software versions. 6. The explicit location during the boot process of the boot image file, /srv/tftp/tftpboot/rhel_as_4_u3.img is defined by two entries: – The setting of the public TFTP boot directory (for example, /srv/tftp/) defined in the TFTP server configuration file (for example, /etc/xinet.d/tftpd). See the bold text in Figure 9-2 on page 287 or Figure 9-3 on page 287. – Adjust the file name parameter in /etc/dhcpd.conf to: filenname “/tftpboot/rhel_as_4_u3.img For more information, see 9.2.2, “Configuring a BOOTP or DHCP service” on page 285.

Chapter 9. Installing Linux

325

7. If there are any changes in the configuration files of a service, restart the services to activate the changes. See 9.2, “Basic preparations for a Linux network installation” on page 283. After this step, all basic preparations to install Red Hat Enterprise Linux using the network are finished.

9.4.3 Basic attended Red Hat Enterprise Linux network installation
At this point, you must have prepared all the network services. If this is not the case, see 9.2.1, “Installing Linux using the network: General remarks” on page 284, and 9.4.2, “Preparing the installation source for Red Hat Enterprise Linux” on page 324. If you have fulfilled the prerequisites, the integrated TFTP client of the BladeCenter JS21 requests a bootable image7 during the boot process. The preparations to provide this bootable image using the prepared TFTP server are described in this section. Attention: The Red Hat Enterprise Linux installation does not set the boot sequence to a hard drive after the installation using the network. You must manually set this with the BladeCenter management module or one of the boot menus. You can also configure the BladeCenter JS21 to boot using the hard drive, but select the boot menu and select boot using the network manually before the installation.

7

This bootable image is also called zimage or bzimage

326

IBM BladeCenter JS21: The POWER of Blade Innovation

1. After the reboot, the SoL output is similar to Figure 9-24. It is almost similar to SLES. . . BOOTP: BOOTP: BOOTP: BOOTP: BOOTP: BOOTP: BOOTP: BOOTP:

chosen-network-type = ethernet,auto,none,auto server IP = 0.0.0.0 requested filename = client IP = 0.0.0.0 client HW addr = 0 11 25 c9 b a7 gateway IP = 0.0.0.0 device /pci@8000000f8000000/pci@2/ethernet@4,1 loc-code U788D.001.23A1137-P1-T8

BOOTP R = 1 BOOTP S = 2 FILE: /tftpboot/netboot.img FINAL Packet Count = 12149 FINAL File Size = 6220088 bytes. load-base=0x4000 real-base=0xc00000 Elapsed time since release of system processors: 1 mins 46 secs zImage starting: loaded at 0x400000 Allocating 0x66a000 bytes for kernel ... trying: 0x01400000 trying: 0x01500000 . .
Figure 9-24 SoL output of RHEL network installation during startup using BOOTP function of the BladeCenter JS21

Chapter 9. Installing Linux

327

2. The SoL output after Red Hat Enterprise Linux has booted from the bootable image is shown in Figure 9-25. Choose a language. Welcome to Red Hat Enterprise Linux +---------+ Choose a Language +---------+ | | | What language would you like to use | | during the installation process? | | | | Catalan # | | Chinese(Simplified) # | | Chinese(Traditional) # | | Croatian # | | Czech # | | Danish # | | Dutch # | | English # | | | | +----+ | | | OK | | | +----+ | | | | | +---------------------------------------+ <Tab>/<Alt-Tab> between elements | <Space> selects | <F12> next screen

Figure 9-25 SoL after the operating system boot from the RHEL bootable image

328

IBM BladeCenter JS21: The POWER of Blade Innovation

3. The screen shown in Figure 9-26 opens. Choose an installation method. We choose FTP. Welcome to Red Hat Enterprise Linux +------+ Installation Method +------+ | | | What type of media contains the | | packages to be installed? | | | | Local CDROM | | Hard drive | | NFS image | | FTP | | HTTP | | | | +----+ +------+ | | | OK | | Back | | | +----+ +------+ | | | | | +-----------------------------------+ <Tab>/<Alt-Tab> between elements | <Space> selects | <F12> next screen

Figure 9-26 Red Hat Enterprise Linux 4: Installation method menu

Chapter 9. Installing Linux

329

4. You see the Networking Device menu as shown in Figure 9-27. Choose eth1 because eth0 is used for SoL. Welcome to Red Hat Enterprise Linux +----------------------+ Networking Device +----------------------+ | | | You have multiple network devices on this system. Which | | would you like to install through? | | | | eth0 - Broadcom Corporation NetXtreme BCM5780S Gigabit Ethernet | | eth1 - Broadcom Corporation NetXtreme BCM5780S Gigabit Ethernet | | | | +----+ +------+ | | | OK | | Back | | | +----+ +------+ | | | | | +-----------------------------------------------------------------+ <Tab>/<Alt-Tab> between elements | <Space> selects | <F12> next screen

Figure 9-27 Red Hat Enterprise Linux 4: Network device menu

330

IBM BladeCenter JS21: The POWER of Blade Innovation

5. The screen now prompts you to configure TCP/IP as shown in Figure 9-28. In this example, the installation process uses DHCP. Press Enter. Welcome to Red Hat Enterprise Linux +--------------------+ Configure TCP/IP +---------------------+ | | | Please enter the IP configuration for this machine. Each | | item should be entered as an IP address in dotted-decimal | | notation (for example, 1.2.3.4). | | | | [*] Use dynamic IP configuration (BOOTP/DHCP) | | | | IP address: ________________ | | Netmask: ________________ | | Default gateway (IP): ________________ | | Primary nameserver: ________________ | | | | +----+ +------+ | | | OK | | Back | | | +----+ +------+ | | | | | +-------------------------------------------------------------+

<Tab>/<Alt-Tab> between elements

| <Space> selects | <F12> next screen

Figure 9-28 Red Hat Enterprise Linux 4: Configure TCP/IP menu

Chapter 9. Installing Linux

331

6. You are prompted for a DNS as shown in Figure 9-29. If all the information is not presented by the DHCP server, you see another message box. Welcome to Red Hat Enterprise Linux +--------------------+ Configure TCP/IP +---------------------+ | +-------------+ Nameserver +--------------+ | | Please e| | Each | | item sho| Your dynamic IP request returned IP | cimal | | notation| configuration information, but it did | | | | not include a DNS nameserver. If you | | | [| know what your nameserver is, please | | | | enter it now. If you don't have this | | | | information, you can leave this field | | | | blank and the install will continue. | | | | | | | | Nameserver IP _________________________ | | | | | | | | +----+ +------+ | | | | | OK | | Back | | | | | +----+ +------+ | | | | | | | | | | +---------+-----------------------------------------+ --------+ <Tab>/<Alt-Tab> between elements | <Space> selects | <F12> next screen

Figure 9-29 RHEL4: Example for a missing name server definition in the DHCP server configuration file

332

IBM BladeCenter JS21: The POWER of Blade Innovation

7. Configure the FTP server information as shown in Figure 9-30. Welcome to Red Hat Enterprise Linux +------------------------+ FTP Setup +-------------------------+ | | | Please enter the following information: | | | | o the name or IP number of your FTP server | | o the directory on that server containing | | Red Hat Enterprise Linux for your | | architecture | | | | FTP site name: 192.168.1.254___________ | | Red Hat Enterprise Linux directory: rhel4___________________ | | | | [x] Use non-anonymous ftp | | | | +----+ +------+ | | | OK | | Back | | | +----+ +------+ | | | | | +--------------------------------------------------------------+ <Tab>/<Alt-Tab> between elements | <Space> selects | <F12> next screen

Figure 9-30 Red Hat Enterprise Linux AS 4: FTP setup

Chapter 9. Installing Linux

333

After this step, the installation data is transferred through FTP and Anaconda. The Red Hat Enterprise Linux installer Anaconda starts as shown in Figure 9-31. Red Hat Enterprise Linux AS (C) 2004 Red Hat, Inc. +--------+ Red Hat Enterprise Linux AS +--------+ | | | Welcome to Red Hat Enterprise Linux AS! | | | | | | +----+ +------+ | | | OK | | Back | | | +----+ +------+ | | | | | +-----------------------------------------------+ <Tab>/<Alt-Tab> between elements | <Space> selects | <F12> next screen

Figure 9-31 Red Hat Enterprise Linux 4: Anaconda welcome screen

334

IBM BladeCenter JS21: The POWER of Blade Innovation

8. After some more menus to set up Red Hat Enterprise Linux 4, the last screen before a reboot is shown in Figure 9-32. At this point, use the BladeCenter management module to change the boot sequence from Network - BOOTP to Hard drive X, if it is not prepared already. Red Hat Enterprise Linux AS (C) 2004 Red Hat, Inc.

+----------------------+ Complete +-----------------------+ | | | Congratulations, your Red Hat Enterprise Linux AS | | installation is complete. | | | | Remove any installation media (diskettes or CD-ROMs) | | used during the installation process and press | | <Enter> to reboot your system. | | | | | | +--------+ | | | Reboot | | | +--------+ | | | | | +---------------------------------------------------------+

<Enter> to reboot
Figure 9-32 SoL after Red Hat Enterprise Linux installation is complete

In the case of a Red Hat Enterprise Linux installation, it is not possible to reduce the effort or avoid the SoL connection during a network installation because of the lack of a tool such as mkzimage_cmdline, but you can prepare an unattended installation.

Chapter 9. Installing Linux

335

9.4.4 Unattended installation with Red Hat Enterprise Linux
To run an unattended installation, there are two main tasks: 1. All the configuration options that are typically chosen by a human during the installation must be defined in a Kickstart file. Therefore, Red Hat Enterprise Linux provides a tool called Kickstart Configurator to create a text file, which holds all predefined configurations options. 2. After you create the configuration file, you have to perform some more steps to enable an unattended installation. However, there is no tool such as mkzimage_cmdline to adjust the boot image file.

Preparing an unattended installation using Kickstart
Note for Red Hat Enterprise Linux 3 user: Be aware of the changes from Red Hat Enterprise Linux 3 to Red Hat Enterprise Linux 4. For example, Red Hat Enterprise Linux 3 uses redhat-config-xxx to start most of the administration tools; Red Hat Enterprise Linux 4 uses system-config-xxx instead.

Restriction: It is necessary to have a running X Server to use the administration tools provided by Red Hat, because in most cases the tools use a graphical user interface (GUI). Red Hat provides a utility called system-config-kickstart to assist with the creation of the configuration file for an unattended installation. This tool is a good starting point to create an initial configuration file, but the provided options are not sophisticated enough to create a working configuration. Therefore, after you create the configuration file with Kickstart, you have to perform some manual changes in this file. Kickstart is not included in the default software installation. If the Red Hat Enterprise Linux installation source is stored in the directory rhel4 on an FTP server with the IP address 192.168.1.254, the first step is to start the package management by issuing the following command: system-config-packages -t ftp://192.168.1.254/rhel4

336

IBM BladeCenter JS21: The POWER of Blade Innovation

This connects to an FTP server with the IP address 192.168.1.254 with the installation source directory rhel4. In the Package Management window, the Kickstart tool is located under System → Administration Tools. 1. After you install Kickstart, issue the system-config-kickstart command to launch this utility. 2. A window opens showing the Basic Configuration panel. The most important configuration setting is highlighted with a rectangle in Figure 9-33. It is also important to define a root password to enable SSH login after installation. This password is saved encrypted in the configuration file.

Figure 9-33 Kickstart main window with Basic Configuration panel (RHEL4)

Chapter 9. Installing Linux

337

3. In the Installation Method panel (shown in Figure 9-34), all the basic parameters for a network installation using FTP are shown.

Figure 9-34 Installation Method panel (RHEL4)

338

IBM BladeCenter JS21: The POWER of Blade Innovation

4. The next editable panel is the Partition Information panel, as shown in Figure 9-35. The provided options are not specific enough to create a valid partition configuration for a BladeCenter JS21. You can leave the configuration as it is. After you create the configuration, adjust the settings manually.

Figure 9-35 Partition Information panel (RHEL4)

5. The next panel is the Authentication panel. But in this case, we use the default setting.

Chapter 9. Installing Linux

339

6. Figure 9-36 shows the Firewall Configuration panel. As an example, it is a good idea to enable SSH using interface eth1 to access the system later using network.

Figure 9-36 Firewall Configuration panel (RHEL4)

340

IBM BladeCenter JS21: The POWER of Blade Innovation

7. Figure 9-37 shows the Package Selection panel. We do not explain the other panels in this context, because they are not important to create a basic configuration file. After the package selection, save the configuration using the Save option in the menu. When you save the file, this runs an automatic basic check for any missing options.

Figure 9-37 Package Selection panel (RHEL4)

8. Manually adjust the Kickstart configuration file that you have created. The basic Kickstart configuration file created with the Kickstart Configurator is shown in Example 9-15.
Example 9-15 Basic Kickstart configuration file

#Generated by Kickstart Configurator #platform=IBM pSeries #System language lang en_US

Chapter 9. Installing Linux

341

#Language modules to install langsupport en_US #System keyboard keyboard us #System mouse mouse #Sytem timezone timezone America/New_York #Root password rootpw --iscrypted $1$/7IU0tSr$QmQtfRtKWulXPC8cmk2kf. #Reboot after installation reboot #Use text mode install text #Install OS instead of upgrade install #Use Web installation url --url ftp://192.168.1.254/rhel4 #System bootloader configuration bootloader --location=mbr #Clear the Master Boot Record zerombr yes #Partition clearing information clearpart --all --initlabel #System authorization infomation auth --useshadow --enablemd5 #Network information network --bootproto=dhcp --device=eth0 network --bootproto=dhcp --device=eth1 #Firewall configuration firewall --enabled --trust=eth1 --ssh #Do not configure XWindows skipx #Package install information %packages --resolvedeps Tip: Red Hat Enterprise Linux 4: System Administration Guide is a good source of information regarding all Kickstart file options. It is also available at the following Web site: http://www.redhat.com/docs/manuals/enterprise/RHEL-4-Manual/sysadmin -guide/

342

IBM BladeCenter JS21: The POWER of Blade Innovation

If you have not defined any partition options, the unattended installation process stops. It is necessary to include the following new section after the #Disk partitioning information section and before the #System authorization information section: #Disk partitioning information autopart Important: The order of the main sections in the Kickstart configuration file is important for the functionality. You can find the fully functional Kickstart configuration file with some additional packages and partitioning information in Appendix B, “Red Hat Enterprise Linux 4 Kickstart file” on page 427.

Performing an unattended Red Hat Enterprise Linux installation
For an unattended installation, pass the location of the configuration file during boot. BOOTP does not have the facility to provide anything more than the location to the bootable image and the TFTP server IP address. To pass the required parameters, use the Open Firmware prompt. 1. During the boot process, a screen output that is readable using SoL is available. Type 8 to go to the Open Firmware prompt, as shown in Figure 9-38. 1 = SMS Menu 8 = Open Firmware Prompt 5 = Default Boot List 6 = Stored Boot List

Memory 0 > _

Keyboard

Network

SCSI

Speaker

ok

Figure 9-38 Open Firmware prompt

Chapter 9. Installing Linux

343

2. To start the unattended installation, type the following command at the Open Firmware prompt if the configuration file is served using FTP: boot net ks=ftp://192.168.1.254/conf/ks.cfg ksdevice=eth1 The nvsetenv command is provided by the ppc64-utils Red Hat Package Manager (RPM) in Red Hat Enterprise Linux and the util-linux RPM in SLES. The syntax to clear the boot-file variable is: /sbin/nvsetenv boot-file "" The unattended Red Hat Enterprise Linux installation is now complete.

344

IBM BladeCenter JS21: The POWER of Blade Innovation

10

Chapter 10.

System management scenarios
This chapter primarily addresses two system management applications that support BladeCenter environments: IBM Director and Cluster Systems Management (CSM).

Cluster Systems Management and IBM Director positioning
CSM is recommended for managing UNIX-only clusters (Linux or AIX) when the client prefers a scripting interface to perform cluster management tasks. CSM provides full installation for Linux and a flexible command-line interface (CLI). You can use IBM Director and CSM together to manage clustered systems. This is especially true in the case of clustered blades.

© Copyright IBM Corp. 2006. All rights reserved.

345

10.1 BladeCenter Web interfaces
The BladeCenter Web interface allows system administrators to easily and effectively manage up to 14 blades from an integrated interface. From trivial tasks such as powering blades on or off, to more complex tasks such as firmware management, the Web interface allows powerful control over all blades and input/output (I/O) modules that are attached to the BladeCenter chassis. You can use the Web interface for the management of other BladeCenter resources, such as I/O modules, and the retrieval of system health information. You can also configure BladeCenter-specific features such as the Serial over LAN (SoL) from the Web interface. For more information, see IBM eServer BladeCenter Systems Management, REDP-3582.

10.2 IBM Director
IBM Director is an integrated suite of tools that provides you with comprehensive system-management capabilities to maximize system availability and lower IT costs. Its open, industry-standard design supports the management of a variety of hardware and operating systems, including most Intel microprocessor-based systems and certain IBM System p5, IBM eServer iSeries™, IBM eServer pSeries, IBM System z9™, and IBM eServer zSeries® servers. IBM Director automates many of the processes that are required to manage systems proactively, including capacity planning, asset tracking, preventive maintenance, diagnostic monitoring, troubleshooting, and more. It has a graphical user interface (GUI) that provides easy access to both local and remote systems. At the time of writing this book, IBM Director V5.10.2 was the latest version available and is the version we used for our testing for this topic. This latest version includes: Broader platform coverage for use in a heterogeneous environment that includes IBM System p5, eServer p5, eServer i5, and eServer pSeries A new streamlined interface to boost productivity A new command-line interface in addition to the graphical interface Lightweight agents for easy deployment

346

IBM BladeCenter JS21: The POWER of Blade Innovation

Important: Though you can use multiple versions of IBM Director in the same environment, ensure that you use like versions of IBM Director Server and IBM Director Agent. For example, use IBM Director Server V4.0 with IBM Director Agent V4.0 and use IBM Director Server V5.10 to manage systems with IBM Director Agent V5.10.

10.2.1 Installing the components for IBM Director
The primary components for IBM Director are: Prerequisite applications IBM Director Core Services IBM Director Server IBM Director Agent IBM Director Console (Not to be confused with a Hardware Management Console (HMC).) IBM Director extensions (This includes extensions for HMCs) Step-by-step instructions for installing the components on the various supported hardware and software platforms (with the exception of HMCs) are provided in detail on the following Web site: http://publib.boulder.ibm.com/infocenter/eserver/v1r2/index.jsp From the IBM Systems Software Information Center link, select Topic overview → IBM Director. The IBM Director CD is shipped with IBM BladeCenter chassis but is not shipped with IBM blade servers. For the purposes of our review and testing, we downloaded the necessary files and also the images as instructed on the Web site. IBM Director can gather some information from a blade server before the IBM Director Agent or IBM Director Core Services is installed on the blade server. The information is gathered from the blade server by way of the BladeCenter management module. In the IBM Director Console, the blade server is represented by a physical platform managed object. However, after you install IBM Director Agent or IBM Director Core Services on the blade server, it is a managed object, and the features and functions that you can use on the blade server are comparable to those that you can use on any managed object.

Chapter 10. System management scenarios

347

The IBM Director tasks that you can use on your BladeCenter unit can vary, depending on the features and options that you have installed. See Table 10-1 for a list of the IBM Director tasks, and information about whether you can use a task on the chassis, network device, or a blade server without IBM Director Agent or IBM Director Core Services installed. Unless otherwise noted in this documentation, a task behaves in the same way for blade servers as for any managed system. Note: When you install IBM Director Agent or IBM Director Core Services on a blade server, the supported tasks depend on the operating system (OS) that you install on the blade server.
Table 10-1 IBM Director task support for BladeCenter products Tasks and subtasks Chassis Network device No Yes No Yes Yes No Yes Yes Yes Yes Blade server without IBM Director Agent or IBM Director Core Services installed Not applicable Yes Yesa Yes Not applicable Yes No No No Yesb

BladeCenter Configuration Manager Event action plans Hardware status Y Inventory Network Device Manager (formerly Switch Management launch pad) Power Management™ Rack Manager Remote session Remote monitors Simple Network Management Protocol (SNMP) browser

Yes Yes Yes Yes Not applicable No Yes Not applicable No No

a. You can obtain an inventory of the chassis, network device, and blade servers through the management module. Blade server inventory that is collected through the management module is a subset of the total inventory that is available if IBM Director Agent or IBM Director Core Services is installed on the blade server. b. To use the SNMP browser task, install the operating system SNMP agent on the blade server.

348

IBM BladeCenter JS21: The POWER of Blade Innovation

The following sections provide additional instructions that augment the details in the Web site provided previously.

Installing IBM Director V5.10.2 on AIX
For this exercise, we used a desktop system running Windows 2000 for the IBM Director Console. We installed the server and extensions code on a BladeCenter JS20 running AIX V5.3 and installed the agent code on several BladeCenter JS21s running AIX V5.3, Red Hat, and SUSE Linux in both symmetric multiprocessor (SMP) and virtual input/output (VIO) settings (logical partitions (LPARs)). Prerequisite applications There are two primary prerequisites that are described in the instructions cited previously. The first is the IBM AIX Pegasus CIM Server. You can find this on the AIX V5.3 Expansion Pack. Install this before the IBM Director Server for AIX or the IBM Director Agent for AIX. The second application is the OpenSSL code that is required by the CIM Server. You can install it from the AIX Toolbox for Linux Applications. Console code We downloaded the console code directly to our desktop workstation running Windows 2000. There is an IBM Director Console V5.10 compressed file and a V5.10.2 patch compressed file. You can find the IBM Director Console V5.10 for download at: http://www.ibm.com/servers/eserver/xseries/systems_management/ibm_di rector/ On this Web site, select Downloads in the left menu. Select IBM Director 5.10 for xSeries and BladeCenter and complete the registration as requested and click Submit. The next screen provides a list of selectable versions of the console code. Select IBM Director 5.10 Upward Integration Modules (UIM) to install the initial version and follow the installation instructions including selection of the Bladecenter Management Extension and the Rack Manager. Install the IBM Director 5.10.2 patch for Windows found at: http://www-307.ibm.com/pc/support/site.wss/document.do?lndocid=MIGR63788 Server and extensions code You can find the IBM Director V5.10.2 Server installation code at the following Web site. You have the option of downloading either a tar file or an ISO image: https://www14.software.ibm.com/webapp/iwm/web/preLogin.do?source=dmp

Chapter 10. System management scenarios

349

We downloaded the ISO image, created a CD and proceeded to install the server code and the BladeCenter and rack extensions using smitty as instructed. Depending on your network setup, you might have to follow the instructions to modify Secure Sockets Layer (SSL) port settings after completing your installation. See the section “Configuring SSL settings for IBM Director Console and IBM Director Server” in the IBM Systems Software Information Center cited previously. Agent code: The agent code is available at: https://www14.software.ibm.com/webapp/iwm/web/reg/download.do?source =dmp&S_PKG=diragent&cp=UTF-8 We downloaded the code and placed it on a CD. We extracted the AIX-specific code from the tar file using the following command: tar -xvf /mnt/*r director/agent/aix We followed the instructions to run the proper file. The instructions provided for downloading and installing from the ISO CD image for the IBM Director Agent for AIX also worked as shown in the Web site referenced previously. We followed the installation instructions on the Web site provided at the beginning of this section and started the console on our PC. The IBM Director server discovered both the JS20 and JS21 blades and BladeCenter modules. We also viewed vital product data and started a remote session on the one blade with the IBM Director Server installed. Figure 10-1 shows the IBM Director Console on our Windows system after the IBM Director automatically discovered all systems in the network.

350

IBM BladeCenter JS21: The POWER of Blade Innovation

Figure 10-1 IBM Director BladeCenter discovery

Figure 10-1 reflects the selection of Chassis and Chassis Members group. The only editing necessary for the discovered objects is to change the names for convenience. At this point the IBM Director Agent (Level 2) is not installed on any of the blades except the one JS20 functioning as the IBM Director Server. As you can see, the management module interface provides information for the management module, assorted blades, and Ethernet switches. Basic functions such as power management (on or off) for the blades is available. However, restarting the OS, remote sessions, file transfer, and other options are not available without installing the IBM Director Agent. Refer to Table 10-1 on page 348 for the functions available without the agent on the blades to be managed.

Chapter 10. System management scenarios

351

Installing IBM Director Agent V5.10.2 on Linux
This version supports the JS21. The following Web site provides specific instructions that are sufficient for a successful installation: http://publib.boulder.ibm.com/infocenter/eserver/v1r2/index.jsp We downloaded the IBM Director V5.10.2 for Linux on Advanced Performance Optimization with Enhanced RISC (POWER) ISO file and burned a CD for the initial installation. We recommend that you use a Linux installation server where you have multiple blades to install. Follow the preparation guidelines closely. Prerequisite RPMs are identified.

For IBM Director Core Services
The prerequisites for IBM Director Core Services are: Red Hat Enterprise Linux AS, Version 4.0, for IBM POWER only: Before you install IBM Director on a system running Red Hat Enterprise Linux AS, Version 4.0, for IBM POWER, ensure that the compat-libstdc++-33-3.2.3-47.3.ppc.rpm Red Hat Package Manager (RPM) file is installed. Linux on POWER only: Ensure that the following RPM files (or later versions) are installed: – librtas-1.2-1.ppc64.rpm – ppc64-utils-2.5-2.ppc64.rpm – lsvpd-0.12.7-1.ppc.rpm You can download these RPM files from the IBM Service and productivity tools for Linux on POWER Web site at: https://www14.software.ibm.com/webapp/set2/sas/f/lopdiags/home.html Select the appropriate tab for your Linux distribution. Tip: SUSE Linux Enterprise Server 9 (SLES9) for IBM POWER only: Disable the Service Location Protocol daemon (SLPD) before you install IBM Director Core Services. IBM Director Server does not discover Level 1 managed objects that are running SLPD.

IBM Director Agent
Red Hat Enterprise Linux AS, Version 4.0, for IBM POWER only: Before you install IBM Director on a system running Red Hat Enterprise Linux AS, Version 4.0, for IBM POWER, ensure that the compat-libstdc++-33-3.2.3-47.3.ppc.rpm RPM file is installed.

352

IBM BladeCenter JS21: The POWER of Blade Innovation

After you fulfill the prerequisites, use the IBM Director V5.10.2 for Linux on POWER CD to install both SLES9 Linux and Redhat Linux blades and LPARs. 1. To start the installation from the CD, perform the following steps: a. Insert the IBM Director Version 5.10.2 for Linux on POWER or IBM Director Agents for AIX 5L, IBM i5/OS®, Windows, Linux on xSeries, IBM System z™ and POWER CD into the drive. b. If the CD automounts, go to step c. If the CD does not automount, type the following command and press Enter: mount /dev/cdrom /mnt/cdrom Here dev/cdrom is the specific device file for the block device and mnt/cdrom is the mount point of the drive. For our purposes, we used /dev/sr0 as the device and /media/cdrom as a mount point. c. Change to the directory in which the installation script is located. Type the following command and press Enter: cd /mnt/cdrom/director/agent/linux/ppc/FILES Here mnt/cdrom is the mount point of the drive. 2. If you want to customize the installation, go to step 3. If you want to accept the default settings for the installation, type the following command before moving to step 7. ./dir5.10_agent_linppc.sh Go to step 7. 3. To customize the installation, copy the response file (diragent.rsp) to a local directory by entering this command: cp diragent.rsp /directory Here directory is the local directory. 4. Open an American Standard Code for Information Interchange (ASCII) text editor and modify the installation settings in the copy of the diragent.rsp file. This file is fully commented. You can specify the location of the RPM files and select log file options. 5. Save the modified response file with a new file name. 6. To install IBM Director Agent using the response file, type the following command and press Enter: ./dir5.10_agent_linppc.sh -r /directory/response.rsp Here directory is the local directory to which you copied the response file, and response.rsp is the name of the response file saved in step 5.

Chapter 10. System management scenarios

353

7. By default, encryption using the Advanced Encryption Standard (AES) algorithm is enabled during installation. An optional step is to configure encryption to a different setting. To disable encryption or change security settings, type the following command and press Enter: /opt/ibm/director/bin/cfgsecurity 8. To start IBM Director Agent, type the following command and press Enter: /opt/ibm/director/bin/twgstart 9. If you installed IBM Director Agent from the CD you can now unmount the CD-ROM. Note: After we installed all the prerequisite RPM files, we experienced the following scenarios: With Red Hat V4 Update 3, we experienced an error at the end of the installation of the IBM Director Agent shell script: Failed Dependencies compat-libstdc++-33.3.2.3-47.3 lsvpd v0.12.7 librtas 1.2 ppc64-utils 2.5 After ensuring that we have already installed these files at the correct level, we issued the twgstart command. The IBM Director Agent processes started and performed as expected. With SLES9 Upgrade 3, we experienced no errors with the installation shell script, but we received an error when we issued the twgstart command. Failed dependencies: lsvpd>=0.12.7 needed by pSeriesCoreServices-level1-5.10.2.1-SLES9 We confirmed that we had already installed a more recent lsvpd level and reissued the twgstart command. The IBM Director Agent started up with no other problems.

354

IBM BladeCenter JS21: The POWER of Blade Innovation

10.2.2 Using the IBM Director Console in a BladeCenter context
After we installed the Agent on all blades and LPARs that we expected to work with, we started the console on our Windows desktop system and began managing the BladeCenter chassis, blades, and network devices. The environment shown in Figure 10-1 on page 351 is a single chassis with only six blades powered up. You can see from the following section that managing multiple chassis with full complements of blades is much easier than attempting to monitor and manage them solely with the BladeCenter Web Interface to each individual chassis. While the initial labeling of the BladeCenter elements is fairly intuitive after discovery, we re-labeled some of the blades for easier tracking as shown in Table 10-2.
Table 10-2 Blade location and names Bay 1 2 3 4 5 6 Name xSeries Bay1 JS21 VIOS Bay 2 JS21 Bay 3 JS20 Director Bay 4 JS20 NIM Bay 5 JS20 SLES Bay 6 Blade type - machine HS20 8678-21X JS21 8844-51X JS21 8844-5CZ JS20 8842-41X JS20 8842-21X JS20 8842-41X

Chapter 10. System management scenarios

355

An example of the inventory information available on a blade is shown in Figure 10-2. This window is generated when we double-click the entry for JS21 VIOS Bay 2.

Figure 10-2 Inventory data

Blade management
When you install IBM Director Agent on a blade or LPAR, there are additional functions that you can access. They include remote sessions (consoles), process management, power management (restart), and others. The IBM Director documentation available on the Web provides excellent instructions on implementation. The objective of this section is to provide a brief view of the basic features that are useful for the BladeCenter environment.

356

IBM BladeCenter JS21: The POWER of Blade Innovation

Discovering IBM Director Level 2 Agents
Figure 10-3 shows the display for systems discovered with Level 2 Director Agents installed. You can see that one item shows a lock (linux.site). This is how a newly discovered agent appears before any access to the agent is requested. Also notice the right panel that displays the various options for selection. You can initiate discovery by selecting Tasks → Discover → Level 2: IBM Director Agents from the top menu bar. Alternatively, you can click the Flashlight icon to perform the same task.

Figure 10-3 Level 2 Agents

The systems that we used for Agent installation and management as shown are: linux.site (a SLES9 Linux LPAR on a JS21 blade) js21a1.itsc.austin.ibm.com (an AIX LPAR on a JS21 blade) js20ibmdirector2 (the IBM Director Server on a JS20 blade) js20:linuxinstall1 (Red Hat Linux on a JS20 blade)

Chapter 10. System management scenarios

357

Power management
Figure 10-4 shows the selections available for actions on Level 2 Agents. You can see this menu when you right-click any one of the entries in the center pane. In this particular example, you see the power management selection which only offers restart. This is only for rebooting the operating system and does not power on or power off the blade itself.

Figure 10-4 Level 2 Agent: Menu selections

358

IBM BladeCenter JS21: The POWER of Blade Innovation

Remote console
Figure 10-5 displays the remote console that is provided when you select the remote session option. IBM Director tries to access the system first through Secure Shell (SSH) and then using Telnet if SSH fails. This screen reflects the successful use of SSH on a SLES9 Linux LPAR.

Figure 10-5 Remote session

Chapter 10. System management scenarios

359

Resource monitors
Another useful task is the resource monitor. Select the system (agent) and then the task either in the right task pane or right-click a particular system. This opens a set of resources to select as shown in Figure 10-6. You can specify any number of resources to monitor. In this case, we selected CPU Utilization and Disk Monitors.

Figure 10-6 Resource monitors

360

IBM BladeCenter JS21: The POWER of Blade Innovation

If you determine that you have critical threshold for a given resource that is important for your application environment, you can identify it as a selected resource (for example, Space Available), and right-click the metric (for example, 30). This shows the data sheet to identify the threshold and notification data as shown in Figure 10-7.

Figure 10-7 Resource threshold

BladeCenter management becomes much more efficient when you enlist the aid of IBM Director.

Chapter 10. System management scenarios

361

For further information for the effective use of IBM Director, refer to Implementing IBM Director 5.10, SG24-6188, and the documentation provided on the IBM Director CD shipped with your IBM BladeCenter.

10.3 Cluster Systems Management
This section provides you with the CSM setup and installation tasks that are relevant to the BladeCenter and BladeCenter JS21 environment. Cluster implementation tasks are varied and dependent largely on your network and switch usage and application parameters. We do not cover cluster implementation in this section other than to show the successful establishment of a CSM cluster using JS20 and JS21 blades over Ethernet. The BladeCenter JS21 is now supported by CSM V1.5.1. There are specific program temporary fixes (PTFs) for AIX V5.3 that you have to install along with this version of CSM. Refer to the following Web site and ensure that you have the authorized program analysis reports (APARs) or the referenced filesets installed: https://www14.software.ibm.com/webapp/set2/sas/f/csm/download/csmaix_1. 5.1.1down.html For this exercise, we created a directory on our selected management server /usr/sys/inst.images/csm and placed all the filesets in this directory and then installed all the necessary filesets. We closely followed the updated instructions in CSM 1.5.0 for AIX 5L and Linux: Planning and Installation Guide, SA23-1344, and applied the V1.5.1 updates available on: http://publib.boulder.ibm.com/infocenter/clresctr/vxrx/index.jsp?topic= /com.ibm.cluster.csm.doc/clusterbooks.html With the updates applied, you can follow the steps as outlined in CSM 1.5.0 for AIX 5L and Linux: Planning and Installation Guide, SA23-1344. The basic steps are as follows: 1. 2. 3. 4. 5. 6. Set up the management server. Set up one or more installation servers (optional). Define the nodes in the cluster. Define non-node devices to the cluster (optional). Install the nodes of the cluster (optional). Add the nodes to the cluster. You can add AIX, Linux, or both AIX and Linux nodes.

Note that for this exercise, we did not run steps 2 and 4.

362

IBM BladeCenter JS21: The POWER of Blade Innovation

10.3.1 Installing and setting up the CSM management server
This section describes the steps to install and set up the CSM management server.

Step 1: Start CSM installation
After you identify and install all the prerequisites noted previously, download the full CSM V1.5.1 Update ISO image (60-day try-and-buy) and burn a CD. Use the CD to install the CSM management server on a BladeCenter JS21 running AIX V5.3. After you assign the CD drive to the JS21 using the management module interface, use the usual AIX commands to assign and mount /dev/cd0. Use the smitty csm command and you see the screen shown in Figure 10-8.

Figure 10-8 SMIT CSM installation

Chapter 10. System management scenarios

363

When you select CSM Install, this generates a screen with multiple CSM installation and configuration options as shown in Figure 10-9. When you select Install the Management Server, this provides the options for selecting your installation source (in our case, the CD), the filesets you want to include, and the normal System Management Interface Tool (SMIT) software installation options.

Figure 10-9 CSM installation options

Step 2: Register the host name
Confirm the registration of the server, client, and management module host names and addresses with your name server. This is an important task that is easily overlooked. Refer to “Installing the management server on AIX” of CSM 1.5.0 for AIX 5L and Linux: Planning and Installation Guide, SA23-1344. For our exercise, we used a JS21 as the management server and set up an AIX LPAR and SLES9 LPAR on a JS21 and another JS21 running AIX as compute nodes.

364

IBM BladeCenter JS21: The POWER of Blade Innovation

Step 3: Update the $PATH and $MANPATH variables
Add /opt/csm/bin to the root user’s $PATH variable on the management server: export PATH=$PATH:/opt/csm/bin To access the CSM main pages, add /opt/csm/man to the root user’s $MANPATH variable on the management server: export MANPATH=$MANPATH:/opt/csm/man To verify that this step is completed successfully, issue the following commands: echo $PATH echo $MANPATH Note: The examples provided previously only show how to change the $PATH and $MANPATH variables in the current login session. To permanently change them, edit your login environment.

Step 4: Create the /csminstall file system
This is an optional step but we chose to implement a separate file system of 5 GB to anticipate AIX and Linux operating system resources. We also increased the /VAR file system by 250 MB.

Step 5: Download CSM 1.5.1.1 updates
Download the csm-aix-1.5.1.1.power.tar.gz file from the following Web site to a local PC: http://www14.software.ibm.com/webapp/set2/sas/f/csm/home.html Use File Transfer Protocol (FTP) to copy the file to the management server. You can extract the file into the /tmp/csm directory and use the tar -xvf command to extract the files. Use smit install with /tmp/csm as the source directory to complete the update installation.

Step 6: Downloading open source software
Cluster Systems Manager uses several types of open source software to manage various hardware devices.

Downloading openCIMOM for hardware control
Tip: This is only necessary for HMC hardware control. It was not necessary for our purposes in a BladeCenter environment.

Chapter 10. System management scenarios

365

Downloading autoupdate
The autoupdate program is required to support the CSM Software Maintenance System (SMS) feature that is available for updating software on Linux nodes. You only require the software if you add Linux nodes to your CSM cluster. You can download the autoupdate software from the following Web site: http://freshmeat.net/projects/autoupdate To download the software, select the link under RPM package, then download autoupdaterelease.noarch.rpm (for example, autoupdate-5.2.5-1.noarch.rpm). Copy the RPM to a temporary directory, for example /tmp/csm/RPMS/ppc. You do not have to install the RPM on the management server. The autoupdate RPM is required only when you add Linux nodes to the cluster. You can postpone downloading the autoupdate RPM until you are ready to follow the procedure to add the Linux node. In this exercise, because we have a Linux LPAR node we downloaded the package to our PC and then used FTP to copy it to the suggested directory.

Step 7: Verify that the prerequisite software is installed
When AIX is installed, the following CSM base filesets are also automatically installed: csm.core csm.client csm.dsh csm.diagnostics If the filesets are not available, use AIX CD-ROM #1 to reinstall them. You also have to check for the latest Reliable Scalable Cluster Technology (RSCT) filesets. You can download the latest available from the following Web site and install them: https://techsupport.services.ibm.com/server/aix.fdc

366

IBM BladeCenter JS21: The POWER of Blade Innovation

Figure 10-10 shows a sample of the current output from the lslpp -L rsct* command. Fileset Level State Type Description (Uninstaller) ---------------------------------------------------------------------------rsct.core.auditrm 2.4.5.0 C F RSCT Audit Log Resource Manager rsct.core.errm 2.4.5.0 C F RSCT Event Response Resource Manager rsct.core.fsrm 2.4.5.0 C F RSCT File System Resource Manager rsct.core.gui 2.4.5.0 C F RSCT Graphical User Interface rsct.core.hostrm 2.4.5.0 C F RSCT Host Resource Manager rsct.core.lprm 2.4.5.0 C F RSCT Least Privilege Resource Manager rsct.core.rmc 2.4.5.0 C F RSCT Resource Monitoring and Control rsct.core.sec 2.4.5.0 C F RSCT Security rsct.core.sensorrm 2.4.5.0 C F RSCT Sensor Resource Manager rsct.core.sr 2.4.5.0 C F RSCT Registry rsct.core.utils 2.4.5.0 C F RSCT Utilities
Figure 10-10 Output from lslpp -L rsct* command

Step 8: Install CSM management server software
The following AIX filesets and RPM packages are available on AIX CD-ROM #1 product media. CSM on AIX filesets: – – – – csm.server csm.gui.dcem csm.gui.websm csm.hpsnm csm.gui.websm is a prerequisite for csm.hpsnm. If you have a High Performance Switch (HPS) network in your cluster, csm.hpsnm is required. Open source software: – – – – conserver expect tcl tk

Chapter 10. System management scenarios

367

To install this software, insert CD-ROM #1 and use AIX Web-based system manager, SMIT panels, or the AIX command-line interface. For example, you can use the AIX geninstall command to install selected CSM software as follows: geninstall -IaX -d /dev/cd0 csm.server csm.gui.dcem \ csm.gui.websm csm.hpsnm R:expect R:tcl R:tk R:conserver

Step 9: Apply CSM management server updates
This reconfirms that the updates we downloaded in “Step 5: Download CSM 1.5.1.1 updates” on page 365, have been applied. This has been done already.

Step 10: Check the remote shell attribute
Run the csmconfig command to check this attribute. Remember that your PATH statement must include /opt/csm/bin. The output from this command on the management server is as follows: RemoteShell=/usr/bin/rsh SetupRemoteShell = 1 (yes) RemoteShell uses rsh as the default executable to run dsh for remote commands. In our initial setup, we accepted the default and were not able to communicate with our nodes. We first used a temporary override to OpenSSH by using the command export DSH_REMOTE_CMD=/usr/bin/ssh. This allowed immediate communication with those nodes where we had installed SSH and had the daemon running. To make this the standard remote executable, we issued the following command: csmconfig RemoteShell=/usr/bin/ssh SetupRemoteShell=1 SetupRemoteShell Indicates to CSM that you want the remote shell to be automatically configured (1 = yes, 0= no). Read more about this in CSM 1.5.0 for AIX 5L and Linux: Planning and Installation Guide, SA23-1344.

Step 11: Install OpenSSH and OpenSSL (optional)
We installed CSM on a server that also functions as an IBM Director Server. Part of this exercise is to install OpenSSH and OpenSSL. OpenSSH is available on the AIX Expansion Pack CD. OpenSSL is available on the Linux Toolbox CD. Use your standard Web-based Systems Manager (WebSM) or SMIT process to install these from their respective CDs.

Step 12: Install Kerberos Version 5 for remote commands (optional)
We chose not to implement this step initially.

368

IBM BladeCenter JS21: The POWER of Blade Innovation

Step 13: Accept the CSM license
See CSM 1.5.0 for AIX 5L and Linux: Planning and Installation Guide, SA23-1344, for the options if you have a full license. We used a 60-day try-and-buy license. To accept the 60-day try-and-buy license agreement, issue the csmconfig command: csmconfig -L You can check the success of the csmconfig command by running it with no flags, and then checking the output as shown in Figure 10-11. You can also run the license acceptance using the smit csm command. AddUnrecognizedNodes = 0 (no) BMCConsoleEncryptAuth = 1 (yes) BMCConsoleKeepAlive = 0 (no) BMCConsolePerMsgAuth = 0 (no) ClusterSNum = ClusterTM = 9078-160 DeviceStatusFrequency = 12 DeviceStatusSensitivity = 8 ExpDate = Sat Jul 22 18:59:59 2006 HAMode = 0 HeartbeatFrequency = 12 HeartbeatSensitivity = 8 MaxNumNodesInDomain = -1 (unlimited) NetworkInstallProtocol = nfs PowerPollingInterval = 300 PowerStatusMode = 1 (Events) Properties = RegSyncDelay = 1 RemoteCopyCmd = /usr/bin/rcp RemoteShell = /usr/bin/rsh SetupKRB5 = 0 SetupNetworkInstallProtocol = 1 (yes) SetupRemoteShell = 1 (yes) TFTPpackage = tftp-hpa
Figure 10-11 Output from the csmconfig command

Notice the ExpDate entry because we ran a 60-day license.

Chapter 10. System management scenarios

369

Step 14: Copy CSM files into the /csminstall subdirectories
Issue the following csmconfig command to copy the CSM files into the proper /csminstall subdirectories: csmconfig -c These files are primarily used when CSM system management scripts are run on the nodes of the cluster. Note: You can combine the -c option with the -L option mentioned in the previous step. Therefore, for example, you can just run csmconfig -c -L instead of running the command twice. Use the following command to set up the management server to place Linux CSM installation files into their appropriate directories: copycsmpkgs -p /csminstall/Linux InstallCSMVersion=1.5.1 InstallOSName=Linux InstallDistributionName=SLES InstallDistributionVersion=9 InstallPkgArchitecture=ppc64 We already downloaded the Linux CSM software into the directory /csminstall/Linux. You can download the file csm-linux-1.5.1.1.ppc64.tar.gz from the following Web site using the 60-day try-and-buy license: https://www14.software.ibm.com/webapp/set2/sas/f/csm/download/home.html Extract the file in a temporary directory using the gunzip command and then extract the files from the resulting tar file using the tar -xvf csm-linux-1.5.1.1.ppc64.tar command. The tar file is 106 MB in size. Therefore, you must ensure that your temporary directory or your /csminstall file system has sufficient space to accommodate it.

Step 15: Set the cluster ID (optional)
This step is only appropriate for 1600 clusters that are assigned a serial number. We did not use it in our context.

Step 16: Store hardware control point user IDs and passwords
This step covers several configuration options including HMC managed IBM System p devices. BladeCenter HS20 (other than HS20-8678), HS40, JS20, and JS21 servers use the Serial over LAN (SoL) feature to provide remote console access. For our BladeCenter environment, we followed these specific instructions from the CSM 1.5.0 for AIX 5L and Linux: Planning and Installation Guide, SA23-1344.

370

IBM BladeCenter JS21: The POWER of Blade Innovation

BladeCenter 8677: To configure BladeCenter management modules, change the default hardware control point login profile user ID and password using the management module Web interface and documentation provided with the hardware. For BladeCenter management modules, the default login profile user ID is USERID, and the default password is PASSW0RD. (P-A-S-S-W-zero-D). BladeCenter blade servers, or IBM eServer xSeries 336, and xSeries 346 servers using SoL to access remote console: If the BladeCenter contains HS20 (other than HS20-8678), HS40, JS20, or JS21 blade servers, or xSeries 336 or xSeries 346 servers using SoL instead of a console server device, run the systemid command with the -c option to store the login profile user ID and password required to access the SoL feature. For BladeCenter advanced management modules, the default console user ID is RMTCON, and the default console password is RMTC0N (R-M-T-C-zero-n). Change the default console login profile user ID and password using the management module Web interface and documentation provided with the hardware. You must not use a login profile with supervisor privileges for console access. If the default console login profile user ID is not used, the new login profile must only be granted Blade Console remote access privileges. Use the systemid -c command or the smit systemid command to complete this step. We used the systemid -c command, as shown in Figure 10-12. # systemid -c mmext.itsc.austin.ibm.com ausres08 New Password: Verifying, please re-enter password: systemid: following entries were created 9.3.5.220_console
Figure 10-12 systemid command

The command syntax in this case is systemid -c <management module external ethernet hosthame> <userid>

Step 17: Install any user-defined power methods
We do not provide any details about this step. See CSM 1.5.0 for AIX 5L and Linux: Planning and Installation Guide, SA23-1344, for further instructions, if necessary.

Chapter 10. System management scenarios

371

Step 18: Verify the installation
To verify that the management server is installed correctly and is ready for use, you can run the ibm.csm.ms probe, which is shipped with CSM. To run the probe issue the following command probemgr -p ibm.csm.ms -l 0 The output from the command provides a view of each of the checks performed and a final summary statement. The output in Figure 10-13 shows the end of the output from our exercise. Probe ibm.csm.ms returned the following information. ibm.csm.ms:trace:Checking that directory /opt/csm exists. ibm.csm.ms:trace:Checking that directory /opt/csm/bin exists. ibm.csm.ms:trace:Checking that directory /opt/csm/csmbin exists. ibm.csm.ms:trace:Checking that directory /opt/csm/install exists. ibm.csm.ms:trace:Checking that directory /usr/sbin/rsct/bin exists. ibm.csm.ms:trace:Checking that directory /csminstall exists. ibm.csm.ms:trace:Checking that directory /var/log/csm exists. ibm.csm.ms:trace:Checking that directory /opt/csm/install/defs exists. ibm.csm.ms:trace:Checking that directory /opt/csm/install/pkgdefs exists. ibm.csm.ms:trace:Checking that directory /csminstall/csm/config/ exists. ibm.csm.ms:trace:Checking that directory /csminstall/csm/status exists. ibm.csm.ms:trace:Checking for packages : csm.client*, csm.core*, csm.deploy*, csm.diagnostics*, csm.dsh*, csm.server*. ibm.csm.ms:trace:Checking for packages : conserver-8.1.7, expect*, openCIMOM-0.8-1, tcl*, tk*. ibm.csm.ms:trace:Check if the CFM cronjob is enabled. Probe ibm.csm.ms was run successfully.
Figure 10-13 Sample output of the probemgr command

There are additional steps included in CSM 1.5.0 for AIX 5L and Linux: Planning and Installation Guide, SA23-1344, such as installing Kerberos as an option, which we did not use. It also provides a step-by-step instruction about how to install CSM on a Linux Management Server. We did not implement this for our test.

372

IBM BladeCenter JS21: The POWER of Blade Innovation

10.3.2 Creating a CSM installation server
CSM 1.5.0 for AIX 5L and Linux: Planning and Installation Guide, SA23-1344, provides clear instructions on the various options for setting up installation servers. With multiple blades, LPARs, or both to contend with, you might want to give this serious attention. We did not set up an installation server for the few nodes that we already installed with their respective operating systems.

10.3.3 Defining cluster nodes
Although CSM 1.5.0 for AIX 5L and Linux: Planning and Installation Guide, SA23-1344, supplies both general and specific attributes for the myriad platform options available, we focus on those that apply to the BladeCenter context and the JS20 or BladeCenter JS21 nodes that we tested.

Determining node attributes
As part of the CSM cluster installation process, gather information that is used to define the cluster nodes. The pieces of information that make up a node definition are referred to as node attributes. The complete list of node attributes and descriptions are documented in the node attributes man page. This section provides information about the attributes that are required when you define nodes and how to determine which values to use. The information is divided into three categories; general attributes, hardware control information, and information about installation software.

General attributes
The general node attributes are: Hostname The resolvable host name or IP address of the node, as known by the management server. It represents the network adapter host name or IP address on the cluster virtual local area network (VLAN). Host name is always required, and you must specify it when you define the node. In a pure Linux cluster, this attribute can be an unresolved IP address. ManagementServer The host name of the CSM management server. ManagementServer is always required. Set it to the host name of the management server as it is known by the node. Because the management server can have multiple interfaces, different nodes might use different interfaces to communicate with the management server. If a route to the node exists at the time that the node is defined, CSM attempts to set the value to the IP address of the management server automatically. If a route to the node does not exist, CSM

Chapter 10. System management scenarios

373

cannot set the value automatically. Therefore, set the name explicitly to the host name of the management server in the network that installs the node. InstallServer The host name or IP address of the installation server as known by the management server, and an optional directory that serves the CSM installation files to this node. InstallAdapterHostname This is the host name or IP address of the node as known by the installation server. That is, it is the host name by which the node is connected to during the installation and that is assigned to the installation adapter when it is configured on the node during an installation. If this attribute is left blank, the Hostname attribute value is used by default. InstallServerAKBNode This is the host name or IP address of the installation server as known by the node. If there are multiple Linux installation servers on the same subnet, this attribute is ignored, and the network connection on the installation server that is used by the node is dynamically determined. If left blank, CSM determines this value dynamically during run time. If there are multiple connections from a Linux installation server to the installation subnet that the node is on, one connection is chosen at random. UserComment This is any useful information concerning this node. Not required. Mode The node’s current phase in the installation process. Set this attribute to MinManaged or run the definenode command with the minmanaged option when you define a node to be minimally managed. If you do not set the Mode attribute of the node, CSM sets it automatically during the different phases of the installation.

Hardware control attributes
You must define the hardware-related attributes for the nodes if hardware control is to be configured on the cluster. In some cases, default values are provided. If these defaults are acceptable, you do not have to provide the attribute values when you define the node. See CSM 1.5.0 for AIX 5L and Linux: Planning and Installation Guide, SA23-1344. For a complete description of the meaning and use of these attributes, see IBM CSM for AIX 5L and Linux: Administration Guide, SA23-1343.

374

IBM BladeCenter JS21: The POWER of Blade Innovation

For BladeCenter blade servers, the following attributes apply to models HS20, HS40, JS20, and JS21 blade servers: PowerMethod Set this value to blade. HWControlPoint Use the host name of the BladeCenter management module. HWControlNodeId Use the blade Advanced System Management (ASM) name. The following attributes apply only to the JS20 and JS21 blade servers, and are valid only if the SoL feature is enabled on the BladeCenter management module: ConsoleMethod Set this value to blade. ConsoleServerName Use the host name of the BladeCenter management module. ConsolePortNum Use the blade slot number within the BladeCenter chassis. ConsoleSerialDevice Leave this field blank. This field is not used for JS20 blade servers. If the SoL feature is not enabled, set the ConsoleSerialDevice attribute for all blades to NONE, and leave the ConsoleMethod, ConsoleServerName, and ConsolePortNum attributes blank.

Using the definenode command
Use the definenode command to establish and set up individual nodes, one at a time. With a large number of nodes, you might want to consider the multinode options as explained in the CSM 1.5.0 for AIX 5L and Linux: Planning and Installation Guide, SA23-1344, and in the CSM 1.5.0 for AIX 5L and Linux: Command and Technical Reference, SA23-1345. To establish one of the JS21 AIX LPARs as a node, issue the following command: definenode -n l2_aix_1 Hostname=l2_aix_1 CSMVersion=1.5.1 ConsoleMethod=blade ConsolePortNum=2 CondoleServerName=mmext HWControlNodeId=JS21_VIO HWControlPoint=mmext InstallDistributionVersion=5.3.0 InstallOSName=AIX ManagementServer=ibmdirector2 PowerMethod=blade

Chapter 10. System management scenarios

375

The output of the lsnode -Al l2_aix_1 command (the command which lists the node attributes of a defined note) is shown in Figure 10-14. # lsnode -Al l2_aix_1 Hostname = l2_aix_1 AdapterStanzaFile = AllowManageRequest = 0 (no) CSMVersion = 1.5.1 ChangedAttributes = {} ConfigChanged = 0 (no) ConsoleMethod = blade ConsolePortNum = 2 ConsoleSerialDevice = ConsoleSerialSpeed = 9600 ConsoleServerName = mmext ConsoleServerNumber = FWSvcProc = FWSysBIOS = HWControlNodeId = JS21_VIOS HWControlPoint = mmext HWModel = HWSerialNum = InstallOSName = AIX InstallStatus = PreManaged InstallTemplate = LICManagedSystemLevel = LICPowerSubsystemLevel = LParID = LastCFMUpdateTime = ManagementServer = 9.3.5.236 Mode = PreManaged NFSServer = Name = l2_aix_1 NodeNameList = {js20ibmdirector2} PhysicalLocation = PowerMethod = blade PowerStatus = 127 (unknown) Properties = Status = 127 (unknown) UUID = UpdatenodeFailed = 0 (false) UserComment =
Figure 10-14 Output of the lsnode command

376

IBM BladeCenter JS21: The POWER of Blade Innovation

As you can see from the preceding section, defining and managing more than a few nodes individually is time consuming. If many nodes are similar in construction, you can use more efficient methods such as host name mapping files and node definition files to reduce the administrative work. This is described in detail in CSM 1.5.0 for AIX 5L and Linux: Planning and Installation Guide, SA23-1344. When you have an environment with multiple blades in multiple chassis, you can use the installation server setup and appropriate InstallServer attributes to enable you to install the operating system and updates to the CSM software as necessary. In our test, we had a limited number of nodes and did not use an installation server for our exercise.

10.3.4 Adding AIX nodes to the cluster
There are multiple preparatory steps that you must consider before adding a node. These are spelled out in CSM 1.5.0 for AIX 5L and Linux: Planning and Installation Guide, SA23-1344. See Chapter 11, “Adding AIX nodes to the cluster” of this guide for information about this topic. For the purposes of this exercise, we completed all the prerequisites and focus on the updatenode command. The updatenode command adds the AIX nodes to the cluster. Run the updatenode command for the AIX nodes that you have defined. The updatenode command does the following functions for an AIX node: If remote shell authentication is not already set up, automatically sets up remote shell authentication for OpenSSH or rsh Distributes configuration files if the configuration file manager (CFM) is set up Set up CFM before you install your nodes to avoid customizing the nodes later. For information about how to configure CFM, see IBM CSM for AIX 5L and Linux: Administration Guide, SA23-1343. Runs any user customization scripts Sets up the Kerberos Version 5 options for remote commands if requested on the csmconfig command

Chapter 10. System management scenarios

377

Before you run updatenode, ensure the following prerequisites: The management server is installed and the AIX nodes are defined The AIX nodes have all the prerequisite AIX software. Ensure that the following CSM software is installed on the AIX nodes (This software is installed automatically with AIX. If for some reason it is uninstalled, you must ensure that it is reinstalled): – csm.core – csm.client If you plan to use Kerberos remote command processing, the necessary software is installed The AIX nodes are up and running It is not necessary to update the cluster nodes with the latest available CSM software updates unless the updates are specifically required. See the following CSM support Web site for information about any required updates for the nodes: http://techsupport.services.ibm.com/server/cluster/fixes We updated the CSM node files csm.core and csm.client to V1.5.1 on all nodes. For our exercise, we issued the updatenode command for each node independently. In each case, we were prompted to supply the root password for the node as shown in Figure 10-15. # updatenode l2_aix_1 updatenode: 2653-206 dsh, using protocol /usr/bin/ssh, cannot connect to nodes: l2_aix_1.itsc.austin.ibm.com. Please enter the password for the current user (normally root) to access the nodes l2_aix_1.itsc.austin.ibm.com:(Password entered) Setup complete for remote shell: /usr/bin/ssh Now running updatenode.client on the nodes. l2_aix_1.itsc.austin.ibm.com: Setting Management Server to 9.3.5.236. l2_aix_1.itsc.austin.ibm.com: Node Install - Successful. l2_aix_1.itsc.austin.ibm.com: Output log is being written to "/var/log/csm/install.log". Now running CFM to push /cfmroot files the nodes. There are no files in /cfmroot.
Figure 10-15 updatenode command applied to AIX node

378

IBM BladeCenter JS21: The POWER of Blade Innovation

To validate that the node is successfully added to the cluster, issue the lsnode command with the node name and appropriate attributes to be shown. Figure 10-16 shows the output for this command. # lsnode -n js21nim -a Mode,UpdatenodeFailed js21nim: Managed, 0
Figure 10-16 lsnode command

You can see that the Mode is now Managed and the UpdatenodeFailed is 0 (for No). The /var/log/csm/updatenode.log on the management server also provides clues to recent updatenode activity.

10.3.5 Adding Linux nodes to the cluster
For this exercise, we established a SLES9 SP3 LPAR on a JS21 blade. We defined it using the definenode command and ensured that all the attributes followed the instructions in CSM 1.5.0 for AIX 5L and Linux: Planning and Installation Guide, SA23-1344.

Chapter 10. System management scenarios

379

Figure 10-17 shows the output of lsnode -Al l4_sles9_1 command, where l4_sles9_1 is the host name for the LPAR. We edited the output so that only the essential attributes are shown. # lsnode -Al l4_sles9_1 Hostname = l4_sles9_1.itsc.austin.ibm.com CSMVersion = 1.5.1 ChangedAttributes = {AllowManageRequest} ConsoleMethod = blade ConsolePortNum = 2 ConsoleSerialDevice = ConsoleSerialSpeed = 9600 ConsoleServerName = mmext HWControlNodeId = JS21_VIOS HWControlPoint = mmext InstallCSMVersion=1.5.1 InstallDistributionName = SLES InstallDistributionVersion = 9 InstallKernelVersion = InstallMethod = autoyast InstallOSName = Linux InstallPkgArchitecture = ppc64 InstallServer = InstallServerAKBNode = InstallServiceLevel = SP3 InstallStatus = PreManaged ManagementServer = 9.3.5.236 Mode = PreManaged NFSServer = Name = l4_sles9_1.itsc.austin.ibm.com NodeNameList = {js20ibmdirector2} PowerMethod = blade PowerStatus = 127 (unknown) Properties = Status = 127 (unknown) UUID = UpdatenodeFailed = 1 (true) UserComment =
Figure 10-17 lsnode -Al output for host l4_sles9_1

380

IBM BladeCenter JS21: The POWER of Blade Innovation

Using the copycsmpkgs command
The next step is to ensure that the management server is prepared to install either the operating system or CSM or both. In this case, the OS is installed. Ensure that the CSM software and open source software identified in CSM 1.5.0 for AIX 5L and Linux: Planning and Installation Guide, SA23-1344, is loaded in the correct directories. Download the CSM code for Linux and placed it in the /csminstall/Linux directory on the management server. Keep the SLES9 Service Pack 3 (SP3) Linux CD ready for use. They contain the required open source software. Issue the copycsmpkgs -p /csminstall/Linux -n l4_sles9_1 command and follow the prompts as it completes the task. The command process prompts you for the distribution media. Tip: Though you might want to mount the distribution media in the CD-ROM tray, do not mount it. The copycsmpkgs command scripts mount it for you. There will be confusion if you mount it before issuing the command.

Using the updatenode command
When the copycsmpkgs command is run successfully, you are ready to run the updatenode command. The command runs with a default -I flag against the node selected and begins to install the necessary CSM for Linux files and open source files. If any files that are considered to be necessary are not found, the command notifies you.

Chapter 10. System management scenarios

381

Figure 10-18 shows the output of the command run on our Linux LPAR. # updatenode -n l4_sles9_1 updatenode: 2653-073 The Autoupdate RPM is missing from /csminstall/Linux/SLES/csm/1.5.1/packages/. This means that Autoupdate is probably not installed on the nodes. Please download the Autoupdate RPM from http://freshmeat.net/projects/autoupdate and place it in /csminstall/Linux/SLES/csm/1.5.1/packages/. Now running updatenode.client on the nodes. l4_sles9_1.itsc.austin.ibm.com: Installing autoupdate-5.4.1-1.noarch.rpm. l4_sles9_1.itsc.austin.ibm.com: The following OPTIONAL RPMs will not be copied or installed (because they could not be found). This may prevent the use of some CSM functionality or optional features. Please consult the CSM Planning and Installation Guide for more information: l4_sles9_1.itsc.austin.ibm.com: perl-RPM2 l4_sles9_1.itsc.austin.ibm.com: Setting Management Server to 9.3.5.236. l4_sles9_1.itsc.austin.ibm.com: Node Install - Successful. l4_sles9_1.itsc.austin.ibm.com: Output log is being written to "/var/log/csm/install.log". Now running CFM to push /cfmroot files the nodes. There are no files in /cfmroot.
Figure 10-18 Output from the updatenode command

You can see from Figure 10-18 that at least one fileset (perl_RPM2) was not found. However, the command completed successfully and the node is now shown as a managed node.

382

IBM BladeCenter JS21: The POWER of Blade Innovation

10.3.6 Testing cluster installation and basic command
Because the distributed shell is used for cluster commands, you can use it to test your cluster. Issue the dsh -a date command. Figure 10-19 shows the output. # dsh -as date l4_sles9_1.itsc.austin.ibm.com: Tue Jun 6 06:17:52 PDT 2006 js21nim.itsc.austin.ibm.com: Tue Jun 6 08:16:11 CDT 2006 l2_aix_1.itsc.austin.ibm.com: Tue Jun 6 08:11:39 CDT 2006 #
Figure 10-19 Output from the dsh command

Node connectivity test
To ensure that the nodes are reachable, issue the lsnode -p command. You see an output similar to Figure 10-20. # lsnode -p js21nim: 1 (alive) l2_aix_1: 1 (alive) l4_sles9_1: 1 (alive)
Figure 10-20 Output from the lsnode command

Your cluster is now active. You can begin normal cluster implementation tasks that are appropriate for you application environment.

Chapter 10. System management scenarios

383

384

IBM BladeCenter JS21: The POWER of Blade Innovation

11

Chapter 11.

Performance tuning
This chapter discusses some of the useful tools and commands to maximize the performance of your JS21. Some of the sections provide suggested configuration changes with expected results to aid your implementation. There are many factors that you have be address when looking at the performance of a server. These include both software and hardware configurations and how they interact with the workload.

© Copyright IBM Corp. 2006. All rights reserved.

385

11.1 Performance tuning for AIX
Performance tuning for AIX is discussed in detail in AIX 5L Version 5.3 Performance Management Guide, SC23-4905. You can find this on your AIX documentation CD (see Infocenter) or online at: http://publib.boulder.ibm.com/infocenter/pseries/v5r3/index.jsp On this Web site, on the left pane, select AIX documentation → AIX PDFs. On the right pane, under the topic Performance management and tuning, select Performance Management Guide. Because there is another IBM Redbook that is written for AIX 5L, this chapter concentrates on Linux. The AIX 5L performance tuning redbook is called: AIX 5L Practical Performance Tools and Tuning Guide, SG24-6478.

11.2 Tuning options for Linux on POWER: Selected examples
The Linux distributions that support the BladeCenter JS21 are built on the 2.6 kernel. The 2.6 kernel can take advantage of many Advanced Performance Optimization with Enhanced RISC (POWER) processor features. We look at how you can configure the Linux operating system (OS) to operate efficiently on this platform.

11.2.1 Disabling unused processes
There are daemons (background services) running on every server that are probably not required. Disabling these daemons frees memory, decreases startup time, and decreases the number of processes that the central processing unit (CPU) has to handle. A side benefit to this is increased security of the server because fewer daemons mean fewer exploitable processes.

386

IBM BladeCenter JS21: The POWER of Blade Innovation

Table 11-1 shows some of the possible daemon processes that you can disable.
Table 11-1 Daemon processes that can be disabled Daemons apmd cups hpoj isdn netfs nfslock pcmcia portmap rhnsd sendmail xfs Description Advanced power management daemon Common UNIX printing system Hewlett-Packard (HP) OfficeJet Support Integrated Services Digital Network (ISDN) modem support Used in support of exporting Network File System (NFS) shares Used for file locking with NFS Personal Computer Memory Card International Association (PCMCIA) support on a server Dynamic port assignment for Remote Procedure Call (RPC) services (such as network information services (NIS) and NFS) Red Hat Network update service for checking for updates and security errata Mail transport agent Font server for X Windows

Disabling daemons on Red Hat
In Red Hat, use the /sbin/chkconfig command to work with daemons. For example, to stop the sendmail daemon immediately, enter the following command as root: /sbin/service sendmail stop If you do not want the daemon to start the next time the machine boots, issue either one of the following commands as root because they both accomplish the same results: /sbin/chkconfig --levels 2345 sendmail off /sbin/chkconfig sendmail off Similarly, there is a graphical user interface (GUI)-based program to choose what daemons are started. To run the GUI, from the Red Hat desktop, select Main Menu → System Settings → Server Settings → Services. Alternatively, issue the following command: /usr/bin/system-config-services

Chapter 11. Performance tuning

387

Disabling daemons on SUSE Linux
In SUSE Linux, you can temporarily stop most daemons with the stop parameter. For example, to stop the sendmail daemon immediately, enter the following command as root: /etc/init.d/sendmail stop In addition, SUSE Linux Enterprise Server (SLES) has three ways to work with daemons: A text-based UI: /sbin/yast runlevel A GUI, Yet Another Setup Tool 2 (YaST2), which you can be start with the following command: /sbin/yast2 runlevel Alternatively, you can open Yast2 by clicking Browse: YaST/ → YaST modules → System → Runlevel editor. The /sbin/chkconfig command If you do not want the daemon to start the next time the machine boots, enter the following command as root: /sbin/chkconfig -s sendmail off

11.2.2 Disabling the graphical user interface
Whenever possible, do not run the GUI on a Linux server. When you do not run a GUI, you have more system resources, including memory, for the applications. Often, a GUI is not required on a Linux server, especially a JS21. You can perform all administration tasks using the command line, redirecting the X display, or through a Web browser interface. There are several useful Web-based tools (for example, webmin, Linuxconf, and Samba Web Administration Tool (SWAT)). If a GUI is required, then start and stop it as necessary rather than running it all the time. In most cases, the server must be running at run level 3, which does not start the GUI when the machine boots. If you want to restart the X Server, use startx from a command prompt. Run level 3 is multi-user mode without a GUI.

388

IBM BladeCenter JS21: The POWER of Blade Innovation

Setting the run level
To set the run level: 1. Determine the run level at which the machine is running with the following command: runlevel This prints the previous and current run level (for example, N 5 means that there was no previous run level (N) and that the current run level is 5). 2. To switch between run levels, use the init command. For example, to switch to run level 3, enter the command: init 3 The following short description provides the different run levels that are used in Linux: – 0: Halt (Do not set initdefault to this because the server immediately shuts down after boot.) – 1: Single user mode – 2: Multi-user, without NFS (the same as 3, if you do not have networking) – 3: Full multi-user mode – 4: Unused – 5: X11 – 6: Reboot (Do not set initdefault to this because the server machine continuously reboots.) 3. To set the initial run level of a machine at boot, modify the /etc/inittab file with the line as shown in Figure 11-1. id:3:initdefault:

Chapter 11. Performance tuning

389

Figure 11-1 Run level and getty edits

11.2.3 Tuning the kernel parameters
You can make changes to the kernel by modifying parameters that control the OS. Make these changes on the command line using the sysctl command. In addition, Red Hat offers a graphical method of modifying these sysctl parameters. To launch the tool, as shown in Figure 11-2, issue the following command: /usr/bin/system-config-proc

390

IBM BladeCenter JS21: The POWER of Blade Innovation

Tip: By default, the kernel includes the necessary module to enable you to make changes using sysctl without having to reboot. However, If you choose to remove this support (during the operating system installation), then you have to reboot Linux before the change takes effect.

Figure 11-2 system-config-proc

The kernel parameters that control how the kernel behaves are stored in /proc (and, in particular, /proc/sys), as shown in Table 11-2. Reading the files in the /proc directory tree provides a simple way to view configuration parameters that are related to the kernel, processes, memory, network, and other components. Each process running in the system has a directory in /proc with the process ID (PID) as name.
Table 11-2 Kernel parameters File/directory /proc/sys/vm/* /proc/stat /proc/cpuinfo Purpose Management of cache memory and buffer Kernel statistics as process, swap, and disk input/output (I/O) Information about the installed CPUs

Chapter 11. Performance tuning

391

File/directory /proc/meminfo /proc/sys/abi/*

Purpose Information about memory usage. The free command uses this information. Used to provide support for foreign binaries to Linux: Those compiled under other UNIX variants such as SCO Unixware 7, SCO OpenServer, and SUN Solaris™ 2. By default, this support is installed, although you can remove it during installation. Used to increase the number of open files that the OS allows and to handle quota For tuning purposes, you can enable hotplug, manipulate shared memory, and specify the maximum number of pid files and level of debug in syslog. Tuning of network in general, Internet Protocol Version 4 (IPV4) and IPV6

/proc/sys/fs/* /proc/sys/kernel/* /proc/sys/net/*

An example of using sysctl
The sysctl commands use the names of files in the /proc/sys directory tree as parameters. For example, to modify the shmmax kernel parameter, you can display (using cat) and change (using echo) the file /proc/sys/kernel/shmmax as shown in Example 11-1.
Example 11-1 Manually updating /proc/sys/kernel/shmmax

#cat /proc/sys/kernel/shmmax 33554432 #echo 33554430 > /proc/sys/kernel/shmmax #cat /proc/sys/kernel/shmmax 33554430 However, using these commands can easily introduce errors. Therefore, we recommend that you use the sysctl command because it checks the consistency of the data before it makes any change, as shown in Example 11-2.
Example 11-2 sysctl command

#sysctl kernel.shmmax kernel.shmmax = 33554432 #sysctl -w kernel.shmmax=33554430 kernel.shmmax = 33554430 #sysctl kernel.shmmax kernel.shmmax = 33554430

392

IBM BladeCenter JS21: The POWER of Blade Innovation

This change to the kernel stays in effect only until the next reboot. If you want to make the change permanently, then you can edit the /etc/sysctl.conf file and add the appropriate command, as shown in Example 11-3.
Example 11-3 /ec/sysctl.conf

kernel.shmmax = 33554439 The next time you reboot, the parameter file is read. You can obtain the same result without rebooting by issuing the following command: #sysctl -p Table 11-3 shows a list of sysctl parameters that you can configure.
Table 11-3 sysctl parameters Parameter net.ipv4.inet_peer_gc-maxtime Description or example of use Sets how often the garbage collector (GC) must pass over the inet peer storage memory pool during low or absent memory pressure. Default is 120, measured in jiffies:

sysctl -w.ipv4.inet_peergc_maxtime=240
net.ipv4.inet_peer-gc-mintime Sets the minimum time that the GC can pass cleaning memory. If your server is heavily loaded, you might want to increase this value. Default is 10, measured in jiffies:

sysctl -w net.ipv4.inet_peer_gc_mintime=80
net.ipv4.inet_peer_maxttl The maximum time-to-live for the inet peer entries. New entries expire after this period of time. Default is 600, measured in jiffies:

sysctl -w net.ipv4.inet_peer_maxttl=500
net.ipv4.inet_peer_minttl The minimum time-to-live for inet peer entries. Set to a high enough value to cover fragment time-to-live in the reassembling side of fragmented packets. This minimum time must be smaller than net.ipv4.inet_peer_threshold. Default is 120, measured in jiffies:

sysctl -w net.ipv4.inet_peer_minttl=80
net.ipv4.inet_peer_threshold Sets the size of inet peer sotrage. When this limit is reached, peer entries are thrown away, using the inet_peer_gc_mintime timeout. Default is 65644:

sysctl -w net.ipv4.inet_peer_threshold=65644
net.core.rmem_max Maximum receive window. Default is 131071:

sysctl -w net.core.rmem_max 16777216
net.core.wmem_max Maximum Transmission Control Protocol (TCP) send window. Default is 131071:

sysctl -w net.core.wmem_max 16777216

Chapter 11. Performance tuning

393

Parameter net.ipv4.tcp_rmem

Description or example of use Memory reserved for TCP receive buffers. Default is 4096 87380 174760:

sysctl -w net.ipv4.tcp_rmem 4096 87380 16777216
net.ipv4.tcp_wmem Memory reserved for TCP send buffers. Default is 4096 65536 174760:

sysctl -w net.ipv4.tcp_wmem 4096 65536 16777216
vm.hugetlb_pool The hugetlb feature works the same way as bigpages, but after hugetlb allocates memory, the physical memory can only be accessed by hugetlb or shm allocated with SHM_HUGETLB. It is normally used with databases such as Oracle or IBM DB2®. Default is 0:

sysctl -w vm.hugetlb_pool=4608
vm.inactive_clean_percent Designates the percent of inactive memory that must be cleaned. Default is 5%:

sysctl -w vm.inactive_clean_percent=30
vm.pagecache Designates how much memory must be used for page cache. This is important for databases such as Oracle and DB2. Default is 1 15 100. The three values of this parameter are: Minimum percent of memory used for page cache. Default is 1% The initial amount of memory for cache. Default is 15% Maximum percent of memory used for page cache. Default is 100%

sysctl -w vm.pagecache=1 50 100

11.2.4 Disabling simultaneous multithreading
If an application's performance does not meet expectations and it is suspected that the application may not be multithreading efficiently, it might be helpful to disable the simultaneous multithreading (SMT) feature on a POWER system. You can do this by modifying /etc/lilo.conf and adding the smt-enable=off parameter to the list of kernel boot parameters.

11.2.5 Tuning memory
Improvements have been made in the 2.6 kernel. You do not have bdflush to tune any more, which was a 2.4 kernel tuning option. Look at your application installation procedure to determine whether there are recommended changes to configure with the sysctl command.

394

IBM BladeCenter JS21: The POWER of Blade Innovation

11.2.6 Tuning the file system
Disk accesses are usually measured in milliseconds and are thousands of times slower than other components (such as memory or Peripheral Component Interconnect (PCI) operations, which are measured in nanoseconds or microseconds). Different file systems are available for Linux that differ in performance and scalability. Besides storing and managing data on the disks, file systems are also responsible for guaranteeing data integrity. The newer Linux distributions include journaling file systems as part of their default installation. Journaling, or logging, prevents data inconsistency in case of a system crash. All modifications to the file system metadata have been maintained in a separate journal or log and can be applied after a system crash to bring it back to its consistent state. Journaling also improves recovery time, because there is no need to perform file system checks at system reboot. As with other aspects of computing, you find that there is a trade-off between performance and integrity. However, as Linux servers make their way into corporate data centers and enterprise environments, requirements such as high availability can be addressed. A server’s disk subsystems can be a major component of overall system performance. Understanding the function of the server is key to determining whether the I/O subsystem has a direct impact on performance. Examples of servers where disk I/O is most important: A file and print server must move data quickly between users and disk subsystems. Because the purpose of a file server is to deliver files to the client, the server must initially read all data from a disk. A database server’s ultimate goal is to search and retrieve data from a repository on the disk. Even with sufficient memory, most database servers perform large amounts of disk I/O to bring data records into memory and flush modified data to disk. Examples of servers where disk I/O is not the most important subsystem: An e-mail server acts as a repository and router for electronic mail and tends to generate a heavy communication load. Networking is more important for this type of server. A Web server that is responsible for hosting Web pages (static, dynamic, or both) benefits from a well-tuned network and memory subsystem.

Chapter 11. Performance tuning

395

11.2.7 Selecting the disk technology
Table 11-4 lists the disk technologies that are commonly used in desktops and servers. The JS21 uses the Serial-Attached SCSI (SAS) interface for the optional internal drives.
Table 11-4 Disk technologies Cost Lowest cost Function Direct-attached storage; for example, low-end servers, local storage (IBM eServer x305) Direct-attached storage; for example, mid-range to high-end server with local storage (x346, x365) Limitations and capabilities An extension of Integrated Drive Electronics (IDE) that is used for connecting internal storage. Maximum: Two drives per EIDE controller Although the standard for more than 10 years, current I/O demands on high-end servers have stretched the capabilities of SCSI. Limitations include cable lengths, transfer speeds, maximum number of attached drives, and limits on the number of systems that can actively access devices on one SCSI bus, affecting clustering capabilities. Generally available since late 2002, this new standard in hard disk drive (HDD) or system board interface is the follow-on technology to EIDE. With its point-to-point protocol, scalability improves as each drive has a dedicated channel. Sequential disk access is comparable to SCSI but random access is less efficient. Redundant Array of Independent Disks (RAID) functionality is also available. Generally available since late 2003, this new standard in HDD or system board interface provides performance and also reliability.

Technology Enhanced Integrated Drive Electronics (EIDE) Small Computer System Interface (SCSI)

Low cost

Serial ATA

Low cost

Midrange data-storage applications

SerialAttached SCSI

Low cost

Direct-attached storage; for example, mid-range to high-end server with local storage (JS21)

396

IBM BladeCenter JS21: The POWER of Blade Innovation

Technology iSCSI

Cost Medium cost

Function Mid-end storage; for example, File/Web server

Limitations and capabilities Became a Request For Comment (RFC) recently. Currently being targeted toward mid-end storage and remote booting. Primary benefits are savings in infrastructure cost and diskless servers. It also provides the scalability and reliability associated with TCP/IP/Ethernet. High latency of TCP/IP limits performance. Note: Red Hat Enterprise Linux currently does not support iSCSI. Provides low latency and high throughput capabilities and removes the limitations of SCSI by providing cable distances of up to 10 km with fiber optic links; 2 Gbps transfer rate, redundant paths to storage to improve reliability. In theory, can connect up to 16 million devices; in loop topologies, up to 127 storage devices or servers can share the same Fibre Channel connection allowing implementation of large clusters.

Fibre Channel

High cost

Enterprise storage; for example, databases

The number of disk drives significantly affects performance because each drive contributes to total system throughput. Capacity requirements are often the only consideration that is used to determine the number of disk drives that are configured in a server. Throughput requirements are usually not well understood or are completely ignored. The key to a good performing disk subsystem depends on maximizing the number of read-write heads that can service I/O requests. With RAID technology, you can spread the I/O over multiple spindles.

Accessing time updates
The Linux file system keeps records of when files are created, updated, and accessed. Default operations include updating the last-time-read attribute for files during reads and writes to files. Because writing is an expensive operation, eliminating unnecessary I/O can lead to overall improved performance. Mounting file systems with the noatime option eliminates the inode access times from being updated. If file update times are not critical to your implementation, as in a Web-serving environment, a user can choose to mount file systems with the noatime flag in the /etc/fstab file. Update the /etc/fstab file with noatime option set: /dev/sdb1 /mountlocation ext3 defaults,noatime 1 2

Chapter 11. Performance tuning

397

It is generally a good idea to have a separate /var partition and mount it with the noatime option. This is applicable to both SUSE and Red Hat Linux.

Selecting the journaling mode of an ext3 file system
You can set three different journaling options in the ext3 file system with the data option in the mount command: data=journal This journaling option provides the highest form of data consistency by causing both file data and metadata to be journaled. It also has the highest performance cost of the three options. data=ordered (default) In this mode only metadata is written. However, file data is guaranteed to be written first. This is the default setting. data=writeback This journaling option provides the fastest access to the data at the expense of data consistency. The data is guaranteed to be consistent as the metadata is still being logged. However, no special handling of actual file data is done and this might lead to old data appearing in files after a system crash. There are two ways to change the journaling mode on a file system: When issuing the mount command: mount -o data=writeback /dev/sdb1 /mnt/mountpoint Including it in the options section of the /etc/fstab file: /dev/sdb1 /testfs ext3 defaults,journal=writeback 0 0 If you want to modify the default data=ordered option on the root partition, make the change to the file listed previously, then issue the mkinitrd command to scan the changes in the /etc/fstab file and create a new image. Update /etc/lilo.conf to point to the new image. For more information about ext3, refer to the following Web site: http://www.redhat.com/support/wpapers/redhat/ext3/

398

IBM BladeCenter JS21: The POWER of Blade Innovation

ReiserFS: The default SUSE Linux file system
The default file system on a SUSE installation since SUSE Linux 7.1 has been ResierFS, developed by Hans Reiser. From its initial design, key performance aspects have included the following features: Journaling that is designed into the file system from the beginning improves reliability and recovery Provides faster access through the use of balanced tree data structures that allow the storing of both content data and security metadata Efficient use of disk space because, unlike other file systems, this file system does not rely on block sizes

Tuning ReiserFS
One of the strengths of the ReiserFS is its support for a large number of small files. Instead of using the traditional block structure of other Linux file systems, ReiserFS uses a tree structure that has the capability to store the actual contents of small files or the tails of those that are larger in the access tree itself. This file system does not use fixed block sizes, therefore only the space that is required to store a file is consumed, leading to less wasted space. There is an option when mounting a ReiserFS file system that improves performance but at the expense of space. When mounting a ReiserFS, you can disable this tail packing option by specifying notail so that the file system performs a little faster but uses more disk space. An example of mounting a ReiserFS file system with the notail option is: /dev/sdb1 /testfs resierfs notail 0 0

11.2.8 Tuning in Transmission Control Protocol and User Datagram Protocol
Use the following commands for tuning servers that support a large number of multiple connections: For servers that receive many connections at the same time, re-use the TIME-WAIT sockets for new connections. This is useful in Web servers, for example: sysctl -w net.ipv4.tcp_tw_reuse=1 If you enable this command, you must also enable fast recycling of TIME-WAIT sockets status: sysctl -w net.ipv4.tcp_tw_recycle=1

Chapter 11. Performance tuning

399

With these parameters enabled, the number of connections is significantly reduced. This is good for performance because each TCP transaction maintains a cache of protocol information about each of the remote clients. In this cache, information such as round-trip time, maximum segment size, and congestion window are stored. For more details, review RFC 1644. The parameter tcp_fin_timeout is the time to hold a socket in state FIN-WAIT-2 when the socket is closed at the server. A TCP connection begins with a three-segment synchronization SYN sequence and ends with a three-segment FIN sequence, both of which do not hold data. If you change the tcp_fin_timeout value, the time from the FIN sequence to when the memory can be freed for new connections can be reduced, thereby improving performance. However, change this value only after careful monitoring, because there is a risk of overflowing memory due to the number of dead sockets. sysctl -w net.ipv4.tcp_fin_timeout=30 One of the problems found in servers with many simultaneous TCP connections is the large number of connections that are open but unused. TCP has a keepalive function that probes these connections and, by default, drops them after 7200 seconds (2 hours). This length of time might be too large for your server and can result in excess memory usage and a decrease in server performance. Setting it to 1800 seconds (30 minutes), for example, might be more appropriate: sysctl -w net.ipv4.tcp_keepalive_time=1800 Set the maximum OS send buffer size (wmem) and receive buffer size (rmem) to 8 MB for queues on all protocols: – sysctl -w net.core.wmem_max=8388608 – sysctl -w net.core.rmem_max=8388608 These specify the amount of memory that is allocated for each TCP socket when it is created. In addition, you must also use the following commands for send and receive buffers. They specify three values: minimum size, initial size, and maximum size: – sysctl -w net.ipv4.tcp_rmem="4096 87380 8388608" – sysctl -w net.ipv4.tcp_wmem="4096 87380 8388608" The third value must be the same as or less than the value of wmem_max and rmem_max.

400

IBM BladeCenter JS21: The POWER of Blade Innovation

11.2.9 Performance tuning tools
This section describes some of the performance tuning tools.

The uptime command
Use the uptime command to verify how long the server has been running and how many users are logged on. You can also use it for a quick overview of the average load of the server. The system load average is displayed for the last 1-minute, 5-minute, and 15-minute intervals. The load average is not a percentage but the number of processes in queue waiting to be processed. If processes that request CPU time are blocked (which means the CPU has no time to process them), the load average increases. Alternatively, if each process gets immediate access to CPU time and there are no CPU cycles lost, the load decreases. The optimal value of the load is 1, which means that each process has immediate access to the CPU and there are no CPU cycles lost. The typical load can vary from system to system: for a uniprocessor workstation, 1 or 2 might be acceptable, but you might probably see values of 8 to 10 on multiprocessor servers. A sample output of the uptime command: 1:57am up 4 days 17:05, 2 users, load average: 0.00, 0.00, 0.00

The dmesg command
The main purpose of dmesg is to display kernel messages. It can provide helpful information in case of hardware problems or problems with loading a module into the kernel. In addition, with dmesg, you can determine what hardware is installed in your server. During every boot, Linux checks your hardware and logs information about it. You can view these logs using the /bin/dmesg command. Example 11-4 shows a partial output from the dmesg command.
Example 11-4 Partial output from the dmesg command

EXT3 FS 2.4-0.9.19, 19 August 2002 on sd(8,1), internal journal EXT3-fs: mounted filesystem with ordered data mode. IA-32 Microcode Update Driver: v1.11 <tigran@veritas.com> ip_tables: (C) 2000-2002 Netfilter core team 3c59x: Donald Becker and others. www.scyld.com/network/vortex.html See Documentation/networking/vortex.txt 01:02.0: 3Com PCI 3c980C Python-T at 0x2080. Vers LK1.1.18-ac 00:01:02:75:99:60, IRQ 15 product code 4550 rev 00.14 date 07-23-00 Internal config register is 3800000, transceivers 0xa.

Chapter 11. Performance tuning

401

8K byte-wide RAM 5:3 Rx:Tx split, autoselect/Autonegotiate interface. MII transceiver found at address 24, status 782d. Enabling bus-master transmits and whole-frame receives. 01:02.0: scatter/gather enabled. h/w checksums enabled divert: allocating divert_blk for eth0 ip_tables: (C) 2000-2002 Netfilter core team Intel(R) PRO/100 Network Driver - version 2.3.30-k1 Copyright (c) 2003 Intel Corporation divert: allocating divert_blk for eth1 e100: selftest OK. e100: eth1: Intel(R) PRO/100 Network Connection Hardware receive checksums enabled cpu cycle saver enabled ide-floppy driver 0.99.newide hda: attached ide-cdrom driver. hda: ATAPI 48X CD-ROM drive, 120kB Cache, (U)DMA Uniform CD-ROM driver Revision: 3.12 Attached scsi generic sg4 at scsi1, channel 0, id 8, lun 0, type 3

The top command
The top command shows you actual processor activity. By default, it displays the most CPU-intensive tasks running on the server and updates the list every 5 seconds. You can sort the processes by PID (numerically), age (newest first), time (cumulative time), and resident memory usage and time (time the process has occupied the CPU since startup). Example 11-5 shows the output from the top command.
Example 11-5 Sample output from the top command

top - 02:06:59 up 4 days, 17:14, 2 users, load average: 0.00, 0.00, 0.00 Tasks: 62 total, 1 running, 61 sleeping, 0 stopped, 0 zombie Cpu(s): 0.2% us, 0.3% sy, 0.0% ni, 97.8% id, 1.7% wa, 0.0% hi, 0.0% si Mem: 515144k total, 317624k used, 197520k free, 66068k buffers Swap: 1048120k total, 12k used, 1048108k free, 179632k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 13737 root 17 0 1760 896 1540 R 0.7 0.2 0:00.05 top 238 root 5 -10 0 0 0 S 0.3 0.0 0:01.56 reiserfs/0 1 root 16 0 588 240 444 S 0.0 0.0 0:05.70 init 2 root RT 0 0 0 0 S 0.0 0.0 0:00.00 migration/0 3 root 34 19 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/0 4 root RT 0 0 0 0 S 0.0 0.0 0:00.00 migration/1 5 root 34 19 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/1 6 root 5 -10 0 0 0 S 0.0 0.0 0:00.02 events/0

402

IBM BladeCenter JS21: The POWER of Blade Innovation

7 root 5 -10 0 0 0 S 0.0 0.0 0:00.00 events/1 8 root 5 -10 0 0 0 S 0.0 0.0 0:00.09 kblockd/0 9 root 5 -10 0 0 0 S 0.0 0.0 0:00.01 kblockd/1 10 root 15 0 0 0 0 S 0.0 0.0 0:00.00 kirqd 13 root 5 -10 0 0 0 S 0.0 0.0 0:00.02 khelper/0 14 root 16 0 0 0 0 S 0.0 0.0 0:00.45 pdflush 16 root 15 0 0 0 0 S 0.0 0.0 0:00.61 kswapd0 17 root 13 -10 0 0 0 S 0.0 0.0 0:00.00 aio/0 18 root 13 -10 0 0 0 S 0.0 0.0 0:00.00 aio/1 You can further modify the processes using renice to give a new priority to each process. If a process stops or occupies too much CPU, you can end the process (kill command). The columns in the output are as follows: PID: Process identification USER: Name of the user who owns (and perhaps started) the process PRI: Priority of the process NI: Niceness level (that is, whether the process tries to be nice by adjusting the priority by the number given; see the following section for details) SIZE: Amount of memory (code + data + stack), in KB, that is used by the process RSS: Amount of physical RAM used, in KB SHARE: Amount of memory shared with other processes, in KB STAT: State of the process: S=sleeping, R=running, T=traced, D=interruptible sleep Z=zombie %CPU: Share of the CPU usage (since the last screen update) %MEM: Share of physical memory TIME: Total CPU time used by the process (since it started) COMMAND: Command line used to start the task (including parameters)

Process priority and nice levels
Process priority is a number that determines the order in which the process is handled by the CPU. The kernel adjusts this number to higher and lower as necessary. The nice value is a limit on the priority. The priority number is not allowed to go below the nice value (a lower nice value is a more favored priority). It is not possible to change the priority of a process. This is only indirectly possible through the use of the nice level of the process. Note that it may not always be possible to change the priority of a process using the nice level. If a

Chapter 11. Performance tuning

403

process is running too slowly, you can assign more CPU to it by giving it a lower nice level. This means that all other programs have fewer processor cycles and run more slowly. Linux supports nice levels from 19 (lowest priority) to -20 (highest priority). The default value is 0. To change the nice level of a program to a negative number (which makes it a high-priority process), it is necessary to log on or su to root. To start the program xyz with a nice level of -5, issue the command: nice -n -5 xyz To change the nice level of a program already running, issue the command: renice level pid If you to change the priority of the xyz program that has a PID of 2500 to a nice level of 10, issue the following command: renice 10 2500

The iostat command
The iostat command is part of the Sysstat set of utilities, available at the following Web site: http://perso.wanadoo.fr/sebastien.godard/ The iostat command lets you see average CPU times since the system was started, in a way that is similar to uptime. In addition, iostat creates a report about the activities of the disk subsystem of the server. The report has two parts: CPU utilization and device (disk) utilization. Example 11-6 shows a sample output of the iostat command.
Example 11-6 Sample output from the iostat command

Linux 2.4.21-9.0.3.EL (x232) 05/11/2004 avg-cpu: %user %nice %sys %idle 0.03 0.00 0.02 99.95 Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn dev2-0 0.00 0.00 0.04 203 2880 dev8-0 0.45 2.18 2.21 166464 168268 dev8-1 0.00 0.00 0.00 16 0 dev8-2 0.00 0.00 0.00 8 0 dev8-3 0.00 0.00 0.00 344 0

404

IBM BladeCenter JS21: The POWER of Blade Innovation

The CPU utilization report has four sections: %user: Shows the percentage of CPU utilization that is taken up while running at the user level (applications) %nice: Shows the percentage of CPU utilization that is taken up while running at the user level with a nice priority (For details about priority and nice levels, see “Process priority and nice levels” on page 403.) %sys: Shows the percentage of CPU utilization that is taken up while running at the system level (kernel) %idle: Shows the percentage of time the CPU is idle The device utilization report is split into the following sections: Device: This the name of the block device. tps: The number of transfers per second (I/O requests per second) to the device. Multiple single I/O requests can be combined in a transfer request, because a transfer request can have different sizes. Blk_read/s, Blk_wrtn/s: Blocks read and written per second indicate data that is read and written from and to the device in seconds. Blocks might also have different sizes. Typical sizes are 1024 bytes, 2048 bytes, or 4048 bytes, depending on the partition size. For example, you can find the block size of /dev/sda1: dumpe2fs -h /dev/sda1 |grep -F "Block size" This gives an output similar to: dumpe2fs 1.34 (25-Jul-2003) Block size: 1024 Blk_read, Blk_wrtn: This indicates the total number of blocks that are read and written since the boot.

The vmstat command
The vmstat command provides information about processes, memory, paging, block I/O, traps and CPU activity. Example 11-7 shows the output from the vmstat command.
Example 11-7 Sample output from the vmstat command

procs -----------memory------- ---swap-- ---io--- ---system-- ----cpu-----r b swpd free buff cache si so bi bo in cs us sy id wa 2 0 0 154804 77328 910900 0 0 4 6 103 19 0 0 100 0

Chapter 11. Performance tuning

405

The columns in the output are as follows: Process – r: The number of processes waiting for run time – b: The number of processes in uninterruptable sleep Memory – – – – swpd: The amount of virtual memory used (KB) free: The amount of idle memory (KB) buff: The amount of memory used as buffers (KB) cache: The amount of memory used as cache (KB)

Swap – si: Amount of memory swapped from the disk (KBps) – so: Amount of memory swapped to the disk (KBps) IO – bi: Blocks sent to a block device (blocks per second) – bo: Blocks received from a block device (blocks per second) System – in: The number of interrupts per second, including the clock – cs: The number of context switches per second CPU (percentages of total CPU time) – – – – us: Time spent running non-kernel code (user time, including nice time) sy: Time spent running kernel code (system time) id: Time spent idle. Before Linux 2.5.41, this included I/O wait time wa: Time spent waiting for I/O. Before Linux 2.5.41, this appeared as zero

The free command
The /bin/free command displays information about the total amounts of free and used memory (including swap) on the system. It also includes information about the buffers and cache used by the kernel. Example 11-8 shows the output from the free command.
Example 11-8 Sample output from the free command

total used free shared buffers cached Mem: 1291980 998940 293040 0 89356 772016 -/+ buffers/cache: 137568 1154412 Swap: 2040244 0 2040244

406

IBM BladeCenter JS21: The POWER of Blade Innovation

The pmap command
The pmap command reports the amount of memory that one or more processes are using. You can use this tool to determine which processes on the server are being allocated memory and whether this amount of memory is a cause of memory bottlenecks: pmap -x <pid> Tip: Using pipes, it is possible to produce the output in one command. For example, to generate a report in Hypertext Markup Language (HTML), run: cat output_traffic-collector | traffic-sort -Hp | traffic-tohtml -o output_traffic-tohtml.html To generate a report as a GIF file, run: cat output_traffic-collector | traffic-sort -Hp | traffic-togif -o output_traffic-togif.gif -x 600 -y 600 Example 11-9 shows the total amount of memory that the cupsd process is using.
Example 11-9 Total amount of memory used by the cupsd process

linux:~ # pmap -x 1796 1796: /usr/sbin/cupsd Address Kbytes RSS Anon Locked Mode Mapping 08048000 244 - - - r-x-- cupsd ffffe000 4 - - - ----- [ anon ] -------- ------- ------- ------- ------total kB 6364 - - For the complete syntax of the pmap command, issue: pmap -?

The strace command
The strace command intercepts and records the system calls that are called by a process, and the signals that are received by a process. This is a useful diagnostic, instructional, and debugging tool. System administrators might find it valuable for solving problems with programs. To use the command, specify the process ID to be monitored: strace -p <pid>

Chapter 11. Performance tuning

407

Example 11-10 shows the output of the strace monitoring httpd process.
Example 11-10 Output from the strace monitoring httpd process

[root@x232 html]# strace -p 815 Process 815 attached - interrupt to quit semop(360449, 0xb73146b8, 1) = 0 poll([{fd=4, events=POLLIN}, {fd=3, events=POLLIN, revents=POLLIN}], 2, -1) = 1 accept(3, {sa_family=AF_INET, sin_port=htons(52534), sin_addr=inet_addr("9.42.171.197")}, [16]) = 13 semop(360449, 0xb73146be, 1) = 0 getsockname(13, {sa_family=AF_INET, sin_port=htons(80), sin_addr=inet_addr ("9.42.171.198")}, [16]) = 0 fcntl64(13, F_GETFL) = 0x2 (flags O_RDWR) fcntl64(13, F_SETFL, O_RDWR|O_NONBLOCK) = 0 read(13, 0x8259bc8, 8000) = -1 EAGAIN (Resource temporarily unavailable) poll([{fd=13, events=POLLIN, revents=POLLIN}], 1, 300000) = 1 read(13, "GET /index.html HTTP/1.0\r\nUser-A"..., 8000) = 91 gettimeofday({1084564126, 750439}, NULL) = 0 stat64("/var/www/html/index.html", {st_mode=S_IFREG|0644, st_size=152, ...}) = 0 open("/var/www/html/index.html", O_RDONLY) = 14 mmap2(NULL, 152, PROT_READ, MAP_SHARED, 14, 0) = 0xb7052000 writev(13, [{"HTTP/1.1 200 OK\r\nDate: Fri, 14 M"..., 264}, {"<html>\n<title>\n RedPaper Per"..., 152}], 2) = 416 munmap(0xb7052000, 152) = 0 socket(PF_UNIX, SOCK_STREAM, 0) = 15 connect(15, {sa_family=AF_UNIX, path="/var/run/.nscd_socket"}, 110) = -1 ENOENT (No such file or directory) close(15) = 0

11.2.10 Tuning for applications
Many applications specify prerequisite settings for various CPU, memory, and I/O parameters to ensure the best performance. If you have followed these instructions, or an application does not provide instructions on these settings and you consider that your application performance is less than anticipated, check some of the tools and commands that are explained in this chapter. For issues relating to memory and file parameters, and also Web servers with large numbers of network connections, look at the sysctl command. Database applications rely heavily on efficient I/O read and write activities. If this is an area of concern, see 11.2.6, “Tuning the file system” on page 395.

408

IBM BladeCenter JS21: The POWER of Blade Innovation

12

Chapter 12.

Storage area network installation and booting
Installing the operating system (OS) to a storage area network (SAN) drive or booting from a SAN drive in combination with a BladeCenter JS21 has some specialities. This chapter discusses these topics.

© Copyright IBM Corp. 2006. All rights reserved.

409

12.1 Storage area network setup
We assume that the user has basic knowledge about SAN setup and configuration. Several IBM Redbooks are available for more general information about SAN, for example, IBM TotalStorage: SAN Product, Design, and Optimization Guide, SG24-6384. A minimal two-path SAN installation is shown in Figure 12-1. We used the IBM two-port Fibre Channel (FC) switch module in this book.
BladeCenter Chassis BladeCente JS21 FC expansion card (HBA)

FC Switch Module Bay 1

FC Switch Module Bay 2

Controller 1

Controller 2

Path 1 Path 2

Storage Unit Hard drives

Figure 12-1 Basic SAN layout with a two-path configuration for a BladeCenter JS21

Important: The JS21 supports only the Qlogic 4Gb Fibre Channel Small Form Factor expansion card. We recommend that you use at least firmware Version 4.00.22, which includes IBM driver Version 1.14. Additionally, we recommend that you update the firmware of all SAN components to the latest available or possible firmware level.

410

IBM BladeCenter JS21: The POWER of Blade Innovation

To benefit from a two-path configuration, the most important setting is a multipath or failover configuration. The functionality of such a configuration depends on the hardware, firmware, the drivers, and the operating system. Because this is a very complex task, it is beyond the scope if this book to describe a complete solution.

12.1.1 Assigning a disk
Follow the description to assign a disk from a storage unit to a BladeCenter JS21 QLogic 4Gb Fibre Channel Small Form Factor expansion card (host bus adapter (HBA)): 1. Install the FC expansion card in a BladeCenter JS21 and connect the devices as shown in Figure 12-1 on page 410. 2. Update the firmware. At the time of the creation of this book, it is only possible to use AIX command diag or the stand-alone diagnostics CD (see 6.6.9, “Updating the firmware using the stand-alone diagnostics CD” on page 111) to update the firmware. 3. Configure a SAN name server. This function is included in any SAN switch. If more than one switch is used in a fabric1, the entries in each name server are auto-synchronized between all switches in this fabric. Hence you can use each switch to configure the name server. a. To do out-of-band management for Fibre Channel switch modules, use the BladeCenter management module for a basic Ethernet setup, as described in 6.4, “I/O module configuration” on page 85. Important: Make sure that the external ports are enabled, because this affects the Ethernet and Fibre Channel ports. b. To configure McDATA or IBM or QLogic Fibre Channel switch modules, download and install the BladeCenter SAN Utility or SANsurfer Switch Management Software, which is available for different architectures and operating systems.

1

Physical connected SAN

Chapter 12. Storage area network installation and booting

411

The management software shows a screen similar to Figure 12-2.

Figure 12-2 SAN switch module configuration panel using the BladeCenter SAN utility

i. To download the software, go to: http://support.qlogic.com ii. Select Drivers, Software and Manuals. iii. In the OEM MODELS section, click IBM. iv. In the IBM eServer BladeCenter Fibre Channel Switches section, choose IBM / QLogic code or McData code. Note: To configure Brocade Fibre Channel switch modules, use the Web or command-line interface (CLI).

412

IBM BladeCenter JS21: The POWER of Blade Innovation

c. After you connect to one Fibre Channel switch module using the management software, you can use the name server fabric overview of a SAN switch to see all the connected SAN devices in fabric, see Figure 12-3. This example does not correspond with the fabric layout shown in Figure 12-1 on page 410.

Figure 12-3 Name server and fabric overview

Note: In fabrics with multiple switches, each SAN switch requires an explicit identifier, called domain ID. The BladeCenter JS21 HBA and especially the worldwide name (WWN) does not show up in the name server of the SAN Switch without activation. To activate the card, scan for all boot devices using the SMS menu (see 6.8, “System Management Services interface” on page 133). After this procedure, the HBA is visible in the name server list. The scan activates the card, but without any bootable partitions, it is not possible to assign the Fibre Channel expansion card as a boot device. The next step is the configuration of a zone.

Chapter 12. Storage area network installation and booting

413

Note: To activate the BladeCenter JS21 HBA, it is also possible to start an operating system from a local disk and load the HBA driver in the modules named qla2400 and qla2xxx. If the HBA is installed, this occurs automatically.

Important: For a SAN high-availability solution, it is necessary to have two fabrics with two different name servers. Relating to the example in Figure 12-1 on page 410, this means that the configuration of path 1 and path 2 are done completely separated. For the configuration of path 1, use the first switch, and configure path 2 with the second switch. It is not allowed to have identical WWNs in one of the two name servers. If this happens, the configuration is wrong. d. The zone configuration is also done on a SAN switch of the fabric. One zone is like a logical network inside the SAN. All components in one zone are visible to each other. Typically there is a switch for global visibility for all devices and ports which are not in a zone. To create a functional zone for the first path of the example setup shown in Figure 12-1 on page 410, use the Zone configuration panel to create: i. ZoneSet, for example, zone_1 ii. Zone, for example, zone_1 iii. An alias2 for HBA Port 1 (for example, JS21_Bay3_Port1) and Storage Unit Controller 1 (for example, Sage1_Controller 1) e. Assign the alias to the zone.

2

An alias is a selectable name and might be used instead of WWNs

414

IBM BladeCenter JS21: The POWER of Blade Innovation

Figure 12-4 shows the result.

Figure 12-4 Zoning for path 1

f. Now activate the zone, which is usually done in the zone menu. Afterwards the zone configuration must also emerge in the name server. After activation, the zone configuration is distributed to all SAN switches in a fabric. Even if the HBA is deactivated and not listed in the name server anymore, the zone setting itself is persistent and works after a reactivation of the HBA as required. For a two-path configuration, perform the same steps on the other switch. Attention: Zone activation is especially critical with running systems. A misconfigured zone might lead to a locked up server.

Chapter 12. Storage area network installation and booting

415

4. The configuration of the storage unit is only drafted in the following section, because the configuration can be very complex and depends on the storage system: a. Create a physical array of hard disk. In general, this must be a Redundant Array of Independent Disks 10 (RAID 10) configuration for a fast and save configuration or a RAID 5 for large amounts of data in combination with redundancy. b. Map one or more arrays and host ports to a host group or host. i. To add a host port to a host or host group, the WWN is required. In most cases, the WWN is determined automatically by the storage unit. But this process does not always work with a BladeCenter JS21, therefore, you must type in the WWN manually. You can find the WWN of a BladeCenter JS21 HBA port in the name server. To enable a two-path configuration, it is necessary to add both BladeCenter JS21 HBA ports. ii. To add an array to a host group or host, specify a specific logical unit number (LUN) and an operating system type. After this step, at least one host port and one array is assigned to a single host group or host. The SAN configuration is now complete. Important for storage unit OS configuration: If more than one type of operating system is used with a storage unit, make sure that the storage unit supports multiple operating systems at the same time.

Restriction: SAN boot with a BladeCenter JS21 might be an issue if an array is configured as LUN0. To be safe, we recommend that you use LUN1 or later to avoid problems. After the SAN configuration is complete, you can install the operating system. The necessary single-path driver for the 4 GB FC expansion card is provided by AIX, Virtual I/O Server (VIOS), SUSE Linux Enterprise Server (SLES), and Red Hat Enterprise Linux by default. Almost all configurations presented in this book were also tested in a single-path SAN environment.

416

IBM BladeCenter JS21: The POWER of Blade Innovation

12.1.2 SAN boot characteristics
The following list provides some of the characteristics of a SAN boot: The boot device name hard drive 1 and hard drive 2 define the internal hard drive of a BladeCenter JS21. The boot device name hard disk 3 points to port 1 of the HBA and hard drive 4 points to port 2 of the HBA. Even with a two-path configuration, there is no redundancy during the boot process. Therefore, the corresponding multipath driver is loaded during the boot phase of the operating system. If more than one path is configured, but the special multipath driver is not running, in most cases, the same disk is discovered as a second disk. It is not advisable to use more than one path if the multipath driver is not used. AIX always uses an explicit hard disk identifier. Linux uses as default device names such as sda or sdb. These device names are not bound to a specific hard disk. To enable an explicit identification, use Universal Unique Identifiers (UUIDs) for Linux. Specify this during the partitioning of a hard drive. For all operating systems, the boot process can be slow. It might happen that the system is running fine, even if no output through Serial over LAN (SoL) is provided for a long period. During the HBA discovery period before the operating system starts, the output shown in Example 12-1 can be seen using SoL with a two-path and two-LUN configuration. The path and LUNs are highlighted in bold.
Example 12-1 SMS device scan with a two-path and two-LUN configuration

scan /pci@8000000f8000000/pci@1/fibre-channel@2/diskQLogic QMC2462SHost Adapter Driver(IBM): 1.17 03/31/06 Firmware version 4.00.22 check /pci@8000000f8000000/pci@1/fibre-channel@2/disk@200400a0b80ba0ed,0001000000000000 QLogic QMC2462SHost Adapter Driver(IBM): 1.17 03/31/06 Firmware version 4.00.22 QLogic QMC2462SHost Adapter Driver(IBM): 1.17 03/31/06 Firmware version 4.00.22 QLogic QMC2462SHost Adapter Driver(IBM): 1.17 03/31/06 Firmware version 4.00.22 check /pci@8000000f8000000/pci@1/fibre-channel@2/disk@200400a0b80ba0ed,0002000000000000 QLogic QMC2462SHost Adapter Driver(IBM): 1.17 03/31/06 Firmware version 4.00.22 QLogic QMC2462SHost Adapter Driver(IBM): 1.17 03/31/06 Firmware version 4.00.22 QLogic QMC2462SHost Adapter Driver(IBM): 1.17 03/31/06

Chapter 12. Storage area network installation and booting

417

Firmware version 4.00.22 scan /pci@8000000f8000000/pci@1/fibre-channel@2,1/diskQLogic QMC2462SHost Adapter Driver(IBM): 1.17 03/31/06 Firmware version 4.00.22 check /pci@8000000f8000000/pci@1/fibre-channel@2,1/disk@200400a0b80ba0ed,0001000000000000QL ogic QMC2462SHost Adapter Driver(IBM): 1.17 03/31/06 Firmware version 4.00.22 QLogic QMC2462SHost Adapter Driver(IBM): 1.17 03/31/06 Firmware version 4.00.22 QLogic QMC2462SHost Adapter Driver(IBM): 1.17 03/31/06 Firmware version 4.00.22 check /pci@8000000f8000000/pci@1/fibre-channel@2,1/disk@200400a0b80ba0ed,0002000000000000QL ogic QMC2462SHost Adapter Driver(IBM): 1.17 03/31/06 Firmware version 4.00.22 QLogic QMC2462SHost Adapter Driver(IBM): 1.17 03/31/06 Firmware version 4.00.22 QLogic QMC2462SHost Adapter Driver(IBM): 1.17 03/31/06 Firmware version 4.00.22 Keep these specialties in mind and start the installation process using network as usual. Double-check if the host type, which means the operating system, is specified correctly for the LUN that is configured in the storage unit. The installation procedure for AIX and SLES are similar. After the installation, the boot sequence must automatically switch to hard disk 2. For Red Hat, it is important to install the system only with one activated path. After the installation, select hard disk 2 as boot device and reboot the system. After this step, it is not a problem to activate the second path.

418

IBM BladeCenter JS21: The POWER of Blade Innovation

A

Appendix A.

SUSE Linux Enterprise Server 9 AutoYaST XML file
The following settings are used in the Extensible Markup Language (XML) file shown in Example A-1: Install minimum amount of software packages Enable the firewall, but allow Secure Shell (SSH) login after installation Create a root user and define a password (encrypted) Configure two Ethernet devices eth0 and eth1 using Dynamic Host Configuration Protocol (DHCP) Enable host name and Domain Name System (DNS) server assignment using DHCP PowerPC Bootloader (PREP) on partition sda1 Add the max_loop=255 parameter to the boot configuration Use the <use>all</use> switch for the partitioning of sda. It is not a problem to use another hard drive, for example, sdc, but all other partitioning configurations will most likely result in a failed installation. The created partition table is shown in Example A-2 on page 425.

© Copyright IBM Corp. 2006. All rights reserved.

419

Example: A-1 Basic tested AutoYaST XML file

<?xml version="1.0"?> <!DOCTYPE profile SYSTEM "/usr/share/autoinstall/dtd/profile.dtd"> <profile xmlns="http://www.suse.com/1.0/yast2ns" xmlns:config="http://www.suse.com/1.0/configns"> <configure> <firewall> <fw_allow_fw_traceroute>no</fw_allow_fw_traceroute> <fw_autoprotect_services>no</fw_autoprotect_services> <fw_dev_ext>eth1</fw_dev_ext> <fw_dev_int>eth0</fw_dev_int> <fw_ipsec_trust>no</fw_ipsec_trust> <fw_log_accept_all>no</fw_log_accept_all> <fw_log_accept_crit>yes</fw_log_accept_crit> <fw_log_drop_all>no</fw_log_drop_all> <fw_log_drop_crit>yes</fw_log_drop_crit> <fw_masq_nets></fw_masq_nets> <fw_masquerade>no</fw_masquerade> <fw_protect_from_internal>no</fw_protect_from_internal> <fw_route>no</fw_route> <fw_services_ext_tcp>ssh</fw_services_ext_tcp> <fw_services_ext_udp></fw_services_ext_udp> <fw_services_int_ip></fw_services_int_ip> <start_firewall config:type="boolean">true</start_firewall> </firewall> <networking> <dhcp_options> <dhclient_additional_options></dhclient_additional_options> <dhclient_client_id></dhclient_client_id> <dhclient_hostname_option>AUTO</dhclient_hostname_option> </dhcp_options> <dns> <dhcp_hostname config:type="boolean">true</dhcp_hostname> <dhcp_resolv config:type="boolean">true</dhcp_resolv> <domain>site</domain> <hostname>linux</hostname> <nameservers config:type="list"> <nameserver>192.168.1.254</nameserver> </nameservers> <searchlist config:type="list"> <search>js21testnet</search> </searchlist> </dns> <interfaces config:type="list">

420

IBM BladeCenter JS21: The POWER of Blade Innovation

<interface> <bootproto>dhcp</bootproto> <device>eth0</device> <startmode>onboot</startmode> </interface> <interface> <bootproto>dhcp</bootproto> <device>eth1</device> <startmode>onboot</startmode> </interface> </interfaces> <modules config:type="list"> <module_entry> <device>static-0</device> <module>bcm5700</module> <options></options> </module_entry> <module_entry> <device>static-1</device> <module>bcm5700</module> <options></options> </module_entry> </modules> <routing> <ip_forward config:type="boolean">false</ip_forward> </routing> </networking> <security> <console_shutdown>reboot</console_shutdown> <cracklib_dict_path>/usr/lib/cracklib_dict</cracklib_dict_path> <cwd_in_root_path>yes</cwd_in_root_path> <cwd_in_user_path>yes</cwd_in_user_path> <displaymanager_remote_access>no</displaymanager_remote_access> <enable_sysrq>no</enable_sysrq> <fail_delay>3</fail_delay> <faillog_enab>yes</faillog_enab> <gid_max>60000</gid_max> <gid_min>1000</gid_min> <kdm_shutdown>all</kdm_shutdown> <lastlog_enab>yes</lastlog_enab> <obscure_checks_enab>yes</obscure_checks_enab> <pass_max_days>99999</pass_max_days> <pass_max_len>16</pass_max_len> <pass_min_days>0</pass_min_days> <pass_min_len>7</pass_min_len>

Appendix A. SUSE Linux Enterprise Server 9 AutoYaST XML file

421

<pass_warn_age>7</pass_warn_age> <passwd_encryption>des</passwd_encryption> <passwd_use_cracklib>yes</passwd_use_cracklib> <permission_security>secure</permission_security> <run_updatedb_as>nobody</run_updatedb_as> <system_gid_max>499</system_gid_max> <system_gid_min>100</system_gid_min> <system_uid_max>499</system_uid_max> <system_uid_min>100</system_uid_min> <uid_max>60000</uid_max> <uid_min>500</uid_min> <useradd_cmd>/usr/sbin/useradd.local</useradd_cmd> <userdel_postcmd>/usr/sbin/userdel-post.local</userdel_postcmd> <userdel_precmd>/usr/sbin/userdel-pre.local</userdel_precmd> </security> <users config:type="list"> <user> <encrypted config:type="boolean">true</encrypted> <user_password>0jZJQH0JKZajo</user_password> <username>root</username> </user> </users> </configure> <install> <bootloader> <activate config:type="boolean">true</activate> <board_type>chrp</board_type> <default>linux</default> <global config:type="list"> <global_entry> <key>timeout</key> <value config:type="integer">100</value> </global_entry> <global_entry> <key>boot</key> <value>/dev/sda1</value> </global_entry> <global_entry> <key>activate</key> <value config:type="boolean">true</value> </global_entry> <global_entry> <key>default</key> <value>linux</value> </global_entry>

422

IBM BladeCenter JS21: The POWER of Blade Innovation

</global> <initrd_modules config:type="list"> <initrd_module> <module>ipr</module> </initrd_module> <initrd_module> <module>qla2400</module> </initrd_module> <initrd_module> <module>ipr</module> </initrd_module> </initrd_modules> <loader_device>/dev/sda1</loader_device> <loader_type>ppc</loader_type> <location>boot</location> <of_defaultdevice config:type="boolean">true</of_defaultdevice> <prep_boot_partition>/dev/sda1</prep_boot_partition> <sections config:type="list"> <section config:type="list"> <section_entry> <key>image</key> <value>/boot/vmlinux</value> </section_entry> <section_entry> <key>label</key> <value>linux</value> </section_entry> <section_entry> <key>root</key> <value>/dev/sda3</value> </section_entry> <section_entry> <key>initrd</key> <value>/boot/initrd</value> </section_entry> <section_entry> <key>append</key> <value>selinux=0 elevator=cfq max_loop=255</value> </section_entry> </section> </sections> </bootloader> <general> <clock> <hwclock>UTC</hwclock>

Appendix A. SUSE Linux Enterprise Server 9 AutoYaST XML file

423

<timezone>US/Pacific</timezone> </clock> <keyboard> <keymap>english-us</keymap> </keyboard> <language>en_US</language> <mode> <confirm config:type="boolean">false</confirm> </mode> <mouse> <id>none</id> </mouse> </general> <partitioning config:type="list"> <drive> <device>/dev/sda</device> <use>all</use> </drive> </partitioning> <report> <errors> <log config:type="boolean">true</log> <show config:type="boolean">true</show> <timeout config:type="integer">0</timeout> </errors> <messages> <log config:type="boolean">true</log> <show config:type="boolean">true</show> <timeout config:type="integer">0</timeout> </messages> <warnings> <log config:type="boolean">true</log> <show config:type="boolean">true</show> <timeout config:type="integer">0</timeout> </warnings> <yesno_messages> <log config:type="boolean">true</log> <show config:type="boolean">true</show> <timeout config:type="integer">0</timeout> </yesno_messages> </report> <software> <addons config:type="list"> <addon>Base-System</addon> <addon>YaST2</addon>

424

IBM BladeCenter JS21: The POWER of Blade Innovation

</addons> <base>Minimal</base> <packages config:type="list"> <package>dhcp</package> <package>dhcp-server</package> <package>expect</package> <package>inetd</package> <package>iprutils</package> <package>libidl-64bit</package> <package>ppc64-utils</package> <package>rdist</package> <package>rsh-server</package> <package>telnet-server</package> <package>librtas</package> </packages> </software> </install> </profile> Example A-2 shows the created partition table.
Example: A-2 Default partition table created by the AutoYaST configuration file

Disk /dev/sda: 146.7 GB, 146772852736 bytes 255 heads, 63 sectors/track, 17844 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks /dev/sda1 * 1 1 172+ Partition 1 does not end on cylinder boundary. /dev/sda2 4 134 1052257+ /dev/sda3 135 17843 142247542+ Id 41 82 83 System PPC PReP Boot Linux swap Linux

Appendix A. SUSE Linux Enterprise Server 9 AutoYaST XML file

425

426

IBM BladeCenter JS21: The POWER of Blade Innovation

B

Appendix B.

Red Hat Enterprise Linux 4 Kickstart file
The following settings are used in the Kickstart configuration file shown in Example B-1: Install basic amount of software packages Enable the firewall, but allow Secure Shell (SSH) login after installation Create a root user and define a password (encrypted) Configure two Ethernet devices eth0 and eth1 using Dynamic Host Configuration Protocol (DHCP) Use the autopart switch for the partitioning. All other configurations will most likely result in a failed installation. The created partition table is shown in Example B-2. The created logical volume group configuration is shown in Example B-3.
Example: B-1 Basic tested Kickstart configuration file

#Generated by Kickstart Configurator #platform=IBM pSeries #System language lang en_US #Language modules to install

© Copyright IBM Corp. 2006. All rights reserved.

427

langsupport en_US #System keyboard keyboard us #System mouse mouse #Sytem timezone timezone America/New_York #Root password rootpw --iscrypted $1$/7IU0tSr$QmQtfRtKWulXPC8cmk2kf. #Reboot after installation reboot #Use text mode install text #Install OS instead of upgrade install #Use Web installation url --url ftp://192.168.1.254/rhel4 #System bootloader configuration bootloader --location=mbr #Clear the Master Boot Record zerombr yes #Partition clearing information clearpart --all --initlabel #System authorization infomation auth --useshadow --enablemd5 #Disk partitioning information autopart #Network information network --bootproto=dhcp --device=eth0 network --bootproto=dhcp --device=eth1 #Firewall configuration firewall --enabled --trust=eth1 --ssh #Do not configure XWindows skipx #Package install information %packages --resolvedeps @ server-cfg @ ftp-server @ development-tools @ compat-arch-development @ admin-tools @ system-tools @ compat-arch-support

428

IBM BladeCenter JS21: The POWER of Blade Innovation

Example B-2 shows the created partition table.
Example: B-2 Default partition table created by the Kickstart configuration file

Disk /dev/sda: 146.7 GB, 146772852736 bytes 255 heads, 63 sectors/track, 17844 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot /dev/sda1 * /dev/sda2 /dev/sda3 Start 1 2 15 End 1 14 17844 Blocks 8001 104422+ 143219475 Id 41 83 8e System PPC PReP Boot Linux Linux LVM

Example B-3 shows the created logical volume group configuration.
Example: B-3 Default logical volumes created by the Kickstart configuration file

--- Logical volume --LV Name VG Name LV UUID LV Write Access LV Status # open LV Size Current LE Segments Allocation Read ahead sectors Block device

/dev/VolGroup00/LogVol00 VolGroup00 1Vzc3F-6ubR-bCrX-Iv4i-hAtZ-lXMl-AHMLwe read/write available 1 134.59 GB 4307 1 inherit 0 253:0

--- Logical volume --LV Name /dev/VolGroup00/LogVol01 VG Name VolGroup00 LV UUID AkDpUP-q6Wb-MdKl-xDLx-QYXn-SKRd-fhlF7J LV Write Access read/write LV Status available # open 1 LV Size 1.94 GB Current LE 62 Segments 1 Allocation inherit Read ahead sectors 0 Block device 253:1

Appendix B. Red Hat Enterprise Linux 4 Kickstart file

429

430

IBM BladeCenter JS21: The POWER of Blade Innovation

C

Appendix C.

SUSE Linux Installation Server setup
Use Yet Another Setup Tool (YaST) to configure the SUSE Linux Installation Server as described in “Preparing SLES as an installation server using YaST” on page 294. However, if you have a system where you cannot use YaST to perform this function, then you can use the following instructions. The following steps describe the manual approach which the YaST installation server applet does automatically. This might be helpful for troubleshooting and for other operating systems or distributions that do not provide this applet. We assume that a File Transfer Protocol (FTP) service is already configured. See 9.2.4, “Configuring a File Transfer Protocol service” on page 288. The next step is to build an installation directory. 1. Create directories to mount the basic SLES ISOs and service pack ISOs: mkdir -p /mnt/loop/sles9_sp3/SUSE-SLES-Version-9/CD1 And for X in ‘seq 1 5‘; do mkdir -p /mnt/loop/sles9_sp3/SUSE-CORE-Version-9/CD$X; done

© Copyright IBM Corp. 2006. All rights reserved.

431

And for X in ‘seq 1 3‘; do mkdir -p /mnt/loop/sles9_sp3/SUSE-SLES-9-Service-Pack-Version-3/CD$X; done Note on SLES10: Because SUSE Linux Enterprise Server 10 (SLES10) is also available on a DVD it is enough to create a single directory, for example, mkdir -p /mnt/loop/sles10/DVD1. 2. Mount the ISO files using a loop device. The following commands must be written without a newline/return but issued with return at the end: mount -t iso9660 -o loop,ro /srv/data/iso-images/SLES-9-ppc-RC5-CD1.iso /mnt/loop/sles9_sp3/SUSE-SLES-Version-9/CD1 And for X in ‘seq 2 6‘; do mount -t iso9660 -o loop,ro /srv/data/iso-images/SLES-9-ppc-RC5-CD$X.iso /mnt/loop/sles9_sp3/SUSE-CORE-Version-9/CD$((X-1)); done And for X in ‘seq 1 3‘; do mount -t iso9660 -o loop,ro /srv/data/iso-images/SLES-9-ppc-SP3-CD$X.iso /mnt/loop/sles9_sp3/SUSE-SLES-9-Service-Pack-Version-3/CD$X; done

432

IBM BladeCenter JS21: The POWER of Blade Innovation

Tip: If enough loop devices are not available, there are two ways to add loop devices, depending on your kernel configuration. You can add the max_loop=255 parameter in the /etc/lilo file to enable 255 loop-devices: # header section timeout = 100 default = linux # image section image = /pci@8000000f8000000/ide@4,1/disk@0:3,/boot/vmlinux label = linux root = /dev/hda3 append = "selinux=0 elevator=cfq max_loop=255" initrd = /pci@8000000f8000000/ide@4,1/disk@0:3,/boot/initrd Issue the lilo command and restart the system to enable the added parameter. The shown settings might be differ on other systems or distributions. If the loop-device driver is a kernel module, as in Red Hat Enterprise Linux (RHEL), it is necessary to add the following line in /etc/modeprobe.conf: options loop max_loop=255

Note on SLES10 - Mount the DVD: mount -t iso9660 -o loop,ro /srv/data/iso-images/SLES-10-ppc-RC1-DVD1.iso /mnt/loop/sles10/DVD1 3. Copy the now accessible files, create one directory and some soft links by issuing the following commands. The copy process takes some time. The final directory structure is shown in Example C-1: rsync -auv /mnt/loop/sles9_sp3 /srv/ftp And mkdir /srv/ftp/sles9_sp3/yast And ln -s SUSE-SLES-Version-9/CD1/boot /srv/ftp/sles9_sp3/boot And ln -s SUSE-SLES-Version-9/CD1/content /srv/ftp/sles9_sp3/content

Appendix C. SUSE Linux Installation Server setup

433

And ln -s SUSE-SLES-Version-9/CD1/control.xml /srv/ftp/sles9_sp3/control.xml And ln -s SUSE-SLES-9-Service-Pack-Version-3/CD1/driverupdate /srv/ftp/sles9_sp3/driverupdate And ln -s SUSE-SLES-9-Service-Pack-Version-3/CD1/linux /srv/ftp/sles9_sp3/linux And ln -s SUSE-SLES-Version-9/CD1/media.1 /srv/ftp/sles9_sp3/media.1 Restriction: It is not possible to work directly with the loop-mounted ISO files. You have to copy the files to the hard drive to create a functional installation source.

Tip: On SLES10 it is sufficient to copy the content of the DVD using: cp -dbr /mnt/loop/sles10/ /srv/ftp/sles10/
Example: C-1 First level of the installation source directory

/srv/ftp/sles9_sp3/ |-- SUSE-CORE-Version-9 |-- SUSE-SLES-9-Service-Pack-Version-3 |-- SUSE-SLES-Version-9 |-- boot -> SUSE-SLES-Version-9/CD1/boot |-- content -> SUSE-SLES-Version-9/CD1/content |-- control.xml -> SUSE-SLES-Version-9/CD1/control.xml |-- driverupdate -> SUSE-SLES-9-Service-Pack-Version-3/CD1/driverupdate |-- linux -> SUSE-SLES-9-Service-Pack-Version-3/CD1/linux |-- media.1 -> SUSE-SLES-Version-9/CD1/media.1 `-- yast

434

IBM BladeCenter JS21: The POWER of Blade Innovation

4. Create two files instorder and order in /srv/ftp/sles9_sp3/yast with the content shown in Example C-2 and Example C-3.
Example: C-2 The instorder file

/SUSE-SLES-9-Service-Pack-Version-3/CD1 /SUSE-SLES-Version-9/CD1 /SUSE-CORE-Version-9/CD1 Important: The path names in Example C-3 are TAB separated.
Example: C-3 The order file

/SUSE-SLES-9-Service-Pack-Version-3/CD1 /SUSE-SLES-9-Service-Pack-Version-3/CD1 /SUSE-SLES-Version-9/CD1 /SUSE-SLES-Version-9/CD1 /SUSE-CORE-Version-9/CD1 /SUSE-CORE-Version-9/CD1 All basic preparations to install SLES using the network are now finished.

Appendix C. SUSE Linux Installation Server setup

435

436

IBM BladeCenter JS21: The POWER of Blade Innovation

Abbreviations and acronyms
ac AIX BIU BOOTP BOS BPU BSMP CISC CLI CRLF CSM CTR dc DDR DHCP DNS ECC ERAT ESM FPR FPU FTP FXU Gb GB gcc GPR alternating current Advanced Interactive eXecutive bus interface unit Bootstrap Protocol base operating system branch processing unit blade system management processor Complex Instruction Set Computer command-line interface carriage return-line feed Cluster Systems Management count register direct current double data rate Dynamic Host Configuration Protocol Domain Name Server error-checking and correction effective to real address translation Ethernet switch module floating-point register floating point unit File Transfer Protocol fixed point unit gigabit gigabyte GNU C Compiler general purpose register RAM RHEL POST POWER MP NFS NIC NIM PKT POSIX KVM LAN LPAR MAC Mb MB MFLOPS IGESM IP IPL ITSO I/O IBM GUI HTTP HTTPS graphical user interface Hypertext Transfer Protocol Hypertext Transfer Protocol-Secure input/output International Business Machines Corporation CISCO Intelligent Gigabit Ethernet Switch Module Internet Protocol initial program load (boot) International Technical Support Organization keyboard, video, and mouse local area network logical partition Media Access Control megabit megabyte Million Floating Point Operations Per Second multiprocessor Network File System Network Interface Controller Network Installation Manager packet Portable Operating System for UNIX Power On Self Test Performance Optimization With Enhanced RISC random access memory Red Hat Enterprise Linux

© Copyright IBM Corp. 2006. All rights reserved.

437

RISC ROM RTC SAN SCSI SMP SoL SPOT SSH TCP/IP TFTP TLB USB VLAN VPN VXU WAN XER XML YaST

reduced instruction set computer read-only memory Real-Time Clock storage area network Small Computer System Interface symmetric multiprocessor Serial over LAN Shared Product Object Tree Secure Shell Transmission Control Protocol/Internet Protocol Trivial File Transfer Protocol Translation Lookaside Buffer Universal Serial Bus virtual local area network virtual private network vector processing unit wide area network Fixed Point Exception Register Extensible Markup Language Yet Another Setup Tool (SUSE)

438

IBM BladeCenter JS21: The POWER of Blade Innovation

Related publications
The publications listed in this section are considered particularly suitable for a more detailed discussion of the topics covered in this redbook.

IBM Redbooks
For information on ordering these publications, see “How to get IBM Redbooks” on page 444. Note that some of the documents referenced here may be available in softcopy only. NIM: From A to Z in AIX 4.3, SG24-5524 Implementing IBM Director 5.10, SG24-6188 The IBM eServer BladeCenter JS20, SG24-6342 IBM TotalStorage: SAN Product, Design, and Optimization Guide, SG24-6384 AIX 5L Practical Performance Tools and Tuning Guide, SG24-6478 Advanced POWER Virtualization on IBM System p5, SG24-7940 The Cutting Edge: IBM eServer BladeCenter, REDP-3581 IBM eServer BladeCenter Systems Management, REDP-3582 Nortel Networks L2/3 Ethernet Switch Module for IBM eServer BladeCenter, REDP-3586 IBM eServer BladeCenter Layer 2-7 Network Switching, REDP-3755 Cisco Systems Intelligent Gigabit Ethernet Switch Module for the IBM eServer BladeCenter, REDP-3869 Virtual I/O Server Integrated Virtualization Manager, REDP-4061

© Copyright IBM Corp. 2006. All rights reserved.

439

Other publications
These publications are also relevant as further information sources: IBM BladeCenter Solution Assurance Planning Review Guide, Document SA749 IBM CSM for AIX 5L and Linux: Administration Guide, SA23-1343 CSM 1.5.0 for AIX 5L and Linux: Planning and Installation Guide, SA23-1344 CSM 1.5.0 for AIX 5L and Linux: Command and Technical Reference, SA23-1345 AIX 5L Version 5.3 Performance Management, SC23-4905

Online resources
These Web sites and URLs are also relevant as further information sources: Power Module Upgrade Guidelines (Technical Update) - IBM BladeCenter (Type 8677) http://www.ibm.com/pc/support/site.wss/document.do?sitestyle=ibm&lnd ocid=MIGR-53353 ServerProven Web site http://www.ibm.com/servers/eserver/serverproven/compat/us/eserver.ht ml IBM Middleware on Linux http://www.ibm.com/software/os/linux/software Information Center VIOS commands http://publib.boulder.ibm.com/infocenter/eserver/v1r3s/index.jsp?top ic=/iphb1/iphb1_vios_commandslist.htm IBM Virtualization http://www.ibm.com/servers/eserver/about/virtualization/ Advanced POWER Virtualization http://www.ibm.com/servers/eserver/pseries/ondemand/ve/resources.htm l Virtual I/O Server http://techsupport.services.ibm.com/server/vios/home.html

440

IBM BladeCenter JS21: The POWER of Blade Innovation

IBM personal computing support site http://www.ibm.com/pc/support/site.wss IBM Cluster information center http://publib.boulder.ibm.com/infocenter/clresctr/vxrx/index.jsp?top ic=/com.ibm.cluster.csm.doc/clusterbooks.html Dropped sessions with Serial over LAN (SOL) and BOOTP on the same port IBM eServer BladeCenter JS20 http://www.ibm.com/pc/support/site.wss/document.do?lndocid=MIGR-5528 2 Download IBM Director 5.1 http://www.ibm.com/servers/eserver/xseries/systems_management/ibm_di rector/ Download site for AutoUpdate V4.3.4 or later levels http://freshmeat.net/projects/autoupdate Download sg3_utils-1.06-1.ppc64* http://people.redhat.com/pknirsch BladeCenter internal network diagram http://www.research.ibm.com/journal/rd/496/brey.pdf BladeCenter networking http://www.research.ibm.com/journal/rd/496/hunter.pdf BladeCenter processor blades, I/O expansion adapters, and units http://www.research.ibm.com/journal/rd/496/hughes.pdf Software and device drivers - IBM BladeCenter http://www.ibm.com/pc/support/site.wss/document.do?lndocid=MIGR-6301 7 Support for BladeCenter products http://www-03.ibm.com/servers/eserver/support/bladecenter/allproduct s/installing.html Expect scripting (to automate/repeat firmware updates) http://expect.nist.gov IBM Linux On POWER diagnostic software https://www14.software.ibm.com/webapp/set2/sas/f/lopdiags/home.html

Related publications

441

Open Firmware http://www.firmworks.com Download site for OpenSSL Red Hat Package Manager (RPM) http://www-1.ibm.com/servers/aix/products/aixos/linux/download.html Red Hat Enterprise Linux installation tip http://www.redhat.com/search/ui.jsp Automatic Linux Installation and Configuration with YaST2 http://www.suse.com/~ug/autoyast_doc/ Red Hat Enterprise Linux installation tip http://www.redhat.com/search/ui.jsp Red Hat Enterprise Linux 4: System Administration Guide http://www.redhat.com/docs/manuals/enterprise/RHEL-4-Manual/sysadmin -guide/ IBM Director installation instructions http://publib.boulder.ibm.com/infocenter/eserver/v1r2/index.jsp Cluster Systems Manager 1.5.1.1 for AIX Package Information https://www14.software.ibm.com/webapp/set2/sas/f/csm/download/csmaix _1.5.1.1down.html Cluster Systems Manager Document Library http://publib.boulder.ibm.com/infocenter/clresctr/vxrx/index.jsp?top ic=/com.ibm.cluster.csm.doc/clusterbooks.html Cluster Systems Manager Web site http://www14.software.ibm.com/webapp/set2/sas/f/csm/home.html Site for autoupdate RPM http://freshmeat.net/projects/autoupdate IBM pSeries and AIX Information Center http://publib.boulder.ibm.com/infocenter/pseries AIX Information Center http://publib.boulder.ibm.com/infocenter/pseries/v5r3/index.jsp QLogic software download page http://support.qlogic.com

442

IBM BladeCenter JS21: The POWER of Blade Innovation

Unrolling AltiVec, Part 1: Introducing the PowerPC SIMD unit http://www.ibm.com/developerworks/power/library/pa-unrollav1/ High-Performance Processors: Altivec Technology http://www.freescale.com/files/32bit/doc/fact_sheet/ALTIVECFACT.pdf AltiVec - Wikipedia http://en.wikipedia.org/wiki/AltiVec PowerPC Microprocessor Family: AltiVec Technology Programming Environments Manual http://www-306.ibm.com/chips/techlib/techlib.nsf/techdocs/FBFA164F82 4370F987256D6A006F424D IBM Personal computing support http://www-307.ibm.com/pc/support/site.wss/document.do?lndocid=MIGR63788 IBM Director 5.10 Update 2 https://www14.software.ibm.com/webapp/iwm/web/preLogin.do?source=dmp IBM Director 5.10 Update 2: Sign in https://www14.software.ibm.com/webapp/iwm/web/reg/download.do?source =dmp&S_PKG=diragent&cp=UTF-8 IBM Support: Fix Central https://techsupport.services.ibm.com/server/aix.fdc Whitepaper: Red Hat's New Journaling File System: ext3 http://www.redhat.com/support/wpapers/redhat/ext3/ SYSSTAT utilities http://perso.wanadoo.fr/sebastien.godard/ Serial over LAN (SOL) Setup Guide - IBM BladeCenter, T http://www-307.ibm.com/pc/support/site.wss/document.do?sitestyle=ibm &lndocid=MIGR-54666&velxr-layout=print

Related publications

443

How to get IBM Redbooks
You can search for, view, or download IBM Redbooks, Redpapers, Hints and Tips, draft publications and Additional materials, as well as order hardcopy Redbooks or CD-ROMs, at this Web site: ibm.com/redbooks

Help from IBM
IBM Support and downloads ibm.com/support IBM Global Services ibm.com/services

444

IBM BladeCenter JS21: The POWER of Blade Innovation

Index
Symbols
/etc/dhcpd.conf 299 /etc/init.d/dhcpd restart command 286 /etc/init.d/dhcpd start command 286 /etc/modeprobe.conf 433 /etc/vsftpd.con 289 /etc/vsftpd.conf file 289 /etc/xinet.d/vsftpd 288 virtual SCSI disk LUN 251 AltiVec 15 optimized applications 5 Web sites 17 AMD 3DNow! 15 application subnet 51 ARP function 49 array processing 14 AutoYaST 311, 419

Numerics
2-port FC switch module firmware update 108 8677 19 8844-31X 11 8844-51X 11 8852 21

B
baseboard management controller (BMC) 70 bkprofdata command 255 blade management 356 blade server 2 assigning media tray 94 assigning names 90 eth0 port 123 I/O expansion card 169 setting the boot sequence 91 Blade System Management Processor (BSMP) 97 BladeCenter 1, 5 advantages high availability 3 lower cost 2 SAN optimization 3 server consolidation 2 switch technology 3 benefits 2 chassis 19, 64 device drivers 95 documentation 46 high-performance, low-latency interconnection network 51 integrated switches 2 internal management network 69 internal network 66 management interface 32 management module 3, 29–30, 411 managing multiple chassis 33, 53 media tray 24 memory 17 multi-chassis interconnection 47

A
activating NIM master 268 active console, selecting 134 advanced management module 71, 77, 98 command-line interface 72 connectors 73 Ethernet port 74 firmware update 104 management network 77 Advanced Micro Devices (AMD) 5 advanced POWER Virtualization 5, 36, 53 Web site 38 AIX performance tuning 386 AIX 5L 26 environment 244 installation CDs/DVDs 260 levels supported by JS21 26 mirroring with LVM 250 network installation 261 new virtual disks 239 performance tuning 386 RAID 182 RAID configuration 170 technology levels 53

© Copyright IBM Corp. 2006. All rights reserved.

445

physical layout 64 planning resources 46 power modules 20, 23 power requirements 4 product documentation updates 46 schematic 66 server consolidation 2 using IBM Director Console 355 Web interface 32, 346 BladeCenter chassis T 23 BladeCenter H 5, 21 BladeCenter JS20 133 BladeCenter JS21 chassis 19 dedicated processor 39 device alias 157 directed BOOTP 160 hardware setup 63 IBM Director Agent support 58 IBM Director support 58 internal hard disk 18 introduction 4 L2 cache 9 large configuration 53 management module 24 memory features 17 micro-partitioning 29 minimal network requirement 47 minimum and optional features 11 models 11 network boot configuration 135 network installation 54–55 network installation environment 54 network interface selection 56 operating system support 26 operating systems 26 performance tuning 385 physical packaging 10 planning elements 45 power/thermal management architecture 23 processor 13 RAID 18 SLES supported versions 27 SMS interface 133 specification 8 specifications 9 storage 3 technology 1 viewing Open Firmware settings 155

VIOS installation 193 BladeCenter T physical layout 65 boot command 161 boot device CD/DVD 145 order 144 boot image 290 boot parameter 299 definition 298 location 325 mkzimage_cmdline 306 using SSH 308 using VNC 308 with AutoYaST 320 boot process explicit location 299, 325 logical partition 231 boot sequence BOOTP 300 current setting 91 default 135 device alias 157 multipath I/O 418 RHEL installation 321, 326, 335 setting 90–91 updated by SLES install 290 Bootstrap Protocol (BOOTP) 284 coexistence with DHCP 55 definition 55 directed versus broadcast 136 byte sector, RAID format 173

C
C2 security 26 capacity planning 33 CD/DVD drive 120 cfgmgr command 239 chsyscfg command 237, 240 chvg -g command 248 Cluster Systems Management (CSM) 32, 345, 362 AIX nodes 377 automating software installations 54 cluster node installation 373 hardware control attributes 374 hardware management subnet 49 installation server 363, 373 introduction 33

446

IBM BladeCenter JS21: The POWER of Blade Innovation

Linux nodes 379 network considerations 61 node attributes 373 planning 60 supported operating systems 59 verifying installation 383 command /usr/lib/methods/define_rspc 166 /usr/sbin/installios 194 AIX cfgmgr 167, 234 mkcd 256 mksysb 256 odmget 166 boot 130–131, 161 console 130 CSM copycsmpkgs 381 definenode 375 dsh 383 lsnode 376 updatenode 377, 381 diag 99 diagmenu 252 env 130–131 env -T 130 info 96 Linux bcmflashdiag 102 chkconfig 288–289 iprconfig 183 mkzimage_cmdline 301, 306, 320 nvsetenv 344 lscfg 251 lshwres 38 lsmap 250 mkzimage_cmdline 299, 322 netstat 290 Open Firmware devalias 156 printenv 154 power 130, 133, 152, 183 reset 130 smitty nim_power5 194 sol 131 telnetcfg 132 update 106 update_flash 101 update_flash (AIX 5L) 100

VIOS backupios 255 cfgdev 233 chdate 210 chsyscfg 231 chsysstate 231 lsdev 233 mkgencfg 209 mktcpip 210 mkvt 231–232, 257 rmvt 231 command-line interface (CLI) 31 Common Internet File System (CIFS) 284 configuration AIX environment 261 NIM master 263 NIM using command line 201 NIM using SMIT 196 VIOS partition 221 configuration file AutoYaST 311 disable parameter 288 Kickstart 341, 343 TFTP 325 xinet.d 288 csmconfig command 368–369

D
date and time setting 210 default user ID and password, management module 133 DHCP AutoYaST configuration 317 coexistence with BOOTP 55 configuration 285 network planning 46 restarting 286 service 54, 325 TFTP file name 298 disk array 188–189 selecting disk units 191 disk space adding to virtual disk 245 logical partition 217 logical volume 222 virtual disk 247 disk technology 396 diskette drive 24

Index

447

dmesg 401 double data rate 2 (DDR2) 4 DVD drive, VIOS installation media 194 Dynamic Host Configuration Protocol (DHCP) 284 dynamic logical partitioning (DLPAR) 29, 43

gunzip command 370

H
hard disk drives (HDD) 9 hardware alerts 33 vital product data (VPD) 81 hardware management 27, 29, 41, 49, 61 subnet 47 Hardware Management Console (HMC) 27, 29–30, 41, 194 high availability 3 high-performance computing (HPC) 4 ideal blade server solution 4 vector processing 14 host bus adapter (HBA) 411, 413–416 discovery period 417 HS20 60 Hypertext Markup Language (HTML) 284 Hypervisor 29

E
e-mail alert 84 Enhanced Integrated Drive Electronics (EIDE) 396 error-checking and correction (ECC) 4 Ethernet expansion card 68, 103, 162, 169 firmware update process 99 port enumeration 162 changing in AIX 162 switch module 68, 89, 108, 122, 169

F
Fibre Channel (FC) 410–411, 416 field-replaceable unit (FRU) 24 field-replaceable units (FRU) 59P6629 24 file system access time updates 397 ext3 journaling mode 398 tuning 395 File Transfer Protocol (FTP) configuring service 288, 324 installation server 284, 296 installation service 293 firmware committing update 101–102 copying permanent to temporary 101 identifying levels 96 rejecting update 101–102 update_flash (Linux command) 101 updating redundant MM 83 updating with AIX 99 floating-point registers (FPRs) 16 free command 406 fully qualified domain name (FQDN) 203

I
I/O module 57 accessing management interface 68 advanced configuration 85, 88 basic setup 85 differences in bay location 68 Ethernet expansion card 51 Ethernet ports 68 expansion card options 169 external ports 89 firmware update 106 IP address 77 management 32 by management module 77 interface 47, 49, 69, 85–87 IP address 85 port 68 port enabling 88 setting IP address 85 SoL 89, 241 updating firmware 106 IBM diagnostic tools and utilities 96 IBM Director 57, 346 Agent 352 components 58 Core Services 352 disabling SLPD on SLES 352

G
General Parallel File System (GPFS) 52 general purpose register (GPR) 16 geninstall command 368 GNU C Compiler (gcc) 27 graphical user interface (GUI) 31, 388

448

IBM BladeCenter JS21: The POWER of Blade Innovation

hardware management subnet 49 installation on AIX 349 installing components 347 introduction 33 inventory 57 network installation of OS 53 requirements 58 performance planning 33 planning 58 primary functions 57 remote management 57 resource monitoring 57 TCP/IP port 84 IBM General Parallel File System (GPFS) 52 IBM middleware 34 ibmvmc0 31 IEEE 802.1Q 37 input/output (I/O) 2 installation AIX 5L CD/DVD 260 general topics 260 AIX on client 270 CSM management server 363 IBM Director agent on Linux 352 IBM Director components 347 IBM Director on AIX 349 Integrated Virtualization Manager (IVM) 193 Linux from CD/DVD 282 Linux on LPAR 282 operating system on LPAR 232 Red Hat Enterprise Linux 283 RPM package 203 SUSE Linux Enterprise Server (SLES) 282 VIOS image from Linux server 202 VIOS in a JS21 194 Virtual I/O Server (VIOS) 194 installation source 294 configuring with YaST 294, 298 destination directory 296 directory 293, 297, 324 for RHEL 337 tree 296 Red Hat Enterprise Linux 324 undefined 301 integrated development environment (IDE) 3 Integrated Drive Electronics (IDE) 396 integrated switch technology 3

integrated systems management 3 Integrated systems management processor (ISMP) 70 Integrated Virtualization Manager (IVM) 27 architecture 30 command logs 257 definition 27 description 41 design 29 devices tab 233 dynamic logical partitioning 43 functions 29 installation 193 limitations 42 LPAR console 37–38 maintenance 253 navigation area 212 Partition Management menu 223 requirements 42 restrictions 28 starting 212 starting graphical interface 211 Storage Management menu 233 VIOS maintenance 253 VIOS management 211 work area Devices tab 218, 222 Logical Volume tab 222, 238 Optical Devices tab 233 Start Wizard button 223 storage pool 218 Intel Multimedia Extensions (MMX) 15 Intel Streaming SIMD Extensions (SSE) 15 INTx port 68 iostat command 404 iSmall Computer System Interface (iSCSI) 13

J
JS21, see BladeCenter JS21

K
kernel parameter 390 kernel.shmm axe 392 keyboard, video, and mouse (KVM) 56, 94, 121 ports 71 Kickstart configuration file 427

Index

449

L
license -accept command 208 Lightweight Directory Access Protocol (LDAP) 26, 78 lilo command 433 Linux basic preparation 283 IBM diagnostic tools and utilities 96 network installation 281 nodes 379 supported kernel 386 tuning options on POWER 386 logical partition (LPAR) 28, 245, 253–256 adding disks 239 assigning memory 224 optical device 234 benefits 35 configuration change 235, 240 creation 218, 223 disk space 252 dynamic 43 Ethernet adapters in SMS 149 Integrated Virtualization Manager (IVM) 30, 211 Linux installation 282 logical volume 217, 222 maintenance 256 management 29 moving optical device 234 next reboot 240 non-dynamic operations 211 opening virtual terminal 231 operating system installation 232 optical device 229, 234 physical adapters 28 power on 231 processing mode 225 processing unit 240 rootvg 217 secure environment 32 security layer 36 SMS mode 232 storage assignment 228 virtual Ethernet 226 adapaters 240 virtual LAN channel 36 virtual SCSI adapater 42 virtual terminal 231–232, 257 virtualization feature 38

logical unit number (LUN) 216, 250, 416 Logical volume (LV) creation 222 logical volume (LV) advanced storage management 244 assigning 239 commands 215 data mirroring 250 default storage pool 217 RAID support 41 viewing 223 VIOS concepts 216 virtual disk extension 245 lpcfgop command 209 lsdev command 209 lsdev -virtual command 31 lshwres command 38 lspv command 275 lsrefcode command 231

M
man dhcpd command 285 management module 24 10/100BaseT Ethernet interface 49 CLI timeout 132 command-line interface 72, 123, 128, 130 configuration 71 connection drawing 73 controlling SoL using commands 130 default gateway 75 default IP address 74–75 default user ID and password 133 earlier firmware 124 external interface 49, 68, 74, 78 firmware 105 firmware update 104 hardware management subnet 49 hardware VPD 81 initial configuration 75 installation guide 71 interface 49 internal network interface 49 IP address 49, 74 (no DHCP) 75 login session limits 124 management interface 71, 73 network requirements 47 redundancy 82

450

IBM BladeCenter JS21: The POWER of Blade Innovation

resetting 84 restarting 78 setting IP configuration 75 setup task 78 SoL 50, 89 SSH 128 system management interface 97 TCP/IP port usage 83 user 75 Web interface 75, 126 management node 60 Management Processor Assistant (MPA) 58 maximum transmission unit (MTU) 37, 77 Media Access Control (MAC) 73–75, 104, 154 determining address 81 media tray 24, 94, 194, 233, 260 diskette drive 283 Medium Access Control (MAC) 69 memory dual inline memory modules (DIMMs) 11 message Could not find the SUSE Linux Enterprise Server 9 Installation CD 301 SoL session is already active 130 This machine’s CPU ID does not match the CPU ID stored in the NIM database 279 Virtual terminal is already connected 231 message passing interface (MPI) 52 metadata, see file system 395 Micro-Partitioning 36, 39–40 middleware 34 midplane 69 mirrorios command 250 mkgencfg command 209 mkinitrd command 398 mkvt command 216 mount command 398 Myrinet 51–52 expansion card 51

network boot 97, 133, 135, 149, 151, 161, 293, 300 logical layout 48 planning 46 minimal requirements 47 Network Equipment Building Standards (NEBS) 23 Network File System (NFS) 54, 284 export and server daemon 205 network information services (NIS) 387 network installation basic preparation 283 configuration file 285 installation source 284 Linux 281, 284 mkzimage_cmdline 299 NIM 270 Open Firmware 322 options 53 planning 54 preparation 55 Red Hat Enterprise Linux 335 Red Hat Enterprise Linux (RHEL) 322 setting up 55 infrastructure 281 network boot 135 SUSE Linux Enterprise Server 291 Trivial File Transfer Protocol (TFTP) 287 Network Installation Manager (NIM) 40, 194, 281 master 202, 260–261, 263, 266–268, 270 state 269 network planning 46 network services 54 NIM client 266 allocating resources 268 changing from rsh to nimsh 276 Nortel Networks Layer 2-7 Gigabit Ethernet switch module 89, 125

N
name server SAN 411, 415 worldwide name 416 navigation area Partition Management menu 230 Service Management menu 253, 257 Storage Management menu 218–219, 222–223, 238, 249

O
ODM 162–166 Open Firmware activating interface 133, 152 aliases 93 boot parameters 343 boot sequence 92–93 defining an alias 156 device tree 161 directed BOOTP 136, 160

Index

451

IP parameters 160 ls command 161 network installation 322 setenv 321 operating system (OS) boot phase 417 installation source 307 supported by JS21 26 optical device sharing 233 optical pass-thru I/O module 51 option blades 2

P
parallel network installation 33 PCI-X SCSI disk array 173, 177, 182 Performance Optimization with Enhanced RISC (POWER) 4 performance tuning tool 401 Peripheral Component Interconnect (PCI) 3 physical disk 188, 216, 244 physical volume 216–218, 245, 250 commands 215 PKT file 104, 106 Pluggable Authentication Modules (PAM) 26 pmap command 407 POWER Hypervisor 36–37 power management 358 power modules 23 Power On Self Test (POST) 133, 135, 153 power requirements for BladeCenter 4 power/thermal management architecture 23 PowerPC 970MP 13 AltiVec extensions 16 processor features 13 operating frequency 19 proxy ARP 49, 69–70 PUTTY 124

AutoYaST with Open Firmware 321 boot sequence 335 configuration tool 336 CSM support 59 disabling daemons 387 general information 26 importing configuration 313 installation 283 iSCSI support 397 Kickstart 342 maintenance contract 26 supported level 26 sysctl 390 Red Hat Enterprise Linux (RHEL) network installation 322 Red Hat Package Manager (RPM) 59 Redundant Array of Independent Disks (RAID) 9 ReiserFS 399 remote console 359 remote shell (rsh) 276 Request For Comment (RFC) 397 resource monitor 360 resource monitoring 57 RETAIN tip H181655 56 RPM AutoUpdate 59 bcmflashdiag-js20 102 conserver 60 fping-2.4b2-5 60 IBMJava2-JRE 60 openssl 277 sg3_utils-1.06-1.ppc64 59 tftp-HPA 60 rstprofdata command 255 Run-Time Abstraction Services (RTAS) 101

S
Samba Web Administration Tool (SWAT) 388 Secure Shell (SSH) 78, 359 during installation 308 Secure Sockets Layer (SSL) 78 selecting active console 134 Serial Attached SCSI (SAS) 3–4, 18, 396 Serial over LAN (SoL) 32 attaching to open console 130 boot sequence 92 configuring management module 125 description 121

R
RAID 18 configuration tool 170 preparing the disks 170 Red Hat disabling daemons 387 Red Hat Enterprise Linux attended installation 330

452

IBM BladeCenter JS21: The POWER of Blade Innovation

Directed BOOTP 136 enabling 127 Ethernet I/O Switch 56 firmware level requirements 124 I/O module requirement 122 installation without SoL 307 interacting with VNC 311 networking 51 persistent target 131 prerequisites 124 range of IP addresses 124 Red Hat Enterprise Linux (RHEL) installation 335 setting up CISCO switch 90 starting session 123, 128 subnet 47, 50 terminate session 131 User Defined Keystroke Sequences 125 VLAN 89, 124 Serial Storage Architecture (SSA) 41 server consolidation 2 Server Message Block (SMB) 284 ServerProven 12 shared Ethernet adapter (SEA) 41, 242–244 Simple Mail Transfer Protocol (SMTP) 78 simultaneous multithreading (SMT) 394 single-active core 13 single-instruction multiple-data (SIMD) 13 small computer system interface (SCSI) 3 small-form-factor ((SFF) 12 smit csm command 369 smitty manfs command 261 software distribution 57 remote management 33 Software Distribution Premium Edition 33 sshpassword 307 stand-alone diagnostics CD updating firmware 99 storage area network (SAN) 3, 216, 409 boot specialities 417 optimization 3 setup 410 storage pool 215–220, 222, 227, 249–250, 252 advanced configuration 218 creation 218–219 subnet hardware management 47, 49 logical layout 48

SUSE Linux Enterprise Server (SLES) attended network installation 299 AutoYaST 311, 320 compatibility 313 clearing boot-file variable 344 configuring RAID 183 creating installation server with service pack 298 CSM support 59 disabling daemons 388 disabling SLPD 352 importing Kickstart file 313 installation 282 after RAID creation 191 kernel modules 302 installation server 294 introduction 27 mkzimage_cmdline 306 network installation 290–291 no hard drives message 188 preparing installation source 293 rescue mode 170 SMB as installation service 284 support for JS21 27 supported level 26 supported version 53 unattended installation 311, 320 with CSM 60 switch module 68, 410, 412 IP address 106 symmetric multiprocessor (SMP) 349 server 35 synchronous dynamic random access memory (SDRAM)) 9 sysctl command 390 system backup image 256 System Management Interface Tool (SMIT) 196 System Management Service (SMS) 133 activating interface 133 options 152 System Management Services (SMS) 133 system management tools 32 systemid command 371

T
Telnet 132 telnetcfg command 132 top command 402

Index

453

Trivial File Transfer Protocol (TFTP) 54, 56 configuration 287, 299 network installation 284 Red Hat bootable image 325 testing service 290 tuning file system 395 memory 394 ReiserFS 399 Transmission Control Protocol (TCP) 399 User Datagram Protocol (UDP) 399 twgstart command 354

U
Universal Manageability Initiative 32 Universal Serial Bus (USB) 24 uplink command 82 uptime command 401 USB CD-ROM 145 usessh 307

V
varyoff command 246 varyonvg command 248 vector length 13 vector processing 14 vector processor (VXU) 15 vector register file (VRF) 16 Vector/SIMD Multimedia eXtension 13 virtual disk 216, 227, 254 LPAR operation 238 virtual Ethernet adapter 146, 226, 240 boot device 149 bridging 242 with IVM 241 design 37 dynamic operations 237 features 37 Integrated Virtualization Manager (IVM) 211 management 29 network traffic 41 VIOS setup 209 virtual LAN (VLAN) 42 virtual I/O adapters 37 Virtual I/O Server (VIOS) 5, 27, 39 backup and restore 255 command-line interface 214

creating logical partitions 223 data protection 244 default storage pool 219 default user 212 define NIM client 202 definition 39 device sharing 233 disk mirroring 249 installation 193–194 from NIM server 194 on RAID array 252 with NIM 202 without DVD drive 194 Integrated Virtualization Manager (IVM) 28, 42, 211 maintenance 253 micro-partitioning 39–40 moving memory 237 network configuration 242 opening a virtual terminal 231 partition configuration 221 RAID support 41 required version 38 rootvg mirroring 250 storage management 216 storage pool 250 rootvg 217 supported operating systems 38 VIOS partition 221 virtual disks 250 virtual Ethernet 37 virtual processor operation 237 virtual SCSI 37 virtualization 35–36, 209 virtual LAN (VLAN) 46 4095 for SoL 50 planning 46, 70 SoL 89 virtual Ethernet 37 virtualization 36 Virtual Management Channel (VMC) 29–30 Virtual Network Computing (VNC) 130, 307 virtual processor 39 virtual SCSI adapter 42 client adapter 41 introduction 37, 41 server adapter 41 virtual terminal 231

454

IBM BladeCenter JS21: The POWER of Blade Innovation

virtualization 36 Virtualization Engine 27, 41 vital product data (VPD) 24 vmstat command 405 vncpassword 307 vsftpd 288

W
worldwide name (WWN) 413

X
xinetd 289

Y
Yet Another Setup Tool (YaST) 294

Index

455

456

IBM BladeCenter JS21: The POWER of Blade Innovation

IBM BladeCenter JS21: The POWER of Blade Innovation

(1.0” spine) 0.875”<->1.498” 460 <-> 788 pages

Back cover

®

IBM BladeCenter JS21
The POWER of Blade Innovation

High-performance blade server ideal for extremely dense HPC clusters First IBM blade server with native virtualization for server consolidation Exceptional price-performance per watt for WebSphere with AIX or Linux OS

Blade servers have captured the industry focus because of their efficient, powerful technology and their modular design, which reduces cost with a more efficient use of valuable floor space. They offer simplified management, which can help to speed up tasks such as installing, provisioning, updating, and troubleshooting hundreds of blade servers. You can do all of this remotely using one graphical console with IBM Director systems management tools or cluster management software such as Cluster Systems Management (CSM). This IBM Redbook takes an in-depth look at the IBM BladeCenter JS21. This is a 2-core or 4-core blade server for applications requiring 64-bit computing. It is ideal for computer-intensive applications and transactional Internet servers. This book helps you to install, tailor, and configure the IBM BladeCenter JS21 running either IBM AIX 5L or the Linux operating system (OS). This document expands the current set of IBM BladeCenter JS21 documentation by providing a desktop reference that offers a detailed technical description of the JS21 system. This publication does not replace the latest IBM BladeCenter JS21 marketing materials and tools. It is intended as an additional source of information that you can use, together with existing sources, to enhance your knowledge of IBM blade solutions.

INTERNATIONAL TECHNICAL SUPPORT ORGANIZATION

BUILDING TECHNICAL INFORMATION BASED ON PRACTICAL EXPERIENCE IBM Redbooks are developed by the IBM International Technical Support Organization. Experts from IBM, Customers and Partners from around the world create timely technical information based on realistic scenarios. Specific recommendations are provided to help you implement IT solutions more effectively in your environment.

For more information: ibm.com/redbooks
SG24-7273-00 ISBN 0738494968

Sign up to vote on this title
UsefulNot useful