The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

The Underground Oracle VM Manual
underground.jpg The only Oracle Virtualization and Oracle VM Manual. New chapters will be added and posted to ITNewsCast regularly.

The Underground Oracle VM Manual Oracle VM for x86 Deployment Guide
Show a printer friendly version of this book with its sub pages Copyright © 2009 - 2011 Roddy Rodstein. All rights reserved. Author: Roddy Rodstein, CISSP Change Log
Revision 1.0 Change Description First Release of the Underground Oracle VM Manual Second Release of the Underground Oracle VM Manual Third Release of the Underground Oracle VM Manual Forth Release of the Underground Oracle VM Manual Complete rewrite of Chapter 1 Oracle VM Centralized Logging Draft Complete rewrite of Chapter 2 Updated By Roddy Rodstein Roddy Rodstein Roddy Rodstein Roddy Rodstein Roddy Rodstein Roddy Rodstein Roddy Rodstein Date 05/2009

2.0

09/2009

3.0

08/2010

4.0

12/2010

1 1 1

03/08/11 03/29/11 04/03/11

Limits of Liability and Disclaimer of Warranty This publication contains information protected by copyright. This publication may not be duplicated in any way without the express written consent of the publisher, except in the form of brief excerpts or quotations for the purpose of review. The information contained herein is for the personal use of the reader and may not be incorporated in any commercial programs, other books, databases, or any kind of software without the written consent of the publisher. Making copies of this book or any portion, for any purpose other than your own, is a violation of United States copyright laws. Warning and Disclaimer Every effort has been made to make this publication as complete and as accurate as possible, but no warranty or fitness is implied. The information provided is on an "as is" basis. The authors and the publisher shall have neither liability nor responsibility to any person or entity with respect to any loss or damages arising from the information contained in this publication. The information found in this publication was gathered from many different sources in the computing world. It is provided for informational purposes only. Use common sense in applying these concepts and tips. Screen shots may vary from environment to environment. Please verify correctness and applicability in a test environment first and then deploy to your production environment(s). Trademarks Trademarked names appear throughout this publication. Rather than listing the names and entities that own the trademarks or include a trademark symbol with each mention of the trademark name, the publisher states that he is using the name for editorial purposes only and to the benefit of the trademark owner, with no intention of infringing upon that trademark. About the Author Roddy Rodstein (CISSP CEH) has over 10 years of professional experience in the IT industry. In his current role at Oracle Roddy is a , member of the Virtualization and Linux team. Before joining Oracle, Roddy spent six years at Citrix supporting Citrix's core product, XenApp. Earlier in his career Roddy successfully established, owned, and operated an IT consulting business that specialized in server based computing and remote access solutions. His professional achievements also extend to writing and self-publishing industry reference guides currently available on Amazon, Securing Microsoft Terminal Services (ISBN: 061514330X) and Citrix CCA MetaFrame 1.8 for Windows Exam Cram (ISBN: 1576109453). Audience This book is intended to assist solution architects, sales engineers, field engineers and consultants in planning, designing, deploying and supporting Oracle VM for x86 in internal and external clouds. This book assumes that the reader has an architectural understanding of cloud computing, Oracle technologies, storage and network system, and related software.

1 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Objectives This book is intended to articulate the design considerations and validation efforts required to design, deploy and support Oracle VM for x86 in an internal or external cloud. Chapters at a Glance Chapter 1: Oracle VM Introduction Chapter 2: Oracle VM for x86 Reference Design and Architectural Introduction Chapter 3: Hard and Soft Partitioning Oracle Technologies with Oracle VM Chapter 4: Oracle VM Server Sizing, Installation and Updates Chapter 5: Oracle VM Manager Sizing, Installation, Updates and the Oracle VM Manager Command Line Interface ...Oracle VM 2.1.5 to 2.2 Upgrade Chapter 6: Oracle VM 2.2 SAN, iSCSI and NFS Back-end Storage Configurations Chapter 7: Oracle VM Networking from A to 802.1Q Chapter 8: Virtualizing Oracle Database 10g/11g Chapter 9: Virtualizing Oracle Enterprise Manager 10g Oracle VM Centralized Logging The Virtualization Policy Project

Watch this space Oracle OpenWorld 2011 Call for Papers - Submit Today! Temporary Post Used For Theme Detection (409fc5e9-d7ee-4e1a-b645-8b82f256f434 - 3bfe001a-32de-4114-a6b4-4005b770f6d7) Debugging Oracle VM 2.2 & Oracle VM Manager 2.2 errors using the many log files New VDI Training Course now available! HIMSS 2011 and New Press Release Oracle Desktop Virtualization at HIMSS 2011 Oracle VM VirtualBox 4.0.4 Released! Two Virtualization Webinars This Week We're Hiring! - Server and Desktop Virtualization Product Management

Chapter 1: Oracle VM Introduction
An introduction to Oracle VM Copyright © 2009 - 2011 Roddy Rodstein. All rights reserved. Show a printer friendly version of this book with its sub pages

Table of Contents
Change Log The Evolution of Oracle VM for x86 Oracle VM for x86 Introduction ...Oracle VM for x86 Virtualization Modes ...Oracle VM for x86 Packaging and Pricing Accelerating Application Deployments with Oracle VM for x86 ...Virtual Machine Cloning ...Oracle VM Templates …...Oracle VM Templates Packaging and Pricing ...Oracle VM Template Builder …...Oracle VM Template Builder Packaging and Pricing ...Oracle Virtual Assembly Builder …...Oracle Virtual Assembly Builder Packaging and Pricing ...Oracle JRockit Virtual Edition …...Oracle JRockit Virtual Edition Packaging and Pricing The Oracle VM Management Pack Plug-in Oracle's x86 Virtualization Support Policy

Change Log
Revision Change Description 1 Complete rewrite of Chapter 1 Updated By Roddy Rodstein Date 03/08/11

The Evolution of Oracle VM for x86
At Oracle OpenWorld 2007, Oracle announced its entry into the x86 server virtualization market with the first release of Oracle VM for x86, version 2.1. Oracle VM for x86 was introduced as a low cost, high performance, open source server-virtualization solution for

2 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Oracle and non-Oracle workloads, backed by Oracle's world-class support organization. With the release of Oracle VM for x86, Oracle had a flagship x86 server virtualization solution and a clear x86 server virtualization strategy. Note: The first release of Oracle VM for x86 was version 2.1. An Oracle colleague explained to me that Larry Ellison avoids using 1.0 for Oracle product releases to help drive early adoption. In August 2008, Oracle announced the release of Oracle VM templates. Oracle VM templates are self-contained preconfigured virtual machines with key Oracle technologies, which can be downloaded from the Oracle Linux eDelivery portal and quickly deployed into production using Oracle VM for x86. Oracle VM templates completely eliminate the operating system and application installation process, reducing risk and dramatically shortening Oracle application deployment timelines. In August 2009, Oracle announced the release of Oracle VM Template Builder. Oracle VM Template Builder is an applicationpackaging studio that generates preconfigured, reusable Oracle VM templates with Oracle and non-Oracle applications on Oracle Linux JeOS (just enough operating system). Oracle Linux JeOS is a customizable small-footprint x86 and x86-64 Oracle Linux operating system. On January 27, 2010 Oracle completed the acquisition of Sun Microsystems, which added Sun's software and hardware assets to Oracle's product portfolio. Shorty after the Sun acquisition, Oracle rebranded numerous Sun virtualization technologies with the Oracle VM product name and created the Oracle VM product family with the following three separate product lines: Oracle VM for x86, Oracle VM for SPARC (formally LDOMs), and Oracle VM VirtualBox. Each product line has add-on products that extend the functionally of the line. Then, Oracle announced Oracle Virtual Assembly Builder and JRockit Virtual Edition. Oracle Virtual Assembly Builder and JRockit Virtual Edition extended Oracle VM templating into the Oracle Fusion Middleware product portfolio. With the addition of Oracle VM templates, Oracle VM Template Builder, Oracle Virtual Assembly Builder, and JRockit Virtual Edition, Oracle VM for x86 was transformed from a x86 server-virtualization solution into Oracle's next-generation application delivery platform. The next sections will examine Oracle VM for x86, Oracle VM templates, Oracle VM Template Builder, Oracle Virtual Assembly Builder, JRockit Virtual Edition and the Oracle VM Management Pack.

Oracle VM for x86 Introduction
Oracle VM for x86 is Oracle's x86 cloud virtualization and next generation application delivery platform. Oracle VM for x86 ships preinstalled on the Sun x86 server and blade product line and is also fully supported on third-party x86-64 servers. Oracle VM for x86 is the preferred virtualization and application delivery platform for internal and/or external clouds using x86 hardware running Oracle Linux, Solaris x86, Red Hat Enterprise Linux, or Windows virtual machines. With Oracle VM for x86, mission-critical applications can be migrated from physical to virtual machines with unparalleled density, performance, and ease of management. Operating system and application installations can be very time consuming. Oracle VM for x86 offers next-generation operating system and application packaging and deployment features that eliminate the traditional operating system and application installation process. Oracle VM for x86 enables operating system and application configurations to be packaged in preconfigured, reusable virtual-machine templates. Virtual machine templates can consist of a single virtual machine or a group of virtual machines, called an assembly. Virtual machine templates can be rapidly redeployed an unlimited number of times as copies of an original virtual machine or virtual machine assembly, or as a unique new virtual machine or virtual machine assembly. The Oracle VM for x86 is a type 1 hypervisor that installs directly on x86-64 hardware, enabling multiple concurrently running virtual machines to share a single piece of hardware. Each virtual machine has its own operating system, kernel, virtual CPUs, RAM, network interfaces, storage, and applications. Oracle VM for x86 increases server utilization by breaking the traditional one-workload-per-box approach to server provisioning while making operating systems and enterprise applications easier to deploy, manage, and support by eliminating the operating system and application installation process. Oracle VM for x86 consists of an x86 server component and a manager component used to manage one or more clustered servers. The server component is based on the open source Xen.org hypervisor, and is called Oracle VM server. The Xen hypervisor is a thin layer of code that installs directly on hardware that enables multiple guest operating systems to run concurrently on a single piece of hardware. Xen has emerged as the industry open standard for x86 virtualization, with broad adoption from ISVs such as Oracle, IBM, HP Dell, and Citrix, and cloud providers such as Amazon AWS, Savvis Cloud Services, and Rackspace Cloud. Oracle makes subtle , changes to the original Xen.org code that create a unique Xen distribution, which Oracle maintains and redistributes as Oracle VM server. Oracle VM server was designed to support the most demanding high I/O workloads like Oracle Database products and Oracle Exalogic, which other hypervisors and Xen distributions are unable to support. The manager component is a traditional Oracle application, named Oracle VM Manager, that installs on Oracle Linux and Red Hat Enterprise Linux. Oracle VM Manager is used to manage one or more clustered Oracle VM servers, virtual machines, and virtual machine resources. Oracle VM Manager is a traditional Oracle application consisting of an Oracle database, an Oracle application server, and a J2EE application with an OS-/browser-neutral Oracle Application Development Framework (ADF) administrative portal. All of the Oracle VM Manager components can be installed in an all-in-one configuration on an Oracle Linux virtual machine or in a multiple node HA configuration. Oracle VM Manager is distributed from the Oracle Linux eDelivery portal as an ISO file and as a preconfigured, production-ready Oracle VM template. Figure 1 shows the Oracle VM Manager administrative portal.

3 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Oracle VM for x86 Virtualization Modes
Oracle VM for x86 supports two unique virtualization modes, paravirtualization mode (PV mode) and hardware virtualization mode (HVM mode). Oracle VM servers can support both paravirtualization mode and hardware virtualization mode simultaneously on a single x86-64 server using either an Intel or an AMD virtualization technology. Intel and AMD virtualization is a requirement only for hardware virtualized mode, not for paravirtualization mode. Intel and AMD virtualization technologies are enabled and managed using the system BIOS. Paravirtualization mode requires the virtual machine operating system to run a Xen kernel and Xen network and I/O drivers. Xen paravirtualized kernels ship along with the default Linux kernel with Oracle Linux and Red Hat Enterprise Linux operating systems. Paravirtualized virtual machines are hypervisor aware and run without the additional overhead of hardware emulation. Paravirtualization offers much less overhead with timers, interrupts, I/O traffic, and context switches, allowing superior scalability under heavy loads when compared to hardware virtualization mode. Unlike paravirtualization mode, which requires the virtual machine to run a Xen kernel, hardware virtualization mode supports native unmodified operating systems. Virtual machines that run under hardware virtualization mode are called hardware virtualized machines (HVM). Hardware virtualized machines are unaware that they have been virtualized and think they are on physical hardware. To provide acceptable performance, hardware virtualized machines need to use paravirtualized network and I/O drivers. From Oracle Linux and Red Hat Enterprise Linux 4.7 onwards, the stock kernels provide paravirtualized network and I/O drivers for hardware virtualized guests. From Solaris 10 10/09 onwards, the stock kernels provide paravirtualized network and I/O drivers for hardware virtualized machines. Windows does not have native paravirtualization support, although Windows virtual machines can run as hardware virtualized machines using Oracle's paravirtualized network and I/O drivers. Oracle has released paravirtualized network and I/O drivers for the Windows operating system that can be freely downloaded from the Oracle Linux eDelivery portal. In hardware virtualization mode, the hypervisor, the Intel or AMD virtualization technologies, and dom0 with QEMU work in concert to emulate hardware for hardware virtualized machines. Hardware virtualization mode has a higher overhead than paravirtualization mode, due in part to the overhead of emulating hardware for hardware virtualized machines. Table 1 shows the guest operating system support matrix for Oracle VM for x86. 2.2.x

Guest Operating System *Oracle Linux 6.x Oracle Linux 5.x

Hardware Paravirtualized Paravirtualized Virtualized 32-bit 64-bit 32-bit * * * * * *

Hardware Virtualized 64-bit * *

4 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Oracle Linux 4.x *Red Hat Enterprise Linux 6.x Red Hat Enterprise Linux 5.x Red Hat Enterprise Linux 4.x Red Hat Enterprise Linux 3.x Microsoft Windows 2000 Microsoft Windows 2003 Microsoft Windows XP Pro Microsoft Windows Vista Microsoft Windows 7 Microsoft Windows 2008 SP1 Microsoft Windows 2008 R2 Oracle Solaris 10 10/09+ Oracle Solaris 11 Express

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

*

* *

* *

*

*

* *

* *

*

*

*

* *

* *

* Oracle VM 2.2.x does not have native ext4 support, therefore Oracle Linux 6 and Red Hat Enterprise Linux 6 virtual machines must use ext3 for the /boot partition to be able to boot.

Oracle VM for x86 Packaging and Pricing
Oracle VM for x86 is not a licensed Oracle technology product. Support for Oracle VM for x86 is available as an add-on component of Oracle’s enterprise support package. Support for Oracle VM for x86 is available with Sun x86 hardware as an add-on Premier Support for Systems package. Oracle VM for x86 support for third-party hardware is sold as an add-on support component of Oracle’s enterprise support package. Premier Support for Systems costs 12% of the net Sun system purchase price and includes comprehensive support for the system hardware and firmware, as well as operating system support for Solaris x86, Solaris 11 Express, Oracle Linux, and Oracle VM for x86. Along with hardware and firmware support, Premier Support for Systems includes operating system support for one or more virtual instances Oracle Linux, Solaris 10 x86, and Solaris 11 Express x86 running on Oracle VM for x86. Support for Oracle VM for x86 for third-party hardware is sold in two packages: Oracle VM Premier Limited support for servers with up to two sockets and Oracle VM Premier support for servers with more than two sockets. Support for Oracle VM for x86 for third-party hardware only includes support for Oracle VM for x86. Operating System support for Oracle Linux, Red Hat Enterprise Linux, Solaris x86, Solaris 11 Express, and Windows virtual machines running on Oracle VM for x86 must be purchased separately. The next section will examine how to accelerate application deployments using the Oracle VM for x86 product line and add-on products.

Accelerating Application Deployments with Oracle VM for x86
As mentioned in the Oracle VM for x86 introduction, the Oracle VM for x86 product line includes comprehensive virtual machine and application packaging and deployment options that accelerate application deployments by eliminating time-consuming operating

5 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

system and application installations. Oracle offers five unique options for accelerating application deployments using the Oracle VM for x86 product line: List 1: Five options for accelerating application deployments using Oracle VM for x86. 1. 2. 3. 4. 5. Virtual Machine Cloning Oracle VM Templates Oracle VM Template Builder Oracle Virtual Assembly Builder Oracle JRockit Virtual Edition

The next sections will examine virtual machine cloning, Oracle VM Templates, Oracle VM Template Builder, Oracle Virtual Assembly Builder, and will conclude with Oracle JRockit Virtual Edition.

Virtual Machine Cloning
Operating system and application installations can be very time consuming. Virtual machine cloning using Oracle VM Manager can help accelerate application deployments by eliminating the entire operating system installation and configuration process. Applications that can tolerate host name and IP changes can be included within clones and can be rapidly redeployed an unlimited number of times as copies of the original virtual machine. Virtual machine cloning accelerates application development and testing by reducing the time required to deploy a clean operating system environment for application testing. For example, using Oracle VM Manager, a virtual machine can be saved as a clone or saved as a template and then the clone or the template can be rapidly redeployed an unlimited number of times as copies of the original virtual machine or as unique new virtual machines. Note: The minimum requirement to create and Oracle VM templates is an Oracle VM server managed by Oracle VM Manager. Virtual machine cloning is an Oracle VM Manager image management feature. The Oracle VM Manager clone and save-as-template features offer sparse and nonsparse virtual machine image provisioning. The term “sparse” is commonly referred to as “thin provisioning,” in contrast to nonsparse, which is referred to as “thick provisioning”. A thin-provisioned clone will not write the parent virtual machine's zeroed blocks to disk, whereas a thick clone will copy all of the parent virtual machine's blocks to disk. Thin-provisioned disks grow proportionally to the number of writes to the disk by the virtual machine, so that large portions of the unused disk do not consume space. As of this writing, both thin and thick clone provisioning are off-line operations, meaning that the parent virtual machine must be powered off to create a clone. The advantage of selecting thin provisioning for clones is that the storage is allocated only when needed, which reduces the time it takes to create clones in addition to saving disk space. The disadvantage of using thin provisioning is that the file system free-space reports may be misleading. For example, since thin-provisioned disks are allocated only when needed, the file system free space reports may not be accurate since large portions of unused disk, that is, the zeroed blocks, have not yet been written to disk. Tip: Some applications do not support copying sparse files and may copy the entire uncompressed size of the file including the sparse sections. For more information on virtual machine cloning please refer to Chapter 5. For more information about Oracle VM Storage, please refer to Chapter 6.

Oracle VM Templates
Operating system and Oracle application installations can be very time consuming. The Oracle VM for x86 Template program was created to accelerate Oracle application deployments by offering self-contained, preconfigured virtual machines with key Oracle technologies. Oracle VM templates can be downloaded from the Oracle Linux eDelivery portal and rapidly deployed an unlimited number of times using Oracle VM for x86. The minimum requirement to run Oracle VM Template Builder and Oracle VM templates is an x86-64 server with Oracle VM for x86, Oracle VM Manager is optional although highly recommended. List 2: The steps to download and deploy an Oracle VM template. Select and Download the desired Oracle VM template from the Oracle Linux eDelivery portal. 1. 2. 3. 4. 5. Copy the Oracle VM template zip file to your Oracle VM server's /OVS/running_pool directory. Unzip and untar the files in the /OVS/running_pool directory. Import the Oracle VM Template using Oracle VM Manager. Start the Oracle VM template using Oracle VM Manager. Access the Oracle VM template's console and select the initial boot-time options.

An Oracle VM template can consist of one or more virtual machines containing a preconfigured Oracle operating system and Oracle technology products. Using Oracle VM templates eliminates the operating systems and Oracle application installation and configuration process, which dramatically shortens application deployment timelines. There is no other vendor in the x86 virtualization market that can offer pre-packaging of production ready enterprise applications.

6 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

List 3: Oracle’s Oracle VM templates collection as of this writing. Virtualization and Management Oracle VM Manager 2.2 and 2.1.5 Oracle VM Template Builder 2.1 Oracle VM Server 2.2.1 and 2.1.5 SDK Sun Ray Software 5 Oracle Secure Global Desktop Oracle Enterprise Manager Grid Control 10g R5 (10.2.0.5) Oracle Enterprise Manager Grid Control 11g R1 (10.2.0.5) Applications E-Business Suite 12.1.3 E-Business Suite 12.1.1 E-Business Suite 12.X Sparse Middle Tier JD Edwards EnterpriseOne 9.0 Update 1 with ESUs and JD Edwards EnterpriseOne Tools 8.98 Update 3 JD Edwards EnterpriseOne 9.0 Update 1 and JD Edwards EnterpriseOneTools 8.98 Update 2 PeopleSoft ELM 9.1 Bundle #2 with PeopleTools 8.50.09 PeopleSoft FSCM 9.1 Bundle #4 (includes Maintenance Pack 2) with PeopleTools 8.50.10 PeopleSoft CRM 9.1 Bundle #2 with PeopleTools 8.50.09 PeopleSoft Portal Solutions 9.1 and PeopleTools 8.50.09 PeopleSoft HCM 9.1 and PeopleTools 8.50.02 Siebel CRM SIA 8.1.1 Siebel CRM SIA 8.0 Middleware Oracle Oracle Oracle Oracle Oracle Oracle WebLogic Server on JRockit Virtual Edition 11g R1 (10.3.2) WebLogic Server 10g Release 3 Business Intelligence Enterprise Edition 10.1.3.4 Application Server 10g Release 3 WebCenter Identity Management 10g Release 2 Fusion Middleware Service Oriented Architecture (SOA) 10.1.3.4 and 10.1.3.3

Database & Real Application Clusters (RAC) Oracle Real Application Clusters (RAC) 11g Release 2 Oracle Real Application Clusters (RAC) 11g Release 1 Oracle Database 11g and Oracle Database 10g Release 2 Operating System Oracle Linux 4 and 5 Oracle Solaris 10 Third-party Software Zeus Traffic Manager

Oracle VM Templates Packaging and Pricing
Oracle VM templates can be downloaded from the Oracle Linux eDelivery portal. An Oracle VM template license includes a free download and free trial use of the Oracle technologies with the option to purchase a product license. Oracle VM templates do not have time limits or feature limitations. Oracle VM templates can be quickly transitioned from evaluations into test, development. or production by purchasing the Oracle technology product license. Oracle frequently updates Oracle VM templates and adds new Oracle VM templates to the Oracle Linux eDelivery portal. The best way to stay up to date about Oracle VM templates is to visit ITNewsCast's Oracle VM section. For information on how to get and deploy Oracle VM templates, please refer to Chapter 8 and Chapter 9.

Oracle VM Template Builder
Oracle VM Template Builder is a free add-on product for Oracle VM for x86 that packages Oracle and non-Oracle applications on

7 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Oracle Linux JeOS (just enough operating system) as a preconfigured, reusable Oracle VM template. Oracle Linux JeOS is a prepackaged, customizable small-footprint x86 and x86-64 Oracle Linux operating system. Deploying preconfigured Oracle VM templates eliminates the entire operating system and application installation and configuration process. Oracle VM Template Builder dramatically shortens application deployments by packaging unique application configurations within preconfigured Oracle VM templates that can be rapidly deployed one or more times using Oracle VM for x86. The minimum requirement to run Oracle VM Template Builder and Oracle VM templates is an x86-64 server with Oracle VM for x86, Oracle VM Manager is optional although highly recommended. Note: Oracle VM Template Builder is the application Oracle uses to create Oracle VM templates. Oracle VM Template Builder templates are designed to be configured at the initial boot time to ensure that each template has unique operating system and application configurations. The initial boot time configuration selections can be default values and/or actions based on the user’s input. Note: Oracle Linux JeOS is shipped with only English language support. Additional language packages for JeOS are available from Oracle Linux eDelivery portal. Oracle VM templates can be built and used by anyone including Oracle, ISVs, Oracle Partners, VARs, and end users. JeOS Oracle VM templates can be redistributed without an agreement from Oracle. Redistributing templates with applications may or may not be allowed. It is necessary to consult with the ISV to determine if their application can be legally redistributed within an Oracle Linux JeOS template. Packaging software applications with Oracle VM Template Builder requires a thorough understanding of the software application's behavior. This particularly applies to the Oracle VM JeOS template reconfiguration and cleanup scripts used in the Oracle VM JeOS template. Creating Oracle VM JeOS templates is a multistep process. The number of steps to create an Oracle VM template depends on the type and complexity of the application software added to the Oracle VM JeOS template.

Oracle VM Builder Packaging and Pricing
Oracle VM Template Builder is distributed as an RPM package and as a Oracle VM template from the Linux eDelivery portal. Oracle Linux JeOS is distributed with Oracle VM Template Builder and as RPMs from the Linux eDelivery portal. Oracle VM Template Builder and Oracle's JeOS are open source, free to download, free to use, and free to redistribute.

Oracle Virtual Assembly Builder
Oracle Virtual Assembly Builder is an add-on product for Oracle VM for x86 that can profile and package reference physical or virtual multitier Oracle Fusion Middleware environments as prepackaged Oracle VM templates, which are called assemblies. Assemblies contain preconfigured JeOS virtual machines with configurable initial boot-time metadata that allow each assembly to have unique operating system and application configurations. The initial boot-time configuration selections can be default values and/or actions based on the user’s input. Note: Introspection is Oracle's terminology for application profiling. Assemblies are created and managed using the Oracle Virtual Assembly Builder Studio, which is a standalone administrative application. Oracle Virtual Assembly Builder Studio integrates with Oracle VM for x86 using the Oracle VM Manager API. The minimum requirements for Oracle Virtual Assembly Builder are a) Oracle Virtual Assembly Builder Studio, b) Oracle VM for x86 server pool, c) Oracle VM Manager, and d) a physical and/or virtual reference Oracle Fusion Middleware environment to introspect. The current release of Oracle Virtual Assembly Builder is 11g Release 1 (11.1.1). Oracle Virtual Assembly Builder 11g Release 1 works exclusively with Oracle VM for x86 2.x. Oracle describes the this release as a developer-centric release, as opposed to an enterprise release, due to the lack of enterprise security features, such as root access to files and directories and the absence of role-based access. The next release of Oracle Virtual Assembly Builder should be an enterprise release with the enterprise security features, broader Oracle Fusions Middleware application support, and Oracle VM 3.0 integration.

Oracle Virtual Assembly Builder Packaging and Pricing
Oracle Virtual Assembly Builder is distributed from the Oracle Technology Network (OTN). Oracle Virtual Assembly Builder is supported on 32-bit version of Oracle Linux and Red Hat Enterprise Linux 5. Oracle Virtual Assembly Builder is not a standalone Oracle technology product that can be purchased a la carte. The right to use Oracle Virtual Assembly Builder is bundled with WebLogic Enterprise Edition. The only way to purchase Oracle Virtual Assembly Builder is by purchasing WebLogic Enterprise Edition licenses. Note: I intend to add a chapter about the next enterprise release of Oracle Virtual Assembly Builder. Please stay tuned to ITNewsCast.com and the underground Oracle VM Manual for the new Oracle Virtual Assembly Builder chapter.

8 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Oracle JRockit Virtual Edition
Oracle JRockit Virtual Edition is an add-on product for Oracle VM for x86 that is packaged as a virtual appliance that runs exclusively on Oracle VM for x86. Oracle JRockit Virtual Edition appliances contain a full featured implementation of Oracle WebLogic Server 11g that runs directly on the hypervisor with the help of a thin OS layer with networking, hardware device interaction, I/O, and process scheduling. Oracle JRockit Virtual Edition is a single-user, single-process system that is designed to run any JDK 6 compatible application with full support for domains and clusters. With Oracle JRockit Virtual Edition, mission critical Java applications can be quickly deployed or migrated from physical or third-party virtual environments to Oracle JRockit Virtual Edition virtual appliances with unparalleled density, performance, and ease of management. Figure 2 shows an Oracle JRockit Virtual Edition appliance, highlighting the JRockit Virtual Edition OS layer.

The Oracle JRockit Virtual Edition OS layer contains a purpose-built kernel that contains only the essential services to run Oracle WebLogic Server 11g with JDK 6 compatible applications. Oracle WebLogic Server 11g on Oracle JRockit Virtual Edition offers up to 30% better performance for Java applications compared to WebLogic deployments using a traditional operating system, and has a footprint up to five hundred times smaller than that of WebLogic deployments using a traditional operating system. Without a traditional operating system, JRockit Virtual Edition eliminates the operating system installation, management, security, and storage requirements as well as operating system license and support costs. To illustrate just how thin the Oracle JRockit Virtual Edition OS layer is, Table 2 compares a Linux operating system to JRockit Virtual Edition. Numbers Administration Utilities Administrative Commands Linux +-500 +-3000 JRockit Virtual Edition 1 10

Command Parameters +-50000 100 Disk Image Size +-10 GB 1 GB

Oracle JRockit Virtual Edition uses the same administrative infrastructure, tools, and WLST scripts as a traditional WebLogic deployment. Each Oracle JRockit Virtual Edition appliance ships with an administrative server equipped with WebLogic Server Administration Console. Oracle JRockit Virtual Edition appliances support both off-line and on-line management of applications running in WebLogic instances. Oracle JRockit Virtual Edition is intended to be a deployment platform. Oracle recommends that Java application development be done on traditional WebLogic platforms. Once the application is ready to be deployed into production, it can be moved from the WebLogic development system to an Oracle JRockit Virtual Edition appliance.

Oracle JRockit Virtual Edition Packaging and Pricing
Oracle JRockit Virtual Edition is distributed as a virtual appliance from the Oracle eDelivery portal. Oracle JRockit Virtual Edition is not a standalone Oracle technology product that can be purchased a la cart. The right to use Oracle JRockit Virtual Edition is bundled with WebLogic Enterprise Edition. The only way to purchase Oracle JRockit Virtual Edition is by purchasing WebLogic Enterprise Edition licenses.

The Oracle VM Management Pack Plug-in
The Oracle VM Management Pack Plug-in adds Oracle VM Manager functionality to Oracle's system management solution, Oracle Enterprise Manager (OEM). The Oracle VM Management Pack Plug-in is a complete port of Oracle VM Manager to Oracle Enterprise Manager 10g and 11g. Along with the Oracle VM Manager feature set, the Oracle VM Management Pack Plug-in offers advanced monitoring, provisioning, patch management, and configuration management. Oracle VM servers can only be managed from one or

9 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

the other of the two management options, Oracle VM Manager or the Oracle VM Management Pack Plug-in, not both. Note: The Oracle VM Management Pack is licensed software; Oracle VM Server and Oracle VM Manager are not licensed software. Figure 3 shows the Oracle Enterprise Manager administrative console.

It's difficult to compare a full-featured system management solution like Oracle Enterprise Manager with a single-product management solution like Oracle VM Manager, although for completeness Table 3 contrasts the high-level features of the Oracle Enterprise Manager with those of the Oracle VM Management Pack Plug-in and Oracle VM Manager. Capability Oracle VM OEM / Manager Oracle VM Manager Plug-in *

Oracle VM server, Oracle VM pool, virtual machine, and virtual machine resources management, quality monitoring and provisioning. *Oracle Linux, Red Hat Enterprise Linux, Solaris and Oracle VM server provisioning and patch management. Database and application administration, quality management and provisioning. Configuration Management, i.e. tracking of configuration drifts. **Parse Oracle VM server and virtual machine operating system configuration files to track changes/differences and send alerts/notification on when changes are made. Scriptable command Line Interface. *

*

*

*

*

*

* Oracle Enterprise Manager does not support Oracle VM server patch management. ** The Application Change Console. The Application Change Console is a separate standalone application that is licensed with the Oracle VM Management Pack Plug-in. The Application Change Console (ACC) is able to parse configuration files to track changes and differences and send alerts and notifications when changes are made. The Application Change Console has prebuilt parsers for Operating System configuration files.

10 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

After installing the Application Change Console, you can point the Application Change Console to the Oracle VM servers and Oracle VM guests using hostnames. The Application Change Console will connect to a host via SSH to gather and track configurations. No additional agents are required. All of the Application Change Console components can run on Oracle Linux on Oracle VM.

Oracle's x86 Virtualization Support Policy
Since the first release of Oracle VM for x86, which was announced at Oracle Open World in November 2007, Oracle has provided certified support for Oracle technology products on Oracle VM for x86. Oracle VM for x86 is the only x86 virtualization solution that Oracle tests and certifies and is the only one supported by Oracle for Oracle technology products. Oracle does not test, certify, or support Oracle technology products on any third-party x86 virtualization solutions. For example, Oracle does not certify or test Oracle technology products on VMware ESX, Microsoft Hyper-V, Citrix Xen Server, Red Hat with Xen, or KVM or on any Xen or KVM-enabled Linux system. Oracle maintains an up-to-date list of Oracle certified software on Oracle VM via Metalink Doc ID: 464754.1. Oracle’s support policy, regarding Oracle technologies and virtualization, can be referenced via MetaLink documents 794016.1 and 249212.1. In short, MetaLink documents 794016.1 and 249212.1 explain that service requests (SRs) involving uncertified virtualization solutions will receive best effort for known issues on physical systems. Oracle may request that the customer reproduce the problem on native hardware if the third-party virtualization software cannot be ruled out as the root cause of the problem. Major ISVs, for example, Microsoft and IBM, have virtualization support policies similar to Oracle’s. Microsoft and IBM may request that customers reproduce problems on native hardware if the third-party virtualization software cannot be ruled out as the root cause of a problem. Microsoft: "As part of the investigation, Microsoft may require the issue to be reproduced by the customer independently from the non-Microsoft hardware virtualization software." IBM: “Will IBM correct all defects for IBM software products that are running in a supported virtualization environment? A: Not necessarily. As with other operating environments, IBM does not warrant that all code defects will be corrected. IBM will issue defect correction information, a restriction, or a bypass to IBM products if the defect is applicable to a supported virtualized environment or to a native physical machine environment (that is, without the virtualization software).” Watch this space Oracle OpenWorld 2011 Call for Papers - Submit Today! Temporary Post Used For Theme Detection (409fc5e9-d7ee-4e1a-b645-8b82f256f434 - 3bfe001a-32de-4114-a6b4-4005b770f6d7) Debugging Oracle VM 2.2 & Oracle VM Manager 2.2 errors using the many log files New VDI Training Course now available! HIMSS 2011 and New Press Release Oracle Desktop Virtualization at HIMSS 2011 Oracle VM VirtualBox 4.0.4 Released! Two Virtualization Webinars This Week We're Hiring! - Server and Desktop Virtualization Product Management

Chapter 2: Oracle VM for x86 Reference Design and Architectural Introduction
Copyright © 2009 - 2011 Roddy Rodstein. All rights reserved. Show a printer friendly version of this book with its sub pages

The chapter starts with an architectural review of Oracle VM for x86 and ends with an introduction to the Oracle VM for x86 reference design. The goal of this chapter is to articulate the architectural and design considerations for Oracle VM for x86 in order to provide the technical foundation for using the Oracle VM for x86 reference design. Note: This chapter replaces the original “Chapter 2: Oracle VM Architectural Review” that is still available as an Oracle VM for x86 architectural reference. Table of Contents Change Log Oracle VM Server and Xen Architectural Review ...Oracle VM for x86 Server Specifications Oracle VM Manager Architectural Review Oracle VM Agent Oracle VM for x86 Networking Oracle VM for x86 Server Pools, HA and Live Migration Oracle VM for x86 Storage Oracle VM for x86 Intra-component Communication & Firewall Requirements 1.0 – The Oracle VM for x86 System Reference Design Introduction ...1.1 – The Oracle VM for x86 System Reference Design Implementation Overview ...1.2 – The Oracle VM for x86 System Reference Design Document Overview 2.0 – The Oracle VM for x86 System Reference Design Architectural Overview ...2.1 - The Oracle VM for x86 System Reference Design Support Infrastructure Part 1 - The Oracle VM for x86 System Reference Design 3.0 - Cloud Infrastructure Architecture

11 of 243

18.04.2011 16:35

The Underground Oracle VM Manual
...3.1 ...3.2 ...3.3 ...3.4 ...3.5 ...3.6 ...3.7 ...3.8 Oracle Oracle Oracle Oracle Oracle Oracle Oracle Oracle VM VM VM VM VM VM VM VM for for for for for for for for x86 x86 x86 x86 x86 x86 x86 x86 Hardware Architecture Server Pool Configurations Network Topology Network Security Architecture Administration and Monitoring Virtual Machine Operating Systems Application Integration Support Service

http://itnewscast.com/book/export/html/844

Change Log Revision Change Description 1 1 New chapter 3.4 – Oracle VM for x86 Network Security Architecture Added "SSH login banners" Updated By Roddy Rodstein Roddy Rodstein Date 04/03/11 04/11/11

1

3.5 - Oracle VM Roddy for x86 Rodstein Administration and Monitoring Added "Rootkit prevention and monitoring"

04/11/11

Oracle VM Server and Xen Architectural Review
Oracle VM for x86 consists of an x86 server component and a manager component that is used to manage one or more clustered servers. The server component is a type 1 hypervisor based on the Xen.org open source hypervisor, which is called the Oracle VM server. Xen is considered a type 1 hypervisor because Xen installs directly on hardware, whereas a type 2 hypervisor, such as Oracle VM VirtualBox, is a software applications installed on top of a desktop operating system. The Xen hypervisor is responsible for monitoring virtual machine operating systems and for scheduling virtual machine operating system requests to the physical server's CPU and memory. Xen is not responsible for and has no knowledge of networking, external storage, video, or common I/O functions. The Xen hypervisor works in concert with a control domain, which, in Xen terminology, is called dom0 or domain 0. Xen and dom0 are automatically started when an Oracle VM server boots. Dom0’s role includes providing an interface to Xen, hardware detection, device support, domU management, and presenting domUs with networking, storage, video, and I/O interfaces. Xen and dom0 enable multiple concurrently running virtual machines to share a single piece of hardware. In Xen terminology, unprivileged domains are called domU or domain U; the industry term for domU is virtual machine. Dom0 and domUs are all virtual machines. Oracle makes subtle changes to the Xen.org code that create a unique Xen distribution, which is redistributed as Oracle VM server. Oracle has developed a Xen/dom0 configuration for Oracle VM that supports the most demanding high I/O workloads that other Linux distributions with Xen do not use. For example, each Linux distribution ships with Xen as an operating system virtualization feature. When Xen is enabled on a Linux system, the entire Linux operating system, including all of the user space applications, is turned into dom0. Xen-enabled Linux systems do not have an optimized Xen or dom0 configuration. In contrast to Xen/Linux, Oracle VM server is a deliberately built virtualization platform, specifically designed for high I/O workloads such as Oracle's Database and Oracle Fusion Middleware workloads. Oracle VM server supports two unique virtualization modes, paravirtualization mode (PV mode) and hardware virtualization mode (HVM mode). Oracle VM servers can support both paravirtualization mode and hardware virtualization mode simultaneously on a single x86-64 server that has either Intel or AMD virtualization technologies. Intel and AMD virtualization is a requirement only for hardware virtualization mode, not for paravirtualization mode. Intel and AMD virtualization technologies are enabled and managed using the system BIOS. Paravirtualization mode requires the virtual machine operating system to run a Xen kernel and Xen network and I/O drivers. Xen paravirtualized guest kernels are available for the Oracle Linux and Red Hat Enterprise Linux operating systems. Paravirtualized virtual machines are hypervisor aware and run without the additional overhead of hardware emulation. Paravirtualization requires much less overhead for timers, interrupts, I/O traffic, and context switches, allowing superior scalability under heavy loads, when compared to hardware virtualization mode. Unlike paravirtualization mode, which requires the virtual machine to run a Xen kernel, hardware virtualization mode supports unmodified operating systems. Virtual machines that run under hardware virtualization mode are called “hardware virtualized machines” (HVM). Hardware virtualized machines are unaware that they have been virtualized and think they are on physical hardware. To provide acceptable performance, hardware virtualized machines should use paravirtualized network and I/O drivers.

12 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

From Oracle Linux and Red Hat Enterprise Linux 4.7 onwards, the stock kernels have provided paravirtualized network and I/O drivers for hardware virtualized guests. From Solaris 10 10/09 onwards, the stock kernels have provided paravirtualized network and I/O drivers for hardware virtualized machines. Windows does not have native paravirtualization support, although Windows virtual machines can run as hardware virtualized machines using Oracle's paravirtualized network and I/O drivers. Oracle has released paravirtualized network and I/O drivers for the Windows operating system that can be freely downloaded from the Oracle Linux eDelivery portal. Paravirtualization and hardware virtualization modes use very different techniques to provide resources to virtual machines. For example, hardware virtualization mode uses Intel or AMD virtualization technologies for memory management and to emulate the boot environment for hardware virtualized machines. Hardware virtualization mode also uses QEMU in dom0 for device emulation for hardware virtualized machines. Paravirtualization mode leverages the guest operating system's Xen kernel for the boot process using the pygrub bootloader, Xen for memory management, and dom0 without QEMU for device support. With paravirtualization mode, dom0 multiplexes native Linux devices for paravirtualized domUs. dom0 runs the back-end drivers, and domU runs the paravirtualized front-end drivers in a back-end dom0, front-end domU driver model. For networking and I/O, dom0 runs a network back-end driver and a block back-end driver to support network and I/O requests for paravirtualized domUs. The network back-end driver communicates through dom0 to the local hardware device to process all domU networking requests. The block back-end driver communicates through dom0 to the local storage to process all domU storage requests. Dom0 runs a dedicated quemu-dm process for each hardware virtualized machine. Hardware virtualized machines do not use the same dom0 back-end and domU front-end drivers as paravirtualized domUs. Hardware virtualized machines use emulated drivers that are created and managed by the quemu-dm processes in dom0. Hardware virtualization mode has a higher overhead than paravirtualization mode, due in part to the CPU overhead of emulating hardware for hardware virtualized machines. Figure 1 illustrates the Xen architecture explained above, highlighting the hypervisor, dom0, QEMU, two paravirtualized domUs, one hardware virtualized domU, and the direct and virtual I/O paths.

Tip: The only way to determine which virtualization mode will provide the best performance for your environment is to benchmark the same workload using the same operating system in paravirtualization mode and in hardware virtualization mode. If you do not have the time or expertise to conduct the benchmarks, consider only using paravirtualization mode for your virtual machines. To better understand the capabilities of Oracle VM server, List 1 highlights the technical specifications of Oracle VM server.

Oracle VM for x86 Server Specifications
Supported Platforms x86 or x86_64 processors with PAE (Physical Address Extension) support Minimum Processor Class I686 class Minimum Memory 1GB Maximum Memory 1TB CPUs Supporting Paravirtualization Mode Intel Pentium-PRO or newer AMD Athlon/Duron or newer

13 of 243

18.04.2011 16:35

The Underground Oracle VM Manual
At least a Pentium IV or Athlon CPU recommended CPUs Supporting Hardware Virtualization Mode

http://itnewscast.com/book/export/html/844

Some Intel Pentium D, Core, Core2 and Xeon models (/proc/cpuinfo should list "vmx" among the flags) Some AMD Athlon and Opteron models (/proc/cpuinfo should show "svm" among the flags) Maximum Number of CPUs or Threads 128 CPUs or 128 threads Boot Options Local disk, SAN, iSCSI, NFS, flash Oracle VM for x86 Virtual Machine Specifications Maximum Number of Virtual CPUs per Virtual Machine 32 virtual CPUs Maximum Amount of Memory per Virtual Machine x86 (32-bit): 63GB x86_64 (64-bit): 510GB Maximum Number of Virtual Disks per Virtual Machine Paravirtualized Mode: 20 hda, 26 sda, 5 xvd Hardware Virtualization Mode: 4 hda, 7 sda, 5 xvd *Each Oracle VM Server can support a maximum of 128 virtual disks Maximum Number of Virtual NICs per Virtual Machine Paravirtualized Mode: 31 Hardware Virtualization Mode: 8 List 1 shows that an Oracle VM server supports a total of a 1TB of RAM and a total of 128 CPU threads. An Oracle VM server with a 1TB of RAM and 128 CPU threads could allocate the majority of the 1TB of RAM and more than 128 CPU threads to virtual machines. Oracle VM server supports CPU oversubscription, which means that an Oracle VM server with 128 CPU threads can overallocate the total number of CPU threads to virtual machines. Oracle VM server does not support memory oversubscription, which means that an Oracle VM server with 1TB of RAM cannot overallocate RAM. By default, each Oracle VM server reserves 512MB of memory for dom0. The average memory overhead for each running guest on a dom0 is approximately 20MB plus 1% of the guest’s memory size. The remaining physical memory can be allocated to guests. Avoid oversubscribing CPU-bound workloads such as the Oracle Database. CPU oversubscription with CPU-bound workloads negatively effects performance and availability. CPU oversubscription for non-CPU-bound workloads, such as Oracle Fusion Middleware products, is recommended. It is common to oversubscribe CPU cores 3-to-1 with non-CPU-bound workloads. For example, each CPU core could allocate 3 virtual CPUs for non-CPU-bound workloads without a performance penalty. The maximum amount of RAM and CPU cores that an Oracle VM server with a 1TB or RAM and 128 CPU cores could allocate to a single virtual machine is 32 virtual CPUs with 510GB of RAM. The minimum amount of RAM and CPU cores that an Oracle VM server with a 1TB or RAM and 128 CPU threads could allocate to a single virtual machine is 1 virtual CPUs with as little as 256MB of RAM. Note: A virtual machine cannot aggregate CPU and RAM resources from more than one server. That is, virtual machines consume resources only from the host where the virtual machine was started. Oracle VM for x86 consists of an x86 server component and a manager component used to manage one or more clustered servers. Oracle VM server enables multiple concurrently running virtual machines to share a single piece of x86 hardware. The manager component is a traditional Oracle application, named Oracle VM Manager. Oracle VM Manager facilitates centralized management of one or more clustered Oracle VM servers, virtual machines, and virtual machine resources. The next section provides an architectural review of Oracle VM Manager.

Oracle VM Manager Architectural Review
Oracle VM Manager is the default “no cost management component” for Oracle VM for x86. Oracle VM Manager is a traditional Oracle application that installs on Oracle Linux and Red Hat Enterprise Linux. It is used to manage one or more clustered Oracle VM servers, virtual machines, and virtual machine resources. Oracle VM Manager consists of an Oracle database, an Oracle application server, and a J2EE application with an OS and browser neutral Oracle Application Development Framework (ADF) administrative portal. Oracle VM Manager also has a command line interface that allows Oracle VM Manager administrative tasks to be performed from the command line or to be executed as scripts. Oracle VM Manager is distributed from the Oracle Linux eDelivery portal as an ISO file and as a preconfigured, production-ready

14 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Oracle VM template. Oracle VM Manager can be installed in an all-in-one configuration using the default Oracle 10g Express Database or in a more traditional two tier architecture with an OC4J web tier and a 10 or 11g database tier. The Oracle VM Manager template is an example of an all-in-one installation with the Oracle 10g Express Database, an OC4J application server, and the Oracle VM Manager application all installed on the same operating system. Table 1 lists the packaged applications in the Oracle VM Manager ISO file.

Application Oracle Database 10g Express Edition

Capability The Oracle Database 10g Express Edition is packaged in the Oracle VM ISO.

Oracle VM The Oracle VM Manager package contains Manager package the Oracle VM Manager web application. The Oracle VM Manager web application runs on Oracle Application Server 10g Release 3 (10.1.3) and is deployed into an OC4J container. Oracle Containers for J2EE (OC4J) Standalone 10.1.3 packaged with Application Development Framework (ADF) 10.1.3.3 XML-RPC 3.0 Oracle Containers for J2EE (OC4J) is the J2EE runtime environment for the Oracle Application Server. Oracle Application Development Framework (ADF) is a Java EE framework.

The XML-RPC 3.0 package is installed during an Oracle VM Manager installation. XML-RPC is used for Oracle VM Manager to Oracle VM agent communication.

The total number of Oracle VM servers that Oracle VM Manager can manage is limited by the type of database repository used by Oracle VM Manager. The default Oracle Database 10g Express Edition has a 4GB limit for on-disk storage, which is a bottleneck in supporting Oracle VM environments with more than 50 Oracle VM servers. Using Oracle Standard Edition or Oracle Enterprise Edition eliminates the 4GB limit for on-disk storage. For example, if your Oracle VM Manager database repository is not using Oracle Database 10g Express Edition but an Oracle Standard or Enterprise Edition database on a reasonably sized server with a gigE networking, Oracle VM Manager could easily scale up to 1000+ Oracle VM servers with thousands of virtual machines. Tip: The Oracle VM Manager application runs much faster using an Oracle Standard or Enterprise Edition database. The Oracle VM Manager application is a great candidate for virtualization on Oracle VM and Oracle VM VirtualBox. Virtualizing Oracle VM Manager saves on hardware costs and improves application flexibility while reducing data center space. The Oracle VM Manager template can be used to quickly deploy a production all-in-one Oracle VM Manager installation as a virtual machine, without the need to dedicate a server for Oracle VM Manager. For a distributed Oracle VM Manager installation, the web tier and the database tier can both be virtualized on Oracle VM using Oracle Linux virtual machines. Note: Oracle VM Manager is not supported and should not be installed in an Oracle VM server's dom0.

Oracle VM Agent
Figure 2 illustrates the Xen architecture with the addition of the Oracle VM Manager. Oracle VM Manager is running on a virtual machine with an all-in-one Oracle VM Manager installation, managing a server pool with one Oracle VM server hosting three virtual machines.

15 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Oracle VM Manager dispatches administrative commands made in the Oracle VM Manager portal to the Oracle VM agent in dom0. The Oracle VM agent executes the Oracle VM Manager commands in dom0 using python scripts located in the /opt/ovs-agent-latest/ directory. Oracle VM Manager facilitates centralized management of server pools and their resources using an agent-based architecture. When an Oracle VM server is added to a server pool, one or more Oracle VM agent roles are assigned to the Oracle VM server. There are a total of three Oracle VM agent roles; 1) the Server Pool Master, 2) the Utility Server and 3) the Virtual Machine Server. When an Oracle VM server is added to a new server pool, it can be assigned one, two, or all three of the agent roles. List 2 explains each of the three Oracle VM agent roles. Server Pool Master The server pool master is the principal server pool role within a server pool. The server pool master is the only agent role that communicates with Oracle VM Manager. The server pool master dispatches commands received from Oracle VM Manager to other servers within a server pool. There can be only one server pool master in a server pool at any instant. The server pool "Virtual IP" feature in Oracle VM Manager 2.2 and above will detect the loss of the server pool master and automatically failover the pool master server role to the first pool member that can lock the cluster. Tip: To eliminate a single point of failure, always enable the Virtual IP feature. The Virtual IP feature requires a unique IP address. If you have enabled the Virtual IP feature and do not know which server is the server pool master, ssh to the Virtual IP to validate which host is the server pool master. Utility Server The utility server role is responsible for I/O-intensive operations such as virtual machine creation and removal and server pool creation and removal, as well for as copying and moving guest files. The server pool master dispatches operations to utility server agents. There can be one or more utility server agents in a server pool. When there are multiple utility server agents in a pool, the server pool master will select the least loaded utility server to conduct a task. Tip: For production Oracle VM systems, use dedicated utility servers to isolate the impact of I/O intensive operations to utility servers. For example, colocating the utility server agent with the virtual machine server agent will affect guest performance during utility server operations. Virtual Machine Server Servers with the virtual machine server agent role are responsible for allocating CPU, memory, and disk resources to the virtual machines in a server pool. There can be one or more virtual machine servers in a server pool. Oracle VM Manager dispatches commands using XML RPC to each server pool master agent, which, in turn, dispatches commands to other pool members over a dedicated management interface using XML RPC. Oracle VM agent architecture is exceptionally bandwidth efficient, since the intracomponent traffic is limited between a) Oracle VM Manager and each server pool master agent and b) each server pool master agent and its server pool members. For example, an Oracle VM environment with 20 server pools, each server pool having 20 servers, would have a total of 20 communication channels between Oracle VM Manager and each server pool master. If Oracle employed a direct Oracle VM Manager to Oracle VM agent relationships, the same Oracle VM environment with 20 server pools, each server pool having 20 servers, would have a total of 400 communication channels.

Oracle VM for x86 Networking
The default Oracle VM server networking configuration routes all dom0 and virtual machine traffic through a Xen bridge. A Xen

16 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

bridge operates at layer 2 of the OSI model, effectively acting as a layer 2 (L2) switch passing packets to the egress port; it relies on the TCP protocol for rate control and packet loss. Oracle VM's default network configuration pairs each network interface (NIC) with a Xen bridge. An Oracle VM server with two NICs will have two Xen bridges, eth0/xenbr0 and eth1/xenbr1. The first Xen bridge, eth0/xenbr0, is configured with an IP address on xenbr0 and is used for Oracle VM management traffic. The second Xen bridge will not have an IP address assigned; it effectively acts as a layer 2 switch for guest traffic. The default Oracle VM server networking configuration can be used as-is or modified to meet your business requirements, for example, to use 802.3AD NIC bonding with 802.1Q. Figure 3 shows the default Oracle VM Xen bridge configuration: an Oracle VM server with two NICs wired into a single switch, hosting three guests.

Tip: In an HA-enabled pool, the loss of network connectivity for the Oracle VM management interface causes an HA event. When an HA event occurs, an Oracle VM server is fenced from the pool and reboots, then all HA-enabled guests are restarted on a live Oracle VM pool member. As of this writing, Oracle VM network configuration management is not included in Oracle VM Manager; it must therefore be performed by hand in dom0. Both 802.3AD NIC bonding and 802.1Q VLANning are supported by Oracle VM for x86, although bonding and VLANning must also be configured by hand in dom0. Figure 4 shows an example 802.3AD NIC bonding and Xen bridges with 802.1Q VLANning.

For more information on Oracle VM networking please refer to Chapter 7: Oracle VM Networking from A to 802.1Q Oracle VM Manager administers Oracle VM servers, virtual machines, and virtual machine resources in server pools. A server pool is a management boundary that contains one or more clustered Oracle VM servers with virtual machines and virtual machine resources. The next section will examine Oracle VM server pools.

Oracle VM for x86 Server Pools, HA, and Live Migration
Oracle VM Manager uses the concept of a server pool to group together and manage one or more clustered Oracle VM servers. An Oracle VM server pool defines the management boundaries and the feature set of Oracle VM servers, virtual machines, and virtual machine resources. Once a server pool is created, resources such as servers, virtual machines, operating system installation ISO files, Oracle VM templates, administrative users, and groups can be configured and managed within the context of the server pool. For example, an Oracle VM environment with multiple server pools could be managed from one single Oracle VM Manager instance, although each server pool's resources, such as servers, virtual machines, operating system installation ISO files, Oracle VM

17 of 243

18.04.2011 16:35

The Underground Oracle VM Manual
templates, administrative users, and groups are isolated to their respected server pool.

http://itnewscast.com/book/export/html/844

An Oracle VM server with local storage is limited to a server pool with only one server, without high availability (HA) or Live Migration functionally. To add additional Oracle VM Servers to a server pool, and to enable HA and Live Migration, at least one shared SAN, iSCSI, or NFS storage repository is required. Shared SAN, iSCSI, or NFS storage allows virtual machines to start, run, and migrate to any Oracle VM Server within a server pool. Without shared storage, virtual machines can only start and run on one Oracle VM server without HA and Live Migration functionality. Figure 5 shows an Oracle VM environment with four Oracle VM servers managed in two server pools.

Oracle VM for x86 Storage
Oracle VM uses two different types of storage repositories for server pools. The first type of storage repository, called a root repository, is used to host a server pool's Oracle Cluster File System v2 (OCFS2) cluster configurations, HA configurations for HA enabled pools and, optionally, virtual machine configuration files and images. There can only be one OCFS2 or NFS root repository per server pool. The other type of storage repository, called an extended repository, is used exclusively for server pools with more than one server, to host virtual machine configuration files and images. There can be one or more extended OCFS2 and/or NFS repositories in a server pool with more than one server. When a server pool needs more storage capacity, additional extended OCFS2 and/or NFS repositories are added to a server pool to increase capacity. Note: OCFS2 is not integrated or supported with any volume manager (LVM) solutions to manage the back-end block storage. Fibre Channel and iSCSI OCFS2 partitions must be provisioned at static sizes, that is, partition sizes cannot change once a partition is formatted with OCFS2. As of this writing, there are no administrative features in Oracle VM for x86 for managing storage repositories. Oracle VM storage repository configurations are made in dom0 and storage management is done using the native storage-array management tools. Oracle VM Manager is responsible for pool creation, not for storage repository management. For example, storage repository provisioning, storage repository snapshotting, storage repository replication, and storage repository monitoring, as well as storage repository backup and restoration, are preformed using the storage-array administrative functionality. Tip: A best practice with server pools that have more than one server is to dedicate the root repository to host only OCFS2 and HA configurations, without virtual machine configuration files and images. A dedicated root repository without virtual machine configuration files and images reduces the risk of cluster data corruption from virtual machine operations. For more information on Oracle VM storage please refer to Chapter 6: Oracle VM 2.2 SAN, iSCSI and NFS Back-end Storage Configurations Figure 6 shows an Oracle VM server pool with three servers using a dedicated root repository hosting the cluster and HA configurations and two extended repositories hosting virtual machine configuration files and images.

18 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

The Oracle VM for x86 HA feature detects server failures within a server pool and responds by restarting the virtual machines from a failed server on a live server in the server pool. Oracle VM leverages a lightly modified OCFS2 cluster stack to monitor the status of each Oracle VM server, using a network heartbeat over the management interface and a storage heartbeat on the root storage repository. If any node in a server pool fails to update/respond to its network or storage heartbeat, the node is fenced from the pool and promptly reboots, then all HA-enabled guests are restarted on a live server in the pool. Figure 7 shows a server pool with three servers. One of the three servers has failed and the virtual machines from the failed server have been restarted on the live servers.

Oracle VM HA adds negligible traffic to the Oracle VM server management network: 1 packet every 2 seconds for the HA heartbeat traffic plus distributed lock manager (DLM) traffic when locks are taken during guest start and stops. The HA heartbeat traffic is latency sensitive, and the allowable latency for heartbeat traffic is 60 seconds. For example, network heartbeat latency is 30 seconds, but after 30 seconds the cluster will attempt to establish a disk heartbeat. If the cluster establishes a disk heartbeat, it will then try to reconnect via the network. The second disk heartbeat timeout latency is set to 30 seconds, so it takes a full 60 seconds plus the inability to see the root storage repository to trigger an HA event. Note: In an HA-enabled pool, the loss of network connectivity for the Oracle VM management interface causes an HA event. When an HA event occurs, an Oracle VM server is fenced from the pool and reboots, then all HA-enabled guests are restarted on a live Oracle VM pool member. The Oracle VM for x86 Live Migration feature moves running virtual machines between server pool members across a LAN without loss of availability. Live Migration has two primary use cases. The first use case is to eliminate planned downtime by migrating running guests from one server pool member to another during planned maintenance events. The second use case is to migrate running guests from an overutilized pool member to a pool member with available resources. There are three requirements for Live Migration. The first requirement is a shared SAN, iSCSI, or NFS storage repository to host the virtual machine configuration files and images. The second requirement is that the source and target pool member servers CPUs be identical. The final requirement is that the target server pool member must have sufficient memory to accommodate the memory requirements of the virtual machine or machines that will be migrated. Figure 8 shows a server pool with virtual machines Live Migrating between server pool members.

19 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Oracle VM uses an iterative precopy method to migrate running guests over a TCP/SSL connection between two pool members. A Live Migration event starts when the source server sends a migration request to the target server, which contains the guest resource requirements. If the target accepts the migration request, the source starts the iterative precopy phase. The iterative precopy phase starts by iteratively copying the guest’s memory pages from the source to the target server over the management network. If a memory page changes during the precopy phase, it is marked dirty and resent. Once the majority of the pages are copied, the stop-and-copy phase begins. The stop-and-copy phase starts by pausing the guest while the remaining dirty pages are copied to the target, which usually takes 60 to 300 milliseconds. Once the pages are copied to the target, the guest is started on target server. Note: Oracle VM server does not support memory oversubscription, which means that an Oracle VM server cannot accept a Live Migration request unless the server has available RAM for the virtual machines. Oracle VM server supports CPU oversubscription, which means that an Oracle VM server can accept a Live Migration request and overallocate the total number of CPU threads to virtual machines, if necessary.

Oracle VM for x86 Intracomponent Communication & Firewall Requirements
This section will examine Oracle VM for x86's intracomponent communication and firewall requirements. The goal of this section is to illustrate the communication ports and system passwords required to help plan, build, and support an Oracle VM for x86 environment. List 3 highlights Oracle VM communication ports and system passwords used with Oracle VM Manager. Oracle VM Manager: TCP 22/ssh is the default service used to access the Oracle VM Manager CLI. Oracle VM Manager communicates with the Server Pool Master agent using TCP/8899. The default HTTPS port for the Oracle VM Manager portal is 4443. The default HTTP port for Oracle VM Manager portal is 8888. The default HTTP port for the Oracle Database 10g Express Edition portal is 8080. The default HTTP port for the Oracle Application Server 10g portal is 8888. The default Database listening port for the Oracle Database 10g Express Edition repository is 1521. Oracle Database 10g Express Edition accounts are SYS and SYSTEM. Virtual machine VNC console access from the Oracle VM Manager portal uses ports 5900 through 5999. Virtual machine VNC console access requires a VNC password without a user name. Virtual machine VNC console passwords are assigned using Oracle VM Manager and are saved in clear text in each virtual machine's vm.cfg file. Oracle VM Server: The Oracle VM Agent listens on TCP 8899 and requires a password. The agent password is selected during the Oracle VM server installation and can be configured from dom0 by typing “service ovs-agent configure”. The xend-relocation-server service listens for Live Migration requests on TCP 8002. The xend-relocation-server is managed by xend and configured using the /etc/xen/xend-config.sxp file. TCP 8002 must be open to all server pool members. The cluster heartbeat is active on TCP 7777 and must be open to all server pool members. SSH is enabled by default on TCP 22. VNC console access to virtual machines uses TCP 5900 through 5999. The rpcbind process (portmapper) listens on TCP/UDP 111. Figure 9 illustrates Oracle VM for x86 intra-component communication and firewall requirements with Oracle VM Manager.

20 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Next, we will examine the intracomponent communication and firewall requirements for an Oracle VM for x86 environment using the Oracle Enterprise Manager Oracle VM Management Pack Plug-in. List 4 highlights communication ports used with the Oracle Enterprise Manager Oracle VM Management Pack Plug-in. The Oracle Enterprise Manager Oracle VM Management Pack Plug-in: Oracle Enterprise Manager Oracle VM Management Pack Plug-in communicates with the Server Pool Master agent using TCP/8899. The default HTTP port for the Enterprise Manager Grid Control portal is 4889. The default database listening port for the Oracle Database repository is 1521. Virtual machine VNC console access from the Oracle VM Manager portal uses ports 5900 through 5999. Virtual machine VNC console access requires a VNC password without a user name. Virtual machine VNC console passwords are assigned using Oracle VM Manager and are saved in clear text in each virtual machine's vm.cfg file. Oracle VM Server: The Oracle VM Agent listens on TCP 8899 and requires a password. The agent password is selected during the Oracle VM server installation and can be configured from dom0 by typing “service ovs-agent configure”. The xend-relocation-server service listens for Live Migration requests on TCP 8002. The xend-relocation-server is managed by xend and configured using the /etc/xen/xend-config.sxp file. TCP 8002 must be open to all server pool members. The cluster heartbeat is active on TCP 7777 and must be open to all server pool members. SSH is enabled by default on TCP 22. VNC console access to virtual machines uses TCP 5900 through 5999. The rpcbind process (portmapper) listens on TCP/UDP 111. Oracle VM Guests: The Oracle Management Agent to Oracle Management Service communication is HTTPS on 4889. Figure 10 illustrates Oracle VM for x86 intra-component communication and firewall requirements with the Oracle VM Management Pack.

21 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

1.0 – The Oracle VM for x86 System Reference Design Introduction
This section presents the Oracle VM for x86 reference design. The Oracle VM for x86 reference designs encompass the software, hardware, storage, and network components required to deploy a scalable, secure, and supportable Oracle VM for x86 solution in an internal or external cloud. The Oracle VM for x86 reference design is a field-tested best-practice standard, designed with simplicity, reproducibility, usability, scalability, supportability and security. The Oracle VM for x86 reference designs represent a complete Oracle VM for x86 standard that can be leveraged as a vanilla solution or modified to more accurately reflect organization-specific needs. The Oracle VM for x86 reference design includes the following categories and solutions: Cloud Infrastructure Administration and Monitoring Oracle VM Server Pools Oracle VM Server Pool Storage Design Oracle VM Server Pool Network Design Oracle VM Templates Virtual Machine Operating Systems Note: A detailed explanation of each category and solution in the Oracle VM for x86 reference design is presented in the architectural overview section.

1.1 – The Oracle VM for x86 System Reference Design Implementation Overview
The Oracle VM for x86 reference design provides a well defined starting point for each implementation. It also serves as a baseline upon which all solution additions, revisions, and tools will be based. As such, there is an increasing value to Oracle VM for x86 reference design users in keeping implementations as close to the reference design as possible. Prior to implementing Oracle VM for x86 based upon the Oracle VM for x86 reference design, it’s important that an infrastructure assessment (IA) and gap analysis (GA) be performed. During the IA/GA, the architecture of the solution will match the customer’s business needs while maintaining the integrity of the Oracle VM for x86 reference design. Implementation and support will follow the analysis phase after careful consideration has been given to any specific design modifications that deviate from the Oracle VM for x86 reference design. 1.2 – The Oracle VM for x86 System Reference Design Document Overview This document outlines the decision points necessary for implementing the Oracle VM for x86 reference design. For decisions that rely on preexisting factors or specific organizational needs, the appropriate best practice will be discovered in the infrastructure assessment (IA) and gap analysis (GA). The best practices should be analyzed carefully and decisions should be made based on organizational needs, existing architecture, and budget resource availability.

2.0 – The Oracle VM for x86 System Reference Design Architectural Overview

22 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

The Oracle VM for x86 reference design is designed to be scalable and resilient for ease of implementation, high availability, and ease of maintenance for internal and external clouds. The complete solution is made up of six architectural components that work together to provide flexibility and options with respect to server consolidation, application delivery, monitoring, authentication and security requirements, user access scenarios, and component migration/modification. The design breaks down into the following six components: Cloud Infrastructure. The Oracle VM for x86 reference design offers a cloud virtualization and management layer that allows multiple Oracle Linux, Red Hat Linux, Solaris, and Windows virtual machines to run on a shared x86-64 hardware environment allowing x86-64 bit server consolidation, and rapid application provisioning. Administration and Monitoring: The Oracle VM for x86 reference design provides complete transparency from a cloud infrastructure and an application perspective to any Oracle VM server, virtual machine, or application running in the cloud using Oracle Enterprise Manager with Oracle VM Manager. Oracle Enterprise Manager allows organizations to consistently and quantitatively measure performance across the organization for all of the hosted applications in the cloud. Oracle VM Server Pools. The Oracle VM for x86 reference design may have one or more Oracle VM server pools to provide isolation, defense in depth, the principle of least privilege, compartmentalization of information, and security domains, and to accommodate different applications and their performance, authentication, and security requirements. This design builds on the ability to test new applications alongside production applications in isolated Oracle VM server pools, without sacrificing the integrity of the production environment. Oracle VM Templates. The Oracle VM for x86 reference design will support an organization’s entire Oracle application portfolio using Oracle VM templates. An Oracle VM template can consist of one or more virtual machines containing a preconfigured operating system with an application. Using Oracle VM templates eliminates the operating system and application installation and configuration process, allowing applications to be deployed from a library of Oracle VM templates. Oracle VM templates used in conjunction with Oracle VM server pools accommodate different applications and their performance, authentication, and security requirements. This design builds on the ability to test new applications alongside production applications in isolated Oracle VM server pools, without sacrificing the integrity of the production environment. Virtual Machine Operating Systems. The Oracle VM for x86 reference design standardizes on a small number of virtual machine operating systems, operating system versions, and virtualization modes to streamline operations and to increase the level of file duplication on production and archival virtual machine data. This design reduces complexity and increases operational efficiency by limiting the number of supported operating systems. Oracle VM Server Pool Storage Design. The Oracle VM for x86 reference design standardizes one OCFS2 or NFS 4GB root repository that is used to host each server pool's Oracle Cluster File System v2 (OCFS2) cluster and HA configurations, and a maximum of 10 extended repositories used exclusively to host virtual machine configuration files and images. When a server pool needs more storage capacity, additional extended OCFS2 and/or NFS repositories are added to a server pool to increase capacity. Oracle VM Server Pool Network Design. The Oracle VM for x86 reference design standardizes one of two Oracle VM server pool network designs that both support defense in depth, the principle of least privilege, compartmentalization of information, and security domains, and accommodate different applications and their performance, authentication, and security requirements. 1. A single two NIC 802.3AD bond with Xen bridges using 802.1Q-Tagged VLANs to isolate traffic between the Oracle VM Manager management network and the virtual machine networks. Figure 11

2. 4 NICs with two 802.3AD bonds. One two NIC bond is dedicated to Oracle VM management traffic with a fixed IP and the second two NIC bond will use Xen bridges with 802.1Q-Tagged VLANs to isolate virtual machine networks. Figure 12

23 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Figure 13 shows a high-level overview of the Oracle VM for x86 reference design components.

The Oracle VM for x86 reference design isolates Oracle VM server pools into the following four security domains: Controlled: A controlled security domain is used to restrict access between security domains. A controlled security domain could contain groups of users with their network equipment or a demilitarized zone (DMZ). Uncontrolled: An uncontrolled security domain refers to any network not in control of an organization, such as the Internet. Restricted: A restricted security domain can represent an organization’s production, test and development networks. Access is restricted to authorized personnel, and there is no direct access from the Internet. Secured: A secured security domain is a network that is only accessible to a small group of highly trusted users, such as administrators and auditors. Note: The classification of security domains is very similar to data classifications. FIPS PUB 199 is the Standards for Security Categorization of Federal Information and Information Systems. FIPS PUB 199 can be used to determine the security category of systems and within which security domain systems should reside.

2.1 - The Oracle VM for x86 System Reference Design Support Infrastructure
Support is an integral part of the Oracle VM for x86 reference design and includes a combination of an Oracle support agreement and on- and off-site support from the implementing party. Administrators will have several options for support, including live assistance, phone support, and forums.

Part 1 - The Oracle VM for x86 System Reference Design 3.0 Cloud Infrastructure Architecture
This section provides a decision matrix for the Oracle VM for x86 reference design. Implementers of the Oracle VM for x86 reference design can use the decision matrix as quick reference guide to identify settings and configuration decisions to be implemented in the environment. Decisions highlighted in yellow may rely on preexisting environment factors or differ depending on organizational needs. These decisions should be carefully analyzed during a gap analysis phase. 3.1 - Oracle VM for x86 Hardware Architecture

24 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Decision Point Server Vendor

Decision Servers will be procured from a reliable hardware vendor. Two socket multiple-core processors for standard workloads and four socket multiple-core processors for large CPU-bound workloads.

Justification NA

Processors

The Maximum Number of CPUs or threads an Oracle VM server can support is 128. Oracle VM server maps a virtual CPU to a hardware thread on a CPU core in a CPU socket. A CPU core or a hyperthread is considered a physical CPU by the Xen hypervisor. For example, a server with an Intel Xeon processor 5600-series CPU with hyperthreading can have up to six cores and twelve threads per socket. A two socket server with an Intel Xeon processor 5600-series CPU could allocate twenty four virtual CPUs without oversubscribing the physical CPUs. CPU-bound workloads should not be on servers that have oversubscribed CPUs. Two socket multiple-core processors are ideal for standard non-CPU-bound workloads. For example, workloads with up to 8 virtual CPUs. Four socket multiple-core processors are ideal for large CPU-bound workloads. For example, workloads with 16 up to 32 virtual CPUs.

RAM

Servers will be ordered with the maximum amount of physical memory.

Oracle VM server does not support memory oversubscription, which means that an Oracle VM server cannot accept a Live Migration or HA request unless the server has available RAM for the virtual machines. Having available RAM on each server provides flexibility in terms of adding new virtual machines to the server pool, and to allow Live Migration and HA within a server pool. An Oracle VM server installation “without” local virtual machine storage requires 1GB of storage and 1GB of swap space. Oracle VM server can be installed on a 2GB flash drive or on local redundant hard drives.

Internal storage

Unless the Oracle VM server is booting from SAN, redundant internal hard drives or a 2GB flash drive is required. Virtual machine images will be stored on shared SAN, iSCSI, or NFS repositories. A minimum of two and up to four NICs per server.

NICs

For network-interface height availability 802.3AD bonds will be used for each pair of network interfaces up to a maximum of four network interface cards with two bonds.

25 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Build Process PXE boot

Oracle VM server will be installed using an automated PXE boot configuration to ensure that each server has a consistent installation configuration.

3.2 - Oracle VM for x86 Server Pool Configurations

Decision Point Oracle VM for x86 server pool design and management

Decision Use multiple Oracle VM server pools managed from Oracle VM Manager.

Justification Single point of administration for multiple Oracle VM server pools accommodating organizationspecific needs, i.e., defense in depth, the principle of least privilege, compartmentalization of information, security domains, and to accommodate different applications and their performance, authentication, and security requirements. This design builds on the ability to test new applications alongside production applications in isolated Oracle VM server pools, without sacrificing the integrity of the production environment. If more than one location exists, Oracle VM server pools may be dispersed to different locations.

Version of Oracle VM server and Oracle VM Manager Oracle VM agent role configurations

The latest version of Oracle VM server and Oracle VM Manager will be used. Server Pool Master Each Oracle VM server pool will enable the Virtual IP feature. Utility Server Each Oracle VM server pool will have at least one dedicated utility server with only the Utility Server role enabled. Virtual Machine Server Virtual Machine Servers will have only the Virtual Machine Server role enabled.

The latest version of Oracle VM server and Oracle VM Manager will be used to ensure that system has the latest platform and security updates and patches. Server Pool Master To eliminate a single point of failure, each Oracle VM server pool will enable the Virtual IP feature. Utility Server Each Oracle VM server pool will have at least one dedicated utility server to isolate the impact of I/O intensive operations on the Utility Servers. Utility Servers will only have Utility Server role enabled. Virtual Machine Server Virtual Machine Servers will only have the Virtual Machine Server role enabled to dedicate system resources to running virtual machines, eliminating the effect of Utility Server operations. Each Oracle VM server pool uses one OCFS2 or NFS 4GB root repository that is used to host each server pool's OCFS2 cluster and HA configurations to isolate and protect the cluster and HA configurations from virtual machine configuration operations. Up to 10 extended repositories can

Oracle VM Server Pool Storage Configurations

Each Oracle VM server pool will have one OCFS2 or NFS 4GB root repository that is used to host each server pool's Oracle Cluster File System v2 (OCFS2) cluster and HA

26 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

configurations and up to a maximum of 10 extended repositories used exclusively to host virtual machine configuration files and images.

be used exclusively to host virtual machine configuration files and images. The Oracle VM agent can reliably support up to 10 extended repositories. OCFS2 is not integrated or supported with any volume manager (LVM) solutions to manage the back-end block storage. Fibre Channel and iSCSI OCFS2 partitions must be provisioned at static sizes, i.e., partition sizes cannot change once a partition is formatted with OCFS2. When a server pool needs more storage capacity, up to ten extended OCFS2 and/or NFS repositories can be added to a server pool to increase capacity. For network interface link aggregation and height availability 802.3AD bonds will be used for each pair of network interfaces up to a maximum of four network interface cards with two bonds. 802.1Q-Tagged VLANs are used to configure isolated virtual networks using "smart" or managed Ethernet switches, rather than relying on cables wired into physical switches and routers. For example, with VLANs a LAN can be segmented into isolated security domains that offer defense in depth, the principle of least privilege, and compartmentalization of information, and can accommodate different applications and their performance, authentication, and security requirements, without being restricted by physical connections. A key component of patch management is acquiring and vetting patches for production systems. Patches must be researched to identify which patches, security fixes, and application updates are applicable to your environment. Newly released patches, security updates, and application updates will be tested before being deployed in to production using time stamped local custom repositories. Local yum repositories will be maintained for patch testing and production using a point-in-time static channel for each supported operating system to ensure all like operating systems are patched in a consistent manner across the organization. Pre- and post-production audits

Oracle VM Server Pool Network Configurations.

Each Oracle VM pool member will use one of the two following network configurations: 1) A single two NIC 802.3AD bond with Xen bridges using 802.1Q-Tagged VLANs to isolate traffic between the Oracle VM management network and the virtual machine networks. 2) 4 NICs with two 802.3AD bonds. One two NIC bond is dedicated to Oracle VM management traffic with a fixed IP and the second two NIC bond will use Xen bridges with 802.1QTagged VLANs to isolate virtual machine networks. Oracle VM servers will be configured to use local custom yum repositories. All patches will be regression tested in the lab environment before they are deployed on production systems. High-priority patches, security fixes, and application upgrades updates will be applied as needed in accordance with <Company Name>’s Change Management Policy. Noncritical fixes will be applied on a Quarterly basis in accordance with

Patch Management

27 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

<Company Name>’s Change Management Policy. All production systems will undergo security audits in accordance with <Company Name>’s Change Management Policy to validate configuration and patch compliance.

will be conducted in accordance with <Company Name>’s Change Management Policy to validate configuration and patch compliance.

3.3 - Oracle VM for x86 Network Topology

Decision Point

Decision

Justification For network interface link aggregation and height availability 802.3AD bonds will be used for each pair of network interfaces up to a maximum of four network interface cards with two bonds. 802.1Q-Tagged VLANs are used to configure isolated virtual networks using "smart" or managed Ethernet switches, rather than relying on cables wired into physical switches and routers. For example, with 802.1Q-Tagged VLANs a LAN can be segmented into isolated security domains that offer defense in depth, the principle of least privilege, and compartmentalization of information, and can accommodate different applications and their performance, authentication, and security requirements, without being restricted by physical connections. Using static reserved IP addresses eliminates issues associated with duplicated or reassigned IP addresses. Each Oracle VM server pool will restrict Oracle VM management traffic to an isolated network subnet to protect the server pool from broadcast storms that could saturate the management network and cause pool members to loose network connectivity and fence from the pool. Oracle VM Manager will be the single point of administration for all Oracle VM server pools and should be located on a secure isolated network segment to protect access to the Oracle VM Manager portal and to better control and monitor communication to the server pool masters.

NIC Each Oracle VM server Configuration will use 802.3AD bonds for each pair of network interfaces up to a maximum of four network interface cards with two bonds. Switch Port 802.1Q-Tagged VLANs Configuration

IP Addresses

Static

Location of Oracle VM server pool management network

Each production Oracle VM server pool have an isolated management network subnet.

Location of Oracle VM Manager

Oracle VM Manager will be located on a isolated network segment.

28 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Virtual Machine Console Access

Inbound TCP 5900 through 5999 traffic will be allowed to each Oracle VM server pool network subnet.

To facilitate virtual machine console access, inbound TCP 5900 through 5999 traffic must be allowed to each Oracle VM server pool network subnet from the system administrators network subnet. Using the default ports creates consistency and simplifies configuration.

Port Numbers

The default ports will be used for all Oracle VM server and Oracle VM Manager components. If NFS shared storage repositories are used, the storage network will be restricted to the Oracle VM Management network subnet.

NFS Storage Repositories

NFS shared storage repositories will be restricted to the Oracle VM Management network subnet to isolate and secure NFS storage traffic.

3.4 – Oracle VM for x86 Network Security Architecture

Decision Point Oracle VM Server and Oracle VM Manager iptables configurations Root ssh access

Decision The iptables service will be enabled on each Oracle VM server using the default policy and ruleset in /etc/sysconfig/iptables. Root ssh access will be disabled on all Oracle VM servers and sudo will be configured for root access. To disable root ssh access, edit the default /etc/ssh/sshd_config file and uncomment the the “#PermitRootLogin yes” line and change the yes to no; that is, “PermitRootLogin no”. Next, restart the sshd service by typing “service sshd restart” to enable the change. To enable sudo access for a users, from dom0 type “viedit” and add the account names under the following lines: ## Allow root to run any commands anywhere root ALL=(ALL) ALL newuser ALL=(ALL) ALL The above example provides root access to

Justification Host firewalls, for example iptables, are a fundamental part of information security that protect hosts from attacks and intrusions.

By default, Oracle VM servers permit ssh access using the root super user account. One of the most important security measure that can be taken on an Oracle VM server is to prevent unauthorized access to the root user account by disabling root ssh access. Systems administrators will access the Oracle VM servers with user privileges and use sudo for root access. All sudo user access will be tracked and logged in the /var/log/secure file.

29 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

all commands for the newuser account. More restrictive access to commands can be configured for each user. Consult the sudoers man page for details. SSH login banners Pre and post SSH login banners will be configured on each Oracle VM server. Pre-login banner: Edit the /etc/ssh /sshd_config and add the following directive: Banner /etc/banner.net Next, create the /etc/banner.net file and add your login banner, i.e. vi /etc/banner.net This system is restricted to authorized access only. All activities on this system are recorded and logged. Unauthorized access will be fully investigated and reported to the appropriate law enforcement agencies. :wq! Once the file has been created and the banner text is added and saved, restart the sshd by typing: # service sshd restart Post login banner: Edit /etc/motd and add your login banner text, i.e. vi /etc/motd This system is restricted to authorized access only. All activities on this system are recorded and logged. Unauthorized access will be fully investigated and reported to the appropriate law enforcement agencies. :wq! Once the file has been edited and saved, restart the sshd by typing: # service sshd restart Iptables failed Iptables failed connection connection logging will logging be enabled on each Failed connect logging is a fundamental part of information security that allows detection of To be able to successfully prosecute individuals who improperly use a computer, the computer must have a warning banner displayed at all access points. SSH login banners presents a definitive warning or disclaimer to all users that wish to access your systems using SSH. SSH login banners should clarify which types of activities are illegal as well as advise legitimate users of their obligations relating to the acceptable use of the system.

30 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Oracle VM server. The following two lines will be added prior to the last REJECT line in the /etc/sysconfig /iptables file: -A RH-Firewall-1-INPUT -m limit --limit 15/minute -j LOG --log-prefix "FW Drop:" -A RH-Firewall-1-INPUT -j REJECT --reject-with icmp-hostprohibited Central log host A central log host will be used to log all user logins and iptables connection failures. Oracle VM Manager will not be placed in a DMZ.

attacks and intrusions.

Centralized logging for user logins and iptables connection failures simplifies security management for the detection of attacks and intrusions. The Oracle VM Manager application was not designed to be an Internet facing application. If remote access is a requirement for Oracle VM Manager VPN access will be used. Oracle VM servers in a DMZ are restricted from inbound and outbound Internet connectivity to minimize the risk of attack from the Internet. Oracle VM servers in a DMZ are restricted from access to internal network segments except TCP/8899 from Oracle VM Manager. If virtual machine console access is required, inbound TCP/5900 through 5999 will be configured to each Oracle VM server in the DMZ for virtual machine console access.

Oracle VM Manager in a DMZ

Oracle VM Servers in a DMZ

Oracle VM server hosting Internet facing virtual machines will be placed in a DMZ without connectivity to the Internet or internal network segments, except TCP/8899 from Oracle VM Manager.

3.5 - Oracle VM for x86 Administration and Monitoring

Decision Point Oracle VM for x86 server pool administration and monitoring

Decision Oracle VM for x86 server pool administration will be done using Oracle VM Manager with the Oracle VM Manager Command Line Interface. Oracle VM for x86 server pools will be monitored using an SNMP based solution.

Justification Oracle VM Manager with the Oracle VM Manager Command Line Interface has the broadest administration feature set available for Oracle VM for x86. For example, the Oracle Enterprise Manager Oracle VM Management Pack Plug-in does not a have comparable Command Line Interface to automate administrative functions. Oracle VM Manager does include monitoring. An SNMP based solution for monitoring Oracle VM servers is the best option since there are no additional software requirements.

31 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Centralized logging for Oracle VM server agent logs and the Oracle VM Manager oc4j.log.

A central log host will be configured to capture the Oracle VM server agent logs and the Oracle VM Manager OC4J logs.

When things go wrong with an Oracle VM server pool, being able to quickly determine the “root cause” of an issue can eliminate or reduce down time. The most effective way to identify problems with an Oracle VM server pool is to analyze the Oracle VM Manager OC4J log and Oracle VM server' agent log files.

Rootkit prevention and monitoring

Each Oracle VM server Wikipedia describes a rootkit as” A will have the rootkit is software that enables chkrootkit RPM continued privileged access to a installed. computer while actively hiding its presence from administrators by subverting standard operating system functionality or other applications.” Monitoring systems for rootkits is fundamental part of information security that allows the detection of rootkits to prevent attacks and intrusions.

3.6 - Oracle VM for x86 Virtual Machine Operating Systems

Decision Point Virtual Machine Operating Systems

Decision A small number of virtual machine operating systems will be used.

Justification Standardizing on a small number of virtual machine operating systems streamlines operations and increases the level of file duplication on production and archival virtual machine data. This design reduces complexity and increases operational efficiency by limiting the number of supported operating systems. Each application has an operating system support matrix that lists the supported operating systems, patch levels, and software prerequisites. In accordance with <Company Name>’s Application Software Policy and Application Software Standards, applications will determine the operating system type and version. A virtual machine template is a self-contained, preconfigured virtual machine with an operating system and optionally an application installed in accordance with <Company Name>’s Server Policy, Server Security Policy, and Operating System Installation Guidelines. Each time a new virtual machine is deployed using a virtual machine template, <Company Name>’s standards are applied to each new virtual machine.

Virtual Machine Operating System Versioning

In accordance with <Company Name>’s Application Software Policy and Application Software Standards, applications will determine the operating system type and version.

Virtual Machine Operating System Deployments

All new virtual machine operating systems will be deployed using a virtual machine template in accordance with <Company Name>’s Server Policy, and Server Security Policy.

32 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Patch Linux virtual machines Management will be configured to use local custom yum repositories.

A key component of patch management is acquiring and vetting patches for production systems. Patches must be researched to identify which All patches will be patches, security fixes, and regression tested in the application updates are applicable lab environment before to your environment. Newly they are deployed on released patches, security updates, production systems. and application updates will be High-priority patches, tested before being deployed in to security fixes, and production using time stamped application upgrades local custom repositories. updates will be applied as needed in Local yum repositories will be accordance with maintained for patch testing and <Company Name>’s production using a point-in-time Change Management static channel for each supported Policy. operating system to ensure all like operating systems are patched in a Noncritical fixes will be consistent manner across the applied on a Quarterly organization. basis in accordance with <Company Pre- and post-production audits will Name>’s Change be conducted in accordance with Management Policy. <Company Name>’s Change Management Policy to validate All production systems configuration and patch will undergo security compliance. audits in accordance with <Company Name>’s Change Management Policy to validate configuration and patch compliance.

3.7 - Oracle VM for x86 Application Integration

Decision Point Application Support

Decision Applications must be supported by the independent software vendor (ISV) on the latest version of Oracle VM for x86 to be included in the Oracle VM for x86 environment. Applications will be analyzed for requirements and dependencies and tested in accordance with <Company Name>'s Software Installation Standards.

Justification Applications that are incompatible with and unsupported by Oracle VM for x86 cannot be supported by <Company Name> on Oracle VM for x86. Only applications with ISV support for the latest version of Oracle VM for x86 can be deployed and supported by <Company Name>, the ISV, and Oracle on Oracle VM for x86. Applications will be analyzed for requirements and dependencies and tested to ensure compliance with ISV specifications.

Application Requirements and Dependencies

Application Installation, Packaging, and Distribution

Applications should be installed and packaged using Oracle VM Template Builder or Oracle Enterprise Manager.

Application installations that are packaged in Oracle VM templates or deployed using Oracle Enterprise Manager have a consistent installation configuration.

33 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Applications that are installed and packaged using Oracle VM Template Builder will be deployed as Oracle VM templates. Applications that are installed using Oracle Enterprise Manager will be installed on Oracle VM templates. Application sunsetting Applications will be sunsetted in accordance with <Company Name>'s Hardware and Software Sunset Policy Applications that have reached the end of their life cycle and are no longer supported by a vendor will be given a sunset date. The sunset date is when the product is scheduled to be removed from production. Sunsetting applications that have reached the end of their life cycle results in better customer service and reduced costs. Patch Management All patches will be regression tested in the lab environment before they are deployed on production systems in accordance with <Company Name>’s Change Management Policy. Noncritical fixes will be applied on a Quarterly basis in accordance with <Company Name>’s Change Management Policy. All production systems will undergo security audits in accordance with <Company Name>’s Change Management Policy to validate configuration and patch compliance. A key component of patch management is acquiring and vetting patches for production systems. Patches must be researched to identify which patches, security fixes, and application updates are applicable to your environment. Newly released patches, security updates, and application updates will be tested before being deployed in to production using time stamped local custom repositories. Pre- and post-production audits will be conducted in accordance with <Company Name>’s Change Management Policy to validate configuration and patch compliance.

3.8 - Oracle VM for x86 Support Services

Decision Point Oracle Support Agreement

Decision Oracle Support Agreement for Oracle VM and Oracle Linux will be active and up to date.

Justification Support is an integral part of every successful IT project. An Oracle support agreement is necessary to be able to receive RPM patches and updates for Oracle VM and Oracle Linux and to create and manage service requests.

34 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

On-site and Off-site support

On-site and off-site support from the implementing party will be used for maintenance, site reviews, upgrades, and security audits.

On-site and off-site support from the implementing party for problem resolution, system maintenance, site reviews, upgrades, and security audits augments the Oracle support agreement and internal IT operations staff.

Watch this space Oracle OpenWorld 2011 Call for Papers - Submit Today! Temporary Post Used For Theme Detection (409fc5e9-d7ee-4e1a-b645-8b82f256f434 - 3bfe001a-32de-4114-a6b4-4005b770f6d7) Debugging Oracle VM 2.2 & Oracle VM Manager 2.2 errors using the many log files New VDI Training Course now available! HIMSS 2011 and New Press Release Oracle Desktop Virtualization at HIMSS 2011 Oracle VM VirtualBox 4.0.4 Released! Two Virtualization Webinars This Week We're Hiring! - Server and Desktop Virtualization Product Management

Chapter 3: Hard and Soft Partitioning Oracle Technologies with Oracle VM
cube.jpg This chapter clarifies how Oracle VM can be used with hard and soft partitioning to manage your Oracle technology license costs. Last update 12-07-2010 Copyright © 2009 - 2011 Roddy Rodstein. All rights reserved. Show a printer friendly version of this book with its sub pages This chapter will review hard and soft partitioning Oracle technologies with Oracle VM. The goal of this chapter is to clarify how Oracle VM can be used with hard and soft partitioning to help manage your Oracle enterprise technology license costs. The chapter starts with a brief introduction to Oracle licensing. Next, we will review Oracle technology named user plus licensing followed with processor licensing with hard and soft partitioning using Oracle VM. The chapter concludes with hard partitioning examples and virtual CPU binding testing techniques. Note: While Oracle recognizes hard and soft partitioning with Oracle VM, this does not imply that this applies when using other vendor's virtualization technologies. Please refer to the SIG or your Oracle representative if you have questions about the licensing impact of other vendor's virtualization approaches. Table of Contents Oracle Technology Licensing Oracle Technology Licensing with Oracle VM …Named User Plus Licensing …Named User Plus Licensing versus Processor Licensing …Processor Licensing Hard and Soft Partitioning with Oracle VM …Soft Partitioning Examples …Hard partitioning Examples …Oracle VM Manager Manual Placement Policy Configuration Hard Partitioning an Oracle VM Guest …CPU Pinning Examples …CPU Pinning with xm Resources Oracle Technology Licensing Oracle segments its product portfolio into two categories, technology and applications. The Oracle technology and applications license models are very different. The only similarity between the technology and applications licensing models is the ability to execute an unlimited license agreement (ULA). Technology products have three forms of licensing, 1) processor 2) named user plus (NUP) and 3) unlimited license agreement (ULA). Applications licensing also have three forms of licensing, 1) component pricing, 2) custom applications suite pricing, and 3) enterprise pricing aka an unlimited license agreement (ULA). List 1 shows Oracle’s technology offering. Database Enterprise Managers Application and System Management Application Server Fusion Middleware Business Intelligence

35 of 243

18.04.2011 16:35

The Underground Oracle VM Manual
Identity Management Tools Enterprise 2.0 Collaboration Data Warehousing Products Integration products List 2 shows Oracle’s application offering. Oracle Oracle Oracle Oracle Oracle Oracle Oracle Oracle Analytic and Business Intelligence (BI) Applications Customer Relationship Management (CRM) Financial Management Governance, Risk, and Compliance (GRC) Management Human Capital Management (HCM) Procurement Project Management Supply Chain Management (SCM)

http://itnewscast.com/book/export/html/844

The only way to determine the most beneficial licensing model for your Oracle software investment is to evaluate your organization’s Oracle software requirements, along with your hardware, operating system and virtualization configurations. Most organizations initially engage their Oracle sales representatives as a first step, in order to help evaluate and quote license options. Customers typically use the initial licensing evaluation and quotes as a starting point to help determine which licensing model and configuration provides the best value. An important Oracle technology licensing consideration is your organization’s hardware, operating system and virtualization configurations. Oracle recognizes a wide variety of hardware, operating system and virtualization configurations that directly affect the CPU count used to calculate Oracle technology processor licenses. For example, Oracle recognizes various hard and soft partitioning configurations for Oracle VM, as well as the big 3 UNIX platforms, that directly affect how to count Oracle technology CPU licenses. From an Oracle technology licensing perspective, hard partitioning allows customers to license a subset of a server’s CPUs. Conversely, soft partitioning counts the total number of a server’s CPUs. Note: Oracle VM is not a licensed technology product. Oracle Technology Licensing with Oracle VM Of the three Oracle technology licensing options, 1) processor 2) named user plus (NUP) and 3) unlimited license agreement (ULA), Oracle VM can help manage and reduce processor licensing costs with Oracle enterprise edition technology products. Oracle VM helps manage enterprise edition technology processor licensing, by using hard and soft partitioning. Hard partitioning with Oracle VM allows a customer to license a subset of an Oracle VM server’s CPUs. Soft partitioning is used to take advantage of Live Migration, which is not supported with hard partitioning. Along with Live Migration, soft partitioning provides the ability to manage the number of licensed Oracle technology product CPUs within an Oracle VM pool. Hard and soft partitioning with Oracle VM provide the ability to manage the number of licensed Oracle technology CPUs. Named user plus licensing and unlimited license agreements are not CPU regulated, which preclude using Oracle VM as a license management option. The SIG states that Oracle technology standard edition products are limited to 2 or 4 sockets, i.e. installed on a server with no more than 2 or 4 physical CPUs. Standard edition products can run on an Oracle VM as long as the Oracle VM server meets the standard edition’s CPU requirements. Most contemporary virtualization servers are equipped with 2 or 4 sockets, which may preclude hosting standard edition products on Oracle VM. Understanding how and where to use hard and soft partitioning with Oracle VM can help organizations to better manage their Oracle Enterprise technology licensing costs for development and production environments. Named User Plus Licensing Smaller deployment with less than 50 users regularly select named user plus licensing. For smaller environments, named user plus licensing can be more cost effective when compared with processor licensing. Along with named user plus licensing, customers regularly select standard editions products over enterprise editions products to further reduce costs. Named User Plus Licensing versus Processor Licensing Oracle VM with processor licensing could provide an alternative to named user plus licensing by leveraging hard partitioning to manage the number of licensed CPUs. The ability to manage the number of licensed CPUs can help control licensing costs, which may provide a cost advantage over named user plus licensing. You would need to run the numbers to determine if hard partitioning could provide a cost advantage over named user plus licensing. Processor Licensing Processor licensing is when an Oracle customer pays per processor (CPU), to run an Oracle technology product. Larger deployments, with 50 or more users, typically use processor based licensing. Oracle recognizes each CPU core as a separate CPU and each CPU type with a different processor factor. The processor factor determines the CPU count. The CPU count determines the number of CPUs required to license the Oracle technology product. Note: Be sure to refer to the latest Oracle Processor Core Factor Table to find out the processor core factor for your hardware.

36 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Table 1 lists the processor factors. Oracle Processor Licensing Processor Factor UltraSparc T1 AMD/Intel All other Multi-core Servers Single Core Servers 0.25 0.50 0.75 1.00

To better understand how to calculate a processor factor, List 1 shows the processor factor for a single quad core Intel, AMD, Sun Sparc and an IBM Power CPU. Intel or AMD CPU 1 quad core CPU requires 2 processor licenses (4 cores multiplied by a factor of .50 equal 2 processor licenses). Sun SPARC64 VI CPU (* different models of Sun CPUs may have different core factors) 1 quad core SPARC64 VI CPU requires 3 processor licenses (4 cores multiplied by a factor of .75 equal 3 processor license). IBM Power6 CPU (* different models of IBM CPUs may have different core factors) 1 quad core Power6 CPU requires 4 processor licenses (4 cores multiplied by a factor of 1.0 equal 4 processor licenses) Two and four core CPUs are now end of life. New Intel x86 servers’ ship with six or eight core CPUs. AMD plans to ship their 12 core CPUs in the first half of 2010. Both Sun Sparc and IBM Power servers now ship with eight core CPUs. As the chip vendors add more cores to CPUs, Oracle technology licensing costs can increase. To better understand the impact to Oracle processor licensing with multi-core CPUs, let’s review List 2. List 2 shows the processor factor for a single eight core Intel, AMD, Sun Sparc, and IBM Power CPU. Intel or AMD CPU 1 eight core CPU requires 2 processor licenses (8 cores multiplied by a factor of .50 equal 4 processor licenses). Sun SPARC64 VI CPU (* different models of Sun CPUs may have different core factors) 1 eight core SPARC64 VI CPU requires 6 processor licenses (8 cores multiplied by a factor of .75 equal 6 processor license). IBM Power6 CPU (* different models of IBM CPUs may have different core factors) 1 eight core Power6 CPU requires 8 processor licenses (8 cores multiplied by a factor of 1.0 equal 8 processor licenses) As illustrated in the above examples, a single eight core CPU doubled the Oracle technology license CPU count, when compared to the single four cores CPU in List 1. Oracle customers using processor licensing will have to carefully consider the licensing impact of a hardware refresh due to the additional CPU cores. List 3 highlights various options to help manage the Oracle technology license CPU count, with multi-core CPUs. One of the options is to move from processor licensing to an unlimited license agreement. Customers with an unlimited license agreement (ULA) have no CPU restrictions with Oracle technology products. Another option would be to use hard partitioning with processor licensing to control the number of licensed CPUs. Hard partitioning allows a customer to license a subset of a server’s CPUs. However, Live Migration is not supported with hard partitioning. Customers can also use soft partitioning with processor licensing and Oracle VM to limit the number of licensed CPUs within a server pool. Soft partitioning supports Live Migration. Hard and Soft Partitioning with Oracle VM Oracle recognizes both hard and soft partitioning for Oracle technologies with Oracle VM. Hard and soft partitioning with Oracle VM can be used with processor licensing and enterprise edition products to manage the number of licensed CPUs, for development and production environments. Note: Standard edition products can run on an Oracle VM as long as the Oracle VM server meets the standard edition’s CPU requirements. Please refer to the relevant licensing documentation for the Standard Edition product in question to verify if the Standard Edition product can be hosted on your server platform with Oracle VM. The difference between hard and soft partitioning is how Oracle recognizes the Oracle technology CPU license count, and the supported virtualization feature set. For example, Live Migration is not supported with Oracle VM when used with hard partitioning. Conversely, soft partitioning can be used within an Oracle VM pool to take advantage of Live Migration, along with the ability to manage the Oracle technology CPU count. From an Oracle technology licensing perspective, hard partitioning allows customers to license a subset of a server’s CPUs. Conversely, soft partitioning counts the total number of a server’s CPUs. Soft partitioning with Live Migration requires each Oracle VM server, running a guest with an Oracle technology product to be licensed. We can limit the number of soft partitioned pool members, where a guest can run, by configuring an Oracle VM Manager manual placement policy. Table 1 provides an overview of hard and soft partitioning. Overview Requirements

Hard Hard partitioning 1. All hard partitioned guests Partitioning allows a must pin the virtual CPUs to the

37 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

customer to license a subset of a server’s CPUs.

Oracle VM server’s physical CPU cores in the guest’s vm.cfg file. 2. All hard partitioned guests must have an Oracle VM Manager manual placement policy to confine the guests to the pinned Oracle VM server(s). 3. All hard partitioned guests “cannot” use Live Migration. 1. Each Oracle VM server running a guest with an Oracle technology product must be licensed. We can use a manual placement policy to license a subset of pool member servers. For example, in a 10 server pool, you could license 2 of the 10 pool members.

Soft Soft partitioning Partitioning requires the sum of an Oracle VM server’s CPU cores to be licensed.

List 4 shows three hard and soft partitioning examples with Oracle technology licensing. 1. A single Intel server with 16 CPU cores, with Linux installed running 11G has a processor factor of 8 CPUs. The Linux server can run one 11G instance. 2. A single Intel server with 16 CPU cores, with Oracle VM installed using soft partitioning has a processor factor of 8 CPUs. The Oracle VM server could run more than 16 single CPU guests each with 11G. 3. A single Intel server with 16 CPU cores with Oracle VM installed using hard partitioning. The Oracle VM server is capable of running more that 16 single CPU guests, although only one of the guests is running 11G with 2 virtual CPUs. In this example, using hard partitioning, we could license a subset of the 8 CPUs. For example, we could hard partition 1 of the 8 CPUs. A single Oracle VM server with 2 Intel eight core CPUs (16 cores), could run 16 one CPU guests, without oversubscribing the servers’ CPUs. Oracle VM supports both CPU and memory oversubscription, which allows a single Oracle VM server to oversubscribe CPU and memory resources to guests. For example, an oversubscribed host with 2 Intel eight core CPUs (16 cores), could provision more than 16 cores to guests. Figure 1 shows three hosts. The first host has two eight core CPUs with Linux installed running 11G. The second host has two eight core CPUs using soft partitioning with Oracle VM installed, hosting 8 guests running 11G. The third host has two eight core CPUs using hard partitioning with Oracle VM installed, hosting 8 guests. Only one of the guests is running 11G.

As shown in Figure 1, the Linux server requires eight Oracle technology CPU licenses and is hosting one 11G application. Servers that host one application are commonly referred to as application silos. The traditional one application per server deployment methodology, shown in Figure 1, inevitably leads to over-provisioning and underutilization of hardware. Studies show that most servers run at 5-15% of their total capacity. For example, most servers spend the majority of their life idle, consuming electricity and taking up valuable data center space. Underutilized servers can be consolidated using Oracle VM with hard or soft partitioning to provide better license and resource utilization when compared to application silos. The soft partitioning example in Figure 1 shows how an Oracle VM server with eight processor licenses can host multiple isolated guests, running 11G, for the same CPU cost as the application silo. Oracle VM with soft partitioning provides superior license and resource utilization when compared to an application silo. Oracle VM supports CPU and memory oversubscription, which allows you to run even more workloads per server when compared to an application silo. The hard partitioning example in Figure 1 shows how a shared infrastructure can be used to support Oracle technology products along with the ability to license a subset of the server’s CPUs. For example, we can license one of the eight CPUs. Hard partitioning with Oracle VM can be used with processor licensing and enterprise edition products to manage the number of licensed CPUs for

38 of 243

18.04.2011 16:35

The Underground Oracle VM Manual
development and production environments.

http://itnewscast.com/book/export/html/844

Soft Partitioning Examples Soft partitioning with Oracle VM can be used with processor licensing for both development and production environments. The use case for Oracle VM and soft partitioning with development environments is to consolidate application silos to a shared Oracle VM infrastructure. Migrating from application silos to a shared infrastructure can help reduce the total number of licensed CPUs, reduce electricity consumption, consolidate underutilized server, and free up data center space. The use case for Oracle VM and soft partitioning with production environments, is the ability to use Live Migration, along with the ability to manage the number of licensed CPUs. For example, it is not necessary to license the sum of all Oracle VM pool member’s CPU cores when using Live Migration with soft partitioning. We can configure an Oracle VM Manager manual placement policy to control which pool members a guest can run on. Using a placement policy with soft partitioning allows us to license a subset of an Oracle VM pool’s CPU cores. A manual placement policy confines a guest to run on the pool members listed in the manual placement policy. Figure 2 shows an Oracle VM server pool with six Oracle VM servers. Each Oracle VM server has two eight core CPUs. The Oracle VM server pool has a total of 96 cores, and a processor factor of 48 CPUs. In Figure 2, there is a total of 8 guests in the pool running 11G, with the ability to run on all 6 Oracle VM pool members. The example shown in Figure 2 would require 48 Oracle technology processor licenses.

Figure 3 shows the same server pool as in Figure 2, with a total of 96 cores and a processor factor of 48 CPUs. In Figure 3, there is a total of 8 guests in the pool running 11G, with the ability to run on two Oracle VM pool members. The scenario shown in Figure 3 requires 16 Oracle technology processor licenses.

39 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Figure 4 shows the same server pool as in Figure 2, with 96 cores, and a processor factor of 48 CPUs. In Figure 4, there is a total of 96 guests in the pool, each running 11G, with the ability to run on any of the Oracle VM pool members. The scenario shown in Figure 4 would require 48 Oracle technology processor licenses.

We can limit the number of Oracle VM pool members that a guest can run on, by configuring a manual placement policy in Oracle VM Manager. A manual placement policy allows you to limit which Oracle VM pool members a guest is allowed to run on. Once a manual placement policy is configured, HA events and Live Migration will be limited to the Oracle VM pool members listed in the manual placement policy. Tip: An auto placement policy will start a guest on the least busy pool member and does not limit a guest’s ability to HA or Live Migrate to any pool members. Hard Partitioning Examples Hard partitioning, also referred to as sub-capacity licensing, allows Oracle customers to license a subset of a server’s CPUs. Hard partitioning is a two step process. The first step is to create a manual placement policy. The manual placement policy will confine the guest to the pinned Oracle VM server. The second step, is to edit the hard partitioned guest’s vm.cfg file, to pin the guest’s virtual CPU to the Oracle VM server’s physical CPU cores.

40 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Oracle’s hard partitioning policy states that a hard partitioned guest’s virtual CPUs mapping must be hardcoded in the guest’s vm.cfg file. To confine the hard partitioned guest to the mapped host, a manual placement policy must be configured. Oracle restricts the use of Live Migration with hard partitioning. Our first hard partitioning example in Figure 5 shows an Oracle VM server with two eight core Intel CPUs, with one hard partitioned guest running 11G. The guest is pinned to two of the Oracle VM server’s cores (2 cores = 1 CPU). From an Oracle technology licensing perspective, the server has a processor factor of 8. Using hard partitioning, the pinned guest would require only 1 CPU license. The additional 7 CPUs could be used to license other Oracle technologies or be shared to run other workloads on the same Oracle VM server.

Another example with the server in Figure 5 would be to hard partition two guests, each guest running 11G. Each of the two guests is pinned to 1 of the Oracle VM server’s cores (2 cores = 1 CPU). The additional 7 CPUs could be used to license other Oracle technologies or be shared to run other workloads on the same Oracle VM server. Figure 6 shows an Oracle VM server with two eight core Intel CPUs, with two hard partitioned guests running 11G. Each guest is pinned to 1 of the Oracle VM server’s cores, 2 cores = 1 CPU.

Hard partitioning an Oracle VM guest is a two step process. The first step is to create a manual placement policy for the hard partitioned guest using Oracle VM Manager. The manual placement policy will confine the guest to the pinned Oracle VM server. The second step is to edit the hard partitioned guest’s vm.cfg file to pin the guest’s virtual CPU to the Oracle VM server’s physical CPU cores. Oracle VM Manager Manual Placement Policy Configuration The ability to limit a guest to an Oracle VM server is accomplished by configuring an Oracle VM Manager manual placement policy. A manual placement policy allows you to configure which Oracle VM pool members a guest is allowed to run on. Once a manual placement policy is configured, HA events and Live Migration will be limited to the pool members listed in the manual placement

41 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

policy. A manual placement policy is a guest property that can be configured during or after guest creation. In the next section, we will walk through the configuration of a manual placement policy. Please note that guests must be powered off to configure a placement policy. The first step is to access Oracle VM Manager and power off the guest. Next, click on the guest’s Virtual Machine Name as shown in Figure 7 to access the guest’s properties.

From the General Information page, click the Policies link to access the Policies properties page, as shown in Figure 8.

From the Policies page click the Placement Policy tab, as shown in Figure 9.

From the Placement Policy page click the Manual button to access the Prefer Server page, as shown in Figure 10.

42 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

From the Preferred Server page, select the desired Oracle VM server(s) from the preferred server list. When you select an Oracle VM server from the preferred server list, the manual placement policy will limit the guest to the selected server(s). Once you have selected the preferred server, click the Confirm button, as shown in Figure 11.

After clicking the Confirm button, the page refreshes and displays the Placement Policy page. The new manual placement policy will be displayed as shown Figure 12.

We have successfully configured a manual placement policy for an Oracle VM hard partitioned guest. The next and final step to hard partition an Oracle VM guest is to pin the guest’s virtual CPUs to the Oracle VM server’s CPU cores. Each hard partitioned guest should be pinned to the Oracle VM server that is listed in the guest’s manual placement policy. Hard Partitioning an Oracle VM Guest This section will start with a brief review of the credit scheduler. Next, we walk through the procedure to hard partitioning an Oracle VM guest by adding the “cpus=” directive in a guest’s vm.cfg file. We conclude the section with CPU pinning examples, using the xm and virsh commands. Oracle VM’s default CPU scheduler is the credit scheduler. The credit scheduler uses a credit/debit system to fairly share CPU resources between guests. Credits are assigned to each running guest, along with the fraction of CPU resources. The credit scheduler continually increments/decrements credits from running guests, which is how the credit scheduler balances resources. In many ways, the credit scheduler is like the Linux scheduler. The Linux scheduler is used as the default CPU scheduler with the KVM hypervisor. Both schedulers can preempt processes as needed while trying to ensure proportional fair share allocations. The default behavior of the credit scheduler is to bind each virtual CPU to a separate physical core. For example, when you create a guest with two virtual CPUs, the credit scheduler will map the two virtual CPUs to two physical cores. So when pinning virtual CPUs, we should follow the credit scheduler’s default behavior of mapping virtual CPUs to a server’s individual CPU cores.

43 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Unless you have pinned a guest’s virtual CPUs, virtual CPUs will occasionally bind to different physical cores. Virtual CPUs bind to different physical cores, due to the credit scheduler’s use of the credit/debit system, which dynamically re-balances CPU resources. For example, if you where to periodically check an unpinned guest’s CPU mapping, you would see a different CPU mapping throughout the day. There are two methods to pin virtual CPUs. We can use the xm command to pin a guests’s virtual CPUs or we can hardcode the CPU mapping in a guest’s vm.cfg file. The difference between pinning CPUs with xm and hard coding the CPU mapping in a guest’s vm.cfg file is the persistence of the CPU mapping. CPUs that are pinned with xm are not persistent between reboots. Hard coding the CPU mapping in a guest’s vm.cfg file is persistent between reboots. To comply with Oracle’s hard partitioning policy, we must hardcode the CPU mapping in a guest’s vm.cfg file. Please note that hard partitioning could cause guest performance issues. For example, if you pin a guest’s virtual CPU to a specific subset of named CPUs without considering how the lower-level I/O interrupts are being assigned, you can end-up hurting performance. I/O interrupts are typically mapped to a specific CPU. If that CPU is not the same as the pinned CPU, the interrupts have to be "re-directed" to the CPU you pinned, which could cause the performance of the guest to decrease. If a hard partitioned guest is experiencing performance issues, the CPU pinning would be an area to investigate. Next, we will review how to hard partition an Oracle VM guest. After the hard partitioning example, we will review pinning an Oracle VM guest using the xm command. Unfortunately, all CPU cores are not equal, so you may need to test various virtual CPU mappings using the xm and virsh commands. CPU Pinning Examples In the following example, we will hard partition a guest running Oracle Database 10g enterprise edition with two virtual CPUs. Two virtual Intel or AMD CPUs equal one Oracle technology CPU. The guest will be pinned to an Oracle VM server with two four core CPUs. The Oracle VM server has a processor factor of four Oracle technology CPUs. Using hard partitioning, we will license only one of the Oracle VM server’s four licensable CPUs. To comply with Oracle’s hard partitioning policy, we must hardcode a guest’s virtual CPU mapping, by adding the “cpus=” directive in the guest’s vm.cfg file. By adding the “cpus=” directive in the guest’s vm.cfg file, we pin the guest’s virtual CPUs to the Oracle VM server’s cores. Let’s review two different “cpus=” configurations, to help explain how to pin a guest’s virtual CPUs to an Oracle VM server’s CPU cores. In the first vm.cfg example, we add a new line in the vm.cfg file, cpus = '0-3'.The cpus = '0-3' entry pins the guest’s virtual CPUs to the Oracle VM server’s CPU cores 0, 1, 2, and 3. Please note the vcpus = 4 entry, one line above the cpus = '0-3' entry. The vcpus = 4 entry defines the number of virtual CPUs. The vcpus = directive can be edited to select the desired number of virtual CPUs. #vi /OVS/running_pool/v52x6410g1/vm.cfg bootloader = '/usr/bin/pygrub' disk = ['file:/OVS/running_pool/v52x6410g1/System.img,xvda,w', 'file:/OVS/running_pool/v52x6410g1/oracle10g_x86_64_asm.img,xvdb,w', ] memory = '2048' name = 'v52x6410g1' on_crash = 'restart' on_reboot = 'restart' uuid = 'd428ba07-31b9-5667-2085-8753a0342425' vcpus = 4 cpus = '0-3' vfb = ['type=vnc,vncunused=1,vnclisten=0.0.0.0'] vif = ['bridge=xenbr0,mac=00:16:3E:20:18:19,type=netfront'] vif_other_config = [] The above example vm.cfg file shows a hard partitioned guest with 4 virtual CPUs. The guest’s 4 virtual CPUs are pinned to the Oracle VM server’s CPU cores 0, 1, 2, and 3. The same guest could also be pinned using cpus = '0' in the vm.cfg file. Using cpus = '0' would pin all 4 virtual CPUs to the same physical core, number 0, on the Oracle VM server. The same guest could also be pinned using cpus = '0,1' in the vm.cfg file. Using cpus = '0,1’ would pin 2 virtual CPUs to core number 0 and 2 virtual CPUs to core number 1. We can also use regular expression inversion. For example, we could use cpus=’^0-1’, which means any core but 0 and 1. In the second vm.cfg example, we add a new line in the vm.cfg file, cpus = '0,1'.The cpus = '0,1' entry pins the guest’s virtual CPUs to the Oracle VM server’s CPU cores 0 and 1. Please note the vcpus = 2 entry above the cpus = '0,1' entry. The vcpus = 2 entrydefines the number of virtual CPUs. The vcpus = directive can be edited to select the desired number of virtual CPUs. #vi /OVS/running_pool/v52x6410g1/vm.cfg bootloader = '/usr/bin/pygrub' disk = ['file:/OVS/running_pool/v52x6410g1/System.img,xvda,w', 'file:/OVS/running_pool/v52x6410g1/oracle10g_x86_64_asm.img,xvdb,w', ]

44 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

memory = '2048' name = 'v52x6410g1' on_crash = 'restart' on_reboot = 'restart' uuid = 'd428ba07-31b9-5667-2085-8753a0342425' vcpus = 2 cpus = '0,1' vfb = ['type=vnc,vncunused=1,vnclisten=0.0.0.0'] vif = ['bridge=xenbr0,mac=00:16:3E:20:18:19,type=netfront'] vif_other_config = [] The above example vm.cfg file shows a hard partitioned guest with 2 virtual CPUs pinned to the Oracle VM server’s CPU cores 0 and 1. We can also use regular expression inversion. For example, we could use cpus=’^0-1’, which means any core but 0 and 1. Note: We must reboot the virtual machine to enforce any new hard partitioning configurations. To be able to hard partition a guest, we need to know the number of CPUs and the number of cores on the pinned Oracle VM server. There are a number of commands to list the CPU details of an Oracle VM server. From dom0 we could type “xm info” or “virsh nodeifo” to list the CPU and core details as shown in the next example. # virsh nodeinfo libvir: Remote error : No such file or directory libvir: warning : Failed to find the network: Is the daemon running ? CPU model: i686 CPU(s): 8 CPU frequency: 2992 MHz CPU socket(s): 2 Core(s) per socket: 4 Thread(s) per core: 1 NUMA cell(s): 1 Memory size: 16775168 kB The “virsh nodeinfo” example shows that the Oracle VM server has two four core CPUs (sockets) with a total of eight cores. The example Oracle VM server has an Oracle technology license processor factor of 4 CPUs. To list the CPU cores, we can type “grep -i processor /proc/cpuinfo”, as shown in the next example. # grep -i processor /proc/cpuinfo processor :0 processor :1 processor :2 processor :3 processor :4 processor :5 processor :6 processor :7 The “grep -i processor /proc/cpuinfo” example lists the number of all eight CPU cores. If you would like to list all of the CPU details type “cat /proc/cpuinfo”. Once we have the Oracle VM server’s CPU and core details, we can pin the guest’s virtual CPUs to any of the physical cores. We will follow the default behavior of the credit scheduler and bind each virtual CPU to a separate physical core. Before we pin the guest, let’s review the guest’s vm.cfg file. Please note the vcpus = 2 directive, which indicates the number of virtual CPUs for the guest. #vi /OVS/running_pool/v52x6410g1/vm.cfg bootloader = '/usr/bin/pygrub' disk = ['file:/OVS/running_pool/v52x6410g1/System.img,xvda,w', 'file:/OVS/running_pool/v52x6410g1/oracle10g_x86_64_asm.img,xvdb,w', ] memory = '2048' name = 'v52x6410g1' on_crash = 'restart' on_reboot = 'restart' uuid = 'd428ba07-31b9-5667-2085-8753a0342425' vcpus = 2 vfb = ['type=vnc,vncunused=1,vnclisten=0.0.0.0'] vif = ['bridge=xenbr0,mac=00:16:3E:20:18:19,type=netfront'] vif_other_config = [] Next, we will pin the two virtual CPUs to core 7 and 3 on the Oracle VM server. We will add the cpus =’7,3’ directive to pin the guest’s two virtual CPUs to core 7 and 3 on the Oracle VM server. #vi /OVS/running_pool/v52x6410g1/vm.cfg bootloader = '/usr/bin/pygrub'

45 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

disk = ['file:/OVS/running_pool/v52x6410g1/System.img,xvda,w', 'file:/OVS/running_pool/v52x6410g1/oracle10g_x86_64_asm.img,xvdb,w', ] memory = '2048' name = 'v52x6410g1' on_crash = 'restart' on_reboot = 'restart' uuid = 'd428ba07-31b9-5667-2085-8753a0342425' vcpus = 2 cpus =’7,3’ vfb = ['type=vnc,vncunused=1,vnclisten=0.0.0.0'] vif = ['bridge=xenbr0,mac=00:16:3E:20:18:19,type=netfront'] vif_other_config = [] We must reboot the virtual machine to enforce the new hard partitioning configuration. Note: If a hard partitioned guest uses Live Migration, or has the CPU properties edited with the xm command, the hard coded CPU mapping in the vm.cfg file will be lost. If the CPU mappings get removed by Live Migration, or xm, you will need to re-pin the virtual CPUs in the guest’s vm.cfg file. Once we reboot the guest we can validate our new hard partition configuration by accessing dom0 as root and type xm vcpu-list [domain], as shown in the next example. # xm vcpu-list v52x6410g1 Name ID VCPU CPU State Time(s) CPU Affinity v52x6410g1 19 0 7 -b12.8 3,7 v52x6410g1 19 1 3 -b5.6 3,7 The “xm vcpu-list v52x6410g1” command validates that our hard partition configuration is enforced. By adding the cpus =”7,3” directive, we pinned the guest’s two virtual CPUs, one virtual CPU is pinned to core 7 and one virtual CPU is pinned to core 3. The above hard partition example showed how to license a subset of an Oracle VM servers’ CPUs. The example Oracle VM server has a processor factor of four CPUs. Using hard partitioning, we licensed only one of the Oracle VM server’s four licensable CPUs. CPU Pinning with xm We can also manage the number of virtual CPUs for a running guest using the “xm vcpu-set” command. Using the “xm vcpu-set” command allows us to test and troubleshoot virtual CPU mappings. Please note that using the “xm vcpu-set” command to pin virtual CPU is not recognized by Oracle for hard partitioning. Note: If you have hard coded the CPU mapping in a guest’s vm.cfg file and use the xm command to change the CPU properties, the hard coded CPU mapping will be lost. The “xm vcpu-set” command allow us to select any number of virtual CPUs up to number of virtual CPUs listed in the vcpus = n directive in the vm.cfg file. For example, if a guest has four virtual CPUs, (vcpus = 4) we could use the “xm vcpu-set” command to reconfigure a guest to use 1, 2 or all four of the virtual CPUs. To view a guest’s virtual CPU statistics, from dom0 as root, type xm vcpu-list [domain], as shown in the next example. If you type “xm vcpu-list”, it will list all of the running guest’s virtual CPU statistics. # xm vcpu-list v52x6410g1 Name ID VCPU CPU State Time(s) CPU Affinity v52x6410g1 18 0 2 -b351.4 2 v52x6410g1 18 1 6 -b220.7 6 We can also use the virsh command to list a guest’s virtual CPU details by typing virsh vcpuinfo [domain], as shown in the next example. # virsh vcpuinfo v52x6410g1 libvir: Remote error : No such file or directory libvir: warning : Failed to find the network: Is the daemon running ? VCPU: 0 CPU: 2 State: blocked CPU time: 236.2s CPU Affinity: ---y---y VCPU: 1 CPU: 6 State: blocked CPU time: 178.5s CPU Affinity: ---y---y In the above example, the guest has two virtual CPUs, 0 and 1. Virtual CPU 0 is in the "blocked" state on the physical core number 2. Virtual CPU 1 is in the "blocked" state on the physical core number 6. Both virtual CPUs are in the blocked state, which means the guest is waiting on I/O or has gone to sleep. There is a total of six virtual CPU states, r for running, b for blocked, p for paused, s for shutdown, c for crashed and finally, d for dying. The next example shows how to change the virtual CPU count from two virtual CPUs to one virtual CPU using the “xm vcpu-set”

46 of 243

18.04.2011 16:35

The Underground Oracle VM Manual
command. # xm vcpu-set v52x6410g1 1 # xm vcpu-list v52x6410g1 Name ID VCPU CPU State Time(s) CPU Affinity v52x6410g1 18 0 2 -b359.5 2 v52x6410g1 18 1 - --p 227.0 6

http://itnewscast.com/book/export/html/844

As shown in the above example, typing “xm vcpu-set v52x6410g1 1” paused one of the two virtual CPUs. A paused virtual CPU is not eligible for scheduling by the credit scheduler. The paused virtual CPU will remain paused until resumed, for example, by typing “xm vcpu-set v52x6410g1 2”, as shown in the next example. # xm vcpu-set v52x6410g1 2 # xm vcpu-list v52x6410g1 Name ID VCPU CPU State Time(s) CPU Affinity v52x6410g1 19 0 2 -b266.4 2 v52x6410g1 19 1 6 -b190.5 6 Next, we will pin the guest’s virtual CPUs to the Oracle VM server’s physical cores using the “xm vcpu-pin <domain> <vcpu> <pcpu>” command. In the next example, we will pin the guest’s virtual CPU 0 to core 1, and virtual CPU 1 to core 4. # xm vcpu-pin v52x6410g1 0 1 # xm vcpu-pin v52x6410g1 1 4 # xm vcpu-list v52x6410g1 Name ID VCPU CPU State Time(s) CPU Affinity v52x6410g1 19 0 1 -b268.4 4 v52x6410g1 19 1 4 -b190.5 6 As shown in the above example, typing “xm vcpu-pin v52x6410g1 0 1” followed by typing “xm vcpu-pin v52x6410g1 1 4” pinned the guest’s virtual CPU 0 to core 1 and virtual CPU 1 to core 4. Resources: The Oracle Software Investment Guide (SIG) Hard Partitioning with Oracle VM Red Hat Knowledgebase: How do I determine if my x86-compatible Intel system is multi-processor, multi-core or supports hyperthreading?

Watch this space Oracle OpenWorld 2011 Call for Papers - Submit Today! Temporary Post Used For Theme Detection (409fc5e9-d7ee-4e1a-b645-8b82f256f434 - 3bfe001a-32de-4114-a6b4-4005b770f6d7) Debugging Oracle VM 2.2 & Oracle VM Manager 2.2 errors using the many log files New VDI Training Course now available! HIMSS 2011 and New Press Release Oracle Desktop Virtualization at HIMSS 2011 Oracle VM VirtualBox 4.0.4 Released! Two Virtualization Webinars This Week We're Hiring! - Server and Desktop Virtualization Product Management

Chapter 4: Oracle VM Server Sizing, Installation and Updates
Last update 11-15-2010 Copyright © 2009 - 2011 Roddy Rodstein. All rights reserved. Show a printer friendly version of this book with its sub pages Chapter 4 covers the following topics: Introduction to the installation options for Oracle VM server Oracle VM Server sizing and resource requirements Walkthroughs of two Oracle VM server CD-ROM installations Configuring a boot server for an Oracle VM server PXE/kickstart installation How to create the PXE boot and kickstart files for an automated Oracle VM server installation Overview of how to update an Oracle VM server from the Unbreakable Linux Network (ULN) Table of Contents Oracle VM Server Installation Options Oracle VM Media Pack Download Oracle VM Server Sizing and Resource Requirements …High-Availability and Disaster-Recovery Requirements …Separation of Duties …Oracle VM Server CPU Requirements …Oracle VM Server Memory Requirements …Oracle VM Server Storage Requirements …Oracle VM Server Network Requirements

47 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

…Oracle VM Server Boot From SAN Requirements CD-ROM Oracle VM Server Pool Member Installation …Oracle VM Server Boot From SAN …Installing Oracle VM Server From Other Sources …Post Installation Checklist CD-ROM Oracle VM Server Installation with Local Storage …Installing Oracle VM Server From Other Sources …Post Installation Checklist Oracle VM Server PXE/Kickstart Installation ...HTTP Services Configuration ......How to Make the Oracle VM Server Installation Tree Available for a PXE/Kickstart Installation ...DHCP Service Configurations ...TFTP Service Configurations How to Create the PXE boot Files Kickstart File Examples ...How to Generate an Encrypted Password with grub-md5-crypt ...How to Stage a Kickstart File for an Oracle VM Server PXE/kickstart Installation ...PXE Boot Client Configuration ...PXE/Kickstart Oracle VM Server Installation and Troubleshooting Tips How to Register and Update an Oracle VM Server from the Oracle Unbreakable Linux Network ...The Oracle Unbreakable Linux Support Program ...Oracle VM Server Update Options ...up2date Program Proxy Configurations ...up2date Program Kernel Configurations ...Import Oracle’s GPG Key ...The Oracle Unbreakable Linux Network Registration Process …Update an Oracle VM Server Using the up2date Program and the Oracle Unbreakable Linux Network Oracle VM Server Installation Options Oracle VM server can be installed either from a CD-ROM or over the network using a pre-boot execution environment (PXE). To install Oracle VM server using the CD-ROM or PXE boot method, you will need to visit the Oracle eDelivery/Linux site to download the Oracle VM Media Pack. The Oracle VM Media Pack is delivered as a zip file that contains the Oracle VM server ISO file. The Oracle VM server ISO file can be burned as a bootable disk and used to install Oracle VM server from a CD-ROM drive. Note: Oracle VM Manager is not supported and should not be installed in Oracle VM server's dom0. The zip file from the Oracle eDelivery/Linux site contains a single Oracle VM server ISO file; this file is used for both x86 and x86-64 hardware. The Oracle VM server install routine automatically detects the hardware platform and installs the appropriate 32-bit or 64-bit Xen hypervisor. Regardless of the hardware platform, 32-bit or 64-bit, dom0 is 32-bit. A PXE boot installation requires several additional steps; for example, a boot server and a kickstart file to automate the Oracle VM server installation must be created. The boot server allows a bare-metal system to automatically receive an IP address via DHCP load , a kernel via TFTP and then boot without an installed operating system. Once the bare-metal server boots, you can install Oracle VM , server from the installation media or use a kickstart file to automate the Oracle VM server installation. The default behavior of the Oracle VM server installer is to install Oracle VM server on the Oracle VM server's local disk. To enable the boot from SAN option, type “boot: linux mpath [enter]” at the installation boot prompt. The installation boot prompt is visable on the first Oracle VM Server installation screen. Typing “boot: linux mpath [enter]” tells the installer to use the device-mappermultipath drivers. Oracle VM server can be installed directly from a bootable CDROM, as well as from the Oracle VM server media files that have been staged on a) an Oracle VM server's local hard drive b) on an NFS share c) on an FTP server and/or d) on a web server. The Oracle VM madia files are also refered to as the Oracle VM server media installation tree. To install Oracle VM server from the Oracle VM media files, from the installation boot prompt type "boot: linux askmethod [enter]. Typing "linux askmethod [enter] from the installation boot prompt will enable the Install Method installation screen. From the Install Method screen you can select to install Oracle VM server from a) Local CDROM b) Hard Disk c) NFS image d) FTP or e) HTTP . Tip: Installing Oracle VM server using a bootable CDROM with Lights out Management (LOM) solutions may generate file copy installation errors. If you experience file copy errors, stage the Oracle VM server media files on the a) Hard Disk b) NFS image c) FTP or d) HTTP and from the installation boot prompt type "boot: linux askmethod [enter] to enable the Install Method installation screen. Oracle VM Media Pack Download To download the Oracle VM Media Pack, point your browser to the Oracle eDelivery/Linux site and fill out the registration/export regulations form to gain access to the Oracle VM Media Pack. Once the registration/export regulation form is complete, you be redirected to the Media Pack Search page. From the Media Pack Search page, select Oracle VM from the Select a Product Pack dropdown menu, then select x86 32-bit or x86 64-bit from the Platform dropdown menu. Note that selecting x86 32-bit or x86 64-bit will take you to the same download page; there is only one media pack for both x86 32-bit and x86 64-bit platforms. Click the Go button to be taken to the Oracle VM Media Pack download page. Figure 1 shows the result from the Media Pack Search for Oracle VM.

48 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Select the desired Oracle VM Media Pack, then press the Continue button or click the Oracle VM <version> Media Pack hyperlink to go to the download page. On the Oracle VM <version> Media Pack download page, click the Download button to download the Oracle VM Server <version> media pack. Figure 2 shows the Oracle VM Media Pack download page.

The Oracle VM Server media is delivered as a zip file. The zip file name corresponds to the Part Number listed on the download page. Once the zip file is downloaded, use your favorite zip utility to unzip the Oracle VM ISO file. Next, burn the ISO file to a bootable CD or DVD that can be used to install Oracle VM server using a CD-ROM. Oracle VM Server Sizing and Resource Requirements This section starts with an introduction to the resource requirements for Oracle VM server. The section concludes with a discussion of Oracle VM server CPU, memory, storage, and networking considerations. Oracle VM server runs on x86 32-bit and x86 64-bit platforms with Intel or AMD chips. The minimum resource requirement for your Oracle VM hardware depends on the resource requirements of the guests that will run on your Oracle VM servers. For example, Oracle recommends a dual core CPU or multiple CPUs with at least 1GB or 2GB of RAM. Oracle’s minimum resource recommendation for Oracle VM is a great starting point for running a couple guests for an evaluation. To size your Oracle VM server hardware and Oracle VM server pools, you will need document a) the resource requirements of all of your virtual machines b) your organization’s high-availability and disaster-recovery requirements and c) your organization’s separation of duty requirements for the Oracle VM users and groups. To size your Oracle VM hardware, first calculate the CPU, memory, and storage requirements for all of your guests. Understanding the CPU, memory, and storage requirements for all of your guests allows you to accurately determine the CPU, memory, and storage requirements for your Oracle VM servers and Oracle VM server pools. Consider, for example, the case of virtualizing one 11g database that requires 16 CPUs, 128GB of memory, and 1TB of storage. The 11g database guest could run on a single Oracle VM server with 4 quad core CPUs (16 cores), 132GB of memory and 1TB of local or remote storage. The Oracle VM server in the example would run the 11g database guest without oversubscribing CPU or memory resources. Please note that dom0 requires a minimum of 512MB of memory, therefore the Oracle VM server in the example allocated 132GB of memory for the virtual machine plus 4GB for the overhead in dom0. If the example Oracle VM server uses local storage, it would be a single server Oracle VM pool. If the Oracle VM server uses a shared storage repository, the Oracle VM server could be a pool member in a multiserver pool. Tip: An Oracle VM Manager placement policy can be used to restrict the 11g guest to a specific Oracle VM server. To virtualize two 11g databases, each database with 16 CPUs and 128GB of memory with 1TB of storage, the two 11g database guests could both run on a single Oracle VM server with 4 six core CPUs (24 cores), 260GB of memory, and 2TB of local or remote storage. Running two 11g database guests each with 16 CPUs and 128GB on a single server with 4 six core CPUs (24 cores) would

49 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

oversubscribe the Oracle VM server by 8 cores, which may not be an option if your database workload is CPU bound. An example of not oversubscribing CPUs with two 11g database guests would be to run each guest on a dedicated Oracle VM server with 4 quad core CPUs, 132 GB of memory, and 1TB of storage. High-Availability and Disaster-Recovery Requirements To help calculate the numbers of guests, Oracle VM servers, and Oracle VM server pools for your Oracle VM environment, you will need to understand your organization’s high-availability and disaster-recovery requirements. High availability is an integral data center strategy that allows organizations to meet service level agreements (SLAs) by minimizing or eliminating planned and unplanned downtime. An SLA will specify the levels of availability that determine which high-availability strategy will allow you to meet your availability SLA. For example, a mission-critical application would have an SLA that requires operational continuity in the event of system failure. Conversely, a non-mission-critical application might have an SLA that allows several hours of downtime. Oracle offers a wide variety of high-availability solutions for databases, applications, and operating systems that offer different levels of availability. For example, RAC is Oracle’s database high-availability solution that offers operational continuity in the event of node failure. Oracle DataGuard and Oracle ApplicationGuard are two other Oracle high-availability solutions that offer operational continuity in the event of failure. Oracle VM has two high-availability features; a) guest HA and b) Live Migration. Oracle VM HA automatically restarts guests when; a) a guest hangs or b) when an Oracle VM pool member fails or restarts. Oracle VM HA minimizes unplanned downtime by restarting guests. Live Migration is used to eliminate planned downtime by migrating running guests from one Oracle VM pool member to another during a maintenance event, for example, for repairs or an upgrade. Both HA and Live Migration require a pool configuration with a minimum of two Oracle VM servers with sufficient memory to run all the guests on one host and a shared storage repository. An organization’s high-availability and disaster-recovery requirements will directly affect the numbers of guests, Oracle VM servers, and Oracle VM server pools required to meet your availability SLA. For example, a corporate policy that states that a mission-critical database requires operational continuity will require a clustering solution such as RAC. From an Oracle VM perspective, supporting a database that requires operational continuity requires one dedicated Oracle VM server per RAC node for production environments. If disaster recovery is a requirement for an Oracle VM environment, the number of Oracle VM servers at the disaster-recovery site would be the minimum number of Oracle VM servers required to run all the guests in the event of an outage of the primary data center. Separation of Duties Separation of duties is also a consideration that affects the number of Oracle VM servers and Oracle VM server pools in an Oracle VM environment. For example, many organizations require separation of duties between development and production environments or separation of duties based on geography. Oracle VM supports role-based access control that can be used to isolates Oracle VM resources such as guests and Oracle VM pools based on user and group membership. Oracle VM role-based access control can isolate access to guests within a pool or isolate access to an entire a pool based on user and group membership. For example, role-based access controls could be created for a group named “Development”; these controls could restrict access to resources such as guests, ISO files, templates, and Oracle VM pools used by the Development group. Isolating resources within a pool, for example, isolating guests for groups a and b, would not affect the number of Oracle VM servers within a pool. But isolating resources at the Oracle VM pool level, for example creating a production pool for the Production group and development pool for the Development group, would require dedicated Oracle VM servers for each group, which, in turn, would require additional Oracle VM servers and Oracle VM pools. Understanding your organization’s separation of duties requirements will help to accurately determine the total number of Oracle VM servers and Oracle VM pools within your Oracle VM environment. Oracle VM Server CPU Requirements Oracle VM server supports two modes of virtualization, paravirtualization mode and hardware-virtualization mode. To support hardware-virtualization mode, the Oracle VM server requires either an Intel CPU with VT (virtualization technology) extensions or an AMD CPU with SVM extensions (also called AMD‐V). The system BIOS settings on the Oracle VM server will need to be edited to enable hardware-virtualization support. Hardware-virtualization support is disabled by default. If the CPU does not support hardware virtualization, paravirtualization can be used for Red Hat Enterprise Linux and Oracle Enterprise Linux guests. To support Live Migration between Oracle VM pool members, each pool member should have CPUs of the same CPU family and model. Attempts at Live Migration between two Oracle VM servers with CPUs that are not of the same CPU family and model may fail. Although the CPUs should be of the same family and model to support Live Migration, each Oracle VM server may have a different number of sockets and cores. Tip: To validate the CPU family and model of an Oracle VM server’s CPUs, view /proc/cpuinfo on your Oracle VM server. To effectively size the number of CPUs and cores for an Oracle VM server, the first step is to count the total number of guest CPUs. For example, when a guest is allocated a virtual CPU, the virtual CPU is actually a physical CPU core allocated from an Oracle VM server. A guest with eight virtual CPUs will be allocated eight CPU cores from an Oracle VM server. Oracle VM server supports oversubscribing CPUs, which means that a single Oracle VM server can overallocate its CPU cores. For example, an Oracle VM server with four 6 core CPUs (24 cores) could allocate more than 24 cores to guests. Oversubscribing CPUs with CPU-bound workloads, such as the Oracle Database or RAC, can quickly lead to nasty performance problems. Oversubscribing CPUs should be used with workloads that are not CPU bound to allow greater utilization of the Oracle VM server hardware. For example, the Oracle SOA Suite is an Oracle application that is not CPU bound. Oversubscribing CPUs with Oracle SOA Suite guests would allow greater utilization of the Oracle VM server hardware. Oracle VM Server Memory Requirements

50 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

By default, each Oracle VM server reserves 512MB of memory for dom0. The average memory overhead for each running guest on a dom0 is approximately 20MB plus 1% of the guest’s memory size. The remaining physical memory can be allocated to guests. The memory requirement for a guest depends on the guest’s workload, but will not vary from the memory required for the same workload running on bare metal. For example, if you would like to run two Oracle Database 11g guests on a single Oracle VM server and each guest requires 64GB of memory, the Oracle VM server would have to have a minimum of 130GB of memory for the guests plus 512MB for dom0. Note: That 512MB of memory that is reserved for dom0 is configurable by editing the “dom0_mem=” parameter in /boot/grub /menu.lst file. The 512MB fixed memory overhead for dom0 is rarely an issue except with servers that only have 1–4GB of memory. There is a noteworthy difference between the memory overhead of paravirtualized guests verses hardware-virtualized guests. The overhead for hardware-virtualized guests is much higher than for paravirtualized guests due to the internal data structures, for example, the use of shadow page tables and dedicated QEMU processes for each guest. The overhead for paravirtualized guests is approximately 8MB per guest, regardless of the amount of memory assigned to the guest. Tip: To list the total amount of memory on the system type “xm info | grep mem” in dom0. To effectively size the amount of memory for an Oracle VM server, the first step is to calculate the total amount of memory for all of the guests that could run the Oracle VM server. Do not forget to add the overhead for dom0 and the overhead for each guest, that is, 20MB per guest plus 1% of the each guest’s memory size. The total memory requirements for the guests plus dom0’s memory requirements is the amount of memory required for any Oracle VM server. If HA or Live Migration is used, each Oracle VM server in the pool would need to have enough memory to run all the guests that could run on a given server at any time. To determine the amount of memory for an Oracle VM server pool, is is necessary to consider the additional memory overhead from an HA event or a Live Migration. For example, in an HA-enabled pool with two servers it would be necessary to have enough memory on each server to run all of the guests in the pool in the event of an Oracle VM server failure. Oracle VM 2.x does not officially support memory overcommit, which means that an Oracle VM server equipped with xGB of memory can only allocate the available memory. To be able to support an HA event or Live Migration in an Oracle VM server pool, each Oracle VM pool member must have enough free memory to be able to accept any new guests. For example, in an HA-enabled pool with two servers, if one server fails, the available server must have enough free memory to run the guests from the failed server. If an HA event occurrs between two pool members and the target server does not have enough free memory for the guests, the guests will be blocked from starting on the target Oracle VM server. Note: Oracle VM 2.2/Xen 3.4.0 ships with the experimental Xenballoond memory overcommit feature, although Xenballoond memory overcommit is not enabled or supported by Oracle. In January 2009, Dan Magenheimer from Oracle announced the "Transcendent Memory" project, "tmem" for short. Tmem will improve on VMware’s long-available but fatally flawed mechanism for "time-sharing" physical memory between virtual machines, commonly known as "ballooning .” The results of tmem will be better memory utilization and fewer disk accesses, which, in turn, will lead to higher performance and greater virtual-machine density per Oracle VM server. Tmem may be included in the Oracle VM 3.0 release. Oracle VM Server Storage Requirements Unless the Oracle VM server is booting from SAN, some form of local storage is required. A default Oracle VM 2.2 server installation creates a “local” OCFS2 virtual machine file system that is mounted under /var/ovs/mount/UUID and linked to /OVS. Using a local storage repository restricts pool membership to one Oracle VM server without Live Migration or HA functionality. To increase the capacity of an Oracle VM pool past one Oracle VM server, the addition of a shared back-end storage repository is required. To determine your storage requirements for a single- or multiple-server Oracle VM server pool, calculate the disk requirements for all of your guests, ISO files, and templates. To account for growth, consider provisioning at least 30% to 50% more storage for your Oracle VM storage repositories than the expected size. Oracle VM’s installation program does not provide the ability to configure shared storage repositories. Storage administration for storage arrays must be configured after the installation of the server. Oracle VM Server Network Requirements Oracle VM server uses Xen bridging to set up the networking for guest traffic. The Xen bridge configuration allows all domUs to appear on the network as individual hosts. All virtual network device communication is routed through a Xen bridge, which operates at layer 2 of the OSI model, effectively acting as an L2 virtual switch. To ensure that dom0 and your Xen bridges provide sufficient throughput and availability for guests, consider bonding multiple network interfaces to increase throughput and availability. For example, mode4 network bonding (802.3AB), running with tagged VLANs (802.1Q) allows better than single-wire-speed transfers from multiple hosts. Oracle VM’s installation program does not include network interfaces bonding and VLAN configurations. Network interfaces bonding and VLAN configurations must be done after the installation of the server. Oracle VM Server Boot From SAN Requirements The default behavior of the Oracle VM server installer is to install Oracle VM server on the Oracle VM server's local disk. To enable the boot from SAN option, type “boot: linux mpath [enter]” from the installation boot prompt. The installation boot prompt is displayed at the first Oracle VM Server installation screen. Typing “boot: linux mpath [enter]” tells the installer to use the device-

51 of 243

18.04.2011 16:35

The Underground Oracle VM Manual
mapper-multipath drivers.

http://itnewscast.com/book/export/html/844

Before a boot from SAN installation is started, ensure that the blade chassis or servers are configured to support boot from SAN. Next, provision, zone and mask at least two LUNs. One unique LUN per server is the the boot LUN. The boot LUN is where Oracle VM server is installed. Do not use the second LUN during the installation. After the installation, the second LUN will be formatted with OCFS2 and configured as the shared root storage repository. The shared root storage repository is the virtual machine file system that is shared between all of the Oracle VM server pool members. To enable the boot from SAN installation option, when the server boots using the Oracle VM ISO, it is necessary to pass the mpath parameter. The next example shows how to pass the mpath parameter from the Oracle VM server boot prompt. boot: linux mpath [enter] Typing “boot: linux mpath [enter]” from the boot prompt at the first Oracle VM Server screen tells the installer to use the devicemapper-multipath drivers. Once the installer is using the device-mapper-multipath drivers, all zoned and maked LUNs will be visable during the installation process. Be sure to only select the boot LUN for the installation! CD-ROM Oracle VM Server Pool Member Installation This section discusses an Oracle VM server installation for an Oracle VM pool member. The difference between an installation for an Oracle VM server pool member and an Oracle VM server with local storage is that the Oracle VM pool member does not need a dedicated /OVS partition. The pool member installation described below will remove the default /OVS partition and reallocate the disk space from the deleted /OVS partition to the / partition. A default Oracle VM server installation will allocate most of the disk space to the /OVS partition. The local disk requirements foran Oracle VM pool member installation is roughly 2GB. Oracle VM Server Boot From SAN The default behavior of the Oracle VM server installer is to install Oracle VM server on the Oracle VM server's local disk. To enable the boot from SAN option, type “boot: linux mpath [enter]” from the installation boot prompt. The installation boot prompt is displayed at the first Oracle VM Server installation screen. Typing “boot: linux mpath [enter]” tells the installer to use the devicemapper-multipath drivers. Before a boot from SAN installation is started, ensure that the blade chassis or servers are configured to support boot from SAN. Next, provision, zone and mask at least two LUNs. One unique LUN per server is the the boot LUN. The boot LUN is where Oracle VM server is installed. Do not use the second LUN during the installation. After the installation, the second LUN will be formatted with OCFS2 and configured as the shared root storage repository. The shared root storage repository is the virtual machine file system that is shared between all of the Oracle VM server pool members. To enable the boot from SAN installation option, when the server boots using the Oracle VM ISO, it is necessary to pass the mpath parameter. The next example shows how to pass the mpath parameter from the Oracle VM server boot prompt. boot: linux mpath [enter] Typing “boot: linux mpath [enter]” from the boot prompt at the first Oracle VM Server screen tells the installer to use the devicemapper-multipath drivers. Once the installer is using the device-mapper-multipath drivers, all zoned and maked LUNs will be visable during the installation process. Be sure to only select the boot LUN for the installation! Installing Oracle VM Server From Other Sources Oracle VM server can be installed directly from a bootable CDROM, as well as from the Oracle VM server media files (aka the Oracle VM server installation tree) that have been staged on a) an Oracle VM server's local hard drive b) on an NFS share c) on an FTP server and/or d) on a web server. To install Oracle VM server from the Oracle VM media files, from the installation boot prompt type "boot: linux askmethod [enter]. Typing "linux askmethod [enter] from the installation boot prompt will enable the Install Method installation screen. From the Install Method screen you can select to install Oracle VM server from a) Local CDROM b) Hard Disk c) NFS image d) FTP or e) HTTP . Tip: Installing Oracle VM server using a bootable CDROM with Lights out Management (LOM) solutions may generate file copy installation errors. If you experience file copy errors, stage the Oracle VM server media files on the a) Hard Disk b) NFS image c) FTP or d) HTTP and from the installation boot prompt type "boot: linux askmethod [enter] to enable the Install Method installation screen. To install Oracle VM server using a CD-ROM: 1. Insert the Oracle VM server media into the CD-ROM drive. 2. Boot the server with the Oracle VM server media in the CD-ROM drive. 3. The Oracle VM Server screen is displayed, as shown in Figure 3. Figure 3 shows the Oracle VM Server screen.

52 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

4. a) To Install Oracle VM server on the local hard drive: Press the Enter key to start the install program. If the Enter key is not pressed for one minute, the install program will automatically start. 4. b) To install Oracle VM server to Boot from SAN: To enable the boot from SAN option, type “linux mpath [enter]” at the boot prompt. Typing “boot: linux mpath [enter]” tells the installer to use the device-mapper-multipath drivers. The next example show how to enable the boot from SAN option. boot: linux mpath [enter] Typing boot: linux mpath [enter] will continue the installation process. 4. c) To Install Oracle VM server from Other Sources: From the installation boot prompt type "boot: linux askmethod [enter] to enable the Install Method installation screen. From the Install Method installation screen, select and enter the details for the Hard drive, NFS image, FTP or HTTP installation media. The next example show how to enable the Install Method installation screen. boot: linux askmethod [enter] Typing boot: askmethod [enter] will continue the installation process. The CD Found screen On the CD Found screen, you can test the media for errors. To test the media, use the Tab key to select the OK button and press Enter. Once the media test is completed, any errors will be reported. To skip the media test and continue with the install, use the Tab key to select the Skip button and press Enter to continue.

Figure 4 shows the CD Found screen.

The Keyboard Selection screen On the Keyboard Selection screen, use the Tab key to select the list of keyboard models. Then use the UP and DOWN keys (↑ or ↓) to select the desired keyboard model. The keyboard that is selected becomes the default keyboard for dom0. Next, use the Tab key to select OK, and press Enter to continue.

53 of 243

18.04.2011 16:35

The Underground Oracle VM Manual
Figure 5 shows the Keyboard Selection screen.

http://itnewscast.com/book/export/html/844

Warning screen If you see the Warning screen, use the Tab key to select the Yes button, then press Enter to continue. Figure 6 shows the Warning screen.

Partitioning Type screen The Partitioning Type screen offers the following four partitioning options: Remove all partitions and create a new default partition layout Remove all Linux partitions and create a new default partition layout Use the free space on selected drives to create a new default partition layout Create a custom partition layout Use the Tab key to select the Remove all partitions and create a new default partition layout option. Ensure that the appropriate drive is select in the Which drive(s) do you want to use for this installation section. Use the Tab key to select the OK button to continue. Figure 7 shows the Partitioning Type screen.

54 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Warning screen Since we selected the Remove all partitions and create a new default partition layout option, a Warning screen is displayed to confirm that we want to remove the partition(s), including all of the data contained on any of the selected partitions. Use the Tab key to select the YES button to continue. Figure 8 shows the Warning screen.

Review Partition Layout screen On the Review Partition Layout screen, use the Tab key to select the YES button to continue to the Partitioning screen. Figure 9 shows the Review Partition Layout screen.

Partitioning screen On the Partitioning screen, use the Tab key to select the /OVS Mount Point, then use the Tab key to select the Delete button. Press Enter to continue. Figure 10 shows the Partitioning screen.

Confirm Delete screen On the Confirm Delete screen,use the Tab key to select the Delete button, then press Enter to continue.

55 of 243

18.04.2011 16:35

The Underground Oracle VM Manual
Figure 11 shows the Confirm Delete screen.

http://itnewscast.com/book/export/html/844

Partitioning screen On the Partitioning screen, use the Tab key to select the / Mount Point, then use the Tab key to select the Edit button. Press Enter to continue. Figure 12 shows the Partitioning screen.

Add Partition screen On the Add Partition screen, use the Tab key to select the Fill all available space option. Next, press the Space bar to select the Fill all available space option. Use the Tab key to select the OK button to proceed. Figure 13 shows the Add Partition screen

Partitioning screen On the Partitioning screen, use the Tab key to select the OK button. Press Enter to continue. Figure 14 shows the Partitioning screen.

56 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

The Boot Loader Configuration screen On the Boot Loader Configuration screen, use the Tab key to select the Master Boot Record (MBR) or the First sector of boot partition as the location to install the boot loader. For this example, we have selected the Master Boot Record (MBR) option.Next, use the Tab key to select the OK button and press Enter to continue. Figure 15 shows the Boot Loader Configuration screen.

Oracle VM Server Management Interface screen On the Oracle VM Server Management Interface screen, use the Tab key to select the network interface that will be dedicated for the dom0 and the Oracle VM agent management traffic. Once the network interface is selected, use the Tab key to select the OK button and press Enter to continue. Note: dom0’s management interface defaults to eth0, which is controlled in the /etc/ovs-config file. The dom0 management interface can be changed after the installation. Figure 16 shows the Oracle VM Server Management Interface screen.

The IPv4 Configuration for eth0 screen The IPv4 Configuration for eth0 screen offers the following three options: Dynamic IP configuration (DHCP)

57 of 243

18.04.2011 16:35

The Underground Oracle VM Manual
Manual address configuration IP Address and Prefix (netmask)

http://itnewscast.com/book/export/html/844

If your Oracle VM server will use DHCP to assign its IP address, select the Dynamic IP configuration (DHCP) option.To select the Dynamic IP configuration (DHCP) entry, use the Tab key to highlight the Dynamic IP configuration (DHCP) entry, then use the Space bar to select the Dynamic IP configuration (DHCP) entry. Use the Tab key to select the OK button to continue. If your Oracle VM server will use a static IP address, select the Manual address configuration entry.To select the Manual address configuration entry use the Tab key to highlight the Manual address configuration entry, then use the Space bar to select the Manual address configuration entry. Next, use the Tab key to enter the IP Address and Prefix (netmask). Use the Tab key to select the OK button to continue. Figure 17 shows the IPv4 Configuration for eth0 screen.

The Miscellaneous Network Settings screen On the Miscellaneous Network Settings screen, use the Tab key to select the Gateway, Primary DNS and optional Secondary DNS to enter the networking settings for your environment. Use the Tab key to select the OK button and press Enter to continue. Figure 18 shows the Miscellaneous Network Settings screen.

Hostname Configuration screen On the Hostname Configuration screen, select one of the following two options: automatically via DHCP manually If the machine uses DHCP to assign its hostname, select the automatically via DHCP option. Then, use the Tab key to select the OK button to continue. To assign a hostname for your Oracle VM server, select the manually option and enter the fully qualified domain name (FQDN) in the text box. Then, use the Tab key to select the OK button to continue. Figure 19 shows the Hostname Configuration screen.

58 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Time Zone Selection screen On the Time Zone Selection screen select the System clock uses UTC option to use Coordinated Universal Time (UTC), then use the Tab key and the UP or DOWN key (↑ or ↓) to select the time zone closest to your Oracle VM server’s physical location. Next, use the Tab key to select the OK button and press Enter to continue. Figure 20 shows the Time Zone Selection screen.

Oracle VM Agent password screen On the Oracle VM Agent password screen, enter the password for the Oracle VM agent in the Password field. In the Password (confirm) field, reenter the password. Use the Tab key to select the OK button and press Enter to continue. If the two passwords do not match, the installation program will ask you to reenter the passwords. The Oracle VM agent password is used by Oracle VM Manager and the Oracle VM Management Pack to dispatch commands and to retrieve pool-status data. The Oracle VM agent password can be changed after the installation using Oracle VM Manager or the Oracle VM Management Pack and from dom0 by typing “service ovs-agent configure”. Figure 21 shows the Oracle VM Agent password screen

Root Password screen On the Root Password screen, enter a password with at least six characters for the root user in the Password field. In the Password (confirm) field, reenter the password. Use the Tab key to select the OK button and press Enter to continue. If the two passwords do

59 of 243

18.04.2011 16:35

The Underground Oracle VM Manual
not match, the installation program will ask you to reenter the passwords. Figure 22 shows the Root Password screen.

http://itnewscast.com/book/export/html/844

Installation to begin screen On the Installation to begin screen, select OK and press Enter to continue. Figure 23 shows the Installation to begin screen.

Complete screen When the Complete screen appears, remove the Oracle VM Server media from the CD-ROM drive and press Enter to reboot the Oracle VM server. Note: Remain at the Oracle VM servers console until the server reboots in order to accept the End User License Agreement, which is displayed after the server reboots. For example, if you ssh in to the server after the reboot, you will not be presented with the End User License Agreement. If the End User License Agreement is not accepted after the reboot, the Oracle VM agent will not be started. Figure 24 shows the Complete screen.

60 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

End User License Agreement screen On the End User License Agreement screen use the UP or DOWN key (↑ or ↓) to read the End User License Agreement. Then, use the Tab key to select the Agree button and press Enter to continue. TIP: If you ssh in to the server after the reboot, you will not be presented with the End User License Agreement. If the End User License Agreement is not accepted after the reboot, the Oracle VM agent will not be started. Figure 25 shows the End User License Agreement screen.

Oracle VM Server login prompt screen On the Oracle VM Server login prompt screen, enter the root username and the password to access the dom0 console. Figure 26 shows the Oracle VM Server login prompt screen.

Post Installation Checklist 1. All Oracle VM servers must be patched from the Unbreakable Linux Network (ULN) to ensure that the server and pool configurations will not be hampered by unpatched bugs. 2. Ensure that all the Oracle VM servers’ clocks are synchronized using NTP . First, open the “/etc/ntp.conf” file by typing “vi /etc/ntp.conf” and validate that at least two available NTP servers entries are listed. The next example shows two bold NTP server entries in an ntp.conf file. # Use public servers from the pool.ntp.org project. # Please consider joining the pool (http://www.pool.ntp.org/join.html). server myntp1.com server myntp2.com Ping each NTP server listed in the ntp.conf file from each Oracle VM server to ensure network connectivity. Next, type "ntpstat" on each Oracle VM server to validate the NTP configuration. The next example shows the output from typing the ntpstat command on an Oracle VM server that has its time synchronized to an NTP server with the IP address of 192.168.4.251. # ntpstat synchronized to NTP server (192.168.4.251) at stratum 4 time correct to within 54 ms polling server every 1024 s Finally, validate that the time, date and time zone on each Oracle VM server as well as on the Oracle VM Manager host is

61 of 243

18.04.2011 16:35

The Underground Oracle VM Manual
synchronized by typing the "date" command.

http://itnewscast.com/book/export/html/844

3. All Oracle VM servers have consistent name resolution using DNS with both forward and reverse lookups. First, open the “/etc/resolv.conf” file by typing “vi /etc/resolv.conf” and validate that two available DNS servers are listed. The next example shows two DNS servers listed in a resolve.conf file. # vi /etc/resolve.conf nameserver <MY DNS SERVER1 IP ADDRESS> nameserver <MY DNS SERVER2 IP ADDRESS> From each Oracle VM server ping each DNS server listed in the resolv.conf file to ensure network connectivity. Next, validate the forward and reverse lookups for each Oracle VM pool member and the Oracle VM Manager host using the “host” command. For example, to validate server2's forward lookup from server1 type “host server2” as shown in the next example. # host server2 server2 has address 192.168.4.6 Next, to validate server2's reverse lookup from server1 type “host 192.168.4.6” as shown in the next example. # host 192.168.4.6 6.4.168.192.in-addr.arpa domain name pointer server2 Note: Using hosts files without DNS is not advised and may produce unpredictable results. 4. The Oracle VM server’s host name in the /etc/hosts file must be associated with the Oracle VM server's public IP address. If an Oracle VM pool member's host name is associated with 127.0.0.1, the cluster.conf file will be malformed and the Oracle VM pool will not be operational. The next example shows the improper syntax from an Oracle VM server's hosts file entry. 127.0.0.1 servername.com servername localhost.localdomain localhost 192.168.4.8 servername.com servername The next example shows the proper syntax for an Oracle VM server’s hosts file entry. 127.0.0.1 localhost 192.168.4.8 servername localhost.localdomain servername.com

5. ocfs2 network connectivity between all Oracle VM server pool members must be operational before creating a multiple server pool. Check the ocfs2 network connectivity between all Oracle VM pool members by typing "nc -zv <myoraclevmserver1> 7777". For example, if you have two Oracle VM servers named ovs1 and ovs2, from ovs1 type "nc -zv ovs2 7777". Typing "nc -zv ovs2 7777" from ovs1 should return "succeeded!". If you receive a "failed: Connection refused" message between any Oracle VM servers, something (firewall, switch, router, cable, etc..) is restricting communication between the hosts. The iptables firewall on an Oracle VM server may be blocking the ocfs2 connectivity. If iptables is disabled and allowing all connections, the output from typing “iptables -L will look like the next example. ” # iptables -L Chain INPUT (policy ACCEPT) target prot opt source destination Chain FORWARD (policy ACCEPT) target prot opt source destination Chain OUTPUT (policy ACCEPT) target prot opt source destination If typing “iptables -L lists firewall rules, you can a) disable iptables by typing "service iptables stop && chkconfig iptables off" or b) ” add the following bold iptables rule to the /etc/sysconfig/iptables file prior to the last line on all Oracle VM pool members. -A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 7777 -j ACCEPT -A RH-Firewall-1-INPUT -j REJECT --reject-with icmp-host-prohibited COMMIT After you have added the above bold iptables rule, restart the iptables service by typing "service iptables restart". CD-ROM Oracle VM Server Installation with Local Storage

62 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

This section discusses a default Oracle VM installation with local storage. A default Oracle VM server installation will allocate most of the disk space to the /OVS partition. The difference between an installation for an Oracle VM server with local storage and a pool member is that the pool member does not need an /OVS partition with dedicated storage. A pool member installation removes the default /OVS partition and reallocates the disk space from the /OVS partition to the / partition. Installing Oracle VM Server From Other Sources Oracle VM server can be installed directly from a bootable CDROM, as well as from the Oracle VM server media files (aka the Oracle VM server installation tree) that have been staged on a) an Oracle VM server's local hard drive b) on an NFS share c) on an FTP server and/or d) on a web server. To install Oracle VM server from the Oracle VM media files, from the installation boot prompt type "boot: linux askmethod [enter]. Typing "linux askmethod [enter] from the installation boot prompt will enable the Install Method installation screen. From the Install Method screen you can select to install Oracle VM server from a) Local CDROM b) Hard Disk c) NFS image d) FTP or e) HTTP . Tip: Installing Oracle VM server using a bootable CDROM with Lights out Management (LOM) solutions may generate file copy installation errors. If you experience file copy errors, stage the Oracle VM server media files on the a) Hard Disk b) NFS image c) FTP or d) HTTP and from the installation boot prompt type "boot: linux askmethod [enter] to enable the Install Method installation screen. To install Oracle VM server using a CD-ROM: 1. Insert the Oracle VM server media into the CD-ROM drive. 2. Boot the server with the Oracle VM server media in the CD-ROM drive. 3. The Oracle VM Server screen is displayed, as shown in Figure 31. Figure 27 shows the Oracle VM Server screen.

4. a) To Install Oracle VM server on the local hard drive: Press the Enter key to start the install program. If the Enter key is not pressed for one minute, the install program will automatically start. 4. b) To Install Oracle VM server from Other Sources: From the installation boot prompt type "boot: linux askmethod [enter] to enable the Install Method installation screen. From the Install Method installation screen, select and enter the details for the Hard drive, NFS image, FTP or HTTP installation media. The next example show how to enable the Install Method installation screen. boot: linux askmethod [enter] Typing boot: askmethod [enter] will continue the installation process.

The CD Found screen On the CD Found screen, you can test the media for errors. To test the media, use the Tab key to select the OK button and press Enter. Once the media test is completed, any errors will be reported. To skip the media test and continue with the install, use the Tab key to select the Skip button and press Enter to continue. Figure 28 show the CD Found screen.

63 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

The Keyboard Selection screen On the Keyboard Selection screen, use the Tab key to select the list of keyboard models. Then use the UP or DOWN key (↑ or ↓) to select the desired keyboard model. The keyboard that is selected becomes the default keyboard for dom0. Next, use the Tab key to select OK, then press Enter to continue. Figure 29 shows the Keyboard Selection screen.

Warning screen If you see the Warning screen, use the Tab key to select the Yes button, then press Enter to continue. Figure 30 shows the Warning screen.

Partitioning Type screen The Partitioning Type screen offers the following four partitioning options: Remove all partitions and create a new default partition layout Remove all Linux partitions and create a new default partition layout Use the free space on selected drives to create a new default partition layout Create a custom partition layout

64 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Use the Tab key to select the Remove all partitions and create a new default partition layout option. Ensure that the appropriate drive is select in the Which drive(s) do you want to use for this installation section. Use the Tab key to select the OK button to continue. Figure 31 shows the Partitioning Type screen.

Warning screen Since we selected the Remove all partitions and create a new default partition layout option, a Warning screen is displayed to confirm that we want to remove the partition(s), including all of the data contained on any of the selected partitions. Use the Tab key to select the YES button to continue. Figure 32 shows the Warning screen.

Review Partition Layout screen From the Review Partition Layout screen, use the Tab key to select the NO button and press Enter to continue. Figure 33 shows the Review Partition Layout screen.

The Boot Loader Configuration screen On the Boot Loader Configuration screen, use the Tab key to select the Master Boot Record (MBR) or the First sector of boot

65 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

partition as the location to install the boot loader. For this example, we selected the Master Boot Record (MBR) option.Use the Tab key to select the OK button and press Enter to continue. Figure 34 shows the Boot Loader Configuration screen.

Oracle VM Server Management Interface screen On the Oracle VM Server Management Interface screen, use the Tab key to select the network interface that will be dedicated for the dom0 and the Oracle VM agent management traffic. Once the network interface is selected, use the Tab key to select the OK button and press Enter to continue. Note: dom0’s management interface defaults to eth0, which is controlled in the /etc/ovs-config file. The dom0 management interface can be changed after the installation. Figure 35 shows the Oracle VM Server Management Interface screen.

The IPv4 Configuration for eth0 screen The IPv4 Configuration for eth0 screen offers the following three options: Dynamic IP configuration (DHCP) Manual address configuration IP Address and Prefix (netmask) If your Oracle VM server will use DHCP to assign its IP address, select the Dynamic IP configuration (DHCP) option.You can select the Dynamic IP configuration (DHCP) entry by using the Tab key to highlight the Dynamic IP configuration (DHCP) entry then use the Space bar to select the Dynamic IP configuration (DHCP) entry. Use the Tab key to select the OK button to continue. If your Oracle VM server will use a static IP address, select the Manual address configuration entry.You can select the Manual address configuration entry by using the Tab key to highlight the Manual address configuration entry, then use the Space bar to select the Manual address configuration entry. Next, use the Tab key to enter the IP Address and Prefix (netmask). Use the Tab key to select the OK button to continue. Figure 36 shows the IPv4 Configuration for eth0 screen.

66 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

The Miscellaneous Network Settings screen On the Miscellaneous Network Settings screen, use the Tab key to select the Gateway, Primary DNS, and optional Secondary DNS to enter the networking settings for your environment. Use the Tab key to select the OK button and press Enter to continue. Figure 37 shows the Miscellaneous Network Settings screen.

Hostname Configuration screen On the Hostname Configuration screen, select one of the following two options: automatically via DHCP manually If the machine uses DHCP to assign its hostname, select the automatically via DHCP option. Use the Tab key to select the OK button to continue. To assign a hostname for your Oracle VM server, select the manually option and enter the fully qualified domain name (FQDN) in the text box. Use the Tab key to select the OK button to continue. Figure 38 shows the Hostname Configuration screen.

Time Zone Selection screen

67 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

On the Time Zone Selection screen select the System clock uses UTC option to use Coordinated Universal Time (UTC), then use the Tab key and the UP or DOWN key (↑ or ↓) to select the time zone closest to your Oracle VM server’s physical location. Use the Tab key to select the OK button and press Enter to continue. Figure 39 shows the Time Zone Selection screen.

Oracle VM Agent password screen On the Oracle VM Agent password screen, enter the password for the Oracle VM agent in the Password field. In the Password (confirm) field, reenter the password. Use the Tab key to select the OK button and press Enter to continue. If the two passwords do not match, the installation program will ask you to reenter the passwords. The Oracle VM agent password is used by Oracle VM Manager and the Oracle VM Management Pack to dispatch commands and to retrieve pool status data. The Oracle VM agent password can be changed after the installation using Oracle VM Manager or the Oracle VM Management Pack, or from dom0 by typing “service ovs-agent configure”. Figure 40 shows the Oracle VM Agent password screen

Root Password screen On the Root Password screen, enter a password with at least six characters for the root user in the Password field. In the Password (confirm) field, reenter the password. Use the Tab key to select the OK button and press Enter to continue. If the two passwords do not match, the installation program will ask you to reenter the passwords. Figure 41 shows the Root Password screen.

68 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Installation to begin screen On the Installation to begin screen, select OK and press Enter to continue. Figure 42 shows the Installation to begin screen.

Complete screen When the Complete screen appears, remove the Oracle VM Server media from the CD-ROM drive and press Enter to reboot the Oracle VM server. TIP: Remain at the Oracle VM servers console until the server reboots to be able to accept the End User License Agreement, which is displayed after the server reboots. For example, if you ssh in to the server after the reboot you will not be presented with the End User License Agreement. If the End User License Agreement is not accepted after the reboot, the Oracle VM agent will not be started. Figure 43 shows the Complete screen.

End User License Agreement screen On the End User License Agreement screen, use the UP or DOWN key (↑ or ↓) to read the End User License Agreement. Use the Tab key to select the Agree button and press Enter to continue.

69 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

TIP: If you ssh in to the server after the reboot you will not be presented with the End User License Agreement. If the End User License Agreement is not accepted after the reboot the Oracle VM agent will not be started. Figure 44 shows the End User License Agreement screen.

Oracle VM Server login prompt screen On the Oracle VM Server login prompt screen, enter the root username and the password to access the dom0 console. Figure 45 shows the Oracle VM Server login prompt screen.

Post Installation Checklist 1. The Oracle VM servers must be patched from the Unbreakable Linux Network (ULN) to ensure that the server and pool configurations will not be hampered by unpatched bugs. 2. Ensure that all the Oracle VM servers’ clock is synchronized using NTP . First, open the “/etc/ntp.conf” file by typing “vi /etc/ntp.conf” and validate that at least two available NTP servers entries are listed. The next example shows two bold NTP server entries in an ntp.conf file. # Use public servers from the pool.ntp.org project. # Please consider joining the pool (http://www.pool.ntp.org/join.html). server myntp1.com server myntp2.com Ping each NTP server listed in the ntp.conf file from each Oracle VM server to ensure network connectivity. Next, type "ntpstat" on each Oracle VM server to validate the NTP configuration. The next example shows the output from typing the ntpstat command on an Oracle VM server that has its time synchronized to an NTP server with the IP address of 192.168.4.251. # ntpstat synchronized to NTP server (192.168.4.251) at stratum 4 time correct to within 54 ms polling server every 1024 s Finally, validate that the time, date and time zone on each Oracle VM server as well as on the Oracle VM Manager host is synchronized by typing the "date" command. 3. The Oracle VM server must have consistent name resolution using DNS with both forward and reverse lookups. First, open the “/etc/resolv.conf” file by typing “vi /etc/resolv.conf” and validate that two available DNS servers are listed. The next

70 of 243

18.04.2011 16:35

The Underground Oracle VM Manual
example shows two DNS servers listed in a resolve.conf file. # vi /etc/resolve.conf nameserver <MY DNS SERVER1 IP ADDRESS> nameserver <MY DNS SERVER2 IP ADDRESS>

http://itnewscast.com/book/export/html/844

From each Oracle VM server ping each DNS server listed in the resolv.conf file to ensure network connectivity. Next, validate the forward and reverse lookups for each Oracle VM pool member and the Oracle VM Manager host using the “host” command. For example, to validate server2's forward lookup from server1 type “host server2” as shown in the next example. # host server2 server2 has address 192.168.4.6 Next, to validate server2's reverse lookup from server1 type “host 192.168.4.6” as shown in the next example. # host 192.168.4.6 6.4.168.192.in-addr.arpa domain name pointer server2 Note: Using hosts files without DNS is not advised and may produce unpredictable results. 4. The Oracle VM server’s host name in the /etc/hosts file must be associated with the Oracle VM server's public IP address. If an Oracle VM pool member's host name is associated with 127.0.0.1, the cluster.conf file will be malformed and the Oracle VM pool will not be operational. The next example shows the improper syntax from an Oracle VM server's hosts file entry. 127.0.0.1 servername.com servername localhost.localdomain localhost 192.168.4.8 servername.com servername The next example shows the proper syntax for an Oracle VM server’s hosts file entry. 127.0.0.1 localhost 192.168.4.8 servername localhost.localdomain servername.com

Oracle VM Server PXE/Kickstart Installation This section starts with a discussion of an Enterprise Linux boot server configuration using RPM packages from the Unbreakable Linux Network (ULN). After the boot server configuration, we will learn how to create and stage a kickstart file to automate an Oracle VM server installation. Next, we will discuss the PXE boot client requirements and configurations. The section concludes with PXE/kickstart installation troubleshooting tips. This section discusses how to configure an Enterprise Linux boot server with DHCP TFTP and HTTP The boot server allows a , , . bare-metal system to receive an IP address via DHCP load a kernel via TFTP and boot without an operating system. Next, the , , kickstart file orchestrates an automated Oracle VM server installation. Once the installation is completed, a personalized Oracle VM server is booted and can be added to a pool using Oracle VM Manager or the Oracle VM Management Pack. The DHCP service will be configured with network-specific details along with the IP addresses of the Oracle VM servers you will PXE/kickstart install. You will need to change the example network details with your specific environmental networking details. The TFTP service will be configured with a /tftpboot directory populated with: 1. 2. 3. 4. The pxelinux.0 bootloader program in /tftpboot The vmlinuz compressed Linux kernel in /tftpboot The initrd.img initial RAM disk in /tftpboot/ovm/2.2 A subdirectory named pxelinux.cfg with the PXE boot files. The example will have the following two PXE boot files: C0A804C7 C0A804C6

The HTTP server will be configured with default settings and used to host the Oracle VM media files and the kickstart files. HTTP Services Configuration This section starts with the installation and configuration of Apache. The section concludes with the steps to mount and copy the Oracle VM installation media files (aka, the Oracle VM server installation tree) to the /var/www/html/oraclevm2.2/ directory. Installing Apache from a registered Enterprise Linux host from the Unbreakable Linux Network is accomplished by typing “up2date -i httpd” while logged in as root or using sudo. Once Apache is installed, configure Apache to automatically start by typing “chkconfig httpd on”. Next, start Apache by typing “service httpd start”. The next example shows how to install, configure and start Apache.

71 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

# sudo up2date -i httpd # sudo chkconfig httpd on # sudo service httpd start

Once the “up2date -i httpd”, “chkconfig httpd on” and “service httpd start” commands have finished, test your Apache server by pointing a web browser to the fully qualified domain name (FQDN) or the IP address of the Apache server. You will see the default Apache test page as shown in Figure 46.

Tip: If you don’t see the default Apache test page, check if iptables is blocking http traffic on the Apache host. Consider disabling iptables to test Apache by typing “sudo /sbin/service iptables stop”. The “If you are the website administrator:” section of the default Apache test page explains how to disable the test page and to enable the default web root directory at /var/www/html. To disable the Apache test page and to enable the default web root at /var/www /html, access the Apache server and comment out all the entries in the /etc/httpd/conf.d/welcome.conf file. The next example shows the default /etc/httpd/conf.d/welcome.conf file.

# This configuration file enables the default "Welcome" # page if there is no default index page present for # the root URL. To disable the Welcome page, comment # out all the lines below. # <LocationMatch "^/+$"> Options -Indexes ErrorDocument 403 /error/noindex.html </LocationMatch>

The next example shows the /etc/httpd/conf.d/welcome.conf file with all the entries commented out.

# This configuration file enables the default "Welcome" # page if there is no default index page present for # the root URL. To disable the Welcome page, comment # out all the lines below. # #<LocationMatch "^/+$"> # Options -Indexes # ErrorDocument 403 /error/noindex.html #</LocationMatch>

To test the new configuration restart Apache, as shown in the next example.

# sudo service httpd restart Stopping httpd: Starting httpd:

[ OK ] [ OK ]

To view the default root directory of your Apache server located at /var/www/html refresh your browser or point a browser to the fully qualified domain name (FQDN) or the IP address of the Apache server. You will be presented with the Apache root directory as shown in Figure 47.

72 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

How to Make the Oracle VM Server Installation Tree Available for a PXE/Kickstart Installation This section discusses how to make the Oracle VM server installation tree available for a PXE/kickstart installation on a boot server in the /var/www/html/ovs2.2 directory. All of the steps below are completed on the boot server using sudo or root. 1. Create a directory named ovs2.2 in /var/www/html, by typing “mkdir -p /var/www/html/ovs2.2”. 2. Copy the Oracle VM server ISO file to the Apache server. 3. From the directory with the ISO file, mount the ISO file, for example in /tmp/mnt by typing “mkdir –p /tmp/mnt” and then “mount -o loop,ro OracleVM-Server-2.2.0.iso /tmp/mnt/”. 4. Copy the installation files to the /var/www/html/ovs2.2 directory by typing “cp –avr /tmp/mnt/* /var/www/html/ovs2.2”. 5. Change the ownership of the files in the ovs2.2 directory by typing “chown -R apache:apache /var/www/html/ovs2.2”. The next example shows how to create the directories, mount and copy the installation files and change the ownership of the ovs2.2 directory.

# # # # #

sudo sudo sudo sudo sudo

mkdir -p /var/www/html/ovs2.2 mkdir -p /tmp/mnt mount -o loop,ro OracleVM-Server-2.2.0.iso /tmp/mnt/ cp -avr /tmp/mnt/* /var/www/html/ovs2.2 chown -R apache:apache /var/www/html/ovs2.2

To confirm that the Oracle VM server installation tree is available for the PXE/kickstart installation via HTTP point your browser to , the ovs2.2 directory, that is, http://<FQDN>/ovs2.2. You will see the installation files as shown in Figure 48.

DHCP Service Configurations In this section, we will install and configure DHCP with the IP addresses of the Oracle VM servers that will be PXE booted and installed using a kickstart file. 1The first step is to install the DHCP service. Using sudo or root, type “up2date -i dhcp” as shown in the next example.

# sudo up2date -i dhcp

2Next, we will configure the startup parameters for the dhcpd daemon, by typing “chkconfig --list dhcpd”. Then, type “chkconfig dhcpd on” to configure the dhcpd daemon to start at runlevels 2, 3, 4 and 5. Next, type “chkconfig --list dhcpd” to validate the dhcpd

73 of 243

18.04.2011 16:35

The Underground Oracle VM Manual
runlevels, as shown in the next example.

http://itnewscast.com/book/export/html/844

# sudo chkconfig --list dhcpd dhcpd 0:off 1:off 2:off 3:off 4:off 5:off 6:off # sudo chkconfig dhcpd on # sudo chkconfig --list dhcpd dhcpd 0:off 1:off 2:on 3:on 4:on 5:on 6:off

3Next, we will configure the DHCP server’s /etc/dhcpd.conf file. The default DHCP configuration file is located at /etc/dhcpd.conf. An example dhcpd.conf file is located at /usr/share/doc/dhcp-3.0.5/dhcpd.conf.sample. Replace the bold section with your environmental details.

ddns-update-style none; allow booting; # support PXE booting allow bootp; # respond to bootp queries subnet 192.168.4.0 netmask 255.255.255.0 { option routers 192.168.4.254; option subnet-mask 255.255.255.0; option nis-domain "example.com"; option domain-name "example.com"; option domain-name-servers 192.168.4.11; option time-offset -28800; # Pacific Standard Time

default-lease-time 120; max-lease-time 120; next-server 192.168.4.11; pool { range dynamic-bootp 192.168.4.199 192.168.4.230; } #Oracle VM 2.2 Kickstart boxes group { filename "pxelinux.0"; host ovs2 { hardware ethernet 00:30:48:7F:44:6E; fixed-address 192.168.4.199; } host ovs3 { hardware ethernet 00:30:48:7F:35:0A; fixed-address 192.168.4.198; } } }

The example dhcpd.conf file will configure the DHCP server to respond to DHCP requests on the 192.168.4.0 network with a netmask of 255.255.255.0, as shown in the next example.

subnet 192.168.4.0 netmask 255.255.255.0 {

Replace the 192.168.4.0 network address and the 255.255.255.0 subnet mask with your network address and subnet mask. The next section of the dhcpd.conf file configures the router, subnet mask, nis-domain, domain name, name server, and time zone.

option routers 192.168.4.254; option subnet-mask 255.255.255.0; option nis-domain "example.com"; option domain-name "example.com"; option domain-name-servers 192.168.4.11; option time-offset -28800; # Pacific Standard Time

Replace the router, subnet mask, nis-domain (if applicable), domain name (optional), name server (DNS), and time zone with your environmental details. Tip: Time zone details are listed in the /usr/share/zoneinfo/ directory.

74 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

The next section lists the next-server entry, which is the IP address of your boot server.

next-server 192.168.4.11;

Replace 192.168.4.11 with the IP address of your boot server. The next section of the dhcpd.conf file shows the IP address range the DHCP server will assign to DHCP clients. For example, the DHCP server will assign IP addresses from 192.168.4.199 to 192.168.4.230.

range dynamic-bootp 192.168.4.199 192.168.4.230;

Replace the 192.168.4.199 192.168.4.230 address range with your DHCP IP address range. The next section of the dhcpd.conf file lists a descriptive name, MAC address, and the fixed IP address of each Oracle VM server that will PXE boot.

host ovs2 { hardware ethernet 00:30:48:7F:44:6E; fixed-address 192.168.4.199; } host ovs3 { hardware ethernet 00:30:48:7F:35:0A; fixed-address 192.168.4.198; }

Replace the host section with a descriptive name for your Oracle VM server, for example, “host server-name”. Next, replace the MAC address and the fixed IP address with your server’s details. Tip: If your servers have an OS installed, type “ifconfig -a” for nix hosts or “ipconfig /all” for Windows to list each NICs MAC address. If your servers do not have an OS, enable the PXE boot option in the system BIOS. Once PXE boot is enabled, you should be able to see the MAC addresses during the system startup. 4Once you have the entries in the dhcpd.conf file, restart the DHCP service by typing “service dhcpd restart”, as shown in the next example.

# sudo service dhcpd start Starting dhcpd:

[ OK ]

Note: It’s necessary to restart the DHCP service to recognize any modifications made to the dhcpd.conf file. TFTP Service Configurations In this section we will configure the tftp service. The tftp service is managed by xinetd. The default xinetd configuration disables tftp. To enable the tftp service, edit /etc/xinetd.d/tftp and change the “disable = off” line to “disable = on”. Once the tftp service is enabled, configure the startup parameters for the xinetd daemon by typing “chkconfig xinetd on”. Typing “chkconfig xinetd on” configures the xinetd daemon to start at runlevels 2, 3, 4, and 5. Next, type “chkconfig --list xinetd” to list xinetd’s runlevels, as shown in the next example.

# sudo chkconfig xinetd on # sudo chkconfig --list xinetd xinetd 0:off 1:off 2:on

3:on

4:on

5:on

6:off

Next, restart the xinetd service by typing “service xinetd restart”, as shown in the next example.

# sudo service xinetd restart Stopping xinetd: Starting xinetd:

[ OK ] [ OK ]

To confirm that the tftp service is running, type “netstat -l -u | grep tftp”, as shown in the next example.

75 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

# netstat -l -u | grep tftp udp 0 0 *:tftp

*:*

If no output is displayed after typing “netstat -l -u | grep tftp”, the tftp service is not running. The example confirms that the tftp service is up. Tip: Consider disabling iptables during the testing phase by typing “sudo /sbin/service iptables stop”. How to Create the PXE boot Files In this section, we will prepare the PXE boot files on the boot server in the /tftpboot directory. The tftp service will be configured with a /tftpboot directory populated with: a bootloader program “pxelinux.0” in /tftpboot a compressed Linux kernel “vmlinuz” in /tftpboot an initial RAM disk “initrd.img” in /tftpboot/ovm/2.2 a subdirectory “pxelinux.cfg” with PXE boot files for two Oracle VM servers: C0A804C7 C0A804C6 In the next section, we will prepare the PXE boot files in the /tftpboot directory. All of the steps below are completed on the boot server using sudo or root. 1. Create the /tftpboot/ovm/2.2/ directory by typing “mkdir -p /tftpboot/ovm/2.2/”. 2. Create the /tftpboot/pxelinux.cfg directory by typing “mkdir -p /tftpboot/pxelinux.cfg” 3. Copy the bootloader program from the boot server in /usr/lib/syslinux/pxelinux.0 to the /tftpboot/ directory by typing “cp /usr/lib /syslinux/pxelinux.0 /tftpboot/”. 4. Copy the initrd.img file located on the <Oracle VM Server media>/images/pxeboot/initrd.img directory to the /tftpboot/ovm/2.2/ directory by typing cp <Oracle VM Server media>/images/pxeboot/initrd.img /tftpboot/ovm/2.2/. 5. Copy the vmlinuz file from the <Oracle VM Server media>/images/pxeboot/ vmlinuz to the /tftpboot directory by typing “cp <Oracle VM Server media>/images/pxeboot/ vmlinuz /tftpboot/”. The next example shows the commands from Steps 1 through 5.

# # # # #

sudo sudo sudo sudo sudo

mkdir -p /tftpboot/ovm/2.2/ mkdir -p /tftpboot/pxelinux.cfg cp /usr/lib/syslinux/pxelinux.0 /tftpboot/ cp <Oracle VM Server media>/images/pxeboot/initrd.img /tftpboot/ovm/2.2/ cp <Oracle VM Server media>/images/pxeboot/ vmlinuz /tftpboot/

The next example show the contents of the /tftpboot directory after Steps 1 through 5.

# tree /tftpboot/ /tftpboot/ | |-- ovm | `-- 2.2 | `-- initrd.img |-- pxelinux.0 |-- pxelinux.cfg `-- vmlinuz

6. Next, we will create two unique PXE boot files. Each PXE boot file has a unique name that represents the hex number of the IP address of the server. A PXE boot file contains the PXE boot and kickstart configurations. The PXE boot configurations are identical for all Oracle VM servers and allow the bare-metal systems to load a kernel via TFTP and boot without an operating system. The kickstart configuration is different for each PXE boot file. Each PXE boot file parses a unique kickstart file that personalizes the kickstart installation. To convert an IP address to the hex value for a PXE boot file, use the “gethostip” program by typing “gethostip -x <IP ADDRESS>”. The next example shows how to list the hex values for 192.168.4.199 and 192.168.4.198.

# gethostip -x 192.168.4.199 C0A804C7 # gethostip -x 192.168.4.198 C0A804C6

76 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

The first PXE boot file is named C0A804C7 and is for the Oracle VM server in the dhcpd.conf file named ovs2. The next example shows ovs2’s entry in the dhcpd.conf file.

host ovs2 { hardware ethernet 00:30:48:7F:44:6E; fixed-address 192.168.4.199; }

The next example shows how to get the name of the PXE boot file for ovs2, that is, 192.168.4.199.

# gethostip -x 192.168.4.199 C0A804C7

The second PXE boot file named is C0A804C6 and is for the Oracle VM server in the dhcpd.conf file named ovs3. The next example shows ovs3’s entry in the dhcpd.conf file.

host ovs3 { hardware ethernet 00:30:48:7F:35:0A; fixed-address 192.168.4.198; }

The next example shows how to get the name of the PXE boot file for ovs3, that is 192.168.4.198.

# gethostip -x 192.168.4.198 C0A804C6

Next, create the PXE boot file for ovs2 (192.168.4.199) by typing “vi /tftpboot/pxelinux.cfg/C0A804C7”. Enter the configurations shown below and save the file.

# vi /tftpboot/pxelinux.cfg/C0A804C7 default ovsboot label ovsboot kernel vmlinuz append initrd=ovm/2.2/initrd.img ks=http://192.168.4.11/ovs2-ks.cfg ksdevice=eth0 :wq!

Typing “vi /tftpboot/pxelinux.cfg/C0A804C7” creates a PXE boot file C0A804C7 in /tftpboot/pxelinux.cfg/. The default line is required. You can change the description from the default line, that is, replace ovsboot with any descriptive name. The label line is also required. You can change the description from the label line, that is, replace ovsboot with any name. The kernel vmlinuz line configures the PXE boot system to load the compressed Linux kernel “vmlinuz”. The append initrd=ovm/2.2/initrd.img ks=http://192.168.4.11/ovs2-ks.cfg ksdevice=eth0 line configures a) the initial RAM disk “initrd.img” and b) the kickstart file ovs2-ks.cfg on the boot server. Pressing the Esc key followed by :wq! saves the C0A804C7 file. Note: The kickstart file can be staged using http, ftp, or nfs, as shown in the next example. ks=http://192.168.4.11/ovs3-ks.cfg ks=ftp:// 192.168.4.11/ovs3-ks.cfg ks=nfs:192.168.4.11/ovs3-ks.cfg Next, create a PXE boot file for ovs3 (192.168.4.198) by typing “vi /tftpboot/pxelinux.cfg/ C0A804C6”. Enter the configurations shown below and save the file.

vi /tftpboot/pxelinux.cfg/C0A804C6 default ovsboot label ovsboot kernel vmlinuz append initrd=ovm/2.2/initrd.img ks=http://192.168.4.11/ovs3-ks.cfg ksdevice=eth0 :wq!

Finally, make sure all files and directories in /tftpboot have the right permission set. The next example shows how to set the correct permissions for the files and directories in /tftpboot.

77 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

# find /tftpboot/ -type f -exec chmod 644 {} \; # find /tftpboot/ -type d -exec chmod 755 {} \;

The next example shows all of the directories and files in the /tftpboot directory.

# tree /tftpboot/ /tftpboot/ | |-- ovm | `-- 2.2 | `-- initrd.img |-- pxelinux.0 |-- pxelinux.cfg | `-- C0A804C7 | `-- C0A804C6 `-- vmlinuz

Kickstart File Examples This section discusses how to create, stage, test, and troubleshoot a kickstart file to automate an Oracle VM server installation. The section starts with an overview of a default Oracle VM anaconda kickstart file followed by three example kickstart files. Then, we will learn how to generate encrypted passwords for a kickstart file. Next, we review with how to stage a kickstart file on a boot server for a PXE/kickstart installation. The section concludes with PXE client configurations and PXE/kickstart installation troubleshooting. Each Oracle VM server installation generates a generic anaconda kickstart file in the /root directory, named anaconda-ks.cfg. The kickstart file is used by anaconda during the installation process to define the installation parameters. The default anaconda-ks.cfg file lists the answers to each Oracle VM server installation question and field entry required by the installation program. If any of the required fields are missing from a kickstart file the installation halts and waits for input. Example 1 shows a default Oracle VM anaconda kickstart file. The bold sections need to be modified to create a kickstart file for your environment. Example 1 # Kickstart file automatically generated by anaconda. install cdrom lang en_US.UTF-8 network --device eth0 --bootproto static --ip 192.168.4.6 --netmask 255.255.255.0 --gateway 192.168.4.254 --nameserver 192.168.4.11 --hostname ovs1.sf.itnc.com ovsagent --iscrypted xxxxxxxxxxxxxxxxxxxxxxxxxx ovsmgmntif eth0 rootpw --iscrypted xxxxxxxxxxxxxxxxxxxxxxxxxx firewall --enabled --port=21:tcp --port=22:tcp --port=53:udp --port=53:tcp --port=80:tcp --port=2049:tcp --port=5900-5950:tcp --port=8002:tcp --port=8003:tcp --port=8899:tcp --port=7777:tcp authconfig --enableshadow --enablemd5 selinux --disabled timezone --utc America/Los_Angeles bootloader --location=mbr --dom0_mem=569 --driveorder=sda # The following is the partition information you requested # Note that any partitions you deleted are not expressed # here so unless you clear all partitions first, this is # not guaranteed to work #clearpart --all --drives=sda #part /boot --fstype ext3 --size=100 --ondisk=sda #part / --fstype ext3 --size=3072 --ondisk=sda #part /OVS --fstype ocfs2 --size=1024 --grow --ondisk=sda #part swap --size=1024 --ondisk=sda %packages @office @admin-tools @editors @text-internet @gnome-desktop @dialup @core @base @games

78 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

@java @base-x @graphics @printing @sound-and-video @ovs-virtualization @graphical-internet tftp bridge-utils squashfs-tools

The default kickstart file must be modified before you can test a PXE/kickstart installation. The next list show the sections of the default kickstart file that need to be modified for your environment. cdrom network --device eth0 --bootproto static --ip 192.168.4.7 --netmask 255.255.255.0 --gateway 192.168.4.254 --nameserver 192.168.4.11 --hostname ovs2.sf.itnc.com ovsagent --iscrypted xxxxxxxxxxxxxxxxxxxxxxxxxx rootpw --iscrypted xxxxxxxxxxxxxxxxxxxxxxxxxx timezone --utc America/Los_Angeles #clearpart --all --drives=sda #part /boot --fstype ext3 --size=100 --ondisk=sda #part / --fstype ext3 --size=3072 --ondisk=sda #part /OVS --fstype ocfs2 --size=1024 --grow --ondisk=sda #part swap --size=1024 --ondisk=sda %packages Tip: To review an exhaustive list of kickstart file options, please consult the following resources: http://www.redhat.com/docs/manuals/enterprise/RHEL-5-manual/Installation_Guide-en-US/s1-kickstart2-options.html http://fedoraproject.org/wiki/Anaconda/Kickstart Example 2 is a commented kickstart file that was modified from the default anaconda kickstart file and will be used to install an Oracle VM server pool member named ovs2.sf.itnc.com. The Oracle VM server ovs2.sf.itnc.com will not have a local storage repository, that is, an /OVS directory. The bold sections need to be modified for your environment. Note: Timezone details and entries are listed in the /usr/share/zoneinfo/ directory. Example 2 ### Installation Install ### Keyboard layout lang en_US.UTF-8 keyboard us ### Skips using X windows skipx ### Installation source url --url http://192.168.4.11/ovs2.2/ ### Network configurations network --device eth0 --bootproto static --ip 192.168.4.7 --netmask 255.255.255.0 --gateway 192.168.4.254 --nameserver 192.168.4.11 --hostname ovs2.sf.itnc.com ### Oracle VM server agent password ovsagent --iscrypted xxxxxxxxxxxxxxxxxxxxxxxxxx ### Oracle VM agent management interface ovsmgmntif eth0 ### Root password rootpw --iscrypted xxxxxxxxxxxxxxxxxxxxxxxxxx ### dom0 iptables configuration firewall --enabled --port=21:tcp --port=22:tcp --port=53:udp --port=53:tcp --port=80:tcp --port=2049:tcp --port=5900-5950:tcp --port=8002:tcp --port=8003:tcp --port=8899:tcp --port=7777:tcp ### Authentication settings authconfig --enableshadow --enablemd5 ### SELinux configuration selinux –disabled ### Timezone settings timezone --utc America/Los_Angeles ### Bootloader configuration bootloader --location=mbr --dom0_mem=569 --driveorder=sda ### Automatic reboot after install #reboot ## Partitioning

79 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

clearpart --linux part /boot --fstype=ext3 --size=256 part / --fstype=ext3 --size=1 --grow part swap --fstype=swap –recommended ## Packages %packages @core @base @ovs-virtualization tftp bridge-utils

Note: Example 2 uses a web server to host the installation source. The installation source can be hosted using http, nfs, or ftp. The next list shows http, nfs and ftp entries in a kickstart file hosting the installation source. url --url http://192.168.4.11/ovs2.2/ nfs --server=192.168.4.11 --dir=/ovs2.2/ url --url=ftp:// 192.168.4.11/ovs2.2/ Example 3 was modified from the default anaconda kickstart file and is used to install an Oracle VM server pool member named ovs3.sf.itnc.com. ovs3.sf.itnc.com will not have an /OVS directory. The bold sections need to be modified for your environment. Example 3 install key --skip lang en_US.UTF-8 keyboard us skipx url --url http://192.168.4.11/ovs2.2/ network --device eth0 --bootproto static --ip 192.168.4.8 --netmask 255.255.255.0 --gateway 192.168.4.254 --nameserver 192.168.4.11 --hostname ovs3.sf.itnc.com ovsagent --iscrypted xxxxxxxxxxxxxxxxxxxxxxxxxxx ovsmgmntif eth0 rootpw --iscrypted xxxxxxxxxxxxxxxxxxxxxxxxxxx firewall --enabled --port=21:tcp --port=22:tcp --port=53:udp --port=53:tcp --port=80:tcp --port=2049:tcp --port=5900-5950:tcp --port=8002:tcp --port=8003:tcp --port=8899:tcp --port=7777:tcp authconfig --enableshadow --enablemd5 selinux --disabled timezone --utc America/Los_Angeles bootloader --location=mbr --dom0_mem=569 --driveorder=sda #reboot clearpart --linux part /boot --fstype=ext3 --size=256 part / --fstype=ext3 --size=1 --grow part swap --fstype=swap --recommended %packages @core @base @ovs-virtualization tftp bridge-utils

Example 4 was modified from the default anaconda kickstart file and is used to install a stand-alone Oracle VM server named ovs1.sf.itnc.com. ovs1.sf.itnc.com will have a local storage repository in the /OVS directory. The bold sections need to be modified for your environment. Example 4 install key --skip lang en_US.UTF-8 keyboard us skipx url --url http://192.168.4.11/ovs2.2/ network --device eth0 --bootproto static --ip 192.168.4.5 --netmask 255.255.255.0 --gateway 192.168.4.254 --nameserver 192.168.4.11 --hostname ovs1.sf.itnc.com ovsagent --iscrypted xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx ovsmgmntif eth0 rootpw --iscrypted xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx firewall --enabled --port=21:tcp --port=22:tcp --port=53:udp --port=53:tcp --port=80:tcp --port=2049:tcp --port=5900-5950:tcp

80 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

--port=8002:tcp --port=8003:tcp --port=8899:tcp --port=7777:tcp authconfig --enableshadow --enablemd5 selinux --disabled timezone --utc America/Los_Angeles bootloader --location=mbr --dom0_mem=569 --driveorder=sda # The following is the partition information you requested # Note that any partitions you deleted are not expressed # here so unless you clear all partitions first, this is # not guaranteed to work #reboot clearpart --all --drives=sda part /boot --fstype ext3 --size=100 --ondisk=sda part / --fstype ext3 --size=3072 --ondisk=sda part /OVS --fstype ocfs2 --size=1024 --grow --ondisk=sda part swap --size=1024 --ondisk=sda %packages @core @base @ovs-virtualization tftp bridge-utils Note: None of the example kickstart files will automatically reboot the server after the installation. Uncomment the reboot section of the kickstart file to have the Oracle VM server automatically reboot after the installation. After a successful PXE/kickstart installation, either disable the PXE boot feature from the BIOS or comment out the Oracle VM server’s entries in the dhcpd.conf file to disable PXE boot and to preclude the system from performing additional PXE/kickstart installations. How to Generate an Encrypted Password with grub-md5-crypt This section reviews how to generate the encrypted passwords for the root user account and the ovs-agent account for an Oracle VM server kickstart file. The following list shows the two password entries in an Oracle VM server kickstart file that require encrypted passwords. ovsagent --iscrypted xxxxxxxxxxxxxxxxxxxxxxxxxx rootpw --iscrypted xxxxxxxxxxxxxxxxxxxxxxxxxx To generate encrypted passwords for the root user account and the ovs-agent account, use the grub-md5-crypt program. From an Enterprise Linux host , while logged in as root or using sudo, type “grub-md5-crypt”. You will be prompted for a password and then asked to confirm the password. The password will not be visible as you type it. The grub-md5-crypt program will then print out the MD5-encrypted password. Note: The Oracle VM installation program requires at least six characters each for the root and Oracle VM agent passwords. The following example shows the output from grub-md5-crypt.

# grub-md5-crypt Password: Retype password: xxxxxxxxxxxxxxxxxxxxxxxxxx

Once you have generated the encrypted passwords for the root user account and the ovs-agent account, place the encrypted passwords in the kickstart file and stage the kickstart file on the boot server. How to Stage a Kickstart File for an Oracle VM Server PXE/kickstart Installation The section discusses how to stage a kickstart file on a web server for an Oracle VM server PXE/kickstart installation. Kickstart files can be staged using http, ftp, or nfs. The next example shows a PXE boot file named C0A804C7 that points to a kickstart file (ovs2-ks.cfg) staged on a web server (192.168.4.11).

# vi /tftpboot/pxelinux.cfg/C0A804C7 default ovsboot label ovsboot kernel vmlinuz append initrd=ovm/2.2/initrd.img ks=http://192.168.4.11/ovs2-ks.cfg ksdevice=eth0 :wq!

Note: Kickstart files can be staged using http, ftp, or nfs, as shown in the next example. ks=http://192.168.4.11/ovs3-ks.cfg ks=ftp:// 192.168.4.11/ovs3-ks.cfg ks=nfs:192.168.4.11/ovs3-ks.cfg

81 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

To stage kickstart files to a web server, copy the kickstart files to the directory specified in the PXE boot file. The above example uses the web root directory at /var/www/html, although the example could have used any subdirectory with a descriptive name, such as http://192.168.4.11/kickstart-files/ovs2-ks.cfg. Figure 49 shows the staged installation tree in the ovs2.2/ directory along with the kickstart file ovs2-ks.cfg and ovs3-ks.cfg in the web root directory, that is, the /var/www/html of the example 192.168.4.11 boot/web server. Note: A kickstart file can be staged in any directory on a web, ftp, or nfs server. PXE Boot Client Configuration This section discusses how to configure a system’s network interface card (NIC) to be a PXE boot client. The final step in preparing a PXE boot environment is to enable each system on which you want to perform a PXE/kickstart installation to boot using PXE. To enable a system’s PXE boot capability, reboot the system and access the system BIOS. From the system BIOS, locate and enable the PXE boot feature. In some cases, it may be necessary to change the boot order and place PXE boot at the top of the list. Tip: Many systems use the "F12" hotkey to access the boot methods selection screen, while others systems require BIOS access to enable PXE boot. Consult your vendor’s website for details about your hardware. You are now ready to test your PXE/kickstart installation. PXE/Kickstart Oracle VM Server Installation and Troubleshooting Tips This section discusses Oracle VM server PXE/Kickstart installation troubleshooting tips. If you run into problems when testing your boot server and PXE/kickstart configurations, check /var/log/messages, /var/log/httpd /access_log and /var/log/httpd/error_logs on the boot server for errors. The /var/log/messages, /var/log/httpd/access_log and /var/log /httpd/error_log log files on the boot server are where all the DHCP TFTP and HTTP errors are logged. The next list shows a , PXE/kickstart check list. PXE/kickstart check list Check that your boot server is ready to answer requests from the PXE clients. Is your tftp service is running? From the boot server, type “netstat -l -u | grep tftp”, as shown in the next example to confirm that the tftp server is running.

# netstat -l -u | grep tftp udp 0 0 *:tftp

*:*

If no output is displayed after typing “netstat -l -u | grep tftp”, the tftp service is not running. The example confirms that the tftp service is up. Check if iptables is blocking traffic on the boot server. Consider disabling iptables to test the boot server by typing “sudo /sbin/service iptables stop”. Check the BIOS of each PXE client to ensure that PXE boot is enabled. When the system boots, do you see the NIC looking for an IP address via DHCP? If not, PXE boot is not enabled. Check that your PXE client is connected to the same network as the boot server. If the PXE client is located across a router from the boot server, the PXE client may not be able to receive an IP address from the DHCP service. How to Register and Update an Oracle VM Server from the Oracle Unbreakable Linux Network This section discusses how to register and update an Oracle VM server from the Oracle Unbreakable Linux Network. The section starts with a brief introduction to the Oracle Unbreakable Linux support program and the Oracle Unbreakable Linux Network (ULN). The section concludes with an overview of how to register and update an Oracle VM server from the Oracle Unbreakable Linux Network using the up2date program. Tip: Even after a fresh installation of Oracle VM, it’s advisable to patch the system before any testing to avoid troubleshooting patched bugs. The Oracle Unbreakable Linux Support Program The Oracle Unbreakable Linux support program is an add-on component of Oracle’s enterprise support package for Oracle VM and Enterprise Linux. The Oracle Unbreakable Linux support program uses the same support infrastructure, support phone number and My Oracle Support account for Oracle VM and Enterprise Linux as with all other Oracle technologies. Using Oracle support for Oracle VM and Enterprise Linux allows an Oracle support service request (SR) to transition between groups with issues that require cross-stack collaboration. For example, if you open a service request for an application issue and the root cause is at the virtualization

82 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

layer, the service request will automatically transition between the application and Oracle VM support engineers. The Oracle Unbreakable Linux Network is a cloud resource for Oracle Unbreakable Linux support customers that hosts the Oracle VM and Enterprise Linux RPM repositories, including software patches, updates and fixes. The Oracle Unbreakable Linux Network’s web portal is located at http://linux.oracle.com. The Oracle Unbreakable Linux Network web portal provides a dashboard and management interface for all registered Oracle VM and Enterprise Linux systems and RPM channels. The Oracle Unbreakable Linux Network repositories are used to patch and install RPMs for Oracle VM and Enterprise Linux systems. Oracle Unbreakable Linux support customers have the option to patch and install RPMs for Oracle VM and Enterprise Linux systems from the Oracle Unbreakable Linux Network using the up2date program, or from a local yum repository. Note: Before you can access the Oracle Unbreakable Linux Network it's necessary to create an Oracle Single Signon account. Your existing My Oracle Support (MOS) Oracle Single Signon account will not work with the Oracle Unbreakable Linux Network until the account has been registered with the Oracle Unbreakable Linux Network. Click the Register link at the the Oracle Unbreakable Linux Network portal to a) create a new Oracle Single Signon account or to b) associate your existing Oracle Single Signon account with the Oracle Unbreakable Linux Network. Figure 50 highlights the Oracle Unbreakable Linux Network web portal.

The Oracle Unbreakable Linux Network and My Oracle Support, formerly Metalink, are entirely separate systems, accessed by different URLs, and use different customer service identifiers (CSIs). My Oracle Support is used to interface with Oracle’s enterprise support organization, whereas the Oracle Unbreakable Linux Network is used to monitor registered Oracle VM and Enterprise Linux systems and RPMs. A valid customer service identifier (CSI) for Oracle VM or Enterprise Linux is required to access the RPMs at the Oracle Unbreakable Linux Network. The customer service identifiers for Oracle VM and Enterprise Linux are only valid for the Oracle Unbreakable Linux Network, not for My Oracle Support. Note: The customer service identifiers for Oracle VM and Enterprise Linux cannot be entered at the My Oracle Support portal. Oracle VM Server Update Options There are three options for updating an Oracle VM Server. The first option is to use the up2date program to register and update Oracle VM servers from the Oracle Unbreakable Linux Network. The up2date program communicates with the Oracle Unbreakable Linux Network over an SSL encrypted connection and downloads and installs all requested packages in an RPM format. Consult up2date’s man page by typing “man up2date” from dom0 as root to view a comprehensive list of command augments. Note: Oracle Unbreakable Linux Network access requires a valid Oracle VM customer service identifier. Figure 51 shows an Oracle VM server being updated from the Oracle Unbreakable Linux Network using the up2date program.

83 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

The second option is to update Oracle VM servers using a local yum repository. A local yum repository can be hosted on any Enterprise Linux system that has been registered with the Unbreakable Linux Network with internet access and Apache. Local yum repositories are populated and synchronized with RPMs that are hosted at the Unbreakable Linux Network using a script and a local cron job. The up2date program or the yum program can be used with a local yum repository. Figure 52 shows an Oracle VM server being updated from a local yum repository.

The third option is boot an Oracle VM server with the latest Oracle VM media in the CD-ROM drive and to select the update option from the boot prompt. Selecting the update option from the boot prompt will update the Oracle VM Server to the version of the Oracle VM server media. Once the server has been updated, the system must be patched from the Oracle Unbreakable Linux Network or from a local yum repository. In the next section, we discuss the process for registering and updating an Oracle VM Server. The examples will all be performed from dom0 as root using the up2date command. up2date Program Proxy Configurations The up2date program defaults may need to modified to meet your environmental and business requirements. For example, if you’re using a proxy server, you will need to edit the up2date configuration to enter your proxy details. To edit the up2date program defaults, from dom0 as root type “up2date --configure”. Typing “up2date --configure” lists the up2date program defaults. There are five proxy configurations that can be edited to allow access from your Oracle VM server to the internet. The following list shows the up2date proxy configuration items with their default settings and item numbers. 3. enableProxy No 4. enableProxyAuth No 11. httpProxy 21. proxyPassword 22. proxyUser To edit an up2date program item type “up2date –configure” then type the item number. Then, type C to clear the default value or type q to quit without saving. Next, type the new value and press Enter to save the new value and to exit. If you need to enter multiple values, separate them with semicolons (;). Another default up2date configuration for Oracle VM is to not update the Oracle VM server’s kernel. The default configuration avoids a “blind” kernel update, which could affect any installed programs or third-party drivers that are dependent on a specific kernel version. If the default kernel configuration is modified to allow kernel updates, previous kernels are not removed, which allows a system to be rebooted into the previous kernel if things go wrong. up2date Program Kernel Configurations To enable kernel updates, clear the “pkgSkipList” and the “pkgsToInstallNotUp” items by typing the item number and then typing C to clear the item list. Next, select Enter to save and exit up2date. The following list shows the default “pkgSkipList” and the “pkgsToInstallNotUp” configurations. 19. pkgSkipList ['kernel*'] 20. pkgsToInstallNotUp ['kernel', 'kernel-modules', 'kernel-devel'] The next example shows the default up2date --config configurations with the proxy and kernel items in bold.

# up2date --config 0. adminAddress ['root@localhost'] 1. debug No 2. disallowConfChange ['noReboot', 'sslCACert', 'useNoSSLForPackages', 'noSSLSe 3. enableProxy No 4. enableProxyAuth No 5. enableRollbacks No 6. fileSkipList [] 7. forceInstall No 8. gpgKeyRing /etc/sysconfig/rhn/up2date-keyring.gpg

84 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

9. headerCacheSize 40 10. headerFetchCount 10 11. httpProxy 12. isatty Yes 13. keepAfterInstall No 14. networkRetries 5 15. noBootLoader No 16. noReboot No 17. noReplaceConfig Yes 18. noSSLServerURL http://linux-update.oracle.com/XMLRPC 19. pkgSkipList ['kernel*'] 20. pkgsToInstallNotUp ['kernel', 'kernel-modules', 'kernel-devel'] 21. proxyPassword 22. proxyUser 23. removeSkipList ['kernel*'] 24. retrieveOnly No 25. retrieveSource No 26. rhnuuid 3cfb2ee2-6a22-11dd-9022-001c23b73c3a 27. serverURL https://linux-update.oracle.com/XMLRPC 28. showAvailablePacka No 29. sslCACert /usr/share/rhn/ULN-CA-CERT 30. storageDir /var/spool/up2date 31. systemIdPath /etc/sysconfig/rhn/systemid 32. updateUp2date Yes 33. useGPG Yes 34. useNoSSLForPackage No 35. useRhn Yes 36. versionOverride Enter number of item to edit <return to exit, q to quit without saving>:

Import Oracle’s GPG Key Before an Oracle VM server can connect to the Oracle Unbreakable Linux Network, Oracle’s GPG key must be imported using the rpm program. To import the Oracle’s GPG key, from dom0 as root type “rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY”, as shown in the next example.

#rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY

Once the GPG key has been imported, the Oracle VM Server can be registered at the Oracle Unbreakable Linux Network by typing “up2date” to start the registration process. The registration process requires you to select a user name and password and to enter a valid Oracle VM Support Identifier number (CSI). The user name and password that are selected during the registration process become the user name and password to log on to the Oracle Unbreakable Linux Network. The first time you log on and authenticate your account on the Oracle Unbreakable Linux Network, you will be prompted to convert your user name and password to an Oracle SSO user name and password. Oracle SSO accounts can be used across Oracle’s web properties, for example on the Oracle Technical Network (OTN). You can change your user name and password after you have authenticated your account by clicking the Profile link. The Oracle Unbreakable Linux Network Registration Process The 1. 2. 3. 4. 5. 6. following list shows the six steps to register an Oracle VM host with the Oracle Unbreakable Linux Network: Review the Unbreakable Linux Privacy Statement Register a User Account Register a System Profile—Hardware Register a System Profile—Packages Send Profile Information to the Unbreakable Linux Network Finished Registration

Step 1. Review the Unbreakable Linux Privacy Statement From the Review the Unbreakable Linux Privacy Statement screen use the Alt key to select the Next tab, once the Next tab is selected press the Enter key to proceed. Figure 53 shows the Review the Unbreakable Linux Privacy Statement screen.

85 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Step 2. Register a User Account On the Register a User Account screen, enter a User name, Password, Password confirmation and a valid Oracle VM CSI number. Use the Alt key to select the Next tab, and then press the Enter key to proceed. Figure 54 shows the Register a User Account screen.

Step 3. Register a System Profile—Hardware On the Register a System Profile—Hardware screen, accept the defaults and use the Alt key to select the Next tab. Once the Next tab is selected, press the Enter key to proceed. Note: The information gathered from the system profile step is saved in your user profile at the Oracle Unbreakable Linux Network. Figure 55 shows the Register a System Profile—Hardware screen.

Step 4. Register a System Profile— Packages On the Register a System Profile—Packages screen, accept the defaults and use the Alt key to select the Next tab. Once the Next tab is selected, press the Enter key to proceed.

86 of 243

18.04.2011 16:35

The Underground Oracle VM Manual
Figure 56 shows the Register a System Profile—Packages screen.

http://itnewscast.com/book/export/html/844

Step 5. Send Profile Information to the Unbreakable Linux Network From the Send Profile Information to the Unbreakable Linux Network screen, accept the defaults and use the Alt key to select the Next tab. Once the Next tab is selected, press the Enter key to proceed. Figure 57 shows the Send Profile Information to the Unbreakable Linux Network screen.

Step 6 Finished Registration On the Finished Registration screen, accept the defaults and use the Alt key to select the Next tab. Once the Next tab is selected, press the Enter key to proceed. Figure 58 shows the Finished Registration screen.

The Oracle VM server has been successfully registered. Once a system is registered type “up2date -u” to update the system.

Watch this space Oracle OpenWorld 2011 Call for Papers - Submit Today! Temporary Post Used For Theme Detection (409fc5e9-d7ee-4e1a-b645-8b82f256f434 - 3bfe001a-32de-4114-a6b4-4005b770f6d7) Debugging Oracle VM 2.2 & Oracle VM Manager 2.2 errors using the many log files New VDI Training Course now available! HIMSS 2011 and New Press Release Oracle Desktop Virtualization at HIMSS 2011 Oracle VM VirtualBox 4.0.4 Released! Two Virtualization Webinars This Week We're Hiring! - Server and Desktop Virtualization Product Management

87 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Chapter 5: Oracle VM Manager Sizing, Installation, Updates and the Oracle VM Manager CLI
Last update 01-28-2010 Copyright © 2009 - 2011 Roddy Rodstein. All rights reserved. Show a printer friendly version of this book with its sub pages Chapter 5 covers the following topics: Oracle VM Manager Introduction Oracle VM Manager Resource Requirements Oracle VM Manager Installation Prerequisites Oracle VM Manager Installation Deploying the Oracle VM Manager Template Updating and Patching Oracle VM Manager Oracle VM Manager Database Backup and Restoration The Oracle VM Manager Command Line Interface Oracle VM 2.1.5 to 2.2 Upgrade Table of Contents Oracle VM Manager Introduction Oracle VM Manager Resource Requirements ...Supporting Large Oracle VM Manager User Communities Oracle VM Manager Installation Prerequisites ...libaio RPM Package ...Name Resolution ...Host Firewall & Port Test ...SMTP Server ...Oracle VM Manager Guest Console Required Installation Passwords and Password Complexity Requirements Oracle VM Manager Installation Applying the 2.2.16 Oracle VM Manager Patch Updating Oracle VM Manager using the Installation Media ...OVS Database and oc4jadmin Accounts and Passwords ...Updating Oracle VM Manager Oracle VM Manager Database Backup and Restoration ...Oracle VM Manager Database Backup ...Oracle VM Manager Database Restore Deploying the Oracle VM Manager Template ...Oracle VM Manager Template Prerequisites ...SMTP Server ...User Accounts, Passwords and Password Complexity Requirements ...HTTP and Oracle VM Manager XE Database listening Ports ...Oracle VM Server Cluster Configurations Download and Deploy the Oracle VM Manager Template ...Running the Deploy_Manager_Template.sh Script The Oracle VM Manager Command Line Interface Introduction Downloading the Oracle VM Manager Command Line Interface from ULN Oracle VM Manager Command Line Interface Installation & Configuration Oracle VM Manager Command Line Interface Command Examples ...Bulk Commands and Batch Scripting ...Guest Backup ...Create a Server Pool Programming with the Oracle VM Manager Command Line Interface Oracle VM 2.1.5 to 2.2 Upgrade

Oracle VM Manager Introduction
Oracle VM Manager is a bare bones stand-alone management solution for Oracle VM servers, Oracle VM server pools and Oracle VM guests. Oracle VM Manager is a light weight private source cluster ready Oracle Application Development Framework 10g (ADF) application that installs on Enterprise Linux. Oracle VM Manager also has a command line interface that allows Oracle VM Manager administrative tasks to be perform from the command line or executed using scripts. Oracle VM Manager is supported on a physical or virtual Enterprise Linux operating system. Oracle VM Manager is a great candidate for virtualization on Oracle VM or Oracle VM VirtualBox. Virtualizing Oracle VM saves on hardware costs and improves application flexibility while reducing data center space. Note: Oracle VM Manager is not supported and should not be installed in Oracle VM server's dom0. List 1 shows the supported Enterprise Linux operating systems and virtualization platforms for Oracle VM Manager.

88 of 243

18.04.2011 16:35

The Underground Oracle VM Manual
Oracle Enterprise Linux 4 update 5 or later Red Hat Enterprise Linux 4 or later Oracle VM for x86 Oracle VM VirtualBox

http://itnewscast.com/book/export/html/844

Oracle VM Manager is distributed in three formats; a) as an ISO file that installs on Enterprise Linux b) as a pre-packaged Oracle VM template (with the CLI) and c) as the Oracle VM Management Pack plug-in. Oracle VM Manager is considered the leading edge development platform for the Oracle VM Management Pack, which is an Oracle Enterprise Manager plug-in. Oracle VM Manager 2.1.2 was ported to Oracle Enterprise Manager 10.2.0.5 and Oracle VM Manager 2.2 was ported to Oracle Enterprise Manager 11g R1. Expect at least a six month lag for Oracle to port a new Oracle VM Manager release to the Oracle VM Management Pack. Note: The Oracle VM Management Pack is licensed software; Oracle VM Server and Oracle VM Manager are not licensed software. Figure 1 highlights the Oracle VM Manager 2.2 user interface.

The next chart shows the matrix of supported Oracle VM Manager and Oracle VM server combinations.
OVM 2.1.0 2.1.1 2.1.2 2.1.5 2.2.0 OVS --------+------+------+-----+-----2.1.0 O | O | O | O | O 2.1.1 X | O | O | O | O 2.1.2 X | X | O | O | O 2.1.5 X | X | X | O | O 2.2.0 X | X | X | X | O 2.2.1 X | X | X | X | O

Note: Oracle VM Manager can manage many Oracle VM server pools, e.g. Oracle VM Manager 2.2 can manage one pool of Oracle VM 2.1.x servers, another pool of Oracle VM 2.2 servers, and many more server pools; But within the same server pool, all the servers must have the same version of Oracle VM server and the Oracle VM agent. Mixing Oracle VM 2.1.x and 2.2.x servers in the same pool is not supported.

Oracle VM Manager Resource Requirements
Oracle's recommends 2 GB of memory and one 1.83 GHz CPU for VM Manager. Oracle's minimum resource recommendations for VM Manager is appropriate for a small Oracle VM pool with a hand full of Oracle VM servers without the Command Line Interface. Oracle VM Manager resource requirements can be quickly tested with Oracle VM Manager user interface (UI) page response times. For example, if the Oracle VM Manager user interface is painfully slow or if you experience page timeouts adding memory and CPUs will speed up the user interface and eliminate page timeouts. Tip: Oracle VM Manager on VMware is performance challenged and will need substantially more CPU and memory than the minimum

89 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

recommendations. Oracle VM Manager is a great fit running on a virtualized Enterprise Linux operating system on an Oracle VM server. Table 1 lists Oracle's recommended minimum resource requirements for Oracle VM Manager 2.x. Description Memory Processor Swap Space Hard Disk Space Minimum Resources 2 GB 1 1.83 GHz CPU 2 GB 4 GB

The memory and CPU requirements for Oracle VM Manager 2.x scales up as the number of Oracle VM pool members grow. Table 2 shows the recommended CPU and memory allocation for three different Oracle VM Manager deployments. Number of Oracle VM Server Pool Members < 10 > 10 > 32 Memory CPU Swap

2 GB 4 GB 8 GB

1 1.83 GHz 2 2 GHz 4 2 GHz

2 GB 8 GB 8 GB

Supporting Large Oracle VM Manager User Communities
If Oracle VM Manager will be supporting over 10 users ensure that the OVM Manager host has at least 2.5G of RAM with the following oc4j script modifications. On the OVM Manager host, edit the “/opt/oc4j/bin/oc4j” file and change this line: OC4J_JVM_ARGS="-XX:PermSize=256m -XX:MaxPermSize=512m" to: OC4J_JVM_ARGS="-XX:PermSize=512m -XX:MaxPermSize=1024m -Xms2048m -Xmx2048m" Next, restart the OC4J service on the Oracle VM Manager hosts by typing "service oc4j restart".

Oracle VM Manager Installation Prerequisites libaio RPM Package
Oracle VM Manager uses the Oracle Database 10g Express Edition as the default data repository. The Oracle Database 10g Express Edition requires the libaio package release 0.3.96 or above. To validate if your Enterprise Linux system has the libaio 0.3.96 or above package, as root “type rpm -q libaio” as shown in the next example. # rpm -q libaio libaio0.3.106-3.2 If you get no result from typing “rpm -q libaio” you will need to install the libaio package. To install libaio from the Oracle Unbreakable Linux network type “up2date -i libaio” as shown in the next example. # up2date -i libaio

Name Resolution
Before you install Oracle VM Manager ensure that the Oracle VM Manager server's host name is properly entered in the /etc/hosts file. The host name, i.e. fully qualified domain name (FQDN) must be associated with the public IP address. The next example shows an “improper” Enterprise Linux /etc/hosts file entry. 127.0.0.1 servername.com servername localhost.localdomain

90 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

localhost 192.168.4.8 servername.com servername The next example shows the proper syntax for an Enterprise Linux /etc/hosts file entry. 127.0.0.1 localhost.localdomain localhost 192.168.4.8 servername.com servername

Host Firewall & Port Test
Oracle VM Manager will reserve tcp ports 8888, 8899 and 4443 for the Oracle VM Manager application. Before installing Oracle VM Manager on the Oracle VM Manager host, check that ports 8888 and 8899 are free. As root type “netstat -na |grep 8888” and then type “netstat -na |grep 8899”. If you do not get a response by typing “netstat -na |grep 8888” and “netstat -na |grep 8899” the ports are free. If you do get a response the ports must be released before you can install Oracle VM Manager. If your Enterprise Linux host uses iptables you will need to allow tcp traffic on ports 8888, 8899 and 4443 to allow access to the Oracle VM Manager portal. To allow tcp traffic on ports 8888, 8899 and 4443 on the Oracle VM Manager host, as root or with sudo type the following three commands: # system-config-securitylevel-tui -q -p 8888 # system-config-securitylevel-tui -q -p 8899 # system-config-securitylevel-tui -q -p 4443

SMTP Server
Oracle VM Manager has several user notification features that require the use of an SMTP server. During the Oracle VM Manager installation you will be asked to a) enter the SMTP server IP address or fully qualified domain name and b) the SMTP port and c) a valid email address and password for the Oracle VM Manager admin account. Entering the SMTP details during the installation is optional and can be configured after the installation by using the “update_email.sh” script on the Oracle VM Manager host or by editing the values in Oracle VM Manager repository database table ovs_sys_value under the OVS schema.

Oracle VM Manager Guest Console
The semi integrated Oracle VM Manager guest console requires the Oracle VM Manager host to have the TightVNC package installed. The TightVNC package allows non-Linux users, i.e.Windows or MAC clients to use the TightVNC guest console. The TightVNC package can be downloaded from: http://oss.oracle.com/oraclevm/manager/RPMS/ The next example shows how to download and install the TightVNC package on the Oracle VM Manager host as root. The TightVNC package can be downloaded and installed in any directory, i.e. /tmp on the Enterprise Linux host. # wget http://oss.oracle.com/oraclevm/manager/RPMS/tightvncjava-1.3.9-3.noarch.rpm # rpm -ivh tightvnc-java-version.noarch.rpm Note: If you use Firefox on Linux to access the Oracle VM Manager user interface and would like to use the TightVNC guest console you will also need to install the Oracle VM Manager console plug in (ovm-console) on your Linux workstation. The ovm-console package is available at http://oss.oracle.com/oraclevm/manager/RPMS/.

Required Installation Passwords and Password Complexity Requirements
During the Oracle VM Manager Installation you will be prompted for a total of five required passwords. List 2 shows the five required passwords. The The The The The password password password password password for for for for for the the the the the Oracle XE SYS and SYSTEM accounts. Oracle VM Manager OVS database schema. OC4J oc4jadmin account. Web Service keystore file. admin account.

List 3 shows the password complexity requirements for the required passwords.

91 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

The password must begin with an uppercase or lowercase letter. The password may include the following characters: Numbers (such as 1, 2, 3, and so on), letters (from a to z, A to Z), and underscores (_). Examples of valid passwords: Password01, Password_123, password. Tip: Select and write down all the passwords before you start the installation. Unfortunately you may not be able to use the same password for all four services because the Web Service has a slightly different password policy.

Oracle VM Manager Installation
An Oracle VM Manager installation takes approximately 5-15 minutes depending on the performance of the Oracle VM Manager host. During the Oracle VM Manager installation, you are required to set the following ports. Please note that the installer program will provide defaults. The Oracle XE HTTP port. The default port number is 8080. The Oracle XE listening port. The default port number is 1521. 1. Download the Oracle VM Manager ISO file from the Oracle eDelivery Linux portal. 2. Log in to the Oracle VM Manager host as root. 3. Copy the Oracle VM Manager ISO file to a directory on the Oracle VM Manager host, i.e. to the /tmp directory. 4. Create a directory to mount the ISO file, i.e. mkdir -p /tmp/mount-point. 5. Mount the ISO file by typing “mount -o loop,ro OracleVM-Manager-version.iso mount-point” 6. Change into the directory where the ISO file is mounted, i.e. “cd /tmp/mount-point. 7. Type “sh runInstaller.sh” to start the Oracle VM Manager installation and then type “1” to select the “Install Oracle VM Manager” option as shown in the next example. # sh runInstaller.sh Welcome to Oracle VM Manager 2.2 Please enter the choice: [1|2|3] 1. Install Oracle VM Manager 2. Uninstall Oracle VM Manager 3. Upgrade Oracle VM Manager 1 Starting Oracle VM Manager 2.2 installation … 8. From the ”Do you want to install a new database or use an existing one? [1|2]” prompt type ”1” to select the ”Install a new Oracle XE database on localhost” option as shown below. Do you want to install a new database or use an existing one? [1|2] 1. Install a new Oracle XE database on localhost 2. Use an existing Oracle database in my network 1 Prepare to install the Oracle XE database ... Checking the supported platforms ... Done Checking the prerequisite packages are installed ... Done Checking the available disk space ... Done Installing the oracle-xe-univ package (rpm) now ... Done Oracle Database 10g Express Edition Configuration ------------------------------------------------This will configure on-boot properties of Oracle Database 10g Express Edition. The following questions will determine whether the database should be starting upon system boot, the ports it will use, and the passwords that will be used for database accounts. Press <Enter> to accept the defaults. Ctrl-C will abort. 9. Press the enter key to select the default HTTP port 8080 for Oracle Application Express. Specify the HTTP port that will be used for Oracle Application Express [8080]: 10. Press the enter key to select the default database listener port 1521. Specify a port that will be used for the database listener [1521]: 11. Type a secure password for the database accounts. Specify a password to be used for database accounts. Note that the same password will be used for SYS and SYSTEM. Oracle recommends the use of different passwords for each database account. This can be done after initial configuration:

92 of 243

18.04.2011 16:35

The Underground Oracle VM Manual
Confirm the password:

http://itnewscast.com/book/export/html/844

12. Type y and press enter to configure the Oracle Database 10g Express Edition to start on boot. Do you want Oracle Database 10g Express Edition to be started on boot (y/n) [y]:y Starting Oracle Net Listener...Done Configuring Database...Done Starting Oracle Database 10g Express Edition Instance...Done Installation Completed Successfully. To access the Database Home Page go to "http://127.0.0.1:8080/apex" Checking the availability of the database … Set default database schema to 'OVS'. 12. Type a secure password for the OVS account. Please enter the password for account 'OVS': Confirm the password: Creating the Oracle VM Manager database schema ...Done Installing the ovs-manager package (rpm) ... Done Installing the oc4j package (rpm) ... Done Please enter the password for account 'oc4jadmin': Confirm the password: Starting OC4J ... Done. To access the OC4J Home Page and change the password go to http://127.0.0.1:8888/em Deploying Oracle VM Manager application to OC4J container. Creating connection pool ... Done Creating data source ... Done Deploying application help ... Done Deploying application ... Done 13. Type a secure password for the keystore password for the Web Service. Please enter the keystore password for the Web Service: Confirm the password: Setting keystore password for Web Service ... Done 14. You can enable HTTPS for the Oracle VM Manager user interface by typing Y, or n if you do not want to enable HTTPS. If you select Y a private SSL certificate will be generated and used to secure the Oracle VM Manager user interface. In the example we typed Y to enable HTTPS. Do you want to use HTTPS access for Oracle VM Manager (Y|n)?y Configuring OC4J to use HTTPS ... Done Stopping OC4J ... Done Starting OC4J ... Done 15. Type a secure password for the default Oracle VM Manager admin account. You will use the admin user name and the admin password to log into the Oracle VM Manager user interface. Please enter the password for the default account 'admin': Confirm the password: Configuring SMTP server … 16. Type the outgoing SMTP mail server IP address or the fully qualified domain name (optional). Please enter the outgoing SMTP mail server (e.g. - mail.abc.com, mail.abc.com:25): mail.my.domain.net:25 Mail server checking, may need some time, please wait... Setting the SMTP server to mail.my.domain.net... Done 17. Type the e-mail address for the admin account. Please enter an e-mail address for account 'admin': my.admin.account@my.domain.com Confirm the e-mail address: my.admin.account@my.domain.com The console feature is not enabled by default.

93 of 243

18.04.2011 16:35

The Underground Oracle VM Manual
For detailed setup, refer to Oracle VM Manager User's Guide Installation of Oracle VM Manager completed successfully. To access the Oracle VM Manager 2.2 home page go to: https://<Oracle VM Manager Host>:4443/OVS To access the Oracle VM Manager web services WSDL page go to: https://<Oracle VM Manager Host>:4443/OVSWS/LifecycleService.wsdl https://<Oracle VM Manager Host>:4443/OVSWS/ResourceService.wsdl https://<Oracle VM Manager Host>:4443/OVSWS/PluginService.wsdl https://<Oracle VM Manager Host>:4443/OVSWS/ServerPoolService.wsdl https://<Oracle VM Manager Host>:4443/OVSWS/VirtualMachineService.wsdl https://<Oracle VM Manager Host>:4443/OVSWS/AdminService.wsdl To access the Oracle VM Manager help page go to: https://<Oracle VM Manager Host>:4443/help/help If you selected HTTP: To access the Oracle VM Manager 2.2 home page go to: http://<Oracle VM Manager Host>:8888/OVS To access the Oracle VM Manager web services WSDL page go to: http://<Oracle VM Manager Host>:8888/OVSWS/LifecycleService.wsdl http://<Oracle VM Manager Host>:8888/OVSWS/ResourceService.wsdl http://<Oracle VM Manager Host>:8888/OVSWS/PluginService.wsdl http://<Oracle VM Manager Host>:8888/OVSWS/ServerPoolService.wsdl http://<Oracle VM Manager Host>:8888/OVSWS/VirtualMachineService.wsdl http://<Oracle VM Manager Host>:8888/OVSWS/AdminService.wsdl To access the Oracle VM Manager help page go to: http://<Oracle VM Manager Host>:8888/help/help

http://itnewscast.com/book/export/html/844

The Oracle VM Manager application was successfully installed and can be accessed by typing https://<Oracle VM Manager Host>:4443/OVS if you selected HTTPS or http://<Oracle VM Manager Host>:8888/OVS if you did not select HTTPS in a web browser. Enter the admin user name and the admin password to log into the Oracle VM Manager portal. List 2 shows the supported Web browsers for the Oracle VM Manager user interface. Microsoft Internet Explorer 5.5 or later Firefox 1.0.4 or later (Linux and Windows) Safari 2.x (Mac) If you configured the SMTP server the admin account will receive the following email:

OVM Administrator
To access the Oracle VM Manager 2.2 home page go to: https://<Oracle VM Manager Host>:4443/OVS To access the Oracle VM Manager web services WSDL page go to: https://<Oracle VM Manager Host>:4443/OVSWS/LifecycleService.wsdl https://<Oracle VM Manager Host>:4443/OVSWS/ResourceService.wsdl https://<Oracle VM Manager Host>:4443/OVSWS/PluginService.wsdl https://<Oracle VM Manager Host>:4443/OVSWS/ServerPoolService.wsdl https://<Oracle VM Manager Host>:4443/OVSWS/VirtualMachineService.wsdl https://<Oracle VM Manager Host>:4443/OVSWS/AdminService.wsdl To access the Oracle VM Manager help page go to: https://<Oracle VM Manager Host>:4443/help/help Figure 2 shows the Oracle VM Manager Log in page.

94 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

The Oracle Database Express Edition portal can be accessed from the Oracle VM manager server locally by entering http://127.0.0.1:8080/apex or remotely by entering the ip address or the FQDN followed by :8080/apex, i.e. http://<Oracle VM Manager Host>:8080/apex in a web browser. You can use the SYS, SYSTEM or the OVS user account and the associated password to log in to the Oracle Database Express Edition portal. Figure 3 shows the Oracle Database Express Edition portal login page.

The Application Server Control portal is also part of the Oracle VM Manager install. The oc4jadmin account can be maintained from the Application Server Control portal. The Application Server Control portal can only be access locally from the Oracle VM Manager host by typing http://127.0.0.1:8888/em. Use the oc4jadmin account and password to access the Application Server Control portal. Figure 4 shows the Application Server Control login page.

Applying the 2.2.16 Oracle VM Manager Patch
The Oracle VM Manager 2.2.16 patch provides several Oracle VM Manager updates and bug fixes. The Oracle VM Manager 2.2.16 patch is available from the Unbreakable Linux Network in the Oracle Software for Enterprise Linux channels. Table 3 lists the ovs-manager-2.2-16.noarch package and the channels where the ovs-manager-2.2-16.noarch package is hosted. Table 3

95 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Package
ovs-manager-2.2-16.noarch ovs-manager-2.2-16.noarch ovs-manager-2.2-16.noarch ovs-manager-2.2-16.noarch

Channel
Oracle Software for Enterprise Linux 4 (i386) Oracle Software for Enterprise Linux 4 (x86_64) Oracle Software for Enterprise Linux 5 (x86_64) Oracle Software for Enterprise Linux 5 (i386)

Description
Oracle Software for Enterprise Linux 4 (i386) Oracle Software for Enterprise Linux 4 (x86_64) Oracle Software for Enterprise Linux 5 (x86_64) Oracle Software for Enterprise Linux 5 (i386)

While applying the ovs-manager-2.2-16.noarch package you will be prompted for a total of three passwords. List 4 shows the three passwords. The password for the Oracle XE SYS account. The password for the Oracle VM Manager OVS database schema. The password for the OC4J oc4jadmin account. The next example shows how to install the ovs-manager-2.2-16.noarch package as root or using sudo.

Updating Oracle VM Manager using the Installation Media
Unlike Oracle VM Server, which has multiple update options Oracle VM Manager is updated by downloading the media from Oracle’s eDelivery portal and executing an installation script. Oracle VM media is freely downloadable from Oracle’s eDelivery system at http://edelivery.oracle.com/linux. Updating Oracle VM Manager is accomplished by running the Oracle VM Manager installation program on the Oracle VM Manager host and to select the upgrade installation option. The Oracle VM Manager installation program will ask for the passwords for the existing Oracle VM Manager OVS database account and the oc4jadmin account. The Oracle VM Manager OVS database account and the oc4jadmin account passwords are selected during the Oracle VM Manager installation process. The account passwords for the Oracle VM Manager OVS database account and the oc4jadmin account can be managed from their respected web portals. Note: The Oracle Database Express Edition portal and the Application Server Control portal are installed by default with the Oracle VM Manager portal. The Oracle VM Manager Template does not enclude the Oracle Database Express Edition portal and the Application Server Control portal.

OVS Database and oc4jadmin Accounts and Passwords
The OVS account can be maintained from the Oracle Database Express Edition portal. The Oracle Database Express Edition portal is a part of the Oracle VM Manager install and can be accessed locally by entering http://127.0.0.1:8080/apex or remotely by entering the ip address or the FQDN followed by :8080/apex, i.e. http://OracleVM_Manager_fqdn:8080/apex in a web browser. The next Figure shows the Oracle Database Express Edition portal login page.

The Application Server Control portal is a part of the Oracle VM Manager install. The oc4jadmin account can be maintained from the Application Server Control portal. The Application Server Control portal can only be accessed locally from the Oracle VM server console by entering http://127.0.0.1:8888/em in a local web browser. The next Figure shows the Application Server Control portal login page.

96 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Updating Oracle VM Manager
The following section will review each of the steps of a successful Oracle VM Manager upgrade. The update process requires you to select the upgrade option from the install prompt as well as enter the passwords for the Oracle VM Manager OVS database account and the Oracle VM Manager oc4jadmin account. Step 1 is to download the ISO image from http://edelivery.oracle.com/linux. Once the ISO image is downloaded and placed on the Oracle VM Manager host or made available from a network share then it should be mounted as root user. The following example shows how to mount the Oracle VM Manager ISO image from the Oracle VM Manager server console as root user. # mount -o ro,loop OracleVM-Manager-2.1.2.iso mnt/ # cd mnt # ls EULA LICENSE readme.txt runInstaller.sh scripts source TRANS.TBL Step 2 is to execute the runInstaller.sh script within the Oracle VM Manager root directory from the Oracle VM Manager server console as root. The following example shows how to execute the runInstaller.sh from the Oracle VM Manager server console as root user. # sh runInstaller.sh Step 3 is to select the Upgrade option (number 3) from the install prompt. The next example shows the installation choice prompt. Please enter the choice: [1|2|3] 1. Install Oracle VM Manager 2. Uninstall Oracle VM Manager 3. Upgrade Oracle VM Manager Step 4 asks if you would like to proceed or cancel the upgrade. You can enter y to proceed or n to cancel the upgrade. The following example shows the upgrade acceptance prompt. Are you sure you want to upgrade Oracle VM Manager from version 2.1.1 to 2.1.2 ? [y|N] y Step 5 is to enter the OVS password. The following example shows the OVS installation password prompt. Please enter the password for database account 'OVS': Step 6 is to enter the oc4jadmin password. The following example shows the oc4jadmin installation password prompt. Please enter the password for account 'oc4jadmin': Step 7 is to enter yes or no to backup the Oracle VM Manager database. This step is optional, i.e. entering no will not abort the upgrade. The following example shows the Oracle VM Manager database backup prompt. Would you like to back up the Oracle VM Manager database? Now back up the Oracle VM Manager database to /usr/lib/oracle/xe/app /oracle/product/10.2.0/server/dump-2008-10-06-05-24-25-PM.dmp A successful upgrade will exit with the following informational warnings. Export terminated successfully with warnings. Done. Now upgrading the database schema ... Done. Now upgrading the Oracle VM Manager application ... Done. Upgrade Oracle VM Manager successfully. Please check the log /var/log/ovm-manager/upgrade_2.1.1_2.1.2.log for more information.

97 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

To access the Oracle VM Manager 2.1.2 home page go to: http://ovm01.com:8888/OVS To access the Oracle VM Manager help page go to: http://ovm01.com:8888/help/help # Step 8 is to validate the Oracle VM Manager version number from the Oracle VM Manager home page. The Oracle VM home page URL can be accessed locally via http://127.0.0.1:8888/OVS or from a remote web browser via http://ip_address_of_host:8888/OVS or http://fqdn:8888/OVS. From the Oracle VM Manager home page you can validate the version number from the footer of the home page.

Oracle VM Manager Database Backup and Restoration Oracle VM Manager Database Backup
The Oracle VM Manager batabase can be backed up using the backup.sh script. Before you backup the Oracle VM Manager database, access Oracle VM Manager and ensure that the status of all of the Oracle VM servers is "Running". You will need the following details for the backup procedure a) the password for database account 'OVS', which was assigned during the Oracle VM Manager install, b) the path for dump file, i.e. /tmp/ovs.dmp and c) the path for log file, i.e. /tmp/ovs.log. We will conclude Step 1 with a complete Oracle VM Manager database restore. The following example shows how to execute the “/opt/ovs-manager-2.2/bin/backup.sh” script from the Oracle VM Manager host as root user. # sh /opt/ovs-manager-2.2/bin/backup.sh Next to backup the Oracle VM Manager database select the Backup Oracle VM Manager option (number 1) from the backup prompt. The next example shows the backup selection prompt. sh /opt/ovs-manager-2.2/bin/backup.sh Welcome to Oracle VM Manager Completed backup: Please enter the choice: [1|2] 1. Back up Oracle VM Manager 2. Restore Oracle VM Manager 1 The next example reviews a complete Oracle VM Manager database backup. Access the Oracle VM Manager server console as root and type "sh /opt/ovs-manager-2.2/bin/backup.sh" to start the Oracle VM Manager database backup as shown in the following example. # sh /opt/ovs-manager-2.2/bin/backup.sh Welcome to Oracle VM Manager Completed backup: Please enter the choice: [1|2] 1. Back up Oracle VM Manager 2. Restore Oracle VM Manager 1 Back up data now ... Please enter the password for database account 'OVS': Please enter the path and filename for the dump file (eg /tmp/ovs.dmp):/tmp/ovs.dmp Please enter the path and filename for the log file (eg /tmp/ovs.log):/tmp/ovs.log SQL*Plus: Release 10.2.0.1.0 - Production on Sat Oct 17 10:00:51 2009 Copyright (c) 1982, 2005, Oracle. All rights reserved.

Connected to: Oracle Database 10g Express Edition Release 10.2.0.1.0 - Production SQL> PL/SQL procedure successfully completed. SQL> Disconnected from Oracle Database 10g Express Edition Release 10.2.0.1.0 - Production Export: Release 10.2.0.1.0 - Production on Sat Oct 17 10:00:52 2009

98 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Copyright (c) 1982, 2005, Oracle. All rights reserved.

Connected to: Oracle Database 10g Express Edition Release 10.2.0.1.0 - Production Export done in US7ASCII character set and AL16UTF16 NCHAR character set server uses AL32UTF8 character set (possible charset conversion) . exporting pre-schema procedural objects and actions . exporting foreign function library names for user OVS . exporting PUBLIC type synonyms . exporting private type synonyms . exporting object type definitions for user OVS About to export OVS's objects ... . exporting database links . exporting sequence numbers . exporting cluster definitions . about to export OVS's tables via Conventional Path ... . . exporting table OVS_AGENT 2 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_ALERT 11 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_CATEGORY 1 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_CDROM 5 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_CDROM_RESOURCE 0 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_GROUP 4 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_HD_TEMP . . exporting table OVS_IMG_OS 0 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_IMG_TEMP . . exporting table OVS_LOCK 19 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_MAP 35 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_NIC_TEMP . . exporting table OVS_OS_RESOURCE 17 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_PARTNER 0 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_PREFERRED_SERVER 0 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_PRIVILEGE 4 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_PRIVILEGE_ROLE 5 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_RESOURCE 4 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_ROLE 3 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_SERVER 2 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics.

99 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

. . exporting table OVS_SITE 1 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_STATISTIC 16593 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_SYS_VALUE 8 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_TASK 0 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_USER 19 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_USER_GROUP 75 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_USER_ROLE 19 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_USER_SITE 19 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_VD_IMG 39 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_VIRTUAL_DISK 73 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_VM_GEN_INFO 25 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_VM_IMG 25 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_VM_NETWORK 25 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_VM_SNAPSHOT 0 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_WORKSPACE_TEMP . exporting synonyms . exporting views . exporting stored procedures . exporting operators . exporting referential integrity constraints . exporting triggers . exporting indextypes . exporting bitmap, functional and extensible indexes . exporting posttables actions . exporting materialized views . exporting snapshot logs . exporting job queues . exporting refresh groups and children . exporting dimensions . exporting post-schema procedural objects and actions . exporting statistics Export terminated successfully with warnings. SQL*Plus: Release 10.2.0.1.0 - Production on Sat Oct 17 10:01:05 2009 Copyright (c) 1982, 2005, Oracle. All rights reserved.

Connected to: Oracle Database 10g Express Edition Release 10.2.0.1.0 - Production SQL>

100 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

PL/SQL procedure successfully completed. SQL> Disconnected from Oracle Database 10g Express Edition Release 10.2.0.1.0 - Production Done Please check the log file /tmp/ovs.log #

Oracle VM Manager Database Restore
The Oracle VM Manager batabase can be restored from a backup using the backup.sh script. Before you restore the Oracle VM Manager database, access Oracle VM Manager and ensure that the status of all of the Oracle VM servers is "Running". The following details are required to restore the Oracle VM Manager databasee a) the password for database account 'OVS', which was assigned during the Oracle VM Manager install, b) the path for dump file, i.e. /tmp/ovs.dmp and c) the path for log file, i.e. /tmp/ovs.log. The following example shows how to execute the “/opt/ovs-manager-2.2/bin/backup.sh” script from the Oracle VM Manager server console as root user. # sh /opt/ovs-manager-2.2/bin/backup.sh Next, to backup the Oracle VM Manager database, select the Restore Oracle VM Manager option (number 2) from the backup prompt. The next example shows the backup selection prompt. sh /opt/ovs-manager-2.2/bin/backup.sh Welcome to Oracle VM Manager Completed backup: Please enter the choice: [1|2] 1. Back up Oracle VM Manager 2. Restore Oracle VM Manager 2 The next example reviews a complete Oracle VM Manager database restore. Access the Oracle VM Manager server console as root and type "sh /opt/ovs-manager-2.1/bin/backup.sh" to start the Oracle VM Manager database restore as shown in the following example. The next example reviews an Oracle VM Manager database restore procedure. Access the Oracle VM Manager server console as root and type "sh /opt/ovs-manager-2.1/bin/backup.sh" as shown in the following example. # sh /opt/ovs-manager-2.1/bin/backup.sh Welcome to Oracle VM Manager Please enter the choice: [1|2] 1. Back up Oracle VM Manager 2. Restore Oracle VM Manager 2 Restore data now ... Please enter the password for database account 'SYS': Please enter the path and filename for the dump file (eg /tmp/ovs.dmp):/tmp/ovs.dmp Please enter the path and filename for the log file (eg /tmp/ovs.log):/tmp/ovs.log Import: Release 10.2.0.1.0 - Production on Sun Oct 18 11:15:24 2009 Copyright (c) 1982, 2005, Oracle. All rights reserved.

Connected to: Oracle Database 10g Express Edition Release 10.2.0.1.0 - Production Export file created by EXPORT:V10.02.01 via conventional path Warning: the objects were exported by OVS, not by you import done in US7ASCII character set and AL16UTF16 NCHAR character set import server uses AL32UTF8 character set (possible charset conversion) . . importing table "OVS_AGENT" 2 rows imported . . importing table "OVS_ALERT" 11 rows imported . . importing table "OVS_CATEGORY" 1 rows imported . . importing table "OVS_CDROM" 5 rows imported . . importing table "OVS_CDROM_RESOURCE" 0 rows imported . . importing table "OVS_GROUP" 4 rows imported . . importing table "OVS_IMG_OS" 0 rows imported . . importing table "OVS_LOCK" 19 rows imported

101 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

. . importing table "OVS_MAP" 35 rows imported . . importing table "OVS_OS_RESOURCE" 17 rows imported . . importing table "OVS_PARTNER" 0 rows imported . . importing table "OVS_PREFERRED_SERVER" 0 rows imported . . importing table "OVS_PRIVILEGE" 4 rows imported . . importing table "OVS_PRIVILEGE_ROLE" 5 rows imported . . importing table "OVS_RESOURCE" 4 rows imported . . importing table "OVS_ROLE" 3 rows imported . . importing table "OVS_SERVER" 2 rows imported . . importing table "OVS_SITE" 1 rows imported . . importing table "OVS_STATISTIC" 16593 rows imported . . importing table "OVS_SYS_VALUE" 8 rows imported . . importing table "OVS_TASK" 0 rows imported . . importing table "OVS_USER" 19 rows imported . . importing table "OVS_USER_GROUP" 75 rows imported . . importing table "OVS_USER_ROLE" 19 rows imported . . importing table "OVS_USER_SITE" 19 rows imported . . importing table "OVS_VD_IMG" 39 rows imported . . importing table "OVS_VIRTUAL_DISK" 73 rows imported . . importing table "OVS_VM_GEN_INFO" 25 rows imported . . importing table "OVS_VM_IMG" 25 rows imported . . importing table "OVS_VM_NETWORK" 25 rows imported . . importing table "OVS_VM_SNAPSHOT" 0 rows imported Import terminated successfully without warnings. Done Please check the log file /tmp/ovs.log #

Deploying the Oracle VM Manager Template
This section will review how to deploy the Oracle VM Manager Template. The section starts with a review of the Oracle VM Manager Template prerequisites, followed with the procedure to deploy the Oracle VM Manager Template on an Oracle VM server.

Oracle VM Manager Template Prerequisites
Table 4 shows the disk, storage, RAM and networking prerequisits for an Oracle VM server hosting the Oracle VM Manager Template. Oracle VM Server Resource Requirements Disk Space Minimum Requirements

The Oracle VM Manager Template requires at least 4GB of free space on the Oracle VM server for the compressed and uncompressed files. The Oracle VM Manager Template can be configured on an Oracle VM server with a local OCFS2 storage repository or with a shared OCFS2 or NFS storage repository. If the Oracle VM server is using a shared storage repository, mount only the root repository before running the Deploy_Manager_Template.sh script. Only after the Oracle VM Manager Template has been deployed can extended storage repositories be configured and mounted. The Oracle VM Manager Template requires at least 2GB of free memory on the Oracle VM Server. The Oracle VM Server will need to have a working Xen bridge for the Oracle VM Manager Template. The Oracle VM Manager Deploy_Manager_Template.sh script will list the “Available network bridges” and ask to select a bridge for the Oracle VM Manager Teample. The xen bridge that is selected, i.e. xenbr0, xenbr1, xenbr2, etc.. must be active and accessable on the network.

Storage Repositories

RAM

Networking

102 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Note: The Oracle VM Manager Template requires a static IP address that will be assigned to the Oracle VM Manager template virtual machine. The Oracle VM Manager Deploy_Manager_Template.sh script will ask for the following network details for the Oracle VM Manager virtual machine: * Static IP address * Netmask * Default Gateway IP address * DNS Server IP address * Hostname

SMTP Server
Oracle VM Manager has several user notification features that require the use of an SMTP server. During the Oracle VM Manager installation you will be asked to a) enter the SMTP server IP address or fully qualified domain name and b) the SMTP port and c) a valid email address and password for the Oracle VM Manager admin account. Entering the SMTP details during the template configuration is optional and can be configured after the template is deployed using the “update_email.sh” script or by editing the values in Oracle VM Manager repository database table ovs_sys_value under the OVS schema. Tip: The Deploy_Manager_Template.sh script can be run with options “--skipsmtpcheck” and “--skipmailcheck” to skip the SMTP server check and email check.

User Accounts, Passwords and Password Complexity Requirements
The Oracle VM Manager Template configuration will create two operating system users with default passwords; root with password ovsroot and oracle with password oracle. The default passwords for the root and oracle accounts should be chaged once the template is deployed. The oracle account belongs to the oinstall and dba groups. The default vnc console password for the Oracle VM Manager Template is oracle. During the Oracle VM Manager Installation you will be prompted for a total of six required passwords. List 5 shows the six required passwords. The The The The The The password for the Oracle XE SYS and SYSTEM accounts. password for the Oracle VM Manager OVS database schema. password for the OC4J oc4jadmin account. password for the Web Service keystore file. password for the admin account. Oracle VM agent password.

Tip: The Oracle VM agent password can be reset by typing “service ovs-agent configure” from dom0 as root. List 6 shows the password complexity requirements for the required passwords. The password must begin with an uppercase or lowercase letter. The password may include the following characters: Numbers (such as 1, 2, 3, and so on), letters (from a to z, A to Z), and underscores (_). Examples of valid passwords: Password01, Password_123, password. Tip: Select and write down all the passwords before you start the installation. Unfortunately you may not be able to use the same password for all four services because the Web Service has a slightly different password policy.

HTTP and Oracle VM Manager XE Database listening Ports
During the Oracle VM Manager Template configuration phase the installer asks to set the XE HTTP port and XE Database port. The installer program will provide defaults for the XE HTTP port and XE Database port. The Oracle XE HTTP port. The default port number is 8080. The Oracle XE listening port. The default port number is 1521.

Oracle VM Server Cluster Configurations
If the Oracle VM Manager Template is being deployed on an Oracle VM server that had been in an Oracle VM server pool, the Oracle VM servers cluster configurations must be cleaned before deploying the template.

103 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

To clean an Oracle VM server's cluster configurations it is necessary to a) empty the /etc/ocfs/cluster.conf file b) delete and recreate the local BerkleyDB and c) run the cleanup.py script to stop o2cb heartbeat, offline o2cb, remove o2cb configuration file, umount ovs-agent storage repositories and to cleanup ovs-agent local database. To clear /etc/ocfs/cluster.conf file type “cat /dev/null> /etc/ocfs2/cluster.conf” from dom0, as shown in the next example. # cat /dev/null> /etc/ocfs2 /cluster.conf To remove the local BerkleyDB first type “service ovs-agent stop”, which stops the Oracle VM agent. Next, type “rm -fr /etc/ovsagent/db/*” to delete the BerkleyDB. Finally, type “service ovs-agent start” to start the Oracle VM agent, which also recreate a new local BerkleyDB. To stop the o2cb heartbeat, offline o2cb, remove o2cb configuration file, unmount ovs-agent storage repositories and to cleanup ovs-agent local database, type "/opt/ovs-agent-2.3/utils/cleanup.py" and then type “y” as shown in the next example. # /opt/ovs-agent-2.3/utils /cleanup.py This is a cleanup script for ovs-agent. It will try to do the following: *) stop o2cb heartbeat *) offline o2cb *) remove o2cb configuration file *) umount ovs-agent storage repositories *) cleanup ovs-agent local database Would you like to continue? [y/N] y Cleanup done.

Download and Deploy the Oracle VM Manager Template
It takes approximately 15 to 30 minutes to download and deploy the Oracle VM Manager Template, depending on download bandwidth, and the performance of the Oracle VM host. List 7 reviews the five steps to deploy the Oracle VM Manager Template. 1. Download the Oracle VM Manager template named “Oracle VM Manager 2.2.0 for x86 (32 bit) with Oracle Linux 5.4” from “http://edelivery.oracle.com/linux”. Note: Do not unzip the V21186-01.zip file until Step 2. 2. Copy the zip file named V21186-01.zip to your Oracle VM server's /OVS/running_pool directory. 3. Unzip the V21186-01.zip file. The V21186-01.zip file contains two file, Deploy_Manager_Template.sh and OVM_EL5U4_X86_OVM_MANAGER_PVM.tgz. Note: Do not untar the OVM_EL5U4_X86_OVM_MANAGER_PVM.tgz file. 4. From dom0 as root type “sh Deploy_Manager_Template.sh”. The Deploy_Manager_Template.sh script is used to check the prerequisite and gather all the details to configure and create the Oracle VM Manager virtual machine. 5. After the Deploy_Manager_Template.sh script completes and the Oracle VM Manager template is deployed, you will need to wait about 3 to 5 minutes for the Oracle VM Manager application to start. Once the Oracle VM Manager application is running log in to the Oracle VM Manager URL that is displayed at the end of the Deploy_Manager_Template.sh script using the admin account and the password youe selected durring the installation. You can ssh to the Oracle VM Manager Template using the IP address assigned durring the configuration phase using the root account with the default “ovsroot” password.

Running the Deploy_Manager_Template.sh Script
The next example shows a complete Deploy_Manager_Template.sh script session run as root from dom0. The Oracle VM server in the example was configed with a shared root storage repository. The Deploy_Manager_Template.sh and OVM_EL5U4_X86_OVM_MANAGER_PVM.tgz files are in the /OVS/running_pool directory. Tip: The Deploy_Manager_Template.sh script can be run with options “--skipsmtpcheck” and “--skipmailcheck” to skip the SMTP server check and email check. # sh Deploy_Manager_Template.sh --skipsmtpcheck --skipmailcheck Starting prerequisite checking ... Checking Done Checking Done Checking Checking Done the Oracle VM Server ... the xend status ...running the Oracle VM Agent version ...Done the Oracle VM Agent status ...

104 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Checking the Oracle VM Server free memory: 15623 MB Done Checking the available disk space ... [ R ] 6a16e7f3-36aa-475e-9181-37f65c944487 => ca-fileserver4:/vol/vol1_nfs_demo *** Storage repositories initialized. **ca-fileserver4.us.oracle.com:/vol/vol1_nfs_scdemo mounted, set as cluster root. [ R ] 6a16e7f3-36aa-475e-9181-37f65c944487 => ca-fileserver4:/vol/vol1_nfs_demo You have 90 GB in /var/ovs/mount/6A16E7F336AA475E918137F65C944487 Checking the running pool directory .../var/ovs/mount/6A16E7F336AA475E918137F65C944487/running_pool Done Checking the HA cluster prerequisite ... Please enter the password for Oracle VM Agent: cluster_precheck...2.3 Done Checking whether Oracle VM Agent DB is clean ... Done Setting up the network for Oracle VM Manager ... Oracle VM Manager needs to be configured using a static IP address. Follow the prompts to provide your network settings for Oracle VM Manager. Press any key to continue... Available network bridges: 0) xenbr0 Link encap:Ethernet HWaddr 00:1F:29:68:55:FC inet addr:10.211.0.115 Bcast:0.0.0.0 Mask:255.255.252.0 1) xenbr1 Link encap:Ethernet HWaddr 00:1F:29:68:55:B4 Please choose one of the network bridges(default is xenbr0): You selected the network bridge: xenbr0 Enter static IP address: 10.211.0.216 Enter netmask: [255.0.0.0] 255.255.252.0 Enter gateway: 10.211.0.1 Enter DNS server: 139.185.51.1 Enter hostname (e.g, host.domain.com): [ca-scdemo-ovm2.2.1-01] Generating parameter file ... Note following password setting will be used for SYS and SYSTEM. Oracle recommends the use of different passwords for each database account. This can be done after initial configuration. Specify a password to be used for database accounts: Confirm the password: Please enter the password for account 'OVS': Confirm the password: Please enter the password for account 'oc4jadmin': Confirm the password: Please enter the keystore password for Web Service: Confirm the password: Do you want to use HTTPS access for Oracle VM Manager (Y|n)? n Please enter the password for the default account 'admin': Confirm the password: Set Oracle VM Server Information: Enter the Server Pool Name: HQ Enter the Oracle VM Server login user name: root Enter the Oracle VM Server login password: Do you want to enable HA(y|N)? y Please confirm the following information: Server Pool name : HQ Oracle VM Server name : ca-scdemo1 Oracle VM Server login user name : root Enable HA : true Is the Oracle VM Server information correct(Y|n)? Y Untar the Oracle VM Manager template tarball ...

105 of 243

18.04.2011 16:35

The Underground Oracle VM Manual
Done Move the parameter file to Manager.img Done Create Oracle VM Manager virtual machine ... Done The Oracle VM Manager virtual machine is booting. To finish the Oracle VM Manager configuration, connect to the virtual machine console using any VNC Viewer from a desktop machine via the command: vncviewer ca-scdemo1:5900 After Oracle VM Manager has been successfully deployed, access the Oracle VM Manager home page at: http://10.211.0.216:8888/OVS

http://itnewscast.com/book/export/html/844

The Oracle VM Manager Command Line Interface Introduction
The Oracle VM Manager Command Line Interface is an Oracle VM Manager add on that is available to Oracle Unbreakable Linux support customers via the Oracle Unbreakable Linux Network. The Oracle VM Manager Command Line Interface offers a command-line interface for Oracle VM Manager. The Oracle VM Manager Command Line Interface allows Oracle VM Manager functions to be executed from the command-line. Oracle VM Manager Command Line Interface commands can be scripted allowing automation of administrative tasks, such as checking status, performing lifecycle management, and executing bulk actions. List 8 highlights the most common use cases for the Oracle VM Manager Command Line Interface: Guest, server and server pool lifecycle management: Power On, Power Off, Clone, Save as Template, Import, Migrate, Pause, Unpause, Suspend, Resume and Delete virtual machines Manage virtual machine resources, including ISO files, virtual machine templates and virtual machine images Manage Oracle VM Manager users, and Oracle VM Manager groups Create and configure server pools Manage the Oracle VM Agent Checking the status of guests, server, and server pools: Get a list of running guests in server pool Get a list of active pool servers Diagnose and troubleshoot issues with guests, servers, and server pools Get configuration and status information for guests, servers and server pools Performing bulk operations: With multiple guests With multiple servers With multiple server pools The Oracle VM Manager Command Line Interface (ovmcli) is a stand-alone application that is written in Python that leverages the Oracle VM Manager Web Services API to communicate with Oracle VM Manager. The Oracle VM Manager Command Line Interface can be installed on any Oracle Enterprise Linux or RHEL host with connectivity to your Oracle VM Manager system. The Oracle VM Manager Command Line Interface can be accessed from a local or remote console. Figure 6 illustrates the Oracle VM Manager Command Line Interface intra-component communication and firewall requirements.

List 9 highlights the Oracle VM Manager Command Line Interface communication ports and system passwords:

106 of 243

18.04.2011 16:35

The Underground Oracle VM Manual
Oracle VM Manager: The default HTTP port of Oracle VM Manager UI is 8888. The default HTTPS port of Oracle VM Manager UI is 4443 Remote access to the Command Line Interface hosts via ssh on port 22 Guest console access uses ports 5900 through 5999. Guest console access requires a password without a user name.

http://itnewscast.com/book/export/html/844

The Oracle VM Manager Command Line Interface has one prerequisite, namely the python-ZSI RPM. The ovmcli and python-ZSI RPMs are hosted at ULN. The ovmcli RPM is located in the el5_i386_oracle_addons and the el5_x86_64_oracle_addons channel. The python-ZSI RPM is located in the el5_i386_addons and el5_x86_64_addons channel.

Downloading the Oracle VM Manager Command Line Interface from ULN
In this section we will walk through the procedure to download the Oracle VM Manager Command Line Interface. We will start with a visit to ULN, to search for, and download, the ovmcli and python-ZSI RPMs. After the download section, we will walk through the installation and configuration of the Oracle VM Manager Command Line Interface on an Oracle Enterprise Linux host. If you have already downloaded the RPMs from ULN, please skip to the next section. Note: ULN is password protected. To access ULN you will need a valid Oracle Unbreakable Linux support contract and a CSI number. Support contracts for Oracle VM can be purchased from your Oracle VM sales representative or directly from the Oracle Store. Once you have a valid support contract, you can register at the ULN portal or you can register by running the “up2date –register” command from one of your Oracle VM servers. Please review Oracle VM: How to update an Oracle VM Server for details about registering an Oracle VM server with ULN. The Oracle VM Manager Command Line Interface (ovmcli) is hosted at ULN in the el5_i386_oracle_addons and el5_x86_64_oracle_addons channels. The Oracle VM Manager Command Line Interface prerequisite python-ZSI is hosted in the el5_i386_addons and el5_x86_64_addons channel. Table 5 shows the Oracle VM Manager Command Line Interface package names with the associated ULN channel names. Package ovmcli-1.0-1.el5.noarch.rpm python-ZSI-2.1-a1.el5.noarch.rpm ULN Channel el5_i386_oracle_addons and el5_x86_64_oracle_addons el5_i386_addons and el5_x86_64_addons

1- Point your browser to http://linux.oracle.com as shown in Figure 7. Enter you user name and password and click the Login button to access ULN.

2- From the ULN home page click on the Channels tab as shown in Figure 8.

107 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

3- From the Channels page click the i386_oracle_addons channel for the x86 platform or click the el5_x86_64_oracle_addons channel for the x86-64 platform to download the appropriate ovmcli RPM. In our example we are looking for the el5_x86_64_oracle_addons channel which is not displayed on the current page. We will need to access the next page to locate the el5_x86_64_oracle_addons channel. Click the Next button at the bottom of the page as shown in Figure 9 to display the next page.

4- From the Channels page click the el5_x86_64_oracle_addons channel as shown in Figure 10.

108 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

5- From the Oracle Software addons for Enterprise Linux 5 (X86-64) page, click the Packages link as shown in Figure 11.

6- From the Oracle Software addons for Enterprise Linux 5 (X86-64) > Packages page, click the ovmcli-1.0-1.el5.noarch link as shown in Figure 12 to display the ovmcli-1.0-1.el5.noarch RPM Package Details page.

109 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

7- From the Oracle Software addons for Enterprise Linux 5 (X86-64) > Packages > ovmcli-1.0-1.el5.noarch page, click the ovmcli1.0-1.el5.noarch.rpm link as shown in Figure 13 to download the ovmcli-1.0-1.el5.noarch RPM.

Note: The File list link lists the details about all the files for the ovmcli-1.0-1.el5.noarch RPM. The Requires link lists the requirements for the ovmcli-1.0-1.el5.noarch RPM. The Provides link lists what the ovmcli-1.0-1.el5.noarch RPM provides, i.e. the ovmcli. The Obsoletes link lists what application(s) the ovmcli-1.0-1.el5.noarch RPM obsoletes. The Conflicts link lists the conflicts for the ovmcli-1.0-1.el5.noarch RPM. 8- From the Oracle Software addons for Enterprise Linux 5 (X86-64) > Packages > ovmcli-1.0-1.el5.noarch page, click the Channel link as shown in Figure 14 to return to the Channel page to download the python-ZSI RPM.

9- From the Channels page locate and click the el5_x86_64_addons channel link as shown in Figure 15 to access the Enterprise Linux 5 Add ons (X86-64) > Packages page.

10- From the Oracle Enterprise Linux 5 Add ons (x86-64) > Packages page click the python-ZSI-2.1-a1.el5.noarch.rpm link as shown in Figure 16 to display the Enterprise Linux 5 Add ons (X86-64) > Packages > python-ZSI-2.1-a1.el5.noarch.rpm page.

110 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

11- From the Enterprise Linux 5 Add ons (X86-64) > Packages > python-ZSI-2.1-a1.el5.noarch.rpm page, click the pythonZSI-2.1-a1.el5.noarch.rpm link as shown in Figure 17 to download the python-ZSI-2.1-a1.el5.noarch.rpm RPM.

Note: The File list link lists the details about all the files for the python-ZSI-2.1-a1.el5.noarch.rpm RPM. The Requires link lists the requirements for the python-ZSI-2.1-a1.el5.noarch.rpm RPM. The Provides link lists what the python-ZSI-2.1a1.el5.noarch.rpm RPM provides, i.e. python-ZSI. The Obsoletes link lists what application(s) the python-ZSI-2.1a1.el5.noarch.rpm RPM obsoletes. The Conflicts link lists the conflicts for the python-ZSI-2.1-a1.el5.noarch.rpm RPM. In the Oracle VM Manager Command Line Interface Download section we visted ULN and downloaded the ovmcli and python-ZSI RPM. In the next section we will walk through the installation and configuration of the Oracle VM Manager Command Line Interface.

Oracle VM Manager Command Line Interface Installation & Configuration
In the Oracle VM Manager Command Line Interface installation and configuration section we will first walk through the Oracle VM Manager Command Line Interface installation followed by the Oracle VM Manager Command Line Interface configuration. The Oracle VM Manager Command Line Interface installation is a quick and simple process that requires two RPMs, a) ovmcli and b) python-ZSI. As soon as you have downloaded both RPMs and a) placed them on the host or b) mounted a share with the RPMs you can install the Command Line Interface RPMs by running “rpm –Uvh ovmcli-1.0-1.el5.noarch.rpm python-ZSI-2.1-a1.el5.noarch.rpm” on the Enterprise Linux host as root. Table 6 shows a successful Oracle VM Manager Command Line Interface installation. # rpm -Uvh ovmcli-1.0-1.el5.noarch.rpm python-ZSI-2.1-a1.el5.noarch.rpm Preparing... ########################################### [100%] 1:python-ZSI ########################################### [ 50%] 2:ovmcli ########################################### [100%]

111 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

# Once you have installed the Oracle VM Manager Command Line Interface you can configure the Oracle VM Manager Command Line Interface by running "ovm config" as root. The Oracle VM Manager Command Line Interface configuration will ask for the following five pieces of information as shown in List 10. 1. 2. 3. 4. Oracle VM Manager hostname Oracle VM Manager port number (the default is 8888) Deploy path (use the default) Location of vncviewer (required for vncviewer command, e.g. /usr/bin/vncviewer – i.e. run ”which vncviewer” to find the path to vncviewer) 5. Enable or disable HTTPS support (depends on Oracle VM Manager setup) The next example shows a complete "ovm config" session. # ovm config This is a wizard for configuring the Oracle VM Manager Command Line Interface. CTRL-C to exit. Oracle VM Manager hostname: ovm1.sf.seo.com Oracle VM Manager port number: 8888 Deploy path (leave blank for default): Location of vncviewer (leave blank to skip): /usr/bin/vncviewer Enable HTTPS support? (Y/n): n Configuration complete. Please rerun the Oracle VM Manager Command Line Interface. # You have successfully configured the Oracle VM Manager Command Line Interface. The Oracle VM Manager Command Line Interface is now ready for use and can be accessed from local or remote console by typing "ovm". The next section introduces how to use the Oracle VM Manager Command Line Interface starting with a single command, then a bulk command and concluding with batch script examples.

Oracle VM Manager Command Line Interface Command Examples
This section will start with an introduction to using the Oracle VM Manager Command Line Interface. After the Oracle VM Manager Command Line Interface introduction we will walk through various example Oracle VM Manager Command Line Interface use cases starting with single operation command followed by bulk commands and batch scripting examples. The Oracle VM Manager Command Line Interface can be accessed from the local or remote console from the server hosting the Oracle VM Manager Command Line Interface. Commands can be executed directly from the command line or from within the Oracle VM Manager Command Line Interface shell. The Oracle VM Manager Command Line Interface shell can be accessed by typing “ovm –u username –p your_password shell” as shown in the following example: # ovm -u username -p password shell ovm> You would substitute username with a valid Oracle VM Manager user name and substitute password with a valid password to authenticate into the shell. Once you have successfully authenticated in to the Oracle VM Manager Command Line Interface shell you will be presented with the “ovm>” prompt. From the “ovm>” prompt you can enter the desired subcommands. From the Oracle VM Manager Command Line Interface shell, type “ovm help”, to display an abbreviated list of the help messages, as shown below. ovm> help Usage: subcommand [suboptions] Oracle VM Manager Command Line Interface Subcommands: group img iso sd svr svrp tmpl user val vm config connect exit help --- Group management --- Virtual machine image management --- ISO image management --- Shared virtual disk management --- Server management --- Server pool management --- Virtual machine template management --- User management --- System parameter management --- Virtual machine management --- CLI configuration --- Connect to Oracle VM Manager from CLI shell --- Exit --- Show help

112 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

shell use vncviewer

--- Launch an interactive shell --- Specify a server pool to use --- Start a VNC console

"help <subcommand>" displays help message for that subcommand. "help all" displays complete list of subcommands. ovm> Tip: Help messages for subcommands, i.e. "help <subcommand>" will display the help message for that subcommand. The "help all" command displays a complete list of all the subcommands. Subcommands can also be run directly from the command line outside of the Oracle VM Manager Command Line Interface shell. The next example shows how to display the help message from the command line, not from the Oracle VM Manager Command Line Interface shell. # ovm -u username -p password help Usage: ovm [options] subcommand [suboptions] Oracle VM Manager Command Line Interface Subcommands: group img iso sd svr svrp tmpl user val vm config connect exit help shell use vncviewer # --- Group management --- Virtual machine image management --- ISO image management --- Shared virtual disk management --- Server management --- Server pool management --- Virtual machine template management --- User management --- System parameter management --- Virtual machine management --- CLI configuration --- Connect to Oracle VM Manager from CLI shell --- Exit --- Show help --- Launch an interactive shell --- Specify a server pool to use --- Start a VNC console

To recap, subcommands can be entered from the Oracle VM Manager Command Line Interface shell or directly from the server’s console. Table 3 shows the same “help” command run from the Oracle VM Manager Command Line Interface shell and from the Oracle VM Manager Command Line Interface server’s console. Command Line Interface shell Server console

# ovm -u username -p password shell # ovm -u username -p password help ovm> help

Bulk Commands and Batch Scripting
In this section, we will start with an introduction to Oracle VM Manager Command Line Interface bulk commands and batch scripting. After the introduction, we will walk through two Oracle VM Manager Command Line Interface use case examples. The first use case will show how to suspend and then resume four guests, to accommodate a backup schedule. The last example will show how to add an Oracle VM server to an existing server pool. Note: We will be adding new example Oracle VM Manager Command Line Interface use cases to this chapter on a regular basis. Stay tuned! Bulk commands can be run from the command line or via batch scripts. For example, if you wanted to suspend multiple guests with a single command i.e. “vm suspend”, this could be accomplished by typing multiple “vm suspend” commands separated by a semi-colon. Before we execute our fist “vm suspend” bulk command, lets check the status of our virtual machines using the “vm ls” command. The “vm ls” command can be entered from the Oracle VM Manager Command Line Interface shell or directly from the console. The next example shows the “vm ls” command entered at the server console which will list the status of all of the virtual machines in your Oracle VM server farm. # ovm -u admin -p password vm ls Name Size(MB) Mem VCPUs node4 13313 256 1 node1 13313 256 1 node3 13313 256 1 Status Server_Pool Running SF-HQ Running SF-HQ Running SF-HQ

113 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

node2

13313

256

1

Running

SF-HQ

The same command can be executed from the Oracle VM Manager Command Line Interface shell as shown in the next example. # ovm -u admin -p password shell Type "help" for a list of commands. ovm> vm ls Name Size(MB) Mem VCPUs node4 13313 256 1 node1 13313 256 1 node3 13313 256 1 node2 13313 256 1 ovm>

Status Server_Pool Running SF-HQ Running SF-HQ Running SF-HQ Running SF-HQ

Now that we have checked that status of the guests in our Oracle VM farm we will suspend four “Running” guests named node1, node2, node3, and node4 with a single bulk command. Enter the bulk command from the command line, not the Oracle VM Manager Command Line Interface shell to suspend all four guests as shown in the next example. # ovm -u admin -p password vm suspend -s serverpool_name -n node1; ovm -u admin -p password vm suspend -s serverpool_name -n node2; ovm -u admin -p password vm suspend -s serverpool_name -n node3; ovm -u admin -p password vm suspend -s serverpool_name -n node4 Suspending. Suspending. Suspending. Suspending. # The above example shows four separate “vm suspend” commands separated by a semi-colon. You would replace the password, serverpool_name and hostnames with yours to use this bulk command example in your environment. Tip: The help message for “vm suspend” can be displayed by typing “ovm -u admin -p password help vm suspend” or from the shell by typing “help vm suspend”. Let’s validate that the status of the four virtual machines has changed, from “Running” to “Suspended”, by typing “ovm -u admin -p password vm ls” from the server console, as shown in the next example. # ovm -u admin -p password vm ls Name Size(MB) Mem VCPUs Status node4 13313 256 1 Suspended node1 13313 256 1 Suspended node3 13313 256 1 Suspended node2 13313 256 1 Suspended Server_Pool SF-HQ SF-HQ SF-HQ SF-HQ

The above “vm ls” example validated that our “vm suspend” bulk command was successfully completed. Next we will review how to create a vmsuspend batch script. If you would like to test the vmsuspend batch script, you will need to resume your suspended guest. We will cover how to resume your guests in great detail in a couple of paragraphs. If you need to resume your guests to test our next example vmsuspend batch script, please run the following bulk command from the server console. # ovm -u admin -p password vm resume -s serverpool_name -n node1; ovm -u admin -p password vm resume -s serverpool_name -n node2; ovm -u admin -p password vm resume -s serverpool_name -n node3; ovm -u admin -p password vm resume -s serverpool_name -n node4 Resuming. Resuming. Resuming. Resuming. # The same “vm suspend” bulk command could be run as a batch script. The commands in an Oracle VM Manager Command Line Interface batch script are executed in sequence, identical to how sqlplus runs a sql script. There is no error handling with Oracle VM Manager Command Line Interface batch scripts which means when the previous command finishes, regardless of success or failure, the next command will run. The first step to create an Oracle VM Manager Command Line Interface batch script is to create a file. Let’s call the example file vmsuspend. Save the example commands shown below in the vmsuspend file. vm vm vm vm suspend suspend suspend suspend -s -s -s -s serverpool_name serverpool_name serverpool_name serverpool_name -n -n -n -n node1; node2; node3; node4;

Now that we have created the file vmsuspend, we can run the vmsuspend file as a batch script in one of two ways; this assumes that the vmsuspend file is located in the current working directory, i.e. /home/user/vmsuspend. You can run the batch script from the server console or from the Oracle VM Manager Command Line Interface shell. The next example shows how to run our example vmsuspend batch script from the server console. # ovm shell -s vmsuspend Login: admin

114 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Password: Suspending. Suspending. Suspending. Suspending. # The above example runs our batch script from the servers console using the “-s” or script option. As shown in the example, it is necessary to authenticate into the shell to run the vmsuspend batch script. Once the script completes you are dropped back to the server console. Our other option to run the vmsuspend batch script is from the Oracle VM Manager Command Line Interface shell. The next example shows how to run the vmsuspend batch script from the Oracle VM Manager Command Line Interface shell. The example assumes that the vmsuspend file is located in the current working directory, i.e. /home/user/vmsuspend. # ovm -u admin -p password shell ovm> @vmsuspend Suspending. Suspending. Suspending. Suspending. ovm> The above example runs our batch script from the Oracle VM Manager Command Line Interface shell. Once the script completes, you are dropped back to the Oracle VM Manager Command Line Interface shell prompt. Let’s validate the status of the four virtual machines by typing “vm ls” from the Oracle VM Manager Command Line Interface shell as shown on the next example. # ovm> vm ls Name Size(MB) Mem VCPUs Status node4 13313 256 1 Suspended node1 13313 256 1 Suspended node3 13313 256 1 Suspended node2 13313 256 1 Suspended Server_Pool SF-HQ SF-HQ SF-HQ SF-HQ

Typing “vm ls” from the Oracle VM Manager Command Line Interface shell validated that our previous vmsuspend batch script did successfully run and suspend all four of the guests. You can also list the status of your virtual machines from the server console, as shown in the next example. # ovm -u admin -p password vm ls Name Size(MB) Mem VCPUs Status node4 13313 256 1 Suspended node1 13313 256 1 Suspended node3 13313 256 1 Suspended node2 13313 256 1 Suspended Server_Pool SF-HQ SF-HQ SF-HQ SF-HQ

Now that we have reviewed running a “vm suspend” bulk command and a vmsuspend batch script along with “vm ls”, let’s examine how to resume our guests. First we will review a “vm resume” bulk command, followed with a vmresume batch script. Next, we will resume four “Suspended” guests named node1, node2, node3, and node4 with a single bulk command. Enter the bulk command from the command line, not the Oracle VM Manager Command Line Interface shell, to resume all four guests as shown in the next example. # ovm -u admin -p password vm resume -s serverpool_name -n node1; ovm -u admin -p password vm resume -s serverpool_name -n node2; ovm -u admin -p password vm resume -s serverpool_name -n node3; ovm -u admin -p password vm resume -s serverpool_name -n node4 Resuming. Resuming. Resuming. Resuming. # You can validate the status of the four virtual machines by typing “ovm -u admin -p password vm ls” from the server console or “vm ls” from the Oracle VM Manager Command Line Interface shell. The same “vm resume” bulk command could be run as a batch script. As discussed in the “vm suspend” section the commands in an Oracle VM Manager Command Line Interface batch script are executed in sequence, identical to how sqlplus runs a sql script. There is no error handling with Oracle VM Manager Command Line Interface batch scripts which means when the previous command finishes, regardless of success or fail, the next command will run. The first step to create our example “vm resume” Oracle VM Manager Command Line Interface batch script, is to create a file. Let’s call the example file vmresume. Save the example commands shown below in the vmsuspend file. vm vm vm vm resume resume resume resume -s -s -s -s serverpool_name serverpool_name serverpool_name serverpool_name -n -n -n -n node1; node2; node3; node4;

115 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Now that we have created the file vmresume, we can run the vmreseume file as a batch script in one of two ways; this assumes that the vmreseume file is located in the current working directory, i.e. /home/user/vmresume. You can run the batch script from the server console or from the Oracle VM Manager Command Line Interface shell. The next example shows how to run our example vmresume batch script from the server console. # ovm shell -s vmresume Login: admin Password: Resuming. Resuming. Resuming. Resuming. # The above example runs our batch script from the servers console using the “-s” or script option. As shown in the example it is necessary to authenticate into the shell to run the vmsuspend batch script. Once the script completes, you are dropped back to the server console. Our other option to run the vmresume batch script is from the Oracle VM Manager Command Line Interface shell. The next example shows how to run the vmresume batch script from the Oracle VM Manager Command Line Interface shell. The example assumes that the vmresume file is located in the current working directory, i.e. /home/user/vmresume. # ovm -u admin -p password shell ovm> @vmresume Resuming. Resuming. Resuming. Resuming. ovm> The above example runs our batch script from the Oracle VM Manager Command Line Interface shell. Once the script completes, you are dropped back to the Oracle VM Manager Command Line Interface shell prompt. You can validate the status of the four virtual machines by typing “ovm -u admin -p password vm ls” from the console or “vm ls” from the Oracle VM Manager Command Line Interface shell. We have just completed our review of the “vm suspend” and “vm resume” bulk command and batch scripting. The next example expands from our previous “vm suspend” and “vm resume” with a detailed use case called Guest Backup. The Guest Backup use case will use a batch script that suspends four guests, pauses for 120 seconds and then resumes the guests to accommodate an Oracle VM repository scheduled backup job.

Guest Backup
Our first example use case “Guest Backup”, builds upon what we learned in the previous section with regards to the vm suspend and vm resume examples. The use case shows us how to create a batch script to suspend four guests, pause for 120 seconds and then resume the guests to accommodate a scheduled snapshots/cloning of an Oracle VM extended repository. There are numerous strategies to back up Oracle VM guests. For example, you can use the same agent based backup solution you have always used for your physical machines or you could leverage the snapshots/cloning functionality from your storage solution. Our goal, with the Guest Backup use case, is to provide an Oracle VM Manager Command Line Interface solution to work with the snapshots/cloning functionality from your existing storage. To capture a clean snapshot or backup of a running guest, the guest should be in the suspended state when the snapshot or backup job is executed. When a guest is suspended, the status of the guest operating system is written to disk and removed from system memory. Conversely, when a guest operating system is paused, the system state continues to reside in memory. For the Guest Backup use case we will use the “vm suspend” and “vm resume” commands to ensure that the status of the guest operating system is written to disk and removed from system memory for the duration of the snapshot or backup job. Prerequisites: A storage solution that supports snapshots or cloning. Our example batch script named vmbackup will be scheduled to run each night at 3:00 AM as a cron job. The vmbackup job will run 2 minutes before the scheduled snapshot or clone job. The target guests are in the Running state. List 11 shows the work flow to execute the Guest Backup use case. 1. Create the vmbackup batch script 2. Test the vmbackup batch script 3. Automate the vmbackup batch script with cron. 1- The first step is to create an Oracle VM Manager Command Line Interface batch script on the machine hosting the Oracle VM Manager Command Line Interface. Let’s create a file named vmbackup. Save the example commands shown below in the vmbackup

116 of 243

18.04.2011 16:35

The Underground Oracle VM Manual
file. vm suspend -s serverpool_name -n node1; vm suspend -s serverpool_name -n node2; vm suspend -s serverpool_name -n node3; vm suspend -s serverpool_name -n node4; sleep 120; vm resume -s serverpool_name -n node1; vm resume -s serverpool_name -n node2; vm resume -s serverpool_name -n node3; vm resume -s serverpool_name -n node4;

http://itnewscast.com/book/export/html/844

2- Next we will test the example vmbackup batch script from the server console, as shown in the example. # ovm -u admin -p password shell -s vmbackup Suspending. Suspending. Suspending. Suspending. Sleeping for 120 seconds Resuming. Resuming. Resuming. Resuming. # 3- The final step is to automate the vmbackup batch script to run at 3:00 AM every day with cron, as shown in the example. # crontab –e * 3 * * * ovm -u admin -p password shell -s vmbackup :wq! # We have just completed our first use case that showed us how to create a batch script to suspend four guests, pause for 120 seconds, and then resume the guests to accommodate a scheduled snapshot or backup job of an Oracle VM extended repository.

Create a Server Pool
The next example batch script shows how to create a server pool. vm ls; svrp ls; svrp new -H ovstest19.cn.test.com --serverpool_name=ovstest19 -a -A oracle -U root -P password -L BJ -D "ovstest19 serverpool"; use -s ovstest19; svrp info; sleep 10; svrp status; svr ls;

Programming with the Oracle VM Manager Command Line Interface
This section will provide an example of a more advanced method of programming with the Oracle VM Manager Command Line Interface commands. You can program Oracle VM Manager Command Line Interface commands to determine whether a command is successfully executed or not. You’re also able to get the return result of commands in a programming friendly way. The following example script will a) Create a serverpool b) Register all the discoverable ISOs c) Create a hvm guest by one of the ISOs and d) Check the status of the guest created, if the status is "Running", then power off the guest. from ovmcli.Ovm import Ovm from ovmcli.errorcode import * import time ovm = Ovm(username='admin', password='oracle') _ = ovm.runcmd serverpool_name = "ovstest19" vm_name = "hvmbyiso1" server_ip = "ovstest19.cn.oracle.com" # Delete a serverpool if OVM_OK == _("svrp del -s %s --force" % serverpool_name): print "OK serverpool_del" # Create a serverpool

117 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

cmd = 'svrp new -H %s --serverpool_name=%s -a -A oracle -U root -P oracle -L BJ ' % (server_ip, serverpool_name) if OVM_OK == _(cmd): print "OK serverpool_create" # Discover and Register ISOs discoverable_isos = _("iso disc -s %s" % serverpool_name) for iso in discoverable_isos: cmd = "iso reg -s %s -g el51 -l %s" % (serverpool_name, iso) if OVM_OK == _(cmd): print "OK iso reg %s" % iso isos = _("iso ls -s %s" % serverpool_name) iso = isos[0][1] # Create a VM cmd = 'vm new --method=iso -x -s %s -n %s -c 1 -i %s -o Other -y 512 -d 1024' % (serverpool_name, vm_name, iso) if OVM_OK == _(cmd): print "OK vm new"

while 1: cmd = "vm status -s %s -n %s" % (serverpool_name, vm_name) status = _(cmd) if type(status) == type([]): vm_status = status[0] if vm_status != 'Running': print "vm_status:" + vm_status print "sleep 10s" time.sleep(10) else: print vm_status cmd = 'vm poweroff -s %s -n %s' % (serverpool_name, vm_name) if OVM_OK == _(cmd): print "OK poweroff" break

Oracle VM 2.1.5 to 2.2 Upgrade
Last update 10-23-09 Copyright © 2009 - 2011 Roddy Rodstein. All rights reserved. Show a printer friendly version of this book with its sub pages In this chapter we will review the steps to upgrade all of the Oracle VM components from version 2.1.5 to version 2.2. The chapter starts with an introduction to the upgrade process for Oracle VM Manager and Oracle VM server. Next we review the upgrade options and upgrade prerequisites for both Oracle VM Manager and Oracle VM server. Next we will review the Unbreakable Linux Network to support the up2date upgrade option. The chapter concludes with a review of the steps to upgrade Oracle VM Manager and Oracle VM server from version 2.1.5 to version 2.2. Table of Contents Oracle VM 2.1.5 to 2.2 Upgrade Processes Oracle VM Manager Upgrade Process Oracle VM Server Upgrade Options The Unbreakable Linux Network Oracle VM Manager Database Backup Oracle VM Manager Database Restore Backup the OVS Repository Download the Oracle VM Manager Media Unzip and Mount the Oracle VM 2.2 ISO Upgrade Oracle VM Manager Oracle VM Server Upgrade using up2date ...Oracle VM Server Upgrade Prerequisites ...Oracle VM Server Upgrade Commands ...Reboot the Oracle VM Pool Master Server ...Reboot the None Master Oracle VM servers ...Enter the Server Pool Virtual IP using Oracle VM Manager Oracle VM 2.1.5 to 2.2 Upgrade Processes The upgrade process for Oracle VM consists of first upgrading Oracle VM Manager followed by the Oracle VM servers. Oracle VM

118 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Manager is upgraded by using the Oracle VM Manager 2.2 ISO file which is available from the eDelivery portal. Oracle VM servers can be upgraded using either a) the bootable Oracle VM Server 2.2 ISO image or b) with up2date and the Unbreakable Linux Network or c) using up2date or yum with a local yum repository. Note: Please note that a rolling upgrade of the Oracle VM servers from version 2.1 to version 2.2 is not supported. You will need to backup and then power off all your virtual machines before you upgrade the Oracle VM servers. A rolling upgrade is not supported, due in part, to an upgrade of the OCFS2 file system. List 1 shows the correct order to backup and upgrade all of the Oracle VM components from version 2.1.5 to version 2.2. 1. 2. 3. 4. 5. 6. 7. 8. Back up the Oracle VM Manager database Backup the OVS repository Upgrade Oracle VM Manager from version 2.1.5 to version 2.2 using the Oracle VM 2.2.0 ISO Upgrade the none master Oracle VM servers, i.e. the virtual machine and utility server pool members Upgrade the Oracle VM server master agent server Reboot the master agent server Reboot the none master Oracle VM servers If your server pools are HA enabled, access Oracle VM Manager and enter the server pool virtual VIP

Oracle VM Manager Upgrade Process Oracle VM Manager is updated by downloading the media from Oracle’s eDelivery portal and executing an installation script. Oracle VM media is freely downloadable from Oracle’s eDelivery portal. The Oracle VM Manager update process will require you to select an installation option of either install, uninstall, or upgrade, as well as entering the passwords for the existing Oracle VM Manager OVS database account and the oc4j admin account. The Oracle VM Manager OVS database account and the oc4j admin account passwords are selected during the Oracle VM Manager installation process. The account passwords for the Oracle VM Manager OVS database account and the oc4jadmin account can be managed from their respected web portals. The OVS account can be maintained from the Oracle Database Express Edition portal which is a part of the Oracle VM Manager install. The Oracle Database Express Edition portal can be accessed from the Oracle VM manager server locally by entering http://127.0.0.1:8080/apex or remotely by entering the ip address or the FQDN followed by :8080/apex, i.e. http://OracleVM_Manager_fqdn:8080/apex in a web browser. Figure 1 shows the Oracle Database Express Edition portal login page.

The Application Server Control portal is also part of the Oracle VM Manager install. The oc4j admin account can be maintained from the Application Server Control portal. The Application Server Control portal can only be accessed locally from the Oracle VM server console by entering http://127.0.0.1:8888/em in a local web browser. Figure 2 shows the Application Server Control login page.

Oracle VM Server Upgrade Options There are two options to ugrade Oracle VM servers. The first option is to use the up2date command from the Oracle VM server’s dom0 console. Using the up2date command requires Unbreakable Linux Network access. The second option is to upgrade the system to the latest version, using a bootable Oracle VM server ISO image. Oracle VM server ISO is freely downloadable from Oracle’s eDelivery portal. Figure 3 shows the Oracle VM server upgrade options.

119 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

The difference between the two Oracle VM server upgrade options is a) the ability to perform incremental updates or b) platform upgrades. Using the up2date command with the Unbreakable Linux Network allows Oracle VM servers to receive incremental updates, patches, security fixes as well as upgrades. Using the bootable media requires the machine to be restarted and to select the update option from the boot prompt. Selecting the update option from the boot prompt will update the Oracle VM server to the version of the bootable media, e.g. from 2.1.5 to 2.2. Figure 4 shows the System to Upgrade boot prompt. From the System to Upgrade boot prompt administrators can select to Reinstall System or Oracle VM server 2.x to upgrade the system.

When the up2date command is executed from an Oracle VM server’s dom0 console, up2date connects to the ULN repository and downloads the requested packages in RPM format. The up2date command communicates over the internet to “linuxupdate.oracle.com” on port 443. Up2date then installs the packages on the registered Oracle VM server. Please consult up2date’s man page by executing “man up2date” from the Oracle VM server’s dom0 console as root for a comprehensive list of command augments. The Unbreakable Linux Network The Oracle Unbreakable Linux support program allows customers to open Oracle VM and Linux Service Requests (SRs) 24/7 as well as access to the Unbreakable Linux Network. Existing Oracle support customers use the same support infrastructure i.e. support phone number and web portal to log and track Oracle VM Service Requests (SR) as with Tech and App Service Requests. The Unbreakable Linux Network is an online resource for Oracle Unbreakable Linux subscribers that offers access to Linux and Oracle VM software patches, updates and fixes. The Unbreakable Linux Network can be accessed from registered Linux and Oracle VM hosts with the up2date program as well as with a web browser, by pointing the browser to http://linux.oracle.com. The Unbreakable Linux Network and My Oracle Support (formally Metalink) are two separate systems, accessed by different URLs and user name and passwords. ULN is used to access Oracle VM and Linux patches, updates, and fixes, and My Oracle Support is used to manage SRs. Figure 5 shows the ULN portal.

120 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

This section will review the steps to upgrade Oracle VM Manager. We start with a review of the Oracle VM Manager database backup and restore procedure followed by the 2.1.5 to 2.2 Oracle VM Manager upgrade. Oracle VM Manager Database Backup Our first step to upgrade Orace VM Manager is to backup our Oracle VM Manager database using the backup.sh script. Before you backup the Oracle VM Manager database, access Oracle VM Manager and ensure that the status of all of the Oracle VM servers is "Running". You will need the following details for the backup procedure a) the password for database account 'OVS', which was assigned during the Oracle VM Manager install, b) the path for dump file, i.e. /tmp/ovs.dmp and c) the path for log file, i.e. /tmp/ovs.log. We will conclude Step 1 with a complete Oracle VM Manager database restore. The following example shows how to execute the “/opt/ovs-manager-2.1/bin/backup.sh” script from the Oracle VM Manager server console as root user. # sh /opt/ovs-manager-2.1/bin/backup.sh Next to backup the Oracle VM Manager database select the Backup Oracle VM Manager option (number 1) from the backup prompt. The next example shows the backup selection prompt. sh /opt/ovs-manager-2.1/bin/backup.sh Welcome to Oracle VM Manager Completed backup: Please enter the choice: [1|2] 1. Back up Oracle VM Manager 2. Restore Oracle VM Manager 1 The next example reviews a complete Oracle VM Manager database backup. Access the Oracle VM Manager server console as root and type "sh /opt/ovs-manager-2.1/bin/backup.sh" to start the Oracle VM Manager database backup as shown in the following example. # sh /opt/ovs-manager-2.1/bin/backup.sh Welcome to Oracle VM Manager Completed backup: Please enter the choice: [1|2] 1. Back up Oracle VM Manager 2. Restore Oracle VM Manager 1 Back up data now ... Please enter the password for database account 'OVS': Please enter the path and filename for the dump file (eg /tmp/ovs.dmp):/tmp/ovs.dmp Please enter the path and filename for the log file (eg /tmp/ovs.log):/tmp/ovs.log SQL*Plus: Release 10.2.0.1.0 - Production on Sat Oct 17 10:00:51 2009 Copyright (c) 1982, 2005, Oracle. All rights reserved.

Connected to: Oracle Database 10g Express Edition Release 10.2.0.1.0 - Production SQL>

121 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

PL/SQL procedure successfully completed. SQL> Disconnected from Oracle Database 10g Express Edition Release 10.2.0.1.0 - Production Export: Release 10.2.0.1.0 - Production on Sat Oct 17 10:00:52 2009 Copyright (c) 1982, 2005, Oracle. All rights reserved.

Connected to: Oracle Database 10g Express Edition Release 10.2.0.1.0 - Production Export done in US7ASCII character set and AL16UTF16 NCHAR character set server uses AL32UTF8 character set (possible charset conversion) . exporting pre-schema procedural objects and actions . exporting foreign function library names for user OVS . exporting PUBLIC type synonyms . exporting private type synonyms . exporting object type definitions for user OVS About to export OVS's objects ... . exporting database links . exporting sequence numbers . exporting cluster definitions . about to export OVS's tables via Conventional Path ... . . exporting table OVS_AGENT 2 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_ALERT 11 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_CATEGORY 1 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_CDROM 5 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_CDROM_RESOURCE 0 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_GROUP 4 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_HD_TEMP . . exporting table OVS_IMG_OS 0 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_IMG_TEMP . . exporting table OVS_LOCK 19 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_MAP 35 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_NIC_TEMP . . exporting table OVS_OS_RESOURCE 17 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_PARTNER 0 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_PREFERRED_SERVER 0 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_PRIVILEGE 4 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_PRIVILEGE_ROLE 5 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_RESOURCE 4 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_ROLE 3 rows exported

122 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_SERVER 2 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_SITE 1 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_STATISTIC 16593 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_SYS_VALUE 8 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_TASK 0 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_USER 19 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_USER_GROUP 75 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_USER_ROLE 19 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_USER_SITE 19 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_VD_IMG 39 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_VIRTUAL_DISK 73 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_VM_GEN_INFO 25 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_VM_IMG 25 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_VM_NETWORK 25 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_VM_SNAPSHOT 0 rows exported EXP-00091: Exporting questionable statistics. EXP-00091: Exporting questionable statistics. . . exporting table OVS_WORKSPACE_TEMP . exporting synonyms . exporting views . exporting stored procedures . exporting operators . exporting referential integrity constraints . exporting triggers . exporting indextypes . exporting bitmap, functional and extensible indexes . exporting posttables actions . exporting materialized views . exporting snapshot logs . exporting job queues . exporting refresh groups and children . exporting dimensions . exporting post-schema procedural objects and actions . exporting statistics Export terminated successfully with warnings. SQL*Plus: Release 10.2.0.1.0 - Production on Sat Oct 17 10:01:05 2009 Copyright (c) 1982, 2005, Oracle. All rights reserved.

123 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Connected to: Oracle Database 10g Express Edition Release 10.2.0.1.0 - Production SQL> PL/SQL procedure successfully completed. SQL> Disconnected from Oracle Database 10g Express Edition Release 10.2.0.1.0 - Production Done Please check the log file /tmp/ovs.log # We just executed a successful Oracle VM Manager database backup. Oracle VM Manager Database Restore In the event that you need to restore the Oracle VM Manager database walk through the same procedure as in the above example and select 2 to restore Oracle VM Manager database. You will need the following details for the restore procedure a) the password for database account 'OVS', which was assigned during the Oracle VM Manager install, b) the path for dump file, i.e. /tmp/ovs.dmp and c) the path for log file, i.e. /tmp/ovs.log. The following example shows how to execute the “/opt/ovs-manager-2.1/bin/backup.sh” script from the Oracle VM Manager server console as root user. # sh /opt/ovs-manager-2.1/bin/backup.sh Next, to backup the Oracle VM Manager database, select the Restore Oracle VM Manager option (number 2) from the backup prompt. The next example shows the backup selection prompt. sh /opt/ovs-manager-2.1/bin/backup.sh Welcome to Oracle VM Manager Completed backup: Please enter the choice: [1|2] 1. Back up Oracle VM Manager 2. Restore Oracle VM Manager 2 The next example reviews a complete Oracle VM Manager database restore. Access the Oracle VM Manager server console as root and type "sh /opt/ovs-manager-2.1/bin/backup.sh" to start the Oracle VM Manager database restore as shown in the following example. The next example reviews an Oracle VM Manager database restore procedure. Access the Oracle VM Manager server console as root and type "sh /opt/ovs-manager-2.1/bin/backup.sh" as shown in the following example. # sh /opt/ovs-manager-2.1/bin/backup.sh Welcome to Oracle VM Manager Please enter the choice: [1|2] 1. Back up Oracle VM Manager 2. Restore Oracle VM Manager 2 Restore data now ... Please enter the password for database account 'SYS': Please enter the path and filename for the dump file (eg /tmp/ovs.dmp):/tmp/ovs.dmp Please enter the path and filename for the log file (eg /tmp/ovs.log):/tmp/ovs.log Import: Release 10.2.0.1.0 - Production on Sun Oct 18 11:15:24 2009 Copyright (c) 1982, 2005, Oracle. All rights reserved.

Connected to: Oracle Database 10g Express Edition Release 10.2.0.1.0 - Production Export file created by EXPORT:V10.02.01 via conventional path Warning: the objects were exported by OVS, not by you import done in US7ASCII character set and AL16UTF16 NCHAR character set import server uses AL32UTF8 character set (possible charset conversion) . . importing table "OVS_AGENT" 2 rows imported . . importing table "OVS_ALERT" 11 rows imported . . importing table "OVS_CATEGORY" 1 rows imported . . importing table "OVS_CDROM" 5 rows imported . . importing table "OVS_CDROM_RESOURCE" 0 rows imported

124 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

. . importing table "OVS_GROUP" 4 rows imported . . importing table "OVS_IMG_OS" 0 rows imported . . importing table "OVS_LOCK" 19 rows imported . . importing table "OVS_MAP" 35 rows imported . . importing table "OVS_OS_RESOURCE" 17 rows imported . . importing table "OVS_PARTNER" 0 rows imported . . importing table "OVS_PREFERRED_SERVER" 0 rows imported . . importing table "OVS_PRIVILEGE" 4 rows imported . . importing table "OVS_PRIVILEGE_ROLE" 5 rows imported . . importing table "OVS_RESOURCE" 4 rows imported . . importing table "OVS_ROLE" 3 rows imported . . importing table "OVS_SERVER" 2 rows imported . . importing table "OVS_SITE" 1 rows imported . . importing table "OVS_STATISTIC" 16593 rows imported . . importing table "OVS_SYS_VALUE" 8 rows imported . . importing table "OVS_TASK" 0 rows imported . . importing table "OVS_USER" 19 rows imported . . importing table "OVS_USER_GROUP" 75 rows imported . . importing table "OVS_USER_ROLE" 19 rows imported . . importing table "OVS_USER_SITE" 19 rows imported . . importing table "OVS_VD_IMG" 39 rows imported . . importing table "OVS_VIRTUAL_DISK" 73 rows imported . . importing table "OVS_VM_GEN_INFO" 25 rows imported . . importing table "OVS_VM_IMG" 25 rows imported . . importing table "OVS_VM_NETWORK" 25 rows imported . . importing table "OVS_VM_SNAPSHOT" 0 rows imported Import terminated successfully without warnings. Done Please check the log file /tmp/ovs.log # We have just completed an Oracle VM Manager backup and restore. Backup the OVS Repository Next we will back up the OVS repository using your existing storage backup tools. Download the Oracle VM Manager Media Point you browser to http://edelivery.oracle.com/linux to download the Oracle VM Manager ISO files as shown in Figure 6. Click the Continue button to proceed.

From the Registration page enter the following required information then click the Continue button to proceed: Full Name (FIRST LAST) Company Name Email address Select your country from the drop down list Select the check box to agree to the Agreement Terms Select the check box to accept the Export Restrictions Figure 7 shows the Registration Page.

125 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

From the Media Pack Search page select Oracle VM from the Select a Product Pack drop down box, then select the desired platform either x86 or x86-64 from the Platform drop down box. Click the Go button to proceed. Figure 8 shows the Media Pack Search page.

From the Media Pack Search results page select the Oracle VM 2.2 Media Pack radio button. Click the Oracle VM 2.2 Media Pack link to proceed. Figure 9 shows the Media Pack Search results page.

126 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

From the Oracle VM 2.2 Media Pack page click the Oracle VM Manager 2.2 Download button to download the Oracle VM Manager 2.2 media. Figure 10 shows the Oracle VM 2.2 Media Pack page.

We have just completed the steps to download the Oracle VM 2.2 Media Pack from the Oracle eDelivery portal. Unzip and Mount the Oracle VM 2.2 ISO Now that you have the Oracle VM Manager 2.2 media, copy it over to your Oracle VM Manager server. From the Oracle VM Manager server console as root, unzip the V18419-01.zip archive as shown in the next example. #unzip V18419-01.zip Archive: V18419-01.zip inflating: OracleVM-Manager-2.2.0.iso, /*.sqljsp, /*.SQLJSP) # The following example shows how to mount the Oracle VM Manager ISO image from the Oracle VM Manager server console as root user. #mount -o ro,loop OracleVM-Manager-2.1.5.iso /mnt/

Upgrade Oracle VM Manager The following section will start with a review of each of the steps of a successful Oracle VM Manager upgrade. The section will conclude with a complete successful Oracle VM Manager upgrade session. Before you proceed with the Oracle VM Manager 2.2 upgrade you will need the OVS password and the oc4jadmin password, which

127 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

where both selected durring the Oracle VM Manager install. The update process requires you to select an upgrade option from the install prompt, as well as enter the passwords for the Oracle VM Manager OVS database account, and the Oracle VM Manager oc4jadmin account. Access the Oracle VM server console as root and cd in to the directory where you mounted the ISO file. The following example shows how to execute the runInstaller.sh from the Oracle VM Manager server console as root user. # sh runInstaller.sh Select the Upgrade Oracle VM Manager option (number 3) from the install prompt. The next example shows the installation selection prompt. Welcome to Oracle VM Manager 2.2 Please enter the choice: [1|2|3] 1. Install Oracle VM Manager 2. Uninstall Oracle VM Manager 3. Upgrade Oracle VM Manager 3 Starting Oracle VM Manager 2.2 upgrade ... Next, you are asked if you would like to backup the Oracle VM Manager database. If you select “N” from the install prompt, the upgrade will be aborted. The following example shows the Oracle VM Manager database backup prompt. Backup the database before upgrade is highly recommended, to backup the database now, choose 'N' and run: /opt/ovs-manager-2.1/bin/backup.sh Next you are asked if you would like to proceed or cancel the upgrade. You can enter y to proceed or N to cancel the upgrade. The following example shows the upgrade acceptance prompt. Are you sure you want to upgrade Oracle VM Manager from version 2.1.5 to 2.2 ? [y|N] Next you must enter the OVS password. The following example shows the OVS installation password prompt. Please enter the password for database account 'OVS': Next enter the oc4jadmin password. The following example shows the oc4jadmin installation password prompt. Please enter the password for account 'oc4jadmin': A successful upgrade will exit with the following informational warnings. Now upgrading the database schema ... Done. Now upgrading the Oracle VM Manager application ... Done. Upgrade Oracle VM Manager successfully. Please check the log /var/log/ovm-manager/upgrade_2.1.5_2.2.log for more information. To access the Oracle VM Manager 2.1.2 home page go to: http://ovm03.sf.seo.com:8888/OVS To access the Oracle VM Manager help page go to: http://ovm03.sf.seo.com:8888/help/help The following example show a complete Oracle VM Manager 2.2 upgrade session. # sh runInstaller.sh Welcome to Oracle VM Manager 2.2 Please enter the choice: [1|2|3] 1. Install Oracle VM Manager 2. Uninstall Oracle VM Manager 3. Upgrade Oracle VM Manager 3 Starting Oracle VM Manager 2.2 upgrade ... Backup the database before upgrade is highly recommended, to backup the database now, choose 'N' and run: /opt/ovs-manager-2.1/bin/backup.sh Are you sure you want to upgrade Oracle VM Manager from version 2.1.5 to 2.2 ? [y|N] y Please enter the password for database account 'OVS': Now upgrading OC4J ... Please enter the password for account 'oc4jadmin': Starting OC4J ... Done. Now upgrading the database schema ... Done. Now upgrading the Oracle VM Manager application ... Done.

128 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Upgrade Oracle VM Manager successfully. Please check the log /var/log/ovm-manager/upgrade_2.1.5_2.2.log for more information. To access the Oracle VM Manager 2.1.2 home page go to: http://ovm03.sf.seo.com:8888/OVS To access the Oracle VM Manager help page go to: http://ovm03.sf.seo.com:8888/help/help # Oracle VM Server Upgrade using up2date In this section we will review how to update an Oracle VM Server using up2date with the Unbreakable Linux Network. We will conclude with a review on how to upgrade Oracle VM servers using up2date with the Unbreakable Linux Network. It is essential to upgrade and boot the Oracle VM servers in the following order; first upgrade the Oracle VM virtual machine and utlilty servers, followed by the Oracle VM pool master server. After all of the servers are servers upgraded, boot the Oracle VM pool master server followed by the none pool master servers. Note: You can determine the Oracle VM server roles i.e. virtual machine, utlilty and master servers by accessing Oracle VM Manager and displaying the Servers tab as shown in Figure 11.

Oracle VM Server Upgrade Prerequisites Before we upgrade any of the Oracle VM servers ensure that all of the virtual machines are off and verify the /etc/hosts and /etc/resolv.conf file setting as explained in List 1. List 1 1- Verify that the hostname in /etc/hosts is associated with the public IP address, not 127.0.0.1. 2- Verify that all of the Oracle VM servers have the proper entries in /etc/resolv.conf or if DNS is not used, make sure the correct setting are in /etc/hosts. Please note that all the servers in the same pool must have the consistent name resolution, either by DNS or by file (/etc/hosts). The upgrade procees for all of the Oracle VM 2.1.5 pool memeber servers i.e. the virtual machine servers, the utlilty servers and the pool master pool server is idential. We will first upgrade all of the none pool master servers, followed by the pool master pool server, using the following commands. Note: Before we upgrade any of the Oracle VM servers, ensure that all of the virtaul machines are powered off and backed up. Oracle VM Server Upgrade Commands First, from dom0 as root install the ovm22upgrade package as shown in the example. # up2date ovm22upgrade Once the ovm22upgrade package successfully completes, run the ovm22upgrade.py script as shown in the next example. # /usr/local/sbin/ovm22upgrade.py To complete the Oracle VM server upgrade run the following command, as shown in the next example. # up2date -fu Once you have upgraded all of the none pool master servers, upgrade the pool master pool server. Next we need to boot the pool master pool server followed by the none pool master servers. The next example shows each of the three steps from a sucessful Oracle VM server upgrade. First install ovm22upgrade package as shown in the example. # up2date ovm22upgrade Fetching Obsoletes list for channel: ovm2_i386_latest... ######################################## Fetching rpm headers... ########################################

129 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Name Version ---------------------------------------------------------ovm22upgrade 1.0

Rel 1.0.19.el5 noarch

Testing package set / solving RPM inter-dependencies... ######################################## ovm22upgrade-1.0-1.0.19.el5 ########################## Done. Preparing ########################################### [100%] Installing... 1:ovm22upgrade ########################################### [100%] Please run the following command to continue the upgrade: /usr/local/sbin/ovm22upgrade.py [root@ovs3 ~]# /usr/local/sbin/ovm22upgrade.py Please stop all running Virtual Machine Guests before proceeding. Next run python script as suggested by the install process as shown in the next example. # /usr/local/sbin/ovm22upgrade.py Running phase 1 of upgrade... done Installing new ovs-release package for OVM 2.2 Fetching Obsoletes list for channel: ovm22_i386_bootstrap... ######################################## Fetching rpm headers... ######################################## Name Version Rel ---------------------------------------------------------ovs-release 2.2 0.15

i386

Testing package set / solving RPM inter-dependencies... ######################################## ovs-release-2.2-0.15.i386.r ########################## Done. Preparing ########################################### [100%] Installing... 1:ovs-release warning: /etc/issue created as /etc/issue.rpmnew ########################################### [100%] Running phase 2 of upgrade... done Updating fstab and grub.conf with disk uuid information Files /etc/fstab and /boot/grub/grub.conf updated to use UUID instead of LABEL. Original files are saved as /etc/fstab.ovsbak_100918 /boot/grub/grub.conf.ovsbak_100918 Please run the 'up2date -fu' command to continue the upgrade. # We will concluse the upgrade process by running the following command, as show in the next example. # up2date -fu Fetching Obsoletes list for channel: ovm22_i386_latest... ######################################## Name Version Rel ---------------------------------------------------------OpenIPMI 2.0.6 11.el5 i386 OpenIPMI-libs 2.0.6 11.el5 i386 SysVinit 2.86 15.el5 i386 acl 2.2.39 3.el5 i386 alsa-lib 1.0.17 1.el5 i386 anaconda 11.1.2.168 1.1.16 i386 anaconda-runtime 11.1.2.168 1.1.16 i386 anacron 2.3 45.0.1.el5 i386 audit-libs 1.7.7 6.el5 i386 authconfig 5.3.21 5.el5 i386 autofs 5.0.1 0.rc2.102.el5_3.1 i386 bash 3.2 24.el5 i386

130 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

bind-libs 9.3.4 10.P1.el5_3.3 i386 bind-utils 9.3.4 10.P1.el5_3.3 i386 binutils 2.17.50.0.6 9.el5 i386 booty 0.80.6 5.1.5 noarch bzip2 1.0.3 4.el5_2 i386 bzip2-libs 1.0.3 4.el5_2 i386 chkconfig 1.3.30.1 2 i386 compat-python-virtinst 0.99.0 3.el5.2.12 noarch coreutils 5.97 19.el5 i386 cracklib 2.8.9 3.3 i386 cracklib-dicts 2.8.9 3.3 i386 crash 4.0 7.2.3.0.1 i386 createrepo 0.4.11 3.el5 noarch cryptsetup-luks 1.0.3 4.el5 i386 curl 7.15.5 2.1.el5_3.5 i386 dbus 1.1.2 12.el5 i386 dbus-glib 0.73 8.el5 i386 device-mapper 1.02.28 2.el5 i386 device-mapper-multipath 0.4.7 23.el5.4.0.1 i386 dhclient 3.0.5 18.el5 i386 dhcpv6-client 1.0.10 16.el5 i386 diffutils 2.8.1 15.2.3.el5 i386 dmraid 1.0.0.rc13 33.el5 i386 dnsmasq 2.45 1.1.el5_3 i386 dosfstools 2.11 7.el5 i386 e2fsprogs 1.39 20.el5 i386 e2fsprogs-libs 1.39 20.el5 i386 ed 0.2 39.el5_2 i386 elfutils-libelf 0.137 3.el5 i386 elinks 0.11.1 6.el5_4.1 i386 enterprise-linux-ovs 5 0.17 i386 ethtool 6 2.el5 i386 file 4.17 15 i386 filesystem 2.4.0 2.0.1 i386 findutils 4.2.27 5.el5 i386 firstboot 1.4.27.7 1.1.1.el5 i386 firstboot-tui 1.4.27.7 1.1.1.el5 i386 fontconfig 2.4.1 7.el5 i386 ftp 0.17 35.el5 i386 glibc 2.5 34 i686 glibc-common 2.5 34 i386 gnupg 1.4.5 14 i386 grub 0.97 13.2.1.1 i386 gzip 1.3.5 10.el5.0.1 i386 hal 0.5.8.1 38.el5 i386 hwdata 0.213.11 1.1.3.el5 noarch initscripts 8.45.25 1.1.1.el5 i386 iproute 2.6.18 9.el5 i386 iptables 1.3.5 4.el5 i386 iptables-ipv6 1.3.5 4.el5 i386 iputils 20020927 45.el5 i386 iscsi-initiator-utils 6.2.0.871 0.7.el5 i386 kbd 1.12 21.el5 i386 kernel-ovs 2.6.18 128.2.1.4.9.el5 i686 kpartx 0.4.7 23.el5.4.0.1 i386 ksh 20080202 2.el5 i386 kudzu 1.2.57.1.21 1 i386 libX11 1.0.3 9.el5 i386 libacl 2.2.39 3.el5 i386 libbdevid-python 5.1.19.6 44.0.1 i386 libgcc 4.1.2 44.el5 i386 libgcrypt 1.2.4 1.el5 i386 libpcap 0.9.4 14.el5 i386 libselinux 1.33.4 5.1.el5 i386 libstdc++ 4.1.2 44.el5 i386 libtiff 3.8.2 7.el5_3.4 i386 libuser 0.54.7 2.el5.5 i386 libvirt 0.3.3 14.0.2.el5_3.1 i386 libvirt-python 0.3.3 14.0.2.el5_3.1 i386 libxml2 2.6.26 2.1.2.8.0.2 i386 libxml2-python 2.6.26 2.1.2.8.0.2 i386 lm_sensors 2.10.7 4.el5 i386 logrotate 3.7.4 9 i386

131 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

logwatch 7.3 6.el5 noarch lvm2 2.02.40 6.el5 i386 m2crypto 0.16 6.el5.3 i386 make 3.81 3.el5 i386 man-pages 2.39 12.el5 noarch mcstrans 0.2.11 3.el5 i386 mdadm 2.6.4 1.el5 i386 mkinitrd 5.1.19.6 44.0.1 i386 mkisofs 2.01 10.7.el5 i386 module-init-tools 3.3 0.pre3.1.42.el5 i386 nash 5.1.19.6 44.0.1 i386 neon 0.25.5 10.el5_4.1 i386 net-snmp 5.3.2.2 5.el5_3.2 i386 net-snmp-libs 5.3.2.2 5.el5_3.2 i386 net-tools 1.60 78.el5 i386 newt 0.52.2 12.el5_4.1 i386 nfs-utils 1.0.9 40.el5 i386 nscd 2.5 34 i386 nss_db 2.2 35.3 i386 nss_ldap 253 17.el5 i386 ocfs2-tools 1.4.3 4.el5 i386 openldap 2.3.43 3.el5 i386 openssh 4.3p2 29.el5 i386 openssh-clients 4.3p2 29.el5 i386 openssh-server 4.3p2 29.el5 i386 openssl 0.9.8e 7.el5 i686 oprofile 0.9.3 18.el5 i386 ovs-agent 2.3 19 noarch ovs-release 2.2 0.17 i386 ovs-utils 1.0 33 i386 p2v-util 0.10 6 i386 pam 0.99.6.2 4.el5 i386 pam_krb5 2.2.14 10 i386 parted 1.8.1 23.el5 i386 patch 2.5.4 29.2.3.el5 i386 pciutils 2.2.3 5 i386 pcre 6.6 2.el5_1.7 i386 perl 5.8.8 18.el5 i386 pexpect 2.3 1.el5 noarch pm-utils 0.99.3 10.el5 i386 popt 1.10.2.3 9.el5 i386 ppp 2.4.4 2.el5 i386 prelink 0.4.0 2.el5 i386 procmail 3.22 17.1.0.1 i386 procps 3.2.7 11.1.el5 i386 psacct 6.3.2 44.el5 i386 psmisc 22.2 6 i386 pykickstart 0.43.3 1.1.2.el5 noarch pypxeboot 0.0.2 9 noarch python 2.4.3 24.el5_3.6 i386 python-pyblock 0.26 3.el5 i386 python-urlgrabber 3.1.0 5.el5 noarch quota 3.13 1.2.5.el5 i386 rdate 1.4 8.el5 i386 rhnlib 2.2.6 2.el5 noarch rpm 4.4.2.3 9.el5 i386 rpm-libs 4.4.2.3 9.el5 i386 rpm-python 4.4.2.3 9.el5 i386 rsh 0.17 38.el5 i386 setup 2.5.58 4.1.1.el5 noarch shadow-utils 4.0.17 14.el5 i386 stunnel 4.15 2.0.1.el5.1 i386 sudo 1.6.9p17 3.el5 i386 sysklogd 1.4.1 44.el5 i386 system-config-date 1.8.12 3.el5.2.1 noarch system-config-keyboard 1.2.11 1.1.1.el5 noarch system-config-language 1.1.18 2.1.1.el5 noarch system-config-network-tui 1.3.99.12 1.0.1.el5 noarch system-config-rootpassword 1.1.9.1 1.1.1 noarch system-config-securitylevel 1.6.29.1 2.1.1.1.el5 i386 system-config-securitylevel-tui 1.6.29.1 2.1.1.1.el5 i386 tcp_wrappers 7.6 40.6.el5 i386 tcpdump 3.9.4 14.el5 i386

132 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

tcsh tftp tmpwatch traceroute tzdata usermode util-linux vim-minimal vixie-cron vnc words xen xen-64 xen-debugger xen-devel xen-tools ypbind yum yum-metadata-parser zip

6.14 14.el5 i386 0.42 3.1.0.1 i386 2.9.7 1.1.el5.2 i386 2.0.1 5.el5 i386 2009k 1.el5 noarch 1.88 3.el5.2 i386 2.13 0.50.el5 i386 7.0.109 4.el5_2.4z i386 4.1 76.el5 i386 4.1.2 14.el5.1.0.1 i386 3.0 9.1 noarch 3.4.0 0.0.23.el5 i386 3.4.0 0.0.23.el5 noarch 3.4.0 0.0.23.el5 noarch 3.4.0 0.0.23.el5 i386 3.4.0 0.0.23.el5 i386 1.19 11.el5 i386 3.2.19 18.el5 noarch 1.1.2 2.el5 i386 2.31 2.el5 i386

Testing package set / solving RPM inter-dependencies... ######################################## OpenIPMI-2.0.6-11.el5.i386. ########################## Done. OpenIPMI-libs-2.0.6-11.el5. ########################## Done. SysVinit-2.86-15.el5.i386.r ########################## Done. acl-2.2.39-3.el5.i386.rpm: ########################## Done. alsa-lib-1.0.17-1.el5.i386. ########################## Done. anaconda-11.1.2.168-1.1.16. ########################## Done. anaconda-runtime-11.1.2.168 ########################## Done. anacron-2.3-45.0.1.el5.i386 ########################## Done. audit-libs-1.7.7-6.el5.i386 ########################## Done. authconfig-5.3.21-5.el5.i38 ########################## Done. autofs-5.0.1-0.rc2.102.el5_ ########################## Done. avahi-0.6.16-1.el5.i386.rpm ########################## Done. bash-3.2-24.el5.i386.rpm: ########################## Done. bind-libs-9.3.4-10.P1.el5_3 ########################## Done. bind-utils-9.3.4-10.P1.el5_ ########################## Done. binutils-2.17.50.0.6-9.el5. ########################## Done. booty-0.80.6-5.1.5.noarch.r ########################## Done. bzip2-1.0.3-4.el5_2.i386.rp ########################## Done. bzip2-libs-1.0.3-4.el5_2.i3 ########################## Done. chkconfig-1.3.30.1-2.i386.r ########################## Done. compat-python-virtinst-0.99 ########################## Done. coreutils-5.97-19.el5.i386. ########################## Done. cracklib-2.8.9-3.3.i386.rpm ########################## Done. cracklib-dicts-2.8.9-3.3.i3 ########################## Done. crash-4.0-7.2.3.0.1.i386.rp ########################## Done. createrepo-0.4.11-3.el5.noa ########################## Done. cryptsetup-luks-1.0.3-4.el5 ########################## Done. curl-7.15.5-2.1.el5_3.5.i38 ########################## Done. dbus-1.1.2-12.el5.i386.rpm: ########################## Done. dbus-glib-0.73-8.el5.i386.r ########################## Done. dbus-libs-1.1.2-12.el5.i386 ########################## Done. device-mapper-1.02.28-2.el5 ########################## Done. device-mapper-event-1.02.28 ########################## Done. device-mapper-multipath-0.4 ########################## Done. dhclient-3.0.5-18.el5.i386. ########################## Done. dhcpv6-client-1.0.10-16.el5 ########################## Done. diffutils-2.8.1-15.2.3.el5. ########################## Done. dmraid-1.0.0.rc13-33.el5.i3 ########################## Done. dnsmasq-2.45-1.1.el5_3.i386 ########################## Done. dosfstools-2.11-7.el5.i386. ########################## Done. e2fsprogs-1.39-20.el5.i386. ########################## Done. e2fsprogs-libs-1.39-20.el5. ########################## Done. e4fsprogs-1.41.1-2.el5.i386 ########################## Done. ebtables-2.0.8-2.el5.i386.r ########################## Done. ed-0.2-39.el5_2.i386.rpm: ########################## Done. elfutils-libelf-0.137-3.el5 ########################## Done. elinks-0.11.1-6.el5_4.1.i38 ########################## Done. enterprise-linux-ovs-5-0.17 ########################## Done. ethtool-6-2.el5.i386.rpm: ########################## Done.

133 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

file-4.17-15.i386.rpm: ########################## Done. filesystem-2.4.0-2.0.1.i386 ########################## Done. findutils-4.2.27-5.el5.i386 ########################## Done. fipscheck-1.0.3-1.el5.i386. ########################## Done. firstboot-1.4.27.7-1.1.1.el ########################## Done. firstboot-tui-1.4.27.7-1.1. ########################## Done. fontconfig-2.4.1-7.el5.i386 ########################## Done. ftp-0.17-35.el5.i386.rpm: ########################## Done. glibc-2.5-34.i686.rpm: ########################## Done. glibc-common-2.5-34.i386.rp ########################## Done. gnupg-1.4.5-14.i386.rpm: ########################## Done. gnutls-1.4.1-3.el5_3.5.i386 ########################## Done. grub-0.97-13.2.1.1.i386.rpm ########################## Done. gzip-1.3.5-10.el5.0.1.i386. ########################## Done. hal-0.5.8.1-38.el5.i386.rpm ########################## Done. hwdata-0.213.11-1.1.3.el5.n ########################## Done. initscripts-8.45.25-1.1.1.e ########################## Done. iproute-2.6.18-9.el5.i386.r ########################## Done. iptables-1.3.5-4.el5.i386.r ########################## Done. iptables-ipv6-1.3.5-4.el5.i ########################## Done. iputils-20020927-45.el5.i38 ########################## Done. iscsi-initiator-utils-6.2.0 ########################## Done. kbd-1.12-21.el5.i386.rpm: ########################## Done. kernel-ovs-2.6.18-128.2.1.4 ########################## Done. kpartx-0.4.7-23.el5.4.0.1.i ########################## Done. ksh-20080202-2.el5.i386.rpm ########################## Done. kudzu-1.2.57.1.21-1.i386.rp ########################## Done. libX11-1.0.3-9.el5.i386.rpm ########################## Done. libacl-2.2.39-3.el5.i386.rp ########################## Done. libbdevid-python-5.1.19.6-4 ########################## Done. libgcc-4.1.2-44.el5.i386.rp ########################## Done. libgcrypt-1.2.4-1.el5.i386. ########################## Done. libpcap-0.9.4-14.el5.i386.r ########################## Done. libselinux-1.33.4-5.1.el5.i ########################## Done. libstdc++-4.1.2-44.el5.i386 ########################## Done. libtiff-3.8.2-7.el5_3.4.i38 ########################## Done. libuser-0.54.7-2.el5.5.i386 ########################## Done. libvirt-0.3.3-14.0.2.el5_3. ########################## Done. libvirt-python-0.3.3-14.0.2 ########################## Done. libxml2-2.6.26-2.1.2.8.0.2. ########################## Done. libxml2-python-2.6.26-2.1.2 ########################## Done. lm_sensors-2.10.7-4.el5.i38 ########################## Done. logrotate-3.7.4-9.i386.rpm: ########################## Done. logwatch-7.3-6.el5.noarch.r ########################## Done. lvm2-2.02.40-6.el5.i386.rpm ########################## Done. m2crypto-0.16-6.el5.3.i386. ########################## Done. make-3.81-3.el5.i386.rpm: ########################## Done. man-pages-2.39-12.el5.noarc ########################## Done. mcstrans-0.2.11-3.el5.i386. ########################## Done. mdadm-2.6.4-1.el5.i386.rpm: ########################## Done. mesa-libGL-6.5.1-7.7.el5.i3 ########################## Done. mkinitrd-5.1.19.6-44.0.1.i3 ########################## Done. mkisofs-2.01-10.7.el5.i386. ########################## Done. module-init-tools-3.3-0.pre ########################## Done. multiprocessing-2.6.1.1-1.i ########################## Done. nash-5.1.19.6-44.0.1.i386.r ########################## Done. neon-0.25.5-10.el5_4.1.i386 ########################## Done. net-snmp-5.3.2.2-5.el5_3.2. ########################## Done. net-snmp-libs-5.3.2.2-5.el5 ########################## Done. net-tools-1.60-78.el5.i386. ########################## Done. newt-0.52.2-12.el5_4.1.i386 ########################## Done. nfs-utils-1.0.9-40.el5.i386 ########################## Done. nscd-2.5-34.i386.rpm: ########################## Done. nspr-4.7.3-2.el5.i386.rpm: ########################## Done. nss-3.12.3.99.3-1.el5_3.2.i ########################## Done. nss_db-2.2-35.3.i386.rpm: ########################## Done. nss_ldap-253-17.el5.i386.rp ########################## Done. ocfs2-tools-1.4.3-4.el5.i38 ########################## Done. open-ovf-0.1-3.0.7.el5.noar ########################## Done. openldap-2.3.43-3.el5.i386. ########################## Done. openssh-4.3p2-29.el5.i386.r ########################## Done. openssh-clients-4.3p2-29.el ########################## Done. openssh-server-4.3p2-29.el5 ########################## Done.

134 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

openssl-0.9.8e-7.el5.i686.r ########################## Done. oprofile-0.9.3-18.el5.i386. ########################## Done. ovs-agent-2.3-19.noarch.rpm ########################## Done. ovs-release-2.2-0.17.i386.r ########################## Done. ovs-utils-1.0-33.i386.rpm: ########################## Done. p2v-util-0.10-6.i386.rpm: ########################## Done. pam-0.99.6.2-4.el5.i386.rpm ########################## Done. pam_krb5-2.2.14-10.i386.rpm ########################## Done. parted-1.8.1-23.el5.i386.rp ########################## Done. patch-2.5.4-29.2.3.el5.i386 ########################## Done. pciutils-2.2.3-5.i386.rpm: ########################## Done. pcre-6.6-2.el5_1.7.i386.rpm ########################## Done. perl-5.8.8-18.el5.i386.rpm: ########################## Done. pexpect-2.3-1.el5.noarch.rp ########################## Done. pm-utils-0.99.3-10.el5.i386 ########################## Done. popt-1.10.2.3-9.el5.i386.rp ########################## Done. ppp-2.4.4-2.el5.i386.rpm: ########################## Done. prelink-0.4.0-2.el5.i386.rp ########################## Done. procmail-3.22-17.1.0.1.i386 ########################## Done. procps-3.2.7-11.1.el5.i386. ########################## Done. psacct-6.3.2-44.el5.i386.rp ########################## Done. psmisc-22.2-6.i386.rpm: ########################## Done. pykickstart-0.43.3-1.1.2.el ########################## Done. pypxeboot-0.0.2-9.noarch.rp ########################## Done. python-2.4.3-24.el5_3.6.i38 ########################## Done. python-iniparse-0.2.3-4.el5 ########################## Done. python-pyblock-0.26-3.el5.i ########################## Done. python-simplejson-2.0.7-2.e ########################## Done. python-urlgrabber-3.1.0-5.e ########################## Done. quota-3.13-1.2.5.el5.i386.r ########################## Done. rdate-1.4-8.el5.i386.rpm: ########################## Done. redhat-lsb-3.1-12.3.0.1.EL. ########################## Done. rhnlib-2.2.6-2.el5.noarch.r ########################## Done. rpm-4.4.2.3-9.el5.i386.rpm: ########################## Done. rpm-libs-4.4.2.3-9.el5.i386 ########################## Done. rpm-python-4.4.2.3-9.el5.i3 ########################## Done. rsh-0.17-38.el5.i386.rpm: ########################## Done. setup-2.5.58-4.1.1.el5.noar ########################## Done. shadow-utils-4.0.17-14.el5. ########################## Done. stunnel-4.15-2.0.1.el5.1.i3 ########################## Done. sudo-1.6.9p17-3.el5.i386.rp ########################## Done. sysklogd-1.4.1-44.el5.i386. ########################## Done. system-config-date-1.8.12-3 ########################## Done. system-config-keyboard-1.2. ########################## Done. system-config-language-1.1. ########################## Done. system-config-network-tui-1 ########################## Done. system-config-rootpassword- ########################## Done. system-config-securitylevel ########################## Done. system-config-securitylevel ########################## Done. tcp_wrappers-7.6-40.6.el5.i ########################## Done. tcpdump-3.9.4-14.el5.i386.r ########################## Done. tcsh-6.14-14.el5.i386.rpm: ########################## Done. tftp-0.42-3.1.0.1.i386.rpm: ########################## Done. tmpwatch-2.9.7-1.1.el5.2.i3 ########################## Done. traceroute-2.0.1-5.el5.i386 ########################## Done. tzdata-2009k-1.el5.noarch.r ########################## Done. usermode-1.88-3.el5.2.i386. ########################## Done. util-linux-2.13-0.50.el5.i3 ########################## Done. vim-minimal-7.0.109-4.el5_2 ########################## Done. vixie-cron-4.1-76.el5.i386. ########################## Done. vnc-4.1.2-14.el5.1.0.1.i386 ########################## Done. words-3.0-9.1.noarch.rpm: ########################## Done. xen-3.4.0-0.0.23.el5.i386.r ########################## Done. xen-64-3.4.0-0.0.23.el5.noa ########################## Done. xen-debugger-3.4.0-0.0.23.e ########################## Done. xen-devel-3.4.0-0.0.23.el5. ########################## Done. xen-tools-3.4.0-0.0.23.el5. ########################## Done. xml-common-0.6.3-18.0.1.noa ########################## Done. ypbind-1.19-11.el5.i386.rpm ########################## Done. yum-3.2.19-18.el5.noarch.rp ########################## Done. yum-metadata-parser-1.1.2-2 ########################## Done. zip-2.31-2.el5.i386.rpm: ########################## Done. dbus-python-0.70-7.el5.i386 ########################## Done.

135 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

gettext-0.14.6-4.el5.i386.r ########################## Done. libXxf86vm-1.0.1-3.1.i386.r ########################## Done. libdaemon-0.10-5.el5.i386.r ########################## Done. libdrm-2.0.2-1.1.i386.rpm: ########################## Done. m4-1.4.5-3.el5.1.i386.rpm: ########################## Done. Preparing ########################################### [100%] Installing... 1:libgcc ########################################### [100%] 2:tzdata ########################################### [100%] 3:glibc-common ########################################### [100%] 4:glibc ########################################### [100%] 5:bash ########################################### [100%] 6:chkconfig ########################################### [100%] 7:popt ########################################### [100%] 8:audit-libs ########################################### [100%] 9:bzip2-libs ########################################### [100%] 10:libxml2 ########################################### [100%] 11:libstdc++ ########################################### [100%] 12:tcp_wrappers ########################################### [100%] 13:elfutils-libelf ########################################### [100%] 14:perl ########################################### [100%] 15:libX11 ########################################### [100%] 16:libacl ########################################### [100%] 17:binutils ########################################### [100%] 18:nspr ########################################### [100%] 19:nss ########################################### [100%] 20:procps ########################################### [100%] 21:iptables ########################################### [100%] 22:libgcrypt ########################################### [100%] 23:lm_sensors ########################################### [100%] 24:diffutils ########################################### [100%] 25:gzip ########################################### [100%] 26:make ########################################### [100%] 27:iptables-ipv6 ########################################### [100%] 28:iputils ########################################### [100%] 29:iproute ########################################### [100%] 30:dosfstools ########################################### [100%] 31:ethtool ########################################### [100%] 32:file ########################################### [100%] 33:mkisofs ########################################### [100%] 34:tftp ########################################### [100%] 35:setup ########################################### [100%] 36:filesystem ########################################### [100%] 37:enterprise-linux-ovs ########################################### [100%] 38:cracklib-dicts ########################################### [100%] 39:booty ########################################### [100%] 40:gnutls ########################################### [100%] 41:libXxf86vm ########################################### [100%] 42:libtiff ########################################### [100%] 43:pcre ########################################### [100%] 44:ed ########################################### [100%] 45:m4 ########################################### [100%] 46:patch ########################################### [100%] 47:libdaemon ########################################### [100%] 48:libdrm ########################################### [100%] 49:nash ########################################### [100%] 50:grub ########################################### [100%] 51:crash ########################################### [100%] 52:oprofile ########################################### [100%] 53:acl ########################################### [100%] 54:bzip2 ########################################### [100%] 55:anacron ########################################### [100%] 56:ypbind ########################################### [100%] 57:mdadm ########################################### [100%] 58:ovs-release warning: /etc/issue created as /etc/issue.rpmnew ########################################### [100%] 59:procmail ########################################### [100%] 60:ftp ########################################### [100%] 61:rdate ########################################### [100%] 62:rsh ########################################### [100%] 63:traceroute ########################################### [100%] 64:zip ########################################### [100%]

136 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

65:words ########################################### [100%] 66:man-pages ########################################### [100%] 67:libselinux ########################################### [100%] 68:device-mapper ########################################### [100%] 69:e2fsprogs-libs ########################################### [100%] 70:openssl ########################################### [100%] 71:shadow-utils ########################################### [100%] 72:openldap ########################################### [100%] 73:e2fsprogs ########################################### [100%] 74:findutils ########################################### [100%] 75:coreutils ########################################### [100%] 76:python ########################################### [100%] 77:module-init-tools ########################################### [100%] 78:libxml2-python ########################################### [100%] 79:newt ########################################### [100%] 80:kpartx ########################################### [100%] 81:system-config-securityl########################################### [100%] 82:ovs-utils ########################################### [100%] 83:curl ########################################### [100%] 84:net-snmp-libs ########################################### [100%] 85:net-tools ########################################### [100%] 86:psmisc ########################################### [100%] 87:dmraid ########################################### [100%] 88:hwdata ########################################### [100%] 89:pciutils ########################################### [100%] 90:cracklib ########################################### [100%] 91:pam warning: /etc/pam.d/system-auth created as /etc/pam.d/system-auth.rpmnew ########################################### [100%] 92:util-linux ########################################### [100%] 93:initscripts ########################################### [100%] 94:libuser ########################################### [100%] 95:usermode ########################################### [100%] 96:system-config-date ########################################### [100%] 97:yum-metadata-parser ########################################### [100%] 98:OpenIPMI-libs ########################################### [100%] 99:bind-libs ########################################### [100%] 100:libpcap ########################################### [100%] 101:cryptsetup-luks ########################################### [100%] 102:parted ########################################### [100%] 103:SysVinit ########################################### [100%] 104:logrotate ########################################### [100%] 105:authconfig ########################################### [100%] 106:kbd ########################################### [100%] 107:system-config-language ########################################### [100%] 108:system-config-securityl########################################### [100%] 109:dnsmasq ########################################### [100%] 110:mcstrans ########################################### [100%] 111:openssh ########################################### [100%] 112:sysklogd ########################################### [100%] 113:m2crypto ########################################### [100%] 114:python-urlgrabber ########################################### [100%] 115:compat-python-virtinst ########################################### [100%] 116:pykickstart ########################################### [100%] 117:nscd ########################################### [100%] 118:mesa-libGL ########################################### [100%] 119:vim-minimal ########################################### [100%] 120:p2v-util ########################################### [100%] 121:vixie-cron ########################################### [100%] 122:ebtables ########################################### [100%] 123:pam_krb5 ########################################### [100%] 124:device-mapper-multipath########################################### [100%] 125:multiprocessing ########################################### [100%] 126:python-iniparse ########################################### [100%] 127:python-simplejson ########################################### [100%] 128:gettext ########################################### [100%] 129:redhat-lsb ########################################### [100%] 130:alsa-lib ########################################### [100%] 131:fontconfig ########################################### [100%] 132:xml-common ########################################### [100%] 133:fipscheck ########################################### [100%] 134:nss_ldap ########################################### [100%] 135:neon ########################################### [100%] 136:device-mapper-event ########################################### [100%]

137 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

137:lvm2 ########################################### [100%] 138:mkinitrd ########################################### [100%] 139:libbdevid-python ########################################### [100%] 140:python-pyblock ########################################### [100%] 141:e4fsprogs ########################################### [100%] 142:nss_db ########################################### [100%] 143:ocfs2-tools warning: /etc/sysconfig/o2cb created as /etc/sysconfig/o2cb.rpmnew ########################################### [100%] 144:pypxeboot ########################################### [100%] 145:sudo ########################################### [100%] 146:openssh-clients ########################################### [100%] 147:openssh-server ########################################### [100%] 148:ppp ########################################### [100%] 149:bind-utils ########################################### [100%] 150:OpenIPMI ########################################### [100%] 151:system-config-rootpassw########################################### [100%] 152:dhclient ########################################### [100%] 153:dhcpv6-client ########################################### [100%] 154:quota ########################################### [100%] 155:vnc ########################################### [100%] 156:nfs-utils ########################################### [100%] 157:prelink ########################################### [100%] 158:tmpwatch ########################################### [100%] 159:gnupg ########################################### [100%] 160:iscsi-initiator-utils ########################################### [100%] 161:pexpect ########################################### [100%] 162:rhnlib ########################################### [100%] 163:autofs ########################################### [100%] 164:ksh ########################################### [100%] 165:logwatch ########################################### [100%] 166:psacct ########################################### [100%] 167:tcsh ########################################### [100%] 168:tcpdump ########################################### [100%] 169:elinks ########################################### [100%] 170:stunnel ########################################### [100%] 171:dbus-libs ########################################### [100%] 172:xen-devel ########################################### [100%] 173:rpm-libs ########################################### [100%] 174:rpm ########################################### [100%] 175:rpm-python ########################################### [100%] 176:dbus-glib ########################################### [100%] 177:xen-tools warning: /etc/xen/xend-config.sxp created as /etc/xen/xend-config.sxp.rpmnew ########################################### [100%] 178:dbus ########################################### [100%] 179:xen ########################################### [100%] 180:createrepo ########################################### [100%] 181:yum ########################################### [100%] 182:dbus-python ########################################### [100%] 183:avahi ########################################### [100%] 184:libvirt ########################################### [100%] 185:libvirt-python ########################################### [100%] 186:open-ovf ########################################### [100%] 187:ovs-agent ########################################### [100%] 188:kernel-ovs ########################################### [100%] 189:xen-64 ########################################### [100%] 190:xen-debugger ########################################### [100%] 191:net-snmp ########################################### [100%] 192:hal ########################################### [100%] 193:kudzu ########################################### [100%] 194:system-config-network-t########################################### [100%] 195:system-config-keyboard ########################################### [100%] 196:pm-utils ########################################### [100%] 197:firstboot-tui ########################################### [100%] 198:firstboot ########################################### [100%] 199:anaconda ########################################### [100%] 200:anaconda-runtime ########################################### [100%] The following packages were added to your selection to satisfy dependencies: Name Version Release -------------------------------------------------------------avahi 0.6.16 1.el5 dbus-libs 1.1.2 12.el5 device-mapper-event 1.02.28 2.el5

138 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

e4fsprogs ebtables fipscheck gnutls mesa-libGL multiprocessing nspr nss open-ovf python-iniparse python-simplejson redhat-lsb xml-common dbus-python gettext libXxf86vm libdaemon libdrm m4 #

1.41.1 2.el5 2.0.8 2.el5 1.0.3 1.el5 1.4.1 3.el5_3.5 6.5.1 7.7.el5 2.6.1.1 1 4.7.3 2.el5 3.12.3.99.3 1.el5_3.2 0.1 3.0.7.el5 0.2.3 4.el5 2.0.7 2.el5 3.1 12.3.0.1.EL 0.6.3 18.0.1 0.70 7.el5 0.14.6 4.el5 1.0.1 3.1 0.10 5.el5 2.0.2 1.1 1.4.5 3.el5.1

Reboot the Oracle VM Pool Master Server Next we will reboot the Oracle VM pool master server as shown in the next example. # reboot Broadcast message from root (pts/0) (Sun Oct 18 15:03:22 2009): The system is going down for reboot NOW! Reboot the None Master Oracle VM servers Next we reboot all of the none master Oracle VM servers as shown in the next example. # reboot Broadcast message from root (pts/0) (Sun Oct 18 15:03:22 2009): The system is going down for reboot NOW! Enter the Server Pool Virtual IP using Oracle VM Manager If you are using HA, the final step is to access Oracle VM Manager and enter a server pool virtual IP address for your server pool or pools. The server pool virtual IP is used to provide automatic Oracle VM master server failover to an Oracle VM none master pool member. The server pool virtual IP must be a vaild IP address on the same network as the pool servers. Access Oracle VM Manager, click the Server Pools tab then click the Edit button to access the Edit Server Pool page. From the Edit Server Pool page, enter the server pool virtual IP in the Server Pool Virtual IP text box then click OK as shown in Figure 12.

Watch this space Oracle OpenWorld 2011 Call for Papers - Submit Today! Temporary Post Used For Theme Detection (409fc5e9-d7ee-4e1a-b645-8b82f256f434 - 3bfe001a-32de-4114-a6b4-4005b770f6d7) Debugging Oracle VM 2.2 & Oracle VM Manager 2.2 errors using the many log files New VDI Training Course now available! HIMSS 2011 and New Press Release Oracle Desktop Virtualization at HIMSS 2011 Oracle VM VirtualBox 4.0.4 Released! Two Virtualization Webinars This Week We're Hiring! - Server and Desktop Virtualization Product Management

139 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Chapter 6: Oracle VM 2.2 SAN, iSCSI and NFS Back-end Storage Configurations
Last update 11-14-2010 Copyright © 2009 - 2011 Roddy Rodstein. All rights reserved. Show a printer friendly version of this book with its sub pages This chapter outlines how to configure an Oracle VM 2.2 pool with Fibre Channel and iSCSI SANs and NFS back-end storage. The chapter also covers guest front-end storage options and configurations. The chapter starts with an overview of the Oracle VM storage stack, followed by an introduction to Oracle VM back-end storage options, configurations and considerations. Next, we will summarize storage administration with Oracle VM 2.2 followed by example root and extended storage repository (SR) configurations using Fibre Channel and iSCSI SANs and NFS storage arrays. The chapter concludes with a review Oracle VM guest front-end storage options and configurations. Table of Contents The Oracle VM Storage Stack OCFS2 Cluster File System OCFS2 User-Space Management Utilities and Commands Oracle VM Storage Repositories Storage Administration The Storage Array Layer …Swap …Backup and Restoration …SAN and iSCSI Multi-pathing …Sparse Files and Unwritten Extents The Server Layer SAN Storage Repository Configuration …SAN Back-end Storage Prerequisites …SAN Oracle VM Server Prerequisites …Adding an Extended SAN Storage Repository iSCSI Storage Repository Configuration …iSCSI Back-end Storage Prerequisites …iSCSI Oracle VM Server Prerequisites …Adding an Extended iSCSI Storage Repository NFS Storage Repository Configuration …NFS Back-end Storage Prerequisites …NFS Oracle VM Server Prerequisites …Adding an Extended NFS Storage Repository Oracle VM Guest Front-end Storage …File-backed block device …Physical backed block device Appendix A Example multipath.conf Files The Oracle VM Storage Stack An Oracle VM storage solution consists of three distinct layers. Each layer has its own unique requirements, configurations, dependencies and features. The first layer is the storage array, which is referred to as back-end storage. Oracle VM supports local storage, Fibre Channel and iSCSI SANs and NFS back-end storage. The second layer is the server layer, which consists of the Oracle VM server storage configurations and the virtual machine file system, i.e. the Oracle Cluster File System 2 (OCFS2) or NFS. Oracle VM supports a wide variety of configurations for Fibre Channel and iSCSI SANs and NFS storage arrays. The third layer is the guest front-end storage, which consists of multiple guest storage and driver options. Note: Oracle VM supports both local and shared back-end storage. Local storage refers to a file system that can only be accessed by a single Oracle VM server. This chapter covers shared back-end storage supporting a clustered multi server pool environment, not local storage. Figure 1 shows a high-level overview of the three layers of the storage stack with a virtual machine running on an Oracle VM server, connected to a storage array. At the bottom of the stack is the storage array. The storage array layer is where the physical disks are managed and presented to the Oracle VM pool members as logical disks. Above the storage array is the server layer. The server layer is where the storage configurations and the OCSF2 or NFS virtual machine file system (the cluster stack) are managed. At the top of the stack is the virtual machine layer. The virtual machine layer is where virtual machine storage is presented to the virtual machine by the Oracle VM server. Figure 1 shows the Oracle VM storage stack.

140 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

When designing an Oracle VM virtual environment, one of the most important considerations is the back-end storage. There are many back-end storage options ranging from local storage, Fibre Channel and iSCSI SANs and NFS. Each back-end storage option has its own capacity, performance and availability features. The back-end storage is where you store and run the virtual machines. Oracle VM supports the Oracle Cluster File System v2 (OCSF2) and NFS on the back-end storage to store and run the virtual machines. The OCFS2 cluster file system and NFS both have their own unique management, performance and availability features. The next section will review the OCSF2 cluster file system, NFS and the cluster stack. The goal of the OCSF2 section is to provide an overview of the architecture,configurations, dependencies and features of the virtual machine file system, at the server layer of the storage stack. Understanding the architecture, configurations, dependencies and features at the server layer will allow you to design a manageable back-end storage solution for Oracle VM. Note: For the remainder of this chapter, the terms “pool” and “cluster” should be considered to be interchangeable. OCFS2 Cluster File System The Oracle Cluster File System 2 (OCFS2) is a general-purpose journaling file system developed by Oracle. Oracle released OCFS2 under the GNU General Public License (GPL), version 2. The OCSF2 source code and its tool set are part of the mainline Linux 2.6 kernel and above. The OCSF2 source code and its tool set can be downloaded from kernel.org and from the Unbreakable Linux Network. Note: OCFS2 is not integrated or supported with any volume manager (LVM) to manage the back-end block storage. Fibre Channel and iSCSI partitions must be provisioned at static sizes, i.e. partition sizes can not change once a partition is formatted with OCFS2. Many customers try to use LVM to manage the back-end block storage for OCFS2. LVM is not cluster aware, so changes made to the back-end block storage by LVM will not be propagated to the OCFS2 file system. The Oracle VM pool members would continue to write to the old volume layout, and corruption will occur. OCFS2 has two components, a kernel component and a user-space component. The kernel component consists of the file system and the cluster stack. The user-space component consists of the utilities to manage the file system and the cluster stack. A slightly modified version of OCSF2 (o2dlm) is bundled with Oracle VM. The OCFS2 file system and cluster stack are installed and configured as part of an Oracle VM server installation. The o2cb service manages the cluster stack and the ocfs2 service manages the OCSF2 file system. The o2cb cluster service is a set of modules and in-memory file systems that manage the ocfs2 file system service. Once a server pool is created using Oracle VM Manager, two cluster configuration files are shared across the server pool that maintain the cluster layout and cluster timeouts configurations. The /etc/ocfs2/cluster.conf file maintains the cluster layout and the /etc/sysconfig/o2cb file maintains the cluster timeouts. Both configuration files are read by the user-space utility configfs. configfs communicates the list of nodes in the /etc/ocfs2/cluster.conf file to the in-kernel node manager, along with the resource used for the heartbeat to the in-kernel heartbeat thread. The ovs-agent, which is also installed and configured by default, is responsible for propagating the /etc/ocfs2/cluster.conf file to all of the pool members. The ovs-agent is an Oracle VM server service that is used for centralized pool management, orchestrated by Oracle VM Manager or the Oracle VM Management Pack. Each time an ovs-agent starts and stops, it updates the pool status, which is managed by the master pool agent. The master pool agent updates the pool membership status and then propagates an up to date /etc/ocfs2/cluster.conf file to all of the pool’s ovs-agents. An Oracle VM server must be online to be in an OCFS2 cluster. Once the cluster is on-line, each pool member starts a process, o2net. The o2net process creates TCP/IP intra-cluster node communication channels on port 7777 and sends regular keepalive packages to each node in the cluster to validate if the nodes are alive. The intra-cluster node communication uses the Oracle VM management network. The Oracle VM management network is selected during the Oracle VM server installation. If a pool member falls of the network and the keepalive connection becomes silent, the server will self-fence. Fencing forcefully removes dead servers from a pool to ensure that active servers are not obstructed from accessing fenced servers cluster resources. Along with the keepalive packages that check for node connectivity, the cluster stack also employs a disk heartbeat check. o2hb is the process that is responsible for the disk heartbeat component of cluster stack that actively monitors the status of all pool members. The heartbeat system uses a file on the OCSF2 file system, that each pool member periodically writes a block to, along with a time stamp. The time stamps are read by each pool member and are used to check if a pool member is alive or dead. If a pool member’s block stops getting updated, the server is considered dead. When a server dies, the server gets fenced. Fencing forcefully removes dead pool member from the pool to ensure that active pool members are not obstructed from accessing fenced pool members resources.

141 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Another important OCSF2 component is the distributed lock manager. The distributed lock manager (o2dlm) tracks all the locks in the cluster, including lock ownership and lock status. Cluster locking is added at the lowest level, in the xend code. The locking method is defined in the xend-config.sxp file, (xend-domains-lock-path /opt/ovs-agent-2.3/utils/dlm.py) All access methods must take a lock, for . .example, Oracle VM Manager, xm and the XenAPIdlm is also used for Oracle VM HA, which relies on the cluster stack to validate pool member status for HA purposes. For example, as pool members boot, reboot and restart, pool membership status will change across the pool. There is also a virtual filesystem interface (dlmfs) that allows user space processes to access the in-kernel distributed lock manager. dlmfs communicates locking and unlocking for pool wide locks on resources to the in-kernel distributed lock manager. The in-kernel distributed lock manager keeps track of all locks and their owners and status. The o2cb init script mounts the virtual filesystem under /dlm on each Oracle VM server. To provide OCFS2 functionality with an NFS storage repository, Oracle VM uses a hidden OCFS2 file-backed block device that facilitates the use of the OCFS2 distributed lock manager (DLM) with NFS. The ability to use the OCFS2 distributed lock manager with OCFS2 and NFS allows Oracle VM to monitor both OCFS2 and NFS storage repositories with the same interface. Table 1 shows the OCSF2 cluster service stack. Service In Kernel Node Manager (NM) Network and Storage Heartbeat (HB) TCP/IP Description The in kernel node manager tracks all of the pool members in the /etc/ocfs2/cluster.conf file. The network and storage heartbeat dispatches up/down notifications when pool members join or leave the cluster. The TCP/IP protocols handle the communication between pool members. DLM tracks locks in the pool, including lock ownership and lock status. configfs communicates the list of pool members to the in-kernel node manager. configsf also communicates the heartbeat resource to the in-kernel heartbeat thread. Configfs mounts under /sys/kernel/config dlmfs communicates locking and unlocking for pool wide locks on resources to the in-kernel distributed lock manager. The in-kernel distributed lock manager keeps track of all locks and their owners and status. The dlmfs user space virtual filesystem interface mounts under /dlmfs

Distributed Lock Manager (DLM) configfs

dlmfs

Now that we have reviewed the components of the OCFS2 file system and cluster stack, let’s see how OCFS2 works together with Oracle VM. When an Oracle VM 2.2 server boots, the o2cb and ocfs2 services are started which bring up the OCFS2 clusterstack. Once the OCFS2 clusterstack is online, the ovs-agent informs the pool master that the node is online. Next, the pool master updates the nodemap file with the node’s online status. Next, the ovs-agent queries the pool master and pulls down an up-to-date /etc/ocfs2 /cluster.conf configuration. Next, the ovs-agent mounts the root and any extended repositories and checks that /OVS is symlinked correctly. When a pool member stops, starts or dies, the pool master attempts to take an EX lock for the dead pool member’s resources. The master agent then updates the nodemap file to monitor the aliveness of all active pool members. Whenever the pool membership status changes, the mater agent will recreate the cluster.conf file and propagate the changes to all of the pool members. The next section will review the OCFS2 user-space management utilities and commands. OCFS2 User-Space Management Utilities and Commands OCFS2 has a full suite of utilities to manage the OCFS2 file system and the cluster stack. Table 2 lists the OCFS2 file system utilities that are available in dom0. OCFS2 Description

142 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Utility mkfs.ocfs2 mkfs.ocfs2 is used to format an OCFS2 file system on a device. mkfs.ocfs2 requires the O2CB cluster service to be up. tune.ocfs2 is used to manage OCFS2 file system parameters, including the volume label, number of node slots and journal size for all node slots.

tune.ocfs2

mounted.ocfs2 mounted.ocfs2 detects and lists all OCFS2 volumes on an Oracle VM server. fsck.ocfs2 debugfs.ocfs2 fsck.ocfs2 checks and repairs the OCFS2 file system. debugfs.ocfs2 is used to query the state of the OCFS file system for debugging. Description Reports if the o2cb services are loaded and mounted Loads the O2CB modules and in-memory file systems Onlines the cluster named ocfs2. The default name for Oracle VM OCFS2 cluster is ocfs2. The cluster name is defined in the cluster.conf file. At least one pool member must be active for the cluster to be online. Offlines the cluster named ocfs2. The default name for Oracle VM OCFS2 cluster is ocfs2. The cluster name is defined in the cluster.conf file. Unloads the O2CB modules and in-memory file systems Starts the cluster named ocfs2 by loading o2cb and onlining the cluster. The default name for Oracle VM OCFS2 cluster is ocfs2. The cluster name is defined in the cluster.conf file. At least one pool member must be active for the cluster to be online.

Table 3 lists the commands to manage the o2cb services (the clusterstack). Command /etc/init.d/o2cb status /etc/init.d/o2cb load /etc/init.d/o2cb online ocfs2

/etc/init.d/o2cb offline ocfs2 /etc/init.d/o2cb unload /etc/init.d/o2cb start ocfs2

Next, we will review Oracle VM local storage, Fibre Channel and iSCSI SANs and NFS storage repositories. Oracle VM Storage Repositories A default Oracle VM 2.2 server installation creates a “local” OCFS2 virtual machine file system that is mounted under /var/ovs /mount/UUID and linked to /OVS. Using a local storage repository restricts pool membership to “one” Oracle VM server without Live Migration or HA functionality. To increase the capacity of an Oracle VM pool past one Oracle VM server, the addition of a shared back-end storage repository is required. An Oracle VM storage repository can consist of “one” large repository, commonly referred to as “a root repository” or a root repository with multiple extended sub repositories. Oracle VM 2.x does not have volume management, so adding storage to a root repository volume will not grow the root repository. The only option to grow an Oracle VM 2.x storage repository is to add sub repositories beneath the root repository. A best practice is to provision one or more larger repositories to avoid the management overhead of numerous sub repositories. Tip: In general, you should consider provisioning at least 30% to 50% more storage for your Oracle VM storage repositories than the expected size. Configuring an Oracle VM pool’s storage repository is a multi step process. Once the back-end storage is provisioned, the pool master must be connected to the storage from dom0. Next, all of the Oracle VM servers that will be added to the pool should be connected to the storage, again from dom0. Finally, all of the Oracle VM servers should be added to the pool using Oracle VM Manager or the Oracle VM Management Pack. Once the pool has multiple servers, virtual machines can start on and migrate to any server in the pool. To add storage to an Oracle VM storage repository, the first step is to provision the storage. Next, connect the pool master, followed by each pool member to the storage using the /opt/ovs-agent-2.3/utils/repos.py script with the -n (new) followed by the -i (initialize, aka mount) switches, to add and then mount the sub storage repository. Finally the new mount point in /var/ovs/mount/UUID needs to be linked to /OVS/UUID, by typing “ln -nsf /var/ovs/mount/<UUID>/OVS”, again from dom0. The end result is a root repository with an “extended” sub repository mounted under /var/ovs/mount/UUID, linked to /OVS/UUID. The Oracle VM agents will automatically place resources such as virtual machines, templates, or ISO files on the root or sub repository with available space. Figure 2 shows a root storage repository.

143 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Figure 3 shows a root storage repository and an extended sub repository.

Oracle VM 2.2 uses the /opt/ovs-agent-2.3/utils/repos.py script to configure storage repositories and a local Berkley DB to save the storage repository configurations. The Oracle VM agent is also responsible for mounting and linking storage repositories when an Oracle VM server boots or restarts. For example, you will not see entries in /etc/fstab for any Oracle VM storage repositories. Oracle VM storage repository configurations are saved in a local Berkley DB or in a shared Berkley DB in the root storage repository. Oracle VM root and extended storage repositories all share the same directory structure. Oracle VM’s OCFS2 file system, clusterstack, repos.py script, Oracle VM agent, Oracle VM Manager as well as the Oracle VM Management Pack are wired to use the default storage repository directory structure. The following example shows the Oracle VM storage repository directory structure including a brief explanation of each directory. /OVS (Root directory) | B47E850ABA50460882B30645CF051619 (UUID of an extended file system) | iso_pool (ISO files storage, requires VT chip extensions) | lost+found (The lost and found directory) | publish_pool (Public virtual machine storage) | running_pool (Published virtual machine storage) | seed_pool (Virtual Machine template storage) | sharedDisk (Shared virtual disk storage) The next example shows the storage repository directory structure of an extended storage repository. / | | | | | | B47E850ABA50460882B30645CF051619 (UUID of an extended file system) iso_pool (ISO files storage, requires VT chip extensions) lost+found (The lost and found directory) publish_pool (Public virtual machine storage) running_pool (Published virtual machine storage) seed_pool (Virtual Machine template storage) sharedDisk (Shared virtual disk storage)

Storage Administration Now that we know all about the OCFS2 file system, clusterstack and the storage repository directory structure, we will to turn our attention to Oracle VM storage administration. As discussed in the Oracle VM Storage Stacksection, there are three distinct layers of an Oracle VM storage solution. The first layer is the storage array, which is referred to as back-end storage. The second layer is the server layer, which consists of the Oracle VM server storage configurations and the virtual machine file system. The third layer is the

144 of 243

18.04.2011 16:35

The Underground Oracle VM Manual
guest front-end storage, which consists of multiple guest storage and driver options.

http://itnewscast.com/book/export/html/844

The following sections will review storage administration at each layer of the storage stack. The Storage Array Layer Oracle VM storage administration is done at the storage array layer. Oracle VM Manager or the Oracle VM Management pack is responsible for pool creation, not for storage repository management. For example, storage repository provisioning, storage repository snapshotting, storage repository replication, storage repository monitoring as well as storage repository backup and restoration are preformed at the storage array layer. Storage array configurations and storage best practices are vendor specific and out of the scope of this document. Please consult your storage administrators and storage vendor and application owners to help develop a storage solution to meet your business requirements. This section will review Oracle VM specific storage array considerations. Administrators do not always have the luxury to design the best storage solution for their environment. Ultimately management makes the decisions and administrators make the best out of what equipment they get. Ideally, we would like to design a storage solution that allowed us to provision tiered storage for different workloads. Each workload, for example, RAC, Fusion Middleware, and E-Business Suite has different requirements, so depending on the workload, back-end disk configurations and guest front-end disk configurations will affect the performance of the workloads. The only way to validate the best configuration for a workload is to benchmark the workload using a variety of back-end and front-end configurations. Once you know which configurations provide the best performance for a given workload then it is time to provision and configure the back-end and front-end storage accordingly. Swap Swap is another storage array layer component that requires careful consideration. The best practice for guests is to add RAM to a guest to tune the database or application workload to minimize swapping altogether. If some swapping is necessary, placing the guests swap files on the Oracle VM server’s local disk will offer better peformnace than hosting a guests swap file on a SAN. Paging over a SAN in parallel with swap traffic from other guests can easily contribute to an I/O bottleneck. Swap traffic from other guests is especially bad when a common set of physical LUNs is provisioned as swap space for many guests. If several guests load up and start swapping heavily, all the guests on that storage will grind to a halt waiting for the saturated LUNs to respond. Please note that placing a guests swap file on an Oracle VM server’s local disk will eliminate the ability to use Live Migration. Backup and Restoration Identify a backup and restoration strategy for the guests. If the storage array does not offer a suitable guest backup and restoration solution at the storage array layer consider using an OS agent based backup and restoration solution. SAN and iSCSI Multi-pathing Configure the Fibre Channel and iSCSI multi-pathing using dm-multipath. Installing 3 rd party SAN connectivity software in dom0 is not supported by Oracle. Sparse Files and Unwritten Extents Guest virtual disks stored on Oracle VM 2.2 and above OCFS2 file systems use sparse files and unwritten extents by default. When using sparse files and unwritten extents, guest virtual disk files grows proportionally to the number of writes to the disk by the guest, so that large portions of the unused disk do not consume space. The advantage of using sparse files is that storage is allocated only when needed which reduces the time it takes to create sparse files along with saving disk space. The disadvantage of using sparse files is that the file system free space reports may be misleading. For example, since storage is allocated only when needed, the file system free space reports may not be accurate since large portions of unused disk, i.e. the sparse zero sections have not yet been written to disk. Tip: Some application do not support copying sparse files and may copy the entire uncompressed size of the file including the sparse sections. The Server Layer Configuring an Oracle VM pool’s storage repository is a multi step process. Once the back-end storage is provisioned, the pool master must be connected to the storage from dom0. Oracle VM supports SAN, iSCSI and NFS back-end storage. Once the pool master is connected to the storage and an HA pool is created in Oracle VM Manager, each pool member should be configured to access the storage, and then added to the pool using Oracle VM Manager. Next, we walk through example root and extended storage repository (SR) configurations using Fibre Channel and iSCSI SANs and NFS storage arrays. SAN Storage Repository Configuration In this section we will walk through the steps to configure Oracle VM servers using a Fibre Channel SAN storage array. All of the steps will be executed on each Oracle VM server from dom0 as root. Once all of the Oracle VM servers are configured, an HA enabled pool will be created in Oracle VM Manager with the pool master server. Next, all of the other configured Oracle VM server will be added to the HA enabled pool. Tip: An HA enabled pool will automatically add (repos.py -n and -r) and mount (repos.py -i) root and extended storage repositories for all pool members.

145 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

OCFS2 is not integrated or supported with any volume manager (LVM) to manage the back-end block storage. Fibre Channel and iSCSI partitions must be provisioned at static sizes, i.e. partition sizes can not change once a partition is formatted with OCFS2. The challenge with supporting volume management with OCFS2 is that the volume manager needs to be cluster-aware and integrated with the OCFS2 cluster stack. To date there are no supported volume management solutions for OCFS2. For example, many customers use LVM to manage the back-end block storage for OCFS2. LVM is not cluster aware, so changes made to the back-end block storage by LVM will not be propagated to the OCFS2 file system. The Oracle VM pool members would continue to write to the old volume layout, and corruption will occur. SAN connectivity is configured using fiber channel HBAs with dm-multipath in dom0 to allow the Oracle VM server to access a Logical Unit (LU) using multiple paths. Oracle VM also supports boot from SAN. The "linux mpath" install option is used to boot an Oracle VM server from a SAN. By using the "linux mpath" install option, the installer will see the multipath devices and allow you to create the boot/root partitions, along with the master boot record (MBR) on the SAN. Please note that this document will not cover boot from SAN. To connect an Oracle VM server to a Fibre Channel storage array, each Oracle VM server’s HBAs must be zoned and masked to the storage. Once the HBAs are zoned and masked, the next step is to configure dm-multipath to detect the LUNs which are recognized as multipath devices. Once the multipath devices are detected, we need to format the devices on the “pool master” using the mkfs.ocfs2 utility. Next, use the repos.py script to configure the storage repository. Finally, create a pool using Oracle VM Manager or the Oracle VM Management Pack by selecting the pool master server. Once the pool is created add all the other Oracle VM servers to the pool. SAN Back-end Storage Prerequisites: Create the LUN(s) The HBAs must be zoned and masked to the storage. SAN Oracle VM Server Prerequisites: 1. All Oracle VM servers must be patched from the Unbreakable Linux Network (ULN) to ensure that the storage configurations will not be hampered by unpatched bugs. 2. Select an Oracle VM server that will be used as the Oracle VM pool master. After the Oracle VM pool master and all the other Oracle VM pool members meet the prerequisites outlined in the following steps, access Oracle VM Manager and create an HA enabled pool using the Oracle VM pool master server. Note: An HA enabled pool automatically mounts and links root and extended storage repositories for each Oracle VM pool member that is added to a pool. 3. Create a multipath.conf file for the storage array. Please refer to Appendix A for examples multipath.conf files. 4. Ensure that all the Oracle VM servers’ clocks are synchronized using NTP . First, open the “/etc/ntp.conf” file by typing “vi /etc/ntp.conf” and validate that at least two available NTP servers entries are listed. The next example shows two bold NTP server entries in an ntp.conf file. # Use public servers from the pool.ntp.org project. # Please consider joining the pool (http://www.pool.ntp.org/join.html). server myntp1.com server myntp2.com Ping each NTP server listed in the ntp.conf file from each Oracle VM server to ensure network connectivity. Next, type "ntpstat" on each Oracle VM server to validate the NTP configuration. The next example shows the output from typing the ntpstat command on an Oracle VM server that has its time synchronized to an NTP server with the IP address of 192.168.4.251. # ntpstat synchronized to NTP server (192.168.4.251) at stratum 4 time correct to within 54 ms polling server every 1024 s Finally, validate that the time, date and time zone on each Oracle VM server as well as on the Oracle VM Manager host is synchronized by typing the "date" command. 5. All Oracle VM servers have consistent name resolution using DNS with both forward and reverse lookups. First, open the “/etc/resolv.conf” file by typing “vi /etc/resolv.conf” and validate that two available DNS servers are listed. The next example shows two DNS servers listed in a resolve.conf file. # vi /etc/resolve.conf nameserver <MY DNS SERVER1 IP ADDRESS> nameserver <MY DNS SERVER2 IP ADDRESS> From each Oracle VM server ping each DNS server listed in the resolv.conf file to ensure network connectivity. Next, validate the forward and reverse lookups for each Oracle VM pool member and the Oracle VM Manager host using the “host” command. For example, to validate server2's forward lookup from server1 type “host server2” as shown in the next example.

146 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

# host server2 server2 has address 192.168.4.6 Next, to validate server2's reverse lookup from server1 type “host 192.168.4.6” as shown in the next example. # host 192.168.4.6 6.4.168.192.in-addr.arpa domain name pointer server2 Note: Using hosts files without DNS is not advised and may produce unpredictable results. 6. The Oracle VM server’s host name in the /etc/hosts file must be associated with the Oracle VM server's public IP address. If an Oracle VM pool member's host name is associated with 127.0.0.1, the cluster.conf file will be malformed and the Oracle VM pool will not be operational. The next example shows the improper syntax from an Oracle VM server's hosts file entry. 127.0.0.1 servername.com servername localhost.localdomain localhost 192.168.4.8 servername.com servername The next example shows the proper syntax for an Oracle VM server’s hosts file entry. 127.0.0.1 localhost 192.168.4.8 servername localhost.localdomain servername.com

7. ocfs2 network connectivity between all Oracle VM server pool members must be operational before creating a multiple server pool. Check the ocfs2 network connectivity between all Oracle VM pool members by typing "nc -zv <myoraclevmserver1> 7777". For example, if you have two Oracle VM servers named ovs1 and ovs2, from ovs1 type "nc -zv ovs2 7777". Typing "nc -zv ovs2 7777" from ovs1 should return "succeeded!". If you receive a "failed: Connection refused" message between any Oracle VM servers, something (firewall, switch, router, cable, etc..) is restricting communication between the hosts. The iptables firewall on an Oracle VM server may be blocking the ocfs2 connectivity. If iptables is disabled and allowing all connections, the output from typing “iptables -L will look like the next example. ” # iptables -L Chain INPUT (policy ACCEPT) target prot opt source destination Chain FORWARD (policy ACCEPT) target prot opt source destination Chain OUTPUT (policy ACCEPT) target prot opt source destination If typing “iptables -L lists firewall rules, you can a) disable iptables by typing "service iptables stop && chkconfig iptables off" or b) ” add the following bold iptables rule to the /etc/sysconfig/iptables file prior to the last line on all Oracle VM pool members. -A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 7777 -j ACCEPT -A RH-Firewall-1-INPUT -j REJECT --reject-with icmp-host-prohibited COMMIT After you have added the above bold iptables rule, restart the iptables service by typing "service iptables restart". 8. If an Oracle VM server was originally installed with a local ocfs2 storage repository, it is necessary to remove and unmount the local ocfs2 storage repository before adding the Oracle VM server to a pool. To determine if an Oracle VM server is using a local storage repository type "/opt/ovs-agent-2.3/utils/repos.py -l" to list all configured storage repositories. If a storage repository is listed, type "/opt/ovs-agent-2.3/utils/repos.py -d UUID" to remove the local repository from the Oracle VM server. Next, check if the local storage repository is still mounted under /var/ovs/mount/UUID. Type “mount |grep mount”, as shown in the next example to list the mounts. # mount |grep mount /dev/sda3 on /var/ovs/mount/62C757BA5E174DF7B5AB01BBAE0F765D type ocfs2 (rw,heartbeat=none) The above example shows that a storage repository /var/ovs/mount/62C757BA5E174DF7B5AB01BBAE0F765D is still mounted under /dev/sda3. Next, unmount the OCFS2 repository by typing “umount /var/ovs/mount/62C757BA5E174DF7B5AB01BBAE0F765D” as shown in the next example. # umount /var/ovs/mount

147 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

/62C757BA5E174DF7B5AB01BBAE0F765D Tip: A default Oracle VM server installation dedicates the majority of the local disk to the OVS partition and create a small root partition. If your Oracle VM server was installed with the default OVS partition with a small root partition, consider rebuilding the server to create a disk layout that allocates the disk space to the root partition. Type “df -h” to list the size of an Oracle VM servers' partitions. Another consideration for small roots that include /var is the potential for large saves from the xendomains service. If an Oracle VM server crashes, the xendomains service will save the state (the memory foot print of each guest) of all running guests in the /var/lib /xen/save directory, which could fill up a small root partition. If xendomains functionality is not needed disable it. The next example shows how to disable or edit the location of the saved xendomains files. 1. Edit /etc/sysconfig/xendomains 2. find the section: ## Type: string ## Default: /var/lib/xen/save # # Directory to save running domains to when the system (dom0) is # shut down. Will also be used to restore domains from if # XENDOMAINS_RESTORE # is set (see below). Leave empty to disable domain saving on shutdown # (e.g. because you rather shut domains down). # If domain saving does succeed, SHUTDOWN will not be executed. # XENDOMAINS_SAVE=/var/lib/xen/save 3. Clear the XENDOMAINS_SAVE path to disable saves. Or point the XENDOMAINS_SAVE path to a partition with available space. 4. Rest the xendomains services by typing “service xendomains restart” to enable a new configuration. 9. If an Oracle VM server have previously been added to an Oracle VM server pool, the Oracle VM server's cluster configurations will need to be cleaned before added to a new Oracle VM server pool. To clean an Oracle VM server's cluster configurations it is necessary to a) empty the /etc/ocfs/cluster.conf file b) delete and recreate the local BerkleyDB and c) run the cleanup.py script to stop o2cb heartbeat, offline o2cb, remove o2cb configuration file, umount ovs-agent storage repositories and to cleanup ovs-agent local database. To clear /etc/ocfs/cluster.conf file type “cat /dev/null> /etc/ocfs2/cluster.conf” from dom0, as shown in the next example. # cat /dev/null> /etc/ocfs2 /cluster.conf To remove the local BerkleyDB first type “service ovs-agent stop”, which stops the Oracle VM agent. Next, type “rm -fr /etc/ovsagent/db/*” to delete the BerkleyDB. Finally, type “service ovs-agent start” to start the Oracle VM agent, which also recreate a new local BerkleyDB. To stop the o2cb heartbeat, offline o2cb, remove o2cb configuration file, unmount ovs-agent storage repositories and to cleanup ovs-agent local database, type "/opt/ovs-agent-2.3/utils/cleanup.py" and then type “y” as shown in the next example. # /opt/ovs-agent-2.3/utils/cleanup.py This is a cleanup script for ovs-agent. It will try to do the following: *) stop o2cb heartbeat *) offline o2cb *) remove o2cb configuration file *) umount ovs-agent storage repositories *) cleanup ovs-agent local database Would you like to continue? [y/N] y Cleanup done. Step 1: The first step is to validate that the HBAs are listed in the /sys/class/fc_host directory. The goal of this step is to record the host adapter ID number(s) and to troubleshoot any SAN connectivity issues. You can skip this step if you’re able to view the HBAs listed in the /sys/class/fc_host directory. Tip: If there are no host adapters listed in the /sys/class/fc_host directory, check if the HBAs are properly zoned and masked. As shown in the next example, type “ll /sys/class/fc_host” to list the host adapters. # ll /sys/class/fc_host total 0 drwxr-xr-x 3 root root 0 Oct 11 08:24 host6

148 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

drwxr-xr-x 3 root root 0 Jul 5 09:28 host7 The output from “ll /sys/class/fc_host” shows that there are two host adapters, host6 and host7. Once you’re able to list the host adapters in the /sys/class/fc_host directory, cat each host adapter’s “port_name” file to get the host adapter number ID number. #cat /sys/class/fc_host/host6/port_name 0x10000000c0ffee7e The above example shows that 0x10000000c0ffee7e is the host adapter number ID number for host6. Next, cat the host7/port_name file to get the host adapter number ID number. cat /sys/class/fc_host/host7/port_name 0x10000000c0ffee7f The above example shows that 0x10000000c0ffee7f is the host adapter number ID number for host7. If you need to rescan the bus, echo the “/sys” filesystem as shown in the next examples. #echo “- - -” > /sys/class/scsi_host/hostH/scan For example. #echo “- - -” > /sys/class/scsi_host/host6/scan #echo “- - -” > /sys/class/scsi_host/host7/scan We have successfully discovered the host adapter ID numbers from each Oracle VM server. Step 2: Next, validate that multipath daemon is properly configured on each Oracle VM server. From dom0, type “service multipathd status”, as shown in the following example. #service multipathd status multipathd (pid 10333) is running... If your multipath daemon is running, please skip to Step 3. If your system’s multipath daemon is stopped, use chkconfig to configure the multipath daemon, as shown in the below example. Next, type “chkconfig --list multipathd” to view the multipath daemon configuration, as shown in the next example. #chkconfig --list multipathd multipathd 0:off 1:off 2:off 3:off 4:off 5:off 6:off The output of “chkconfig --list multipathd” shows that the multipath daemon is not configured to run at any system run level i.e. run level 0 through 6. Next, type “chkconfig multipathd on” in order to automatically start the multipath daemon at run level 3, 4 and 5, as shown in the next example. #chkconfig multipathd on Next, validate the multipathd startup configuration by typing “chkconfig --list multipathd” as shown in the next example. # chkconfig --list multipathd multipathd 0:off 1:off 2:on

3:on

4:on

5:on

6:off

The output of “chkconfig --list multipathd” validates that the multipath daemon is configured to run at run level 2, 3, 4, and 5. Finally, start the multipath daemon by typing “service multipathd start”, as shown in the following example. # service multipathd start Starting multipathd daemon:

[ OK ]

We have successfully configured and started the multipath daemon on each Oracle VM server.

149 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Step 3: Next, we will configure dm-multipath on each Oracle VM server by replacing or modifying the default /etc/multipath.conf file with a multipath.conf file crafted for your storage solution. multipath.conf file settings can be vendor specific, please check with your storage vendor for a multipath.conf file for Oracle VM or RHEL 5U3. If you already have a working multipath.conf file please skip to Step 4. Each Oracle VM server has an example multipath.conf file located at /etc/multipath.conf. The example multipath.conf file should be modified or replaced with a vendor specific multipath.conf file to support your storage array. A multipath.conf file is divided into four sections. List 4 shows the four sections of a multipath.conf file. 1. 2. 3. 4. blacklist defaults multipaths devices

Next, we will review the four sections of a multipath.conf file. Blacklist The blacklist section lists devices that are to be excluded from multipath control. For example, if the server boots from a local disks i.e. sda, sdb, hda, hdb, etc…then we need to include those disks in the blacklist. The example multipath.conf file has two blacklist entries. As shown below, the first blacklist entry is uncommented and will blacklist all devices. blacklist { devnode "*" } If you are going to test the example multipath.conf file, comment the blacklist entry to allow devices to be managed by dm-multipath. The next example shows the blacklist entries commented. #blacklist { # devnode "*" #} The second blacklist entry is commented and shows how to blacklist WWIDs, ram, raw, loop, fd, md, dm-, sr, scd, st, and hd devices. The next example shows the second black list entry. #blacklist { # wwid 26353900f02796769 # devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*" # devnode "^hd[a-z]" #} The first entry wwid 26353900f02796769 is an example that shows how to blacklist WWID. If you need to blacklist WWIDs, add an entry for each WWID, for example. wwid "3600508b40008dc480000500000670000" wwid "3600508b40008dc480000500000640000" wwid "3600508b40008dc480000500000610000" The second devnode line devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*" blacklists ram, raw, loop, fd, md, dm-, sr, scd, st, and hd devices. The third devnode entry devnode "^hd[a-z]" will blacklist hda disks. To test the default blacklist entry, first comment out the blacklist entry that blacklist all devices. Next, remove the wwid line, then uncomment the blacklist, devnode and the } sections, as shown below. blacklist { devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*" devnode "^hd[a-z]" } You will need to restart the multipath daemon to test any new settings. The next example shows how to restart the multipath daemon. # service multipathd restart Stopping multipathd daemon: Starting multipathd daemon:

[ OK ] [ OK ]

150 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

If your Oracle VM servers use a serial or SCSI controller for local disks then the interface names will be similar to sda, sdb, etc. To exclude the local disks i.e. sda and sdb you need to add a devnode entry for the devices. The example devnode entry below will blacklist the sda and the sdb device. blacklist { devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*" devnode "^hd[a-z]" devnode "^sd[a-b]$" } Defaults The defaults section allows you to configure default settings for dm-multipath that “may” be supported by your storage array. If you determined that your storage array does not use the settings in the defaults section, use the devices section of the multipath.conf file. The defaults section settings are overwritten when the devices and multipaths sections are used. The default “defaults” setting in the example multipath.conf uses the “user_friendly_names yes” entry. The “user_friendly_names yes” entry will allow you to use an alias, instead of WWID (World Wide Identifier) names. Multipath devices can be identified by a WWID names or by an alias. A WWID is a unique identifier for the multipath device that does not change. Device names such as /dev/sdx and /dev/dm-x can change on reboot, so defining multipath devices by their ID is preferred. The multipath device names in the /dev/mapper directory references LUN IDs that do not change and are user friendly, i.e. mpath0, mpath1, etc… Multipaths The multipaths section is where you map devices to a user friendly name. Each multipath entry will specify the UUID or wwid and the alias of a LUN along with path_checker variables, which will regularly check the path. The settings in the multipaths section overwrite the settings specified in the defaults and devices sections. Devices The devices section is used to define vendor specific settings. Consult your storage vendor for the entries for your storage array. If you are using multiple SAN storage systems, several device entries are necessary. After changing settings in a multipath.conf file, administrators must restart the dm multipath daemon by typing service multipathd restart, as shown in the next example. # service multipathd restart Stopping multipathd daemon: Starting multipathd daemon:

[ OK ] [ OK ]

To generate detailed return messages, administrators can type “multipath -ll”. The “multipath -ll” command will list all LUNs by WWID with their multipath device names and the individual paths used to create the multipath. The next example shows Oracle VM’s default multipath.conf file. # # # # # # This is a basic configuration file with some examples, for device mapper multipath. For a complete list of the default configuration values, see /usr/share/doc/device-mapper-multipath-0.4.7/multipath.conf.defaults For a list of configuration options with descriptions, see /usr/share/doc/device-mapper-multipath-0.4.7/multipath.conf.annotated

# Blacklist all devices by default. Remove this to enable multipathing # on the default devices. blacklist { devnode "*" } ## By default, devices with vendor = "IBM" and product = "S/390.*" are ## blacklisted. To enable multipathing on these devices, uncomment the ## following lines. #blacklist_exceptions { # device { # vendor "IBM" # product "S/390.*" # } #} ## Use user friendly names, instead of using WWIDs as names. defaults { user_friendly_names yes

151 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

} ## ## Here is an example of how to configure some standard options. ## # #defaults { # udev_dir /dev # polling_interval 10 # selector "round-robin 0" # path_grouping_policy multibus # getuid_callout "/sbin/scsi_id -g -u -s /block/%n" # prio_callout /bin/true # path_checker readsector0 # rr_min_io 100 # max_fds 8192 # rr_weight priorities # failback immediate # no_path_retry fail # user_friendly_names yes #} ## ## The wwid line in the following blacklist section is shown as an example ## of how to blacklist devices by wwid. The 2 devnode lines are the ## compiled in default blacklist. If you want to blacklist entire types ## of devices, such as all scsi devices, you should use a devnode line. ## However, if you want to blacklist specific devices, you should use ## a wwid line. Since there is no guarantee that a specific device will ## not change names on reboot (from /dev/sda to /dev/sdb for example) ## devnode lines are not recommended for blacklisting specific devices. ## #blacklist { # wwid 26353900f02796769 # devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*" # devnode "^hd[a-z]" #} #multipaths { # multipath { # wwid 3600508b4000156d700012000000b0000 # alias yellow # path_grouping_policy multibus # path_checker readsector0 # path_selector "round-robin 0" # failback manual # rr_weight priorities # no_path_retry 5 # } # multipath { # wwid 1DEC_____321816758474 # alias red # } #} #devices { # device { # vendor "COMPAQ " # product "HSV110 (C)COMPAQ" # path_grouping_policy multibus # getuid_callout "/sbin/scsi_id -g -u -s /block/%n" # path_checker readsector0 # path_selector "round-robin 0" # hardware_handler "0" # failback 15 # rr_weight priorities # no_path_retry queue # } # device { # vendor "COMPAQ " # product "MSA1000 " # path_grouping_policy multibus # } #} Note: If you do not have a working multipath.conf file please reference Appendix A for example multipath.conf files.

152 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Once you have a working multipath.conf file and have restarted the multipath daemon you can list the mapped devices in the /dev/mapper directory. Next we will access dom0 as root and type “ll /dev/mapper” to view the mapped devices, as shown in the following example. # ll /dev/mapper total 0 crw------- 1 root root 10, 62 Jan 23 16:44 control brw-rw---- 1 root disk 253, 0 Jan 23 17:01 mpath0 The "mpath0" entry validates that the mapped device is available from dom0. We can also list the mapped devices with their major and minor numbers by typing “dmsetup ls”. The minor numbers corresponds with the dm device name. In the following example the minor number of 0 corresponds to the multipath device /dev/dm-0. #dmsetup ls mpath0 (253, 0) We can also validate the mapped devices with the corresponding multipath devices by listing the /dev/mpath/ directory as shown in the next example. # ll /dev/mpath/ total 0 lrwxrwxrwx 1 root root 7 Jan 23 17:01 mpath0 -> ../dm-0 To list all the storage devices and the available paths type “multipath –l”, as shown in the next example. # multipath -l mpath0 (2000b080000002369) dm-0 Pillar,Axiom 600 [size=603G][features=0][hwhandler=0] \_ round-robin 0 [prio=0][enabled] \_ 7:0:0:0 sdb 8:16 [active][undef] \_ 7:0:1:0 sdc 8:32 [active][undef] \_ 8:0:0:0 sdd 8:48 [active][undef] The "multipath -l" command queries sysfs and the device mapper only, it does not invoke path checkers. The "multipath -ll" gets information from all relevant sources, including path checkers. In Step 3 we reviewed the syntax of a multipath.conf file and showed how to restart the multipath daemon to view the mapped devices, and the storage devices, along with the available paths. Step 4: Next we will create an OCFS2 storage repository on a LUN or LUNs from “one” Oracle VM server. We will format the OCSF partition from the pool master. The next example shows the syntax to format an OCFS2 volume. #mkfs.ocfs2 -L mylabel -Tdatafiles -N8 <device> In the above example, the mkfs.ocfs2 utility is used to format the device. The “-L parameter is optional and can be used to add a ” descriptive label to the OCFS2 volume. The “-Tdatafiles” parameter makes mkfs.ocfs2 choose the optimal filesystem parameters for the device. The -N parameter selects the number of slots. The number of slots determines the number of pool members that can concurrently mount the OCFS2 volume. The OCFS2 file system can support up to 255 nodes. For example, if your Oracle VM server pool will have 20 pool members, select -N20. The slot number can later be increased or decreased using the tunefs.ocfs2 utility. Next, from dom0, format an OCFS2 volume by typing “mkfs.ocfs2 -L root-sr -Tdatafiles -N16 /dev/mapper/mpath0”, as shown in the next example. Substitute root-sr with your desired label name and /dev/mapper/mpath0 with the proper device path for your environment. # mkfs.ocfs2 -L root-sr -Tdatafiles -N16 /dev/mapper /mpath0 mkfs.ocfs2 1.4.3 Cluster stack: classic o2cb Filesystem Type of datafiles Filesystem label=root-sr Block size=4096 (bits=12) Cluster size=1048576 (bits=20) Volume size=497142464512 (474112 clusters) (121372672 blocks) 15 cluster groups (tail covers 22528 clusters, rest cover

153 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

32256 clusters) Journal size=33554432 Initial number of node slots: 16 Creating bitmaps: done Initializing superblock: done Writing system files: done Writing superblock: done Writing backup superblock: 5 block(s) After the OCFS2 volume has been formatted you can use the mounted.ocfs2 utility to detect and list the OCFS2 volume. To detect and list the OCFS2 volume from the above example, type “mounted.ocfs2 –d” from dom0, as shown in the next example. # mounted.ocfs2 -d Device FS /dev/mapper/mpath0

UUID Label ocfs2 35c601e9-b0da-4950-9a90-ec0193baa205 root-sr

The above mounted.ocfs2 -d example lists the device name, the file system type, the UUID and the label. You can also use the full (-f) mode to lists the status of each OCFS2 volume. # mounted.ocfs2 -f Device /dev/mapper/mpath0

FS Nodes ocfs2 Not mounted

The above mounted.ocfs2 -f example lists the device name, the file system type, and the status of the node. Note: Sparse files and unwritten extents are activated by default when using Oracle VM 2.2’ mkfs.ocfs2 utility. If your system was upgraded from 2.1 to 2.2, it’s necessary to enable sparse files and unwritten extents using the following procedure. # umount <device> # tunefs.ocfs2 --fs-features=sparse,unwritten <device> To validate the enabled OCSF2 features, type “tunefs.ocfs2 -Q "%M %H %O\n" <device>” as shown in the next example. # tunefs.ocfs2 -Q "%M %H %O\n" <device> backup-super strict-journal-super sparse inline-data unwritten We have successfully formatted the /dev/mpath0 device with OCSF2 using the mkfs.ocfs2 utility as well as reviewed mounted.ocfs2 –d, mounted.ocfs2 -f and how to list the OCFS2 features using tunefs.ocfs2. Step 5: Next, on the pool master configure the root repository using the repos script. After the pool master is configured, configure all other pool members. 1. From dom0 type "/opt/ovs-agent-2.3/utils/repos.py -l" to list any configured storage repositories, as shown in the next example.

# /opt/ovs-agent-2.3/utils /repos.py -l # Typing "/opt/ovs-agent-2.3/utils/repos.py -l" should result with an empty entry. If a storage repository is listed, type "/opt/ovs-agent2.3/utils/repos.py -d UUID" to remove the repository. Next, unmout the storage repository in /var/ovs/mount/UUID by typing “umount /var/ovs/mount/UUID”. 2. Next, type "/opt/ovs-agent-2.3/utils/repos.py -n /dev/mpath0" to add the new device to the list of managed devices, as shown in the next example. Substitute /dev/sdb for the correct device path for your environment. # /opt/ovs-agent-2.3/utils/repos.py -n /dev/mapper/mpath0 [ NEW ] 002463a4-8998-4423-a797-8a1544739409 => /dev/mapper/mpath0 3. Next, type "/opt/ovs-agent-2.3/utils/repos.py -r UUID" to tag the storage repository as the root storage repository, as shown in the next example. # /opt/ovs-agent-2.3/utils/repos.py -r 002463a4-8998-4423a797-8a1544739409 [ R ] 002463a4-8998-4423-a797-8a1544739409 => /dev/mapper /mpath0

154 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Note: The UUID will be listed in step 2 or you can list the UUID by typing repos.py -l. 4. Next, "only" on the pool master type /opt/ovs-agent-2.3/utils/repos.py -i to mount the root storage repository, as shown in the next example. This step only needs to be performed on the pool master. # /opt/ovs-agent-2.3/utils/repos.py -i *** Storage repositories initialized. Note: When repos.py -i is run, the new storage repository will be mounted under /var/ovs/mount/UUID, although the new storage repository will not be linked to /OVS. The Oracle VM agent is responsible for mounting and linking storage repositories for pool members. Next, validate that the storage repository has been mounted by typing “mounted.ocfs2 -f“, as shown in the next example. # mounted.ocfs2 -f Device /dev//mapper/mpath0

FS Nodes ocfs2 ovs1.sf.itnc.com

You can also validate the ocfs2 mounts by typing mount|grep ocfs2, as shown in the next example. # mount|grep ocfs2 ocfs2_dlmfs on /dlm type ocfs2_dlmfs (rw) /dev/mapper/mpath0 on /var/ovs/mount /A85D7145957842F988293FDA43F8754D type ocfs2 (rw,_netdev,heartbeat=local) Typing df –h would also validate that the root storage repository is mounted, as shown in the next example. # df -h Filesystem Size Used Avail Use% Mounted on /dev/sda2 451G 961M 426G 1% / /dev/sda1 99M 46M 48M 49% /boot tmpfs 285M 0 285M 0% /dev/shm /dev/mapper/mpath0 463G 541M 463G 1% /var/ovs/mount/A33BF3D2E09B45F3931F830CB1A404AA 5. Next, create the pool in Oracle VM Manager. When creating the pool select the Oracle VM server that was used to format the OCFS2 file system as the pool master. 6. Next, add all of the other configured Oracle VM servers to the pool. Adding an Extended SAN Storage Repository To add storage to an Oracle VM storage repository, the first step is to provision the storage. The HBAs must be zoned and masked to the storage to be able to use the LUNs. Next, connect the pool master and format the storage using the steps outlined in Step 4. Subsequently, connect the other pool member to the storage using the /opt/ovs-agent-2.3/utils/repos.py script with the -n (new) followed by the -i (initialize, aka mount) switches, to add and then mount the sub storage repository. Finally, the new mount point in /var/ovs/mount/UUID needs to be linked to /OVS/UUID, by typing “ln -nsf /var/ovs/mount/<UUID>/ /OVS”, again from dom0. The end result is a root repository with an “extended” sub repository mounted under /var/ovs/mount/UUID which is linked to /OVS/UUID. Once a pool is configured, the Oracle VM agent will automatically place resources such as virtual machines, templates, or ISO files on the storage repository with available space. The Oracle VM agent is also responsible for mounting and linking storage repositories. iSCSI Storage Repository Configuration This section will review how to configure a root and an extended iSCSI storage repository with Oracle VM 2.2. Network interface bonding as well as dm-multipath may be used with iSCSI storage to provide multiple path support with Oracle VM. iSCSI Back-end Storage Prerequisites: Create the LUN(s) Create the masking rules iSCSI Oracle VM Server Prerequisites:
1. All Oracle VM servers must be patched from the Unbreakable Linux Network (ULN) to ensure that the storage configurations will not be hampered by unpatched bugs. 2. Select an Oracle VM server that will be used as the Oracle VM pool master. After the Oracle VM pool master and all the other Oracle VM pool members meet the prerequisites outlined in the following steps, access Oracle VM Manager and create an HA enabled pool using the Oracle VM pool master server. Note: An HA enabled pool automatically mounts and links root and extended storage repositories for each Oracle VM pool member that is added to a pool.

155 of 243

18.04.2011 16:35

The Underground Oracle VM Manual
3. Ensure that all the Oracle VM servers’ clocks are synchronized using NTP .

http://itnewscast.com/book/export/html/844

First, open the “/etc/ntp.conf” file by typing “vi /etc/ntp.conf” and validate that at least two available NTP servers entries are listed. The next example shows two bold NTP server entries in an ntp.conf file. # Use public servers from the pool.ntp.org project. # Please consider joining the pool (http://www.pool.ntp.org/join.html). server myntp1.com server myntp2.com Ping each NTP server listed in the ntp.conf file from each Oracle VM server to ensure network connectivity. Next, type "ntpstat" on each Oracle VM server to validate the NTP configuration. The next example shows the output from typing the ntpstat command on an Oracle VM server that has its time synchronized to an NTP server with the IP address of 192.168.4.251. # ntpstat synchronized to NTP server (192.168.4.251) at stratum 4 time correct to within 54 ms polling server every 1024 s Finally, validate that the time, date and time zone on each Oracle VM server as well as on the Oracle VM Manager host is synchronized by typing the "date" command. 4. All Oracle VM servers have consistent name resolution using DNS with both forward and reverse lookups. First, open the “/etc/resolv.conf” file by typing “vi /etc/resolv.conf” and validate that two available DNS servers are listed. The next example shows two DNS servers listed in a resolve.conf file. # vi /etc/resolve.conf nameserver <MY DNS SERVER1 IP ADDRESS> nameserver <MY DNS SERVER2 IP ADDRESS> From each Oracle VM server ping each DNS server listed in the resolv.conf file to ensure network connectivity. Next, validate the forward and reverse lookups for each Oracle VM pool member and the Oracle VM Manager host using the “host” command. For example, to validate server2's forward lookup from server1 type “host server2” as shown in the next example. # host server2 server2 has address 192.168.4.6 Next, to validate server2's reverse lookup from server1 type “host 192.168.4.6” as shown in the next example. # host 192.168.4.6 6.4.168.192.in-addr.arpa domain name pointer server2 Note: Using hosts files without DNS is not advised and may produce unpredictable results. 5. The Oracle VM server’s host name in the /etc/hosts file must be associated with the Oracle VM server's public IP address. If an Oracle VM pool member's host name is associated with 127.0.0.1, the cluster.conf file will be malformed and the Oracle VM pool will not be operational. The next example shows the improper syntax from an Oracle VM server's hosts file entry. 127.0.0.1 servername.com servername localhost.localdomain localhost 192.168.4.8 servername.com servername The next example shows the proper syntax for an Oracle VM server’s hosts file entry. 127.0.0.1 192.168.4.8 localhost.localdomain localhost servername.com servername

6. ocfs2 network connectivity between all Oracle VM server pool members must be operational before creating a multiple server pool. Check the ocfs2 network connectivity between all Oracle VM pool members by typing "nc -zv <myoraclevmserver1> 7777". For example, if you have two Oracle VM servers named ovs1 and ovs2, from ovs1 type "nc -zv ovs2 7777". Typing "nc -zv ovs2 7777" from ovs1 should return "succeeded!". If you receive a "failed: Connection refused" message between any Oracle VM servers, something (firewall, switch, router, cable, etc..) is restricting communication between the hosts. The iptables firewall on an Oracle VM server may be blocking the ocfs2 connectivity. If iptables is disabled and allowing all connections, the output from typing “iptables -L will look like the next example. ” # iptables -L

156 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Chain INPUT (policy ACCEPT) target prot opt source destination Chain FORWARD (policy ACCEPT) target prot opt source destination Chain OUTPUT (policy ACCEPT) target prot opt source destination If typing “iptables -L lists firewall rules, you can a) disable iptables by typing "service iptables stop && chkconfig iptables off" or b) add the following ” bold iptables rule to the /etc/sysconfig/iptables file prior to the last line on all Oracle VM pool members. -A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 7777 -j ACCEPT -A RH-Firewall-1-INPUT -j REJECT --reject-with icmp-host-prohibited COMMIT After you have added the above bold iptables rule, restart the iptables service by typing "service iptables restart". 7. If an Oracle VM server was originally installed with a local ocfs2 storage repository, it is necessary to remove and unmount the local ocfs2 storage repository before adding the Oracle VM server to a pool. To determine if an Oracle VM server is using a local storage repository type "/opt/ovs-agent2.3/utils/repos.py -l" to list all configured storage repositories. If a storage repository is listed, type "/opt/ovs-agent-2.3/utils/repos.py -d UUID" to remove the local repository from the Oracle VM server. Next, check if the local storage repository is still mounted under /var/ovs/mount/UUID. Type “mount |grep mount”, as shown in the next example to list the mounts. # mount |grep mount /dev/sda3 on /var/ovs/mount/62C757BA5E174DF7B5AB01BBAE0F765D type ocfs2 (rw,heartbeat=none) The above example shows that a storage repository /var/ovs/mount/62C757BA5E174DF7B5AB01BBAE0F765D is still mounted under /dev/sda3. Next, unmount the OCFS2 repository by typing “umount /var/ovs/mount/62C757BA5E174DF7B5AB01BBAE0F765D” as shown in the next example. # umount /var/ovs/mount /62C757BA5E174DF7B5AB01BBAE0F765D Tip: A default Oracle VM server installation dedicates the majority of the local disk to the OVS partition and create a small root partition. If your Oracle VM server was installed with the default OVS partition with a small root partition, consider rebuilding the server to create a disk layout that allocates the disk space to the root partition. Type “df -h” to list the size of an Oracle VM servers' partitions. Another consideration for small roots that include /var is the potential for large saves from the xendomains service. If an Oracle VM server crashes, the xendomains service will save the state (the memory foot print of each guest) of all running guests in the /var/lib/xen/save directory, which could fill up a small root partition. If xendomains functionality is not needed disable it. The next example shows how to disable or edit the location of the saved xendomains files. 1. Edit /etc/sysconfig/xendomains 2. find the section:

## Type: string ## Default: /var/lib/xen/save # # Directory to save running domains to when the system (dom0) is # shut down. Will also be used to restore domains from if # XENDOMAINS_RESTORE # is set (see below). Leave empty to disable domain saving on shutdown # (e.g. because you rather shut domains down). # If domain saving does succeed, SHUTDOWN will not be executed. # XENDOMAINS_SAVE=/var/lib/xen/save 3. Clear the XENDOMAINS_SAVE path to disable saves. Or point the XENDOMAINS_SAVE path to a partition with available space. 4. Rest the xendomains services by typing “service xendomains restart” to enable a new configuration. 8. If an Oracle VM server have previously been added to an Oracle VM server pool, the Oracle VM server's cluster configurations will need to be cleaned before added to a new Oracle VM server pool. To clean an Oracle VM server's cluster configurations it is necessary to a) empty the /etc/ocfs/cluster.conf file b) delete and recreate the local BerkleyDB and c) run the cleanup.py script to stop o2cb heartbeat, offline o2cb, remove o2cb configuration file, umount ovs-agent storage repositories and to cleanup ovs-agent local database. To clear /etc/ocfs/cluster.conf file type “cat /dev/null> /etc/ocfs2/cluster.conf” from dom0, as shown in the next example. # cat /dev/null> /etc/ocfs2 /cluster.conf To remove the local BerkleyDB first type “service ovs-agent stop”, which stops the Oracle VM agent. Next, type “rm -fr /etc/ovs-

157 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

agent/db/*” to delete the BerkleyDB. Finally, type “service ovs-agent start” to start the Oracle VM agent, which also recreate a new local BerkleyDB. To stop the o2cb heartbeat, offline o2cb, remove o2cb configuration file, unmount ovs-agent storage repositories and to cleanup ovs-agent local database, type "/opt/ovs-agent-2.3/utils/cleanup.py" and then type “y” as shown in the next example. # /opt/ovs-agent-2.3/utils/cleanup.py This is a cleanup script for ovs-agent. It will try to do the following: *) stop o2cb heartbeat *) offline o2cb *) remove o2cb configuration file *) umount ovs-agent storage repositories *) cleanup ovs-agent local database Would you like to continue? [y/N] y Cleanup done.
Step 1: The first step is to validate that the iscsi service is running on each Oracle VM server. Access dom0 as root and check the status of the iscsi service by typing “service iscsi status”, as shown in the next example.

# service iscsi status iscsid (pid 2314 2313) is running... If the iscsi service is not running, start the iscsi service by typing “service iscsi start” as shown in the next example. # service iscsi start We have successfully validated that the iscsi service is running on each Oracle VM pool member. Step 2: Next, on each Oracle VM server discover the iSCSI LUNs using the iscsiadm utility. Once the iSCSI LUNs have been discovered, if necessary, remove any entries that will not be used. Next, we will verify that the unused LUNs are removed. List 1 shows the procedure to discover, remove and validate iSCSI LUNs. 1. First, from dom0 type “iscsiadm -m discovery -t sendtargets -p iSCSI-Target-IPADDRESS”, to discover the entries from your iSCSI target. Substitute “iSCSI-Target-IPADDRESS” with the IP address or FQDN of your iSCSI target. 2. Second, remove any unused entries by typing “iscsiadm -m node -p iSCSI Qualified Name -o delete”, for example, iscsiadm -m node -p 192.168.4.10:3260,1 -T iqn.2006-01.com.openfiler:tsn.a83c0838952c -o delete. 3. Finally, validate that only the desired LUNs are discovered by typing “iscsiadm -m node”. As shown in the next example, the output from “iscsiadm -m discovery -t sendtargets -p 192.168.4.10” lists two entries. The first entry, 192.168.4.10:3260,1 iqn.2006-01.com.openfiler:tsn.a83c0838952c will be removed. The second entry, 192.168.4.10:3260,1 iqn.2006-01.com.openfiler:tsn.db29e77712c0 will become the root repository. # iscsiadm -m discovery -t sendtargets -p 192.168.4.10 192.168.4.10:3260,1 iqn.2006-01.com.openfiler:tsn.a83c0838952c 192.168.4.10:3260,1 iqn.2006-01.com.openfiler:tsn.db29e77712c0 Note: Discovered LUNs will appear in /proc/partitions only after restarting the iscsi service. In general, if your Oracle VM server lists entries that you will not use, you will need to remove all of the entries. To remove an unused entry, for example type “iqn.2006-01.com.openfiler:tsn.a83c0838952c” type “iscsiadm -m node -p 192.168.4.10:3260,1 -T iqn.2006-01.com.openfiler:tsn.a83c0838952c -o delete ”, as shown in the next example. #iscsiadm -m node -p 192.168.4.10:3260,1 -T iqn.2006-01.com.openfiler:tsn.a83c0838952c -o delete The next example shows how to verify that the unused entry has been removed by typing “iscsiadm -m node". # iscsiadm -m node 192.168.4.10:3260,1 iqn.2006-01.com.openfiler:tsn.aa231ffd6ef2 As shown in the above example only one entry is listed.

158 of 243

18.04.2011 16:35

The Underground Oracle VM Manual
In Step 2 we reviewed how to discover, remove and validate iSCSI LUNs.

http://itnewscast.com/book/export/html/844

Step 3: Next, on each Oracle VM server list /proc/partitions to review each Oracle VM server’s devices. After we review the devices listed in /proc/partitions, restart the iscsi service to mount the discovered iSCSI LUN. After the iSCSI service is restarted, the iSCSI LUN will be listed in /proc/partitions as a new device, i.e. sdb. Before we restart the iscsi service, review the devices listed on each Oracle VM server by typing “cat /proc/partitions”, as shown in the next example. # cat /proc/partitions major minor #blocks name 8 8 8 sda2 8 sda3 0 488386584 sda 1 104391 sda1 2 487227352 3 1052257

Note the sda, sda1, sda2 and the sda3 devices. Next, type “service iscsi restart” to restart the iscsi service, which also mounts the discoved LUN. # service iscsi restart Stopping iSCSI daemon: iscsid dead but pid file exists Turning off network shutdown. Starting iSCSI daemon:

[ OK ] [ OK ] [ OK ] Setting up iSCSI targets: Logging in to [iface: default, target: iqn.2006-01.com.openfiler:tsn.db29e77712c0, portal: 192.168.4.10,3260] Login to [iface: default, target: iqn.2006-01.com.openfiler:tsn.db29e77712c0, portal: 192.168.4.10,3260]: successful [ OK ] # After the iscsi service is restarted, any discovered iSCSI LUNs will be listed in /proc/partitions as a new device, as shown in the next example. # cat /proc/partitions major minor #blocks name 8 8 8 sda2 8 sda3 8 sdb 0 488386584 sda 1 104391 sda1 2 487227352 3 1052257

16 485490688

Note that the LUN i.e. the sdb device is listed in /proc/partitions. Now the new device can be partitioned using the mkfs.ocfs2 utility. In Step 3 we reviewed /proc/partitions on each Oracle VM server. Next, we restarted the iscsi service which mounted the discovered iSCSI LUN. After the iSCSI LUN was mounted, we validated that the new device was listed in /proc/partitions. Step 4: Next, “only on the pool master” you have to format an OCFS2 volume on the new device (the iSCSI LUN), using the mkfs.ocfs2 utility. The OCSF2 volume should be formatted “only” from one Oracle VM server, i.e. the pool master server. Note: If you already have created a server pool, format the OCSF2 volume “only” on the server pool master server. If you have not created a server pool in Oracle VM manager, use the Oracle VM server that you will select as the pool master to format the OCFS2 volume. The next example shows the syntax to format an OCFS2 volume. #mkfs.ocfs2 -L mylabel -Tdatafiles -N8 /dev/sdx

159 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

In the above example, the mkfs.ocfs2 utility is used to format the device. The “-L parameter is optional and can be used to add a ” descriptive label to the OCFS2 volume. The “-Tdatafiles” parameter makes mkfs.ocfs2 chose the optimal filesystem parameters for the device. The -N parameter selects the number of slots. The number of slots determines the number of pool members that can concurrently mount the OCFS2 volume. The OCFS2 file system can support up to 255 nodes. For example, if your Oracle VM server pool will have 20 pool members, select -N20. The slot number can later be increased or decreased using the tunefs.ocfs2 utility. Next, from dom0, format an OCFS2 volume by typing “mkfs.ocfs2 -L root-sr -Tdatafiles -N16 /dev/sdb”, as shown in the next example. Substitute root-sr with your desired label name and /dev/sdb with the proper device path for your enviroment. # mkfs.ocfs2 -L root-sr -Tdatafiles -N16 /dev/sdb mkfs.ocfs2 1.4.3 Cluster stack: classic o2cb Filesystem Type of datafiles Filesystem label=root-sr Block size=4096 (bits=12) Cluster size=1048576 (bits=20) Volume size=497142464512 (474112 clusters) (121372672 blocks) 15 cluster groups (tail covers 22528 clusters, rest cover 32256 clusters) Journal size=33554432 Initial number of node slots: 16 Creating bitmaps: done Initializing superblock: done Writing system files: done Writing superblock: done Writing backup superblock: 5 block(s) After the OCFS2 volume has been formatted you can use the mounted.ocfs2 utility to detect and list the OCFS2 volume. To detect and list the OCFS2 volume from the above example, type “mounted.ocfs2 –d” from dom0, as shown in the next example. # mounted.ocfs2 -d Device FS UUID Label /dev/sdb ocfs2 35c601e9-b0da-4950-9a90ec0193baa205 root-sr The above mounted.ocfs2 -d example lists the device name, the file system type, the UUID and the label. You can also use the full (-f) mode to lists the status of each OCFS2 volume. # mounted.ocfs2 -f Device FS Nodes /dev/sdb ocfs2 Not mounted The above mounted.ocfs2 -f example lists the device name, the file system type, and the status of the node. Note: Sparse files and unwritten extents are activated by default when using Oracle VM 2.2’ mkfs.ocfs2 utility. If your system was upgraded from 2.1 to 2.2, it’s necessary to enable sparse files and unwritten extents using the following procedure. # umount <device> # tunefs.ocfs2 --fs-features=sparse,unwritten <device> To validate the enabled OCSF2 features, type “tunefs.ocfs2 -Q "%M %H %O\n" <device>” as shown in the next example. # tunefs.ocfs2 -Q "%M %H %O\n" <device> backup-super strict-journal-super sparse inline-data unwritten We have successfully formatted the /dev/sdb volume with OCSF2 using the mkfs.ocfs2 utility as well as reviewed mounted.ocfs2 –d, mounted.ocfs2 -f and how to list the OCFS2 features using tunefs.ocfs2. Step 5: Next, on the pool master configure the root repository using the repos script. After the pool master is configured, configure all other pool members. 1. From dom0 type "/opt/ovs-agent-2.3/utils/repos.py -l" in order to list any configured storage repositories, as shown in the next example.

160 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

# /opt/ovs-agent-2.3/utils /repos.py -l # Typing "/opt/ovs-agent-2.3/utils/repos.py -l" should result with an empty entry. If a storage repository is listed, type "/opt/ovs-agent2.3/utils/repos.py -d UUID" to remove the repository. Next, unmout the storage repository in /var/ovs/mount/UUID by typing “umount /var/ovs/mount/UUID”. 2. Next, type "/opt/ovs-agent-2.3/utils/repos.py -n /dev/sdb" to add the new device to the list of managed devices, as shown in the next example. Substitute /dev/sdb for the correct device path for your environment. # /opt/ovs-agent-2.3/utils/repos.py -n /dev/sdb [ NEW ] 002463a4-8998-4423-a797-8a1544739409 => /dev/sdb 3. Next, type "/opt/ovs-agent-2.3/utils/repos.py -r UUID" to tag the storage repository as the root storage repository, as shown in the next example. # /opt/ovs-agent-2.3/utils/repos.py -r 002463a4-8998-4423a797-8a1544739409 [ R ] 002463a4-8998-4423-a797-8a1544739409 => /dev/sdb Note: The UUID will be listed in step 2 or you can list the UUID by typing repos.py -l. 4. Next, "only" on the pool master type /opt/ovs-agent-2.3/utils/repos.py -i to mount the root storage repository, as shown in the next example. This step only needs to be performed on the pool master. # /opt/ovs-agent-2.3/utils/repos.py -i *** Storage repositories initialized. Note: When repos.py -i is run, the new storage repository will be mounted under /var/ovs/mount/UUID, although the new storage repository will not be linked to /OVS. Next, validate that the storage repository has been mounted by typing “mounted.ocfs2 -f“, as shown in the next example. # mounted.ocfs2 -f Device FS Nodes /dev/sdb ocfs2 ovs2.sf.itnc.com You can also validate the ocfs2 mounts by typing mount|grep ocfs2, as shown in the next example. # mount|grep ocfs2 ocfs2_dlmfs on /dlm type ocfs2_dlmfs (rw) /dev/sdb on /var/ovs/mount/A85D7145957842F988293FDA43F8754D type ocfs2 (rw,_netdev,heartbeat=local) Typing df –h would also validate that the root storage repository is mounted, as shown in the next example. # df -h Filesystem Size Used Avail Use% Mounted on /dev/sda2 451G 961M 426G 1% / /dev/sda1 99M 46M 48M 49% /boot tmpfs 285M 0 285M 0% /dev/shm /dev/sdb 463G 541M 463G 1% /var/ovs/mount /A33BF3D2E09B45F3931F830CB1A404AA 5. Next, create the pool in Oracle VM Manager. When creating the pool select the Oracle VM server that was used to format the OCFS2 file system as the pool master. 6. Next, add all of the other configured Oracle VM servers to the pool. Adding an iSCSI Extended Storage Repository To add storage to an Oracle VM storage repository, the first step is to provision the storage. The storage will need to be zoned and masked to be able to use the LUNs. Next, discover the new device by typing “iscsiadm -m node -T target --rescan”. Note: If you restart the iscsi service to detect the new LUN, the iscsi service will log out of the existing storage repository, discover the new LUN and reboot the server. To avoid a reboot of your Oracle VM server use the rescan option with the iscsiadm utility, i.e.

161 of 243

18.04.2011 16:35

The Underground Oracle VM Manual
“iscsiadm -m node -T target --rescan”.

http://itnewscast.com/book/export/html/844

Next, connect the pool master and format the storage using the steps outlined in Step 4. Then, connect the other pool member to the storage using the /opt/ovs-agent-2.3/utils/repos.py script with the -n (new) followed by the -i (initialize, aka mount) switches, to add and then mount the sub storage repository. Finally, the new mount point in /var/ovs/mount/UUID needs to be linked to /OVS/UUID, by typing “ln -nsf /var/ovs/mount/<UUID>/ /OVS”, again from dom0. The end result is a root repository with an “extended” sub repository mounted under /var/ovs/mount/UUID which is linked to /OVS/UUID. Once a pool is configured, the Oracle VM agent will automatically place resources such as virtual machines, templates, or ISO files on the storage repository with available space. The Oracle VM agent is also responsible for mounting and linking storage repositories. NFS Storage Repository Configuration This section will review how to configure a root and an extended NFS storage repository with Oracle VM 2.2. NFS Back-end Storage Prerequisites: Create the LUN(s) Create the masking rules The NFS share must have the “no_root_squash” option enabled. NFS Oracle VM Server Prerequisites: 1. All Oracle VM servers must be patched from the Unbreakable Linux Network (ULN) to ensure that the storage configurations will not be hampered by unpatched bugs. 2. Select an Oracle VM server that will be used as the Oracle VM pool master. After the Oracle VM pool master and all the other Oracle VM pool members meet the prerequisites outlined in the following steps, access Oracle VM Manager and create an HA enabled pool using the Oracle VM pool master server. Note: An HA enabled pool automatically mounts and links root and extended storage repositories for each Oracle VM pool member that is added to a pool. 3. Ensure that all the Oracle VM servers’ clocks are synchronized using NTP . First, open the “/etc/ntp.conf” file by typing “vi /etc/ntp.conf” and validate that at least two available NTP servers entries are listed. The next example shows two bold NTP server entries in an ntp.conf file. # Use public servers from the pool.ntp.org project. # Please consider joining the pool (http://www.pool.ntp.org/join.html). server myntp1.com server myntp2.com Ping each NTP server listed in the ntp.conf file from each Oracle VM server to ensure network connectivity. Next, type "ntpstat" on each Oracle VM server to validate the NTP configuration. The next example shows the output from typing the ntpstat command on an Oracle VM server that has its time synchronized to an NTP server with the IP address of 192.168.4.251. # ntpstat synchronized to NTP server (192.168.4.251) at stratum 4 time correct to within 54 ms polling server every 1024 s Finally, validate that the time, date and time zone on each Oracle VM server as well as on the Oracle VM Manager host is synchronized by typing the "date" command. 4. All Oracle VM servers have consistent name resolution using DNS with both forward and reverse lookups. First, open the “/etc/resolv.conf” file by typing “vi /etc/resolv.conf” and validate that two available DNS servers are listed. The next example shows two DNS servers listed in a resolve.conf file. # vi /etc/resolve.conf nameserver <MY DNS SERVER1 IP ADDRESS> nameserver <MY DNS SERVER2 IP ADDRESS> From each Oracle VM server ping each DNS server listed in the resolv.conf file to ensure network connectivity. Next, validate the forward and reverse lookups for each Oracle VM pool member and the Oracle VM Manager host using the “host” command. For example, to validate server2's forward lookup from server1 type “host server2” as shown in the next example. # host server2 server2 has address 192.168.4.6 Next, to validate server2's reverse lookup from server1 type “host 192.168.4.6” as shown in the next example.

162 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

# host 192.168.4.6 6.4.168.192.in-addr.arpa domain name pointer server2 Note: Using hosts files without DNS is not advised and may produce unpredictable results. 5. The Oracle VM server’s host name in the /etc/hosts file must be associated with the Oracle VM server's public IP address. If an Oracle VM pool member's host name is associated with 127.0.0.1, the cluster.conf file will be malformed and the Oracle VM pool will not be operational. The next example shows the improper syntax from an Oracle VM server's hosts file entry. 127.0.0.1 servername.com servername localhost.localdomain localhost 192.168.4.8 servername.com servername The next example shows the proper syntax for an Oracle VM server’s hosts file entry. 127.0.0.1 localhost 192.168.4.8 servername localhost.localdomain servername.com

6. ocfs2 network connectivity between all Oracle VM server pool members must be operational before creating a multiple server pool. Check the ocfs2 network connectivity between all Oracle VM pool members by typing "nc -zv <myoraclevmserver1> 7777". For example, if you have two Oracle VM servers named ovs1 and ovs2, from ovs1 type "nc -zv ovs2 7777". Typing "nc -zv ovs2 7777" from ovs1 should return "succeeded!". If you receive a "failed: Connection refused" message between any Oracle VM servers, something (firewall, switch, router, cable, etc..) is restricting communication between the hosts. The iptables firewall on an Oracle VM server may be blocking the ocfs2 connectivity. If iptables is disabled and allowing all connections, the output from typing “iptables -L will look like the next example. ” # iptables -L Chain INPUT (policy ACCEPT) target prot opt source destination Chain FORWARD (policy ACCEPT) target prot opt source destination Chain OUTPUT (policy ACCEPT) target prot opt source destination If typing “iptables -L lists firewall rules, you can a) disable iptables by typing "service iptables stop && chkconfig iptables off" or b) ” add the following bold iptables rule to the /etc/sysconfig/iptables file prior to the last line on all Oracle VM pool members. -A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 7777 -j ACCEPT -A RH-Firewall-1-INPUT -j REJECT --reject-with icmp-host-prohibited COMMIT After you have added the above bold iptables rule, restart the iptables service by typing "service iptables restart". 7. If an Oracle VM server was originally installed with a local ocfs2 storage repository, it is necessary to remove and unmount the local ocfs2 storage repository before adding the Oracle VM server to a pool. To determine if an Oracle VM server is using a local storage repository type "/opt/ovs-agent-2.3/utils/repos.py -l" to list all configured storage repositories. If a storage repository is listed, type "/opt/ovs-agent-2.3/utils/repos.py -d UUID" to remove the local repository from the Oracle VM server. Next, check if the local storage repository is still mounted under /var/ovs/mount/UUID. Type “mount |grep mount”, as shown in the next example to list the mounts. # mount |grep mount /dev/sda3 on /var/ovs/mount/62C757BA5E174DF7B5AB01BBAE0F765D type ocfs2 (rw,heartbeat=none) The above example shows that a storage repository /var/ovs/mount/62C757BA5E174DF7B5AB01BBAE0F765D is still mounted under /dev/sda3. Next, unmount the OCFS2 repository by typing “umount /var/ovs/mount/62C757BA5E174DF7B5AB01BBAE0F765D” as shown in the next example. # umount /var/ovs/mount /62C757BA5E174DF7B5AB01BBAE0F765D Tip: A default Oracle VM server installation dedicates the majority of the local disk to the OVS partition and create a small root partition. If your Oracle VM server was installed with the default OVS partition with a small root partition, consider rebuilding the server to create a disk layout that allocates the disk space to the root partition.

163 of 243

18.04.2011 16:35

The Underground Oracle VM Manual
Type “df -h” to list the size of an Oracle VM servers' partitions.

http://itnewscast.com/book/export/html/844

Another consideration for small roots that include /var is the potential for large saves from the xendomains service. If an Oracle VM server crashes, the xendomains service will save the state (the memory foot print of each guest) of all running guests in the /var/lib /xen/save directory, which could fill up a small root partition. If xendomains functionality is not needed disable it. The next example shows how to disable or edit the location of the saved xendomains files. 1. Edit /etc/sysconfig/xendomains 2. find the section: ## Type: string ## Default: /var/lib/xen/save # # Directory to save running domains to when the system (dom0) is # shut down. Will also be used to restore domains from if # XENDOMAINS_RESTORE # is set (see below). Leave empty to disable domain saving on shutdown # (e.g. because you rather shut domains down). # If domain saving does succeed, SHUTDOWN will not be executed. # XENDOMAINS_SAVE=/var/lib/xen/save 3. Clear the XENDOMAINS_SAVE path to disable saves. Or point the XENDOMAINS_SAVE path to a partition with available space. 4. Rest the xendomains services by typing “service xendomains restart” to enable a new configuration. 8. If an Oracle VM server have previously been added to an Oracle VM server pool, the Oracle VM server's cluster configurations will need to be cleaned before added to a new Oracle VM server pool. To clean an Oracle VM server's cluster configurations it is necessary to a) empty the /etc/ocfs/cluster.conf file b) delete and recreate the local BerkleyDB and c) run the cleanup.py script to stop o2cb heartbeat, offline o2cb, remove o2cb configuration file, umount ovs-agent storage repositories and to cleanup ovs-agent local database. To clear /etc/ocfs/cluster.conf file type “cat /dev/null> /etc/ocfs2/cluster.conf” from dom0, as shown in the next example. # cat /dev/null> /etc/ocfs2 /cluster.conf To remove the local BerkleyDB first type “service ovs-agent stop”, which stops the Oracle VM agent. Next, type “rm -fr /etc/ovsagent/db/*” to delete the BerkleyDB. Finally, type “service ovs-agent start” to start the Oracle VM agent, which also recreate a new local BerkleyDB. To stop the o2cb heartbeat, offline o2cb, remove o2cb configuration file, unmount ovs-agent storage repositories and to cleanup ovs-agent local database, type "/opt/ovs-agent-2.3/utils/cleanup.py" and then type “y” as shown in the next example. # /opt/ovs-agent-2.3/utils/cleanup.py This is a cleanup script for ovs-agent. It will try to do the following: *) stop o2cb heartbeat *) offline o2cb *) remove o2cb configuration file *) umount ovs-agent storage repositories *) cleanup ovs-agent local database Would you like to continue? [y/N] y Cleanup done.
Step 1: On the pool master configure the root repository using the repos script. After the pool master is configured, configure all other pool members.

1.

From dom0 type "/opt/ovs-agent-2.3/utils/repos.py -l" to list any configured storage repositories, as shown in the next example.

# /opt/ovs-agent-2.3/utils /repos.py -l # Typing "/opt/ovs-agent-2.3/utils/repos.py -l" should result with an empty entry. If a storage repository is listed, type "/opt/ovs-agent2.3/utils/repos.py -d UUID" to remove the repository. Next, unmout the storage repository in /var/ovs/mount/UUID by typing “umount /var/ovs/mount/UUID”. 2. Next, type "/opt/ovs-agent-2.3/utils/repos.py -n nfsserver:/mnt/vol1/ovs-root/" to add the new share, as shown in the next example. Substitute nfsserver with the FQDN or IP address of your filer and substitute :/path/to/ovs-root-share with the path to your NFS share.

164 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

# /opt/ovs-agent-2.3/utils/repos.py -n 192.168.4.10:/mnt/vg-931 /nfs-root/nfs-sr/ [ NEW ] ef47b8a9-620f-4ed1-aac1-7ac10f4f7fcf => 192.168.4.10:/mnt/vg-931/nfs-root/nfs-sr/ 3. Next, type "/opt/ovs-agent-2.3/utils/repos.py -r UUID" to tag the storage repository as the root storage repository, as shown in the next example. # /opt/ovs-agent-2.3/utils/repos.py -r ef47b8a9-620f4ed1-aac1-7ac10f4f7fcf [ R ] ef47b8a9-620f-4ed1-aac1-7ac10f4f7fcf => 192.168.4.10:/mnt/vg-931/nfs-root/nfs-sr/ Note: The UUID will be listed in step 2 or you can list the UUID by typing repos.py -l. 4. Next, type /opt/ovs-agent-2.3/utils/repos.py -i to mount the root storage repository, as shown in the next example. This step only needs to be performed on the pool master. When Oracle VM pool members are added to a pool the agent will mount and link the root storage repository. # /opt/ovs-agent-2.3/utils/repos.py -i *** Storage repositories initialized. Note: When repos.py -i is run, the new storage repository will be mounted under /var/ovs/mount/UUID, although the new storage repository will not be linked to /OVS. Once the pool is created the Oracle VM agent will auto mount and link the storage repository. 5. Next, create the pool in Oracle VM Manager. When creating the pool select the Oracle VM server that was used to format the OCFS2 file system as the pool master. 6. Next, add all of the other configured Oracle VM servers to the pool. Adding an Extended NFS Storage Repository To add storage to an Oracle VM storage repository, the first step is to provision the storage. The storage needs to be zoned and masked before it can be added as an extended storage repository. Next, connect the pool master and all the pool members to the share using the /opt/ovs-agent-2.3/utils/repos.py script with the -n (new) as outlined in Step 2. Next, type /opt/ovs-agent-2.3/utils /repos.py -i mount the sub storage repository. Finally the new mount point in /var/ovs/mount/UUID needs to be linked to /OVS/UUID, by typing “ln -nsf /var/ovs/mount/<UUID>/ /OVS”, again from dom0. The end result is a root repository with an “extended” sub repository mounted under /var/ovs/mount/UUID which is linked to /OVS/UUID. Oracle VM Guest Front-end Storage This section will review the virtual machine layer of the storage stack. The virtual machine layer is where the storage is presented to virtual machines as either a flat file, as a LUN, or as a combination of flat files and LUNs. The virtual machine storage layer is referred to as the guest front-end storage. The section starts with a review of file-backed block devices and the file-backed block device driver options. The section concludes with a review of physical backed block devices. File-backed block device A file-backed block device uses a flat file in the storage repository as the guest’s primary storage. By default, Oracle VM Manager and the Oracle VM Management Pack create a file named System.img for each guest. For example, a guest named racnode1 would have a directory named xxx_racnode1 in the in the /OVS/*_pool/ xxx_racnode1/ directory that contains the System.img file. By default, Oracle VM Manager and the Oracle VM Management Pack configure guest storage as file-backed block device using the fast-loopback driver in dom0. You can validate that a guest is using a file-backed block device by looking at the 'disk =' directive in a guest’s vm.cfg file. Each guest has a vm.cfg file in the /OVS/*_pool/vmname/ directory. A 'file:' reference indicates the use of the fast-loopback driver. File-backed block devices can use one of two drivers a) the default fast-loopback driver or b) the blktap driver. In certain circumstances, the blktap driver may provide better performance than the fast-loopback driver. Oracle VM Manager and the Oracle VM Management Pack do not support editing the file-backed block device driver settings. To test the blktap driver you must edit the 'file' directive by hand in the desired guest’s vm.cfg file from ‘file’ to 'tap:aio:'. The next example shows a vm.cfg file from an 11g Oracle VM template that is configured with two virtual disks using file-backed block devices with the fast-loopback driver. The first of two virtual disks is defined in the 'disk =' directive contains the OS and the second virtual disk defined in the 'disk =' directive is an ASM disk. bootloader = '/usr/bin/pygrub' disk = ['file:/OVS/running_pool/v52x8611g1/System.img,xvda,w', 'file:/OVS/running_pool/v52x8611g1/oracle11g_x86_asm.img,xvdb,w', ] memory = '2048' name = 'v52x8611g1' on_crash = 'restart'

165 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

on_reboot = 'restart' uuid = 'f8725f79-c6c8-26d4-a51e-1f32cf010c84' vcpus = 2 vif = ['bridge=xenbr0,mac=00:16:3E:56:20:63,type=netfront'] vif_other_config = [] The next example shows the same vm.cfg file from the above 11g Oracle VM template that is configured using file-backed block devices with the blktap driver. The first of two virtual disks is defined in the 'disk =' directive contains the OS and the second virtual disk defined in the 'disk =' directive is an ASM disk. bootloader = '/usr/bin/pygrub' disk = ['tap:aio:/OVS/running_pool/v52x8611g1/System.img,xvda,w', 'tap:aio:/OVS/running_pool/v52x8611g1/oracle11g_x86_asm.img,xvdb,w', ] memory = '2048' name = 'v52x8611g1' on_crash = 'restart' on_reboot = 'restart' uuid = 'f8725f79-c6c8-26d4-a51e-1f32cf010c84' vcpus = 2 vif = ['bridge=xenbr0,mac=00:16:3E:56:20:63,type=netfront'] vif_other_config = [] You can quickly benchmark guest front-end performance by typing “hdparm -tT <device>” within the guest. Replace <device> with the device you would like to benchmark. Type df to list the devices. For example, if a guest is using the default file-backed block device with the fast-loopback driver, as root from the guest console type “hdparm -tT <device>” to gather cached reads and buffered disk reads statistics, as shown in the next example. # hdparm -tT /dev/xvda2 /dev/xvda2: Timing cached reads: 26968 MB in 1.99 seconds = 13536.94 MB/sec Timing buffered disk reads: 148 MB in 3.03 seconds = 48.87 MB/sec Record the data from “hdparm -tT <device>” and power off the guest. Once the guest is powered of edit the guest’s vm.cfg file and replace the file directive with tap:aio. Power on the guest and run the same “hdparm -tT <device>” command to gather the cached reads and the buffered disk reads statistics using the with the blktap driver. Physical backed block device The second guest storage option is a physical backed block device. A physical backed block device offers the lowest overhead and best performance of the two Oracle VM guest storage options. In most cases, a physical backed block device will be the best option for hight I/O workloads. For example, Oracle’s certified Oracle VM RAC configuration uses physical backed block devices to provide the best performance for RAC. To use a physical backed block device, you export a physical block device e.g. a LUN from dom0 to the guest, as a virtual block device. As of this writing, Oracle VM Manager or the Oracle VM Management Pack cannot manage physical backed block devices. To use physical backed block devices with Oracle VM, you need to edit the guest’s vm.cfg file manually to use a physical backed block device. Note: Oracle VM 2.2 Manager can manage physical multipath devices as Shared Disks. The next example shows a vm.cfg file that uses physical backed block devices. The first of two disks is defined in the 'disk =' directive, and contains the OS, the second disk defined in the 'disk =' directive is an ASM disk. bootloader = '/usr/bin/pygrub' disk = ['phy:/dev/sdu,xvda,w!’, 'phy:/dev/sdv,xvda,w!’ ] memory = '2048' name = 'v52x8611g1' on_crash = 'restart' on_reboot = 'restart' uuid = 'f8725f79-c6c8-26d4-a51e-1f32cf010c84' vcpus = 2 vif = ['bridge=xenbr0,mac=00:16:3E:56:20:63,type=netfront'] vif_other_config = [] The next example shows a vm.cfg file from a guest that uses a file backed block device for the OS and eight physical backed block devices.

166 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

bootloader = '/usr/bin/pygrub' disk = ['file:/var/ovs/mount/ 4C42F6B3FCB841499D595C0CC36D7695/ running_pool/266_lax005112pvm07/System.img,xvda,w', 'phy:/dev/sda,/dev/xvda3,w!', 'phy:/dev/sdb,/dev/xvda4,w!', 'phy:/dev/sdc,/dev/xvda5,w!', 'phy:/dev/sdd,/dev/xvda6,w!', 'phy:/dev/sde,/dev/xvda7,w!', 'phy:/dev/sdf,/dev/xvda8,w!', 'phy:/dev/sdg,/dev/xvda9,w!', 'phy:/dev/sdh,/dev/xvda10,w!', ] keymap = 'en-us' memory = '4096' name = 'oel5u5pv07' on_crash = 'restart' on_reboot = 'restart' uuid = '4fa8e2f0-4514-5169-467a-7fd64fe62147' vcpus = 2 vfb = ['type=vnc,vncunused=1,vnclisten=0.0.0.0'] vif = ['bridge=xenbr0,mac=00:16:3E:69:E4:04,type=netfront', 'bridge=xenbr1,mac=00:16:3E:61:DE:5E,type=netfront', ] vif_other_config = [] Configuring a physical backed block device is a manual multi step process. List 5 shows the steps to configure a physical backed block device. 1. The first step is to create a guest using Oracle VM Manager. Please note, that after the guest is created, the default file backed block device, i.e. the System.img file can be used or deleted and replaced with a physical backed block device. 2. Provision a disk for the guest, i.e. one or more LUNs. The Oracle VM servers must be zoned and masked to be able to access the storage. 3. Configure the storage in each dom0. For example, if the guest will run on 4 servers within a pool, the storage must be configured in dom0 on all four servers. 4. Once the LUN is presented in dom0, export the LUN to the guest by editing the vm.cfg file using a physical backed block device as show in the above examples. Appendix A Example multipath.conf Files Appendix A lists five multipath.conf files from production Oracle VM systems. Appendix A starts with two example multipath.conf files for EMC CLARiiON and SYMMETRIX, followed with multipath.conf example files for Pillar Data Axiom 600, HP EVA SAN, IBM 2145 SAN and a 3PAR 224MU6 Storage Array. 1. EMC CLARiiON and SYMMETRIX Example 1 – Quick and Simple #vi /etc/multipath.conf devnode_blacklist { devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*" devnode "^hd[a-z]" } defaults { user_friendly_names yes }

2. EMC CLARiiON and SYMMETRIX Example 2 – Verbose #vi /etc/multipath.conf ## This is the /etc/multipath.conf file recommended for ## EMC storage devices. ## ## OS : RHEL5 ## Arrays : CLARiiON and SYMMETRIX ## Use user friendly names, instead of using WWIDs as names. defaults { user_friendly_names yes } ## The blacklist is the enumeration of all devices that are to be

167 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

## excluded from multipath control devnode_blacklist { ## Replace the wwid with the output of the command ## 'scsi_id -g -u -s /block/internal scsi disk name' ## Enumerate the wwid for all internal scsi disks. ## Optionally, the wwid of VCM database may also be listed here. # wwid 20010b9fd080b7321 devnode "sd[a]$" devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)0-9*" devnode "^hda-z" devnode "^cciss!c0-9d0-9*" } devices { ## Device attributes for EMC SYMMETRIX device { vendor "EMC " product "SYMMETRIX" path_grouping_policy multibus getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" features "0" hardware_handler "0" failback immediate } ## Device attributes for EMC CLARiiON device { vendor "DGC" product "*" path_grouping_policy group_by_prio getuid_callout "/sbin/scsi_id -g -u -s /block/%n" prio_callout "/sbin/mpath_prio_emc /dev/%n" hardware_handler "1 emc" features "1 queue_if_no_path" no_path_retry 300 path_checker emc_clariion failback immediate } }

3. Pillar Data Axiom 600 First add the following lines to /etc/modprobe.conf alias alias alias alias alias qla2100 qla2200 qla2300 qla2322 qla2400 qla2xxx qla2xxx qla2xxx qla2xxx qla2xxx

#vi /etc/multipath.conf blacklist { devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*" devnode "^hd[a-z]" devnode "^cciss!c[0-9]d[0-9]*" wwid } devices { device { vendor "Pillar" product "Axiom 600" getuid_callout "/sbin/scsi_id -g -u -s /block/%n" prio_callout "/sbin/mpath_prio_alua %n" features "0" hardware_handler "0" path_grouping_policy group_by_prio rr_weight priorities rr_min_io 1000 path_checker tur

168 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

} device { vendor "Pillar" product "Axiom 500" getuid_callout "/sbin/scsi_id -g -u -s /block/%n" prio_callout "/sbin/mpath_prio_alua_pillar %n" features "0" hardware_handler "0" path_grouping_policy group_by_prio rr_weight priorities rr_min_io 1000 path_checker tur }

4. HP EVA SAN # vi /etc/multipath.conf defaults { user_friendly_names yes } blacklist { devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*" devnode "^(hd|xvd)[a-z]*" wwid "*" } # Make sure our multipath devices are enabled. blacklist_exceptions { wwid "3600508b40008dc480000500000550000" wwid "3600508b40008dc4800005000005b0000" wwid "3600508b40008dc4800005000005e0000" wwid "3600508b40008dc480000500000670000" wwid "3600508b40008dc480000500000640000" wwid "3600508b40008dc480000500000610000" } multipath { wwid 3600508b40008dc4800005000005b0000 alias mpath1 } multipath { wwid 3600508b40008dc4800005000005e0000 alias mpath2 } multipath { wwid 3600508b40008dc480000500000610000 alias mpath3 } multipath { wwid 3600508b40008dc480000500000640000 } multipath { wwid 3600508b40008dc480000500000670000 alias mpath5 }

5. IBM 2145 SAN # vi /etc/multipath.conf devnode_blacklist { # wwid 26353900f02796769 devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*|sda" devnode "^hd[a-z]" } multipaths { multipath {

169 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

#

wwid 3600507680194011ef000000000000b46 alias oraclevm-lun0 path_grouping_policy failover path_checker readsector0 path_selector "round-robin 0" failback 0 rr_weight priorities no_path_retry 5

} multipath { wwid 3600507680194011ef000000000000dbd alias oraclevm-lun1 } multipath { wwid 3600507680194011ef000000000000dbe alias oraclevm-lun2 } multipath { wwid 3600507680194011ef000000000000dc4 alias oraclevm-lun3 } multipath { wwid 3600507680194011ef000000000000dbf alias oraclevm-lun4 } multipath { wwid 3600507680194011ef000000000000dc0 alias oraclevm-lun5 } multipath { wwid 3600507680194011ef000000000000eaf alias oraclevm-lun6 } multipath { wwid 3600507680194011ef000000000000eb0 alias oraclevm-lun7 } multipath { wwid 3600507680194011ef000000000000eb1 alias oraclevm-lun8 } multipath { wwid 3600507680194011ef000000000000eb2 alias oraclevm-lun9 } devices { # IBM 2145 device { vendor "IBM" product "2145" path_grouping_policy group_by_prio prio_callout "/sbin/mpath_prio_alua /dev/%n" } } 6. 3PAR 224MU6 Storage Array # Default Blacklist - Ignore Internal Devices blacklist { devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*" devnode "^hd[a-z]" devnode "^cciss!c[0-9]d[0-9]*" } defaults { udev_dir /dev polling_interval 10 selector "round-robin 0" path_grouping_policy multibus getuid_callout "/sbin/scsi_id -g -u -s /block/%n"

170 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

prio_callout /bin/true rr_min_io 100 rr_weight priorities failback immediate } devices { device { vendor "3PARdata" product "VV" path_grouping_policy multibus path_checker tur no_path_retry 60 } } Watch this space Oracle OpenWorld 2011 Call for Papers - Submit Today! Temporary Post Used For Theme Detection (409fc5e9-d7ee-4e1a-b645-8b82f256f434 - 3bfe001a-32de-4114-a6b4-4005b770f6d7) Debugging Oracle VM 2.2 & Oracle VM Manager 2.2 errors using the many log files New VDI Training Course now available! HIMSS 2011 and New Press Release Oracle Desktop Virtualization at HIMSS 2011 Oracle VM VirtualBox 4.0.4 Released! Two Virtualization Webinars This Week We're Hiring! - Server and Desktop Virtualization Product Management

Chapter 7: Oracle VM Networking from A to 802.1Q
Oracle VM Networking from A to 802.1Q
Last update 12-03-2010 Copyright © 2009 - 2011 Roddy Rodstein. All rights reserved. Show a printer friendly version of this book with its sub pages Chapter 7 covers the following topics: Oracle VM Networking Introduction Beware of Baseboard Management Controllers and Broadcom NetXtreme II NICs Oracle VM Guest Networking Oracle VM Server Networking Configuration Examples The Default ethX/xenbX Configuration 802.3AD NIC Bonding with Xen Bridges 802.3AD NIC Bonding with Xen Bridges using 802.1Q—VLANning 802.3AD NIC Bonding with a Single Xen Bridge Configuration 802.3AD NIC Bonding with Xen bridges and 802.1Q Configuration

Oracle VM Networking Introduction
Oracle VM ships with a flexible networking configuration that can be used as-is or modified to meet your business requirements. Oracle VM network configuration management is not supported by Oracle VM Manager or the Oracle VM Management Pack; it must therefore be performed by hand in dom0. Both 802.3AD NIC bonding and 802.1Q VLANning are supported, although they must also be configured by hand in dom0. The default Oracle VM server networking configuration routes all dom0 and guest traffic through a Xen bridge. A Xen bridge operates at layer 2 of the OSI model, effectively acting as a layer 2 (L2) switch passing packets to the egress port; it relies on the TCP protocol for rate control and packet loss. Table 1 shows the OSI model. Note that Xen bridges operate in layer 2. Layer Description 7 6 5 4 Application Layer Presentation Layer Session Layer Transport Layer

171 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

3 2

Network Layer Data Link Layer
LLC sublayer MAC sublayer

1

Physical Layer

Oracle VM's default network configuration pairs each network interface (NIC) with a Xen bridge. The Xen bridges are created by default by the Xen SysV startup script (/etc/init.d/xend). An Oracle VM server with two NICs will have two Xen bridges, eth0/xenbr0 and eth1/xenbr1. The first Xen bridge, eth0/xenbr0, is configured with an IP address on xenbr0 and is dedicated to Oracle VM management and HA traffic. The second Xen bridge will not have an IP address assigned; it effectively acts as a layer-two switch for guest traffic. Any Xen bridge, including the ent0/xenbr0 pair, can be used for guest traffic. Guest virtual NICs can be used with any Xen bridges by editing the guest's vm.cfg file or by editing the guest's network properties using Oracle VM Manager or the Oracle VM Management Pack. Tip: In an HA-enabled pool, the loss of network connectivity for the Oracle VM management interface causes an HA event. When an HA event occurs, an Oracle VM server is fenced from the pool and reboots, then all HA-enabled guests are restarted on a live Oracle VM pool member. It's your prerogative how to physically wire your Oracle VM server's NICs. For example, you could wire each NIC into a managed switch using VLANs to segregate the traffic from each Xen bridge. Another option is to wire your NICs into an unmanaged switch on a flat network, using the Xen bridges to distribute traffic across the switch. Figure 1 shows the default Oracle VM Xen bridge configuration: an Oracle VM server with two NICs wired into a single switch, hosting three guests.

As shown in Figure 1, the default Oracle VM server network configuration will pair eth0 with xenbr0 and use the eth0/xenbr0 pair for Oracle VM agent, OCFS2 cluster heartbeat, and Live Migration traffic, and pair eth1 with xenbr1. If an Oracle VM server had four or more NICs, the default configuration would pair eth0 with xenbr0, used for Oracle VM management traffic, and then create additional Xen bridges, that is, eth1/xenbr1, eth2/xenbr2, eth3/xenbr3, and so on. Any Xen bridge can be used for guest traffic, even eth0/xenbr0. In Figure 1, packets that arrive at the physical NIC (eth0) are handled by dom0’s Ethernet driver and appear on xenbr0. Xen bridges distribute packets like a layer-two switch for dom0 and guests. Xen bridges route guest packets based on the guest's MAC address. The next example shows the output from “ifconfig” and “brctl show” from an Oracle VM server with two NICs without any guests. # ifconfig eth0 Link encap:Ethernet HWaddr 00:30:48:7F:35:0A UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:17459615 errors:0 dropped:0 overruns:0 frame:0 TX packets:15661870 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:100 RX bytes:3807333630 (3.5 GiB) TX bytes:3092948689 (2.8 GiB) eth1 Link encap:Ethernet HWaddr 00:30:48:7F:35:0B UP BROADCAST RUNNING MULTICAST MTU:1500

172 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Metric:1 RX packets:93673 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:100 RX bytes:9902147 (9.4 MiB) TX bytes:0 (0.0 b) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:203179 errors:0 dropped:0 overruns:0 frame:0 TX packets:203179 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:52010419 (49.6 MiB) TX bytes:52010419 (49.6 MiB) xenbr0 Link encap:Ethernet HWaddr 00:30:48:7F:35:0A inet addr:192.168.4.8 Bcast:0.0.0.0 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:17379990 errors:0 dropped:0 overruns:0 frame:0 TX packets:15661772 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:3559117450 (3.3 GiB) TX bytes:3092938757 (2.8 GiB) xenbr1 Link encap:Ethernet HWaddr 00:30:48:7F:35:0B UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:14385 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:4797529 (4.5 MiB) TX bytes:0 (0.0 b) # brctl show bridge name bridge id STP enabled interfaces xenbr0 8000.0030487f350a no eth0 xenbr1 8000.0030487f350b no eth1 The output from “ifconfig” in the above example shows, eth0, eth1, lo, xenbr0, and xenbr1. Next, the output from “brctl show” shows the Xen bridge configurations, that is, xenbr0 is connected to eth0 and xenbr1 is connected to eth1. The Oracle VM server's dom0 interfaces are managed using native Linux network scripts, for example, /etc/sysconfig/network and /etc/sysconfig/network-scripts/ifcfg-*. After an Oracle VM Server 2.2 boots, the Xen /etc/xen/xend-config.sxp script (network-script network-bridge) entry is referenced and creates an independent Xen bridge for each physical NIC. For example, typing "brctl addif xenbr0 eth0" creates the eth0/xenbr0 pair and typing "brcrtl addif xenbr0 eth1" creates the eth1/xenbr1 pair. dom0's management interface is selected during the Oracle VM server installation and can be edited after the installation in the /etc/ovs-config file. From an HA perspective, the NIC used for Oracle VM management traffic is the only network interface that is monitored for HA events. If HA is a requirement for both the Oracle VM hosts and guests, it’s important to design your Oracle VM servers with a single bond used by all Xen bridges for dom0 and guests. The next example shows Oracle VM's default /etc/sysconfig/network-scripts/ifcfg* scripts from an Oracle VM server with two NICs. # cat /etc/sysconfig/network-scripts/ifcfg* # Intel Corporation 80003ES2LAN Gigabit Ethernet Controller (Copper) DEVICE=eth0 BOOTPROTO=static BROADCAST=192.168.4.255 HWADDR=00:30:48:7F:44:6E IPADDR=192.168.4.7 NETMASK=255.255.255.0 NETWORK=192.168.4.0 ONBOOT=yes # Intel Corporation 80003ES2LAN Gigabit Ethernet Controller (Copper) DEVICE=eth1 BOOTPROTO=static HWADDR=00:30:48:7F:44:6F ONBOOT=yes DEVICE=lo

173 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

IPADDR=127.0.0.1 NETMASK=255.0.0.0 NETWORK=127.0.0.0 # If you're having problems with gated making 127.0.0.0/8 a martian, # you can change this to something else (255.255.255.255, for example) BROADCAST=127.255.255.255 ONBOOT=yes NAME=loopback Note: Oracle VM 2.2 does not use the default Xen.org pseudo devices in dom0, for example, no vif or peth devices are used.

Beware of Baseboard Management Controllers and Broadcom NetXtreme II NICs
Beware of Baseboard Management Controllers that virtualize ethernet ports! Baseboard Management Controllers that virtualize ethernet ports may not work well with dom0's bonded interfaces and might result in broadcast storms on the connected switches. A broadcast storm could cause a switch to disable the ports that detect the broadcast storm. HA enabled Oracle VM servers connected to a switch port that gets disabled would promptly fence from the cluster, reboot and the HA enabled guests would restart on available pool members. Bonded Oracle VM server's that use the Broadcom NetXtreme II bnx2x driver on hardware with Baseboard Management Controllers are subject to broadcast storms. The solution to eliminate the broadcast storms is to a) change the network interface driver from the bnx2x driver to the tg3 driver or b) disable the Baseboard Management Controller's management firmware and/or c) both a and b. Typing “dmesg|grep Broad” from dom0 will validate if an Oracle VM server is using Broadcom NetXtreme II network interfaces. The next example shows the result from typing “dmesg|grep Broad” on an Oracle VM server using Broadcom NetXtreme II network interfaces.
# dmesg |grep Broad Broadcom NetXtreme II 5771x 10Gigabit Ethernet Driver bnx2x 1.52.12 ($DateTime: 2009/12/17 12:14:50 $) eth0: Broadcom NetXtreme II BCM57711E XGb (A0) PCI-E x4 5GHz (Gen2) found at mem fb000000, IRQ 22, node addr d8d385d91f88 eth1: Broadcom NetXtreme II BCM57711E XGb (A0) PCI-E x4 5GHz (Gen2) found at mem fa000000, IRQ 23, node addr d8d385d91f8c eth2: Broadcom NetXtreme II BCM57711E XGb (A0) PCI-E x4 5GHz (Gen2) found at mem f9000000, IRQ 23, node addr d8d385d91f89 eth3: Broadcom NetXtreme II BCM57711E XGb (A0) PCI-E x4 5GHz (Gen2) found at mem f8000000, IRQ 24, node addr d8d385d91f8d eth4: Broadcom NetXtreme II BCM57711E XGb (A0) PCI-E x4 5GHz (Gen2) found at mem f7000000, IRQ 24, node addr d8d385d91f8a eth5: Broadcom NetXtreme II BCM57711E XGb (A0) PCI-E x4 5GHz (Gen2) found at mem f6000000, IRQ 25, node addr d8d385d91f8e eth6: Broadcom NetXtreme II BCM57711E XGb (A0) PCI-E x4 5GHz (Gen2) found at mem f5000000, IRQ 25, node addr d8d385d91f8b eth7: Broadcom NetXtreme II BCM57711E XGb (A0) PCI-E x4 5GHz (Gen2) found at mem f4000000, IRQ 22, node addr d8d385d91f8f Broadcom NetXtreme II CNIC Driver cnic v1.9.13b (Dec 16, 2009) Broadcom NetXtreme II iSCSI Driver bnx2i v1.8.12f (Jan 19, 2010)

The next example shows an Oracle VM server using the bnx2x network interface driver for eth0 and eth1. # cat /etc/modprobe.conf alias eth0 bnx2x alias eth1 bnx2x alias scsi_hostadapter lpfc alias scsi_hostadapter1 usb-storage alias bond0 bonding options bonding mode=1 miimon=100 downdelay=200 updelay=200 max_bonds=1 To change the network interface driver from bnx2x to tg3, it is necessary to edit dom0's modprob.conf file as shown in the next example. # cat /etc/modprobe.conf alias eth0 tg3 alias eth1 tg3 alias scsi_hostadapter lpfc alias scsi_hostadapter1 usb-storage alias bond0 bonding options bonding mode=1 miimon=100 downdelay=200 updelay=200 max_bonds=1 After changing the network interface driver in the modprobe.conf file, restart the network service by typing “service network restart”. Typing “service network restart” will unload the bnx2x driver and load the tg3 driver. Broadcom has a DOS utility named xdiag that can be used to disable the Baseboard Management Controller's management firmware (MF) code. If typing "xdiag -ver" shows MF active" on a controller, it is necessary to type "xdiag -c <controller#> -mfw 0" to disable the controler. xdiag can be downloaded as a stand alone application or as a bootable ISO. The next example shows how to disable a Baseboard Management Controller's management firmware using a bootable ISO that contains xdiag. 1- Download and burn this image: http://www.broadcom.com/support/ethernet_nic/driver-sla.php?driver=NX2-diag 2- Boot the Oracle VM server using the bootable cdrom 3- At the boot prompt press enter to continue 4- Select 1 Install to hardisk using Freedos Setup 5- Press enter to select English 6- Run freedos from the ISO image 7- Disable the management firmware on each network interface as shown in the following example:

174 of 243

18.04.2011 16:35

The Underground Oracle VM Manual
>cd nx2 >uxdiag -c 1 -mfw 0 >uxdiag -c 2 -mfw 0 The above example disabled the management firmware on controllers 1 and 2.

http://itnewscast.com/book/export/html/844

Oracle VM Guest Networking
Oracle VM networking can be broken down into two parts. The first part of the networking configuration consists of the Oracle VM servers and the second part of the networking configuration is for the guests. As explained above, Oracle VM server networking is configured in dom0 using native Linux networking scripts along with the default Xen SysV startup script (/etc/init.d/xend). Guest networking is configured in each guest's vm.cfg file and within the each guest’s operating system. Guest networking is configured in each guest's vm.cfg file in the “vif” and “vif_other_config” directives. Each guest will have a unique vm.cfg file, located in the storage repository, for example, the /var/ovs/mount/<UUID>/running_pool/<VIRTUAL MACHINE NAME> directory. The next example shows a vm.cfg file from the Oracle VM 11g template. The “vif” and “vif_other_config” directives are highlighted. bootloader = '/usr/bin/pygrub' disk = ['file:/OVS/running_pool/v52x8611g1 /System.img,xvda,w', 'file:/OVS/running_pool/v52x8611g1 /oracle11g_x86_asm.img,xvdb,w', ] memory = '2048' name = 'v52x8611g1' on_crash = 'restart' on_reboot = 'restart' uuid = 'f8725f79-c6c8-26d4-a51e-1f32cf010c84' vcpus = 2 vif = ['bridge=xenbr0,mac=00:16:3E:56:20:63,type=netfront'] vif_other_config = [] In the above example, the settings in the “vif” directive show that: 1. the guest is using xenbr0 “bridge=xenbr0” 2. the guest's MAC address “mac=00:16:3E:29:D1:49” 3. dom0's front end driver used for the guest “type=netfront” The “vif_other_config” directive is empty, indicating that there is not an Oracle VM Manager network policy. A guest with the proper “vif” and OS networking setup will appear on the network from the assigned Xen bridge in the guest's vm.cfg file. Tip: To generate a unique MAC address for a guest, type the following text while logged in as root: PYTHONPATH=/opt/ovs-agent-2.2 python -c "from OVSCommons import randomMAC; print randomMAC()" The next example shows the output from “ifconfig” and “brctl show” from an Oracle VM server with two NICs hosting three guests; each guest has a single NIC. # ifconfig eth0 Link encap:Ethernet HWaddr 00:30:48:7F:45:AE UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:496976 errors:0 dropped:0 overruns:0 frame:0 TX packets:424315 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:100 RX bytes:78403808 (74.7 MiB) TX bytes:72492155 (69.1 MiB) eth1 Link encap:Ethernet HWaddr 00:30:48:7F:45:AF UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:4178056 errors:0 dropped:0 overruns:0 frame:0 TX packets:4687018 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:100 RX bytes:986681511 (940.9 MiB) TX bytes:166187546 (158.4 MiB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:2238791 errors:0 dropped:0 overruns:0 frame:0

175 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

TX packets:2238791 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:437463449 (417.1 MiB) TX bytes:437463449 (417.1 MiB) vif2.0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:876067 errors:0 dropped:0 overruns:0 frame:0 TX packets:786336 errors:0 dropped:21 overruns:0 carrier:0 collisions:0 txqueuelen:32 RX bytes:121934739 (116.2 MiB) TX bytes:142809517 (136.1 MiB) vif3.0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:53263 errors:0 dropped:0 overruns:0 frame:0 TX packets:57794 errors:0 dropped:61 overruns:0 carrier:0 collisions:0 txqueuelen:32 RX bytes:6934997 (6.6 MiB) TX bytes:10148669 (9.6 MiB) vif4.0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:381583 errors:0 dropped:0 overruns:0 frame:0 TX packets:489057 errors:0 dropped:8 overruns:0 carrier:0 collisions:0 txqueuelen:32 RX bytes:529005642 (504.4 MiB) TX bytes:102502853 (97.7 MiB) xenbr0 Link encap:Ethernet HWaddr 00:30:48:7F:45:AE inet addr:192.168.4.6 Bcast:0.0.0.0 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:543791 errors:0 dropped:0 overruns:0 frame:0 TX packets:397866 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:66334927 (63.2 MiB) TX bytes:73129563 (69.7 MiB) xenbr1 Link encap:Ethernet HWaddr 00:30:48:7F:45:AF UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:16159 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:5373332 (5.1 MiB) TX bytes:0 (0.0 b) # brctl show bridge name bridge id STP enabled interfaces xenbr0 8000.0030487f45ae no vif2.0 eth0 xenbr1 8000.0030487f45af no vif4.0 vif3.0 eth1 The output from “ifconfig” in the above example shows eth0, eth1, lo, vif2.0, vif3.0, vif4.0, xenbr0, and xenbr1. vif2.0, vif3.0, vif4.0 are the guest virtual NICs. Next, the output from “brctl show” shows the bridge configurations, that is, xenbr0 is connected to vif2.0 and eth0, and xenbr1 is connected to vif3.0, vif4.0, and eth1.

Oracle VM Server Networking Configuration Examples
In this section we will examine the following three Oracle VM networking configurations: The default ethX/xenbX configuration 802.3AD NIC bonding with Xen bridges 802.3AD NIC bonding with Xen bridges using 802.1Q - VLANning

The Default ethX/xenbX Configuration
Figure 1 shows the default Oracle VM Xen bridge configuration, with an Oracle VM server with two NICs wired into a single

176 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

unmanaged switch. The Oracle VM server is hosting three guests that use xennbr1; xenbr0 is dedicated to management traffic.

In Figure 1, the Oracle VM management traffic and HA monitoring are isolated to the eth0/xenbr0 interface and the loss of network connectivity for eth0/xenbr0 will cause an HA event. If the eth1/xenbr1 interface looses connectivity and all the guests are dropped from the network, an HA event will not occur. Figure 2 shows the default Oracle VM Xen bridge configuration with a two-server Oracle VM pool. Each Oracle VM server has two NICs wired in to a single unmanaged switch. The Oracle VM server is hosting three guests that use xenbr1; xenbr0 is dedicated to management traffic.

In Figure 2, the Oracle VM management traffic and HA monitoring are isolated to eth0/xenbr0 and the loss of network connectivity for eth0/xenbr0 will cause an HA event. If the eth1/xenbr1 interface loses connectivity and all the guests are dropped from the network an HA event will not occur.

802.3AD NIC Bonding with Xen Bridges
Figure 3 shows an 802.3AD NIC bonding configuration with one Xen bridge on bond1.

177 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

The Oracle VM server in Figure 3 has four NICs configured with two bonds, bond0 and bond1. Each NIC from each bond is wired into a separate switch for high availability. The Oracle VM server is hosting three guests using xenbr1; bond0 and eth0:0 are dedicated to management traffic. Tip: If a bond is dedicated for dom0 management traffic it is not necessary to configure a Xen bridge for dom0; a Xen bridge is only a requirement for guests to access the network. In Figure 3, the Oracle VM management traffic and HA monitoring are isolated to bond0 and the eth0:0 interface and the loss of network connectivity for bond0 or the eth0:0 interface would cause an HA event. If the bond1/xenbr1 interface loses connectivity and all the guests are dropped from the network, an HA event will not occur. Figure 4 shows an 802.3AD NIC bonding configurations with one Xen bridge with a two-server Oracle VM pool.

Each Oracle VM server in Figure 4 has four NICs configured with two bonds, bond0 and bond1. Each NIC from each bond is wired into a separate switch for high availability. Each Oracle VM server is hosting three guests that use xenbr1; bond0 and eth0:0 are dedicated to management traffic. In Figure 4 the Oracle VM management traffic and HA monitoring are isolated to bond0 and the eth0:0 interface and the loss of network connectivity for bond0 or the eth0:0 interface would cause an HA event. If the bond1/xenbr1 interface loses connectivity and all the guests are dropped from the network, an HA event will not occur. 802.3AD NIC Bonding with Xen Bridges and 802.1Q - VLANning Figure 5 shows an 802.3AD NIC bonding configuration that uses Xen bridges with 802.1Q VLANning.

The Oracle VM server in Figure 5 has four NICs configured with one bond, bond0. Each NIC from each bond is wired into a separate switch for high availability. The Oracle VM server is hosting three guests that use vlan2, vlan3, and vlan4; vlan1 is dedicated to management traffic. Note: In Figure 5 vlan1, vlan2, vlan3 and vlan4 are Xen bridges. Xen bridges can have discriptive names, i.e. vlanX, xenbrX, etc... In Figure 5, the Oracle VM management traffic and HA monitoring are isolated to the bond0:1 interface over vlan1. If dom0 loses network connectivity due to bond0, bond0:1 or vlan1 an HA event would occur causing the Oracle VM server to fence from the pool and reboot, all HA-enabled guests would restart on a live Oracle VM pool member. Figure 6 shows an 802.3AD NIC bonding configurations with a two-server Oracle VM pool that uses Xen bridges with 802.1Q VLAN and VLAN tagging.

178 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Each Oracle VM server in Figure 5 has four NICs configured with one bond, bond0. Each NIC from each bond is wired into a separate switch for high availability. Each Oracle VM server is hosting three guests that use vlan2, vlan3, and vlan4, respectively; vlan1 is dedicated to management traffic. The loss of network connectivity with the bond0 interface would cause an HA event that fences the Oracle VM server and reboots it, and restart all HA-enabled guests on a live Oracle VM pool member.

802.3AD NIC Bonding with a Single Xen Bridge Configuration
This section will explain how to configure NIC bonding with a single Xen bridge on an Oracle VM server with two NICs. The example configuration has a single Xen bridge used for dom0 and the guests. In this example, the Oracle VM server and the guests are on the same network and use the same Xen bridge. Each Oracle VM server in a pool should have the identical networking configuration. Figure 7 shows an example configuration with NIC bonding using a single Xen bridge.

The example configuration will create a bond with two NICs, one interface and one Xen bridge, used by dom0 and guests. After configuring the bonding interface, dom0 will have the following interfaces: eth0: the first physical NIC eth1: the second physical NIC bond0: The bonding device bond0 xenbr0: Xen bridge with an IP address for dom0 and the guests Note: This example configuration provides HA for both dom0 and the guests if the Oracle VM server loses network connectivity. Step 1: Disable the default xend network configuration. From dom0 type the following commands while logged in as root to disable the xend network configuration: # cd /etc/xen/scripts/ # ./network-bridges stop Step 2: Create a file in the /etc/xen/scripts directory named “network-bridge-ovs” that contains the following lines: #!/bin/sh /bin/true For example, from dom0 while logged in as root type “vi /etc/xen/scripts/network-bridge-ovs” to create a file named networkbridge-ovs, then enter the text in the next example:

179 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

#!/bin/sh /bin/true Step 3: Make the etc/xen/scripts/network-bridge-ovs file executable by typing the text in the next example. # chmod 755 /etc/xen/scripts/networkbridge-ovs Step 4: Edit the /etc/xen/xend-config.sxp file and change the following line from “network-script network-bridges” to “network-script network-bridge-ovs”. Step 5: Edit /etc/modprobe.conf and load and configure the bonding modules. For example: alias eth0 <DriverName> alias eth1 <DriverName> alias bond0 bonding options bonding mode=1 miimon=100 downdelay=200 updelay=200 Change the <DriverName> to the correct driver for your NICs, for example, tg3, e1000, or the like. We have done extensive testing with the various bonding modes and suggest mode 1 for all Oracle VM configurations. For example, the existing 2.6.18 kernel has known issues with mode 4 and 6. Note: Please review Bonding with LACP does not work if your testing mode 4. Consider testing the "xmit_hash_policy=layer2+3" setting in the above modprobe.conf file. Adding the "xmit_hash_policy=layer2+3" setting in the modprobe.conf file may help spread the load between the NICs in the bond and provide better throughput. To list the NIC drivers, try one of the following commands from dom0 while logged in as root: # dmesg | grep 'Ethernet driver' # grep 'Ethernet driver' /var/log/dmesg # grep ethX /etc/modules.conf Step 6: Configure the network devices and the Xen bridge. To configure the eth0 and eth1 interfaces, edit the /etc/sysconfig/network-scripts/ifcfg-eth0 file as follows: DEVICE=eth0 BOOTPROTO=none ONBOOT=yes MASTER=bond0 SLAVE=yes USERCTL=no HWADDR=<YOUR MAC ADDRESS> For /etc/sysconfig/network-scripts/ifcfg-eth1 DEVICE=eth1 BOOTPROTO=none ONBOOT=yes MASTER=bond0 SLAVE=yes USERCTL=no HWADDR=<YOUR MAC ADDRESS> The HWADDR parameter is the pointer to the NIC's MAC address. The SLAVE parameter defines the network card as a slave of a bond-device, in this case bond0. The MASTER parameter points to the actual bonding device that this network interface will be part of. Next, create the bonding device bond0 by creating a file named /etc/sysconfig/network-scripts/ifcfg-bond0, containing the following lines: DEVICE=bond0 ONBOOT=yes BRIDGE=xenbr0 USERCTL=no The next example shows how to create and save a file named /etc/sysconfig/network-scripts/ifcfg-bond0 with the bond device parameters. # vi /etc/sysconfig/network-scripts/ifcfg-bond0 DEVICE=bond0 ONBOOT=yes

180 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

BRIDGE=xenbr0 USERCTL=no :wq! The BRIDGE parameter specifies that the bond interface does not have an IP address, but will be connected to bridge xenbr0. Next, create the Xen bridge by creating a file named /etc/sysconfig/network-scripts/ifcfg-xenbr0 containing the following lines: DEVICE=xenbr0 ONBOOT=yes STP=off USERCTL=no IPADDR=<YOUR IP ADDRESS> NETMASK=<YOUR NETMASK> The next example shows how to create and save a file named ifcfg-xenbr0 that contains the Xen bridge parameters. # vi /etc/sysconfig/network-scripts/ifcfg-xenbr0 DEVICE=xenbr0 ONBOOT=yes STP=off USERCTL=no IPADDR=<YOUR IP ADDRESS> NETMASK=<YOUR NETMASK> :wq! An IP address is assigned to xenbr0 using the IPADDR and NETMASK lines. The NETWORK and BROADCAST parameters are deprecated and are automatically calculated with ipcalc. Step 5: Restart the network and xend services from dom0 by typing “service network restart” followed by “service xend restart,” as shown in the next example: # service network restart # service xend restart Once xend and the network service have been restarted, the bond with two NICs with one Xen bridge will be ready for use. In Oracle VM Manager each guest should be configured to use xenbr0. To validate the configurations, from dom0 type “ifconfig” to list the interfaces and the Xen bridges, as shown in the next example. bond0 Link encap:Ethernet HWaddr 00:15:17:2D:8A:91 UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 RX packets:11770170 errors:0 dropped:0 overruns:0 frame:0 TX packets:10814791 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:1057328427 (1008.3 MiB) TX bytes:1040946196 (992.7 MiB) eth0 Link encap:Ethernet HWaddr 00:15:17:2D:8A:91 UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1 RX packets:321110 errors:0 dropped:0 overruns:0 frame:0 TX packets:12551 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:41226443 (39.3 MiB) TX bytes:1203300 (1.1 MiB) eth1 Link encap:Ethernet HWaddr 00:15:17:2D:8A:90 BROADCAST MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 b) TX bytes:0 (0.0 b) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:808396 errors:0 dropped:0 overruns:0 frame:0 TX packets:808396 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:140236379 (133.7 MiB) TX bytes:140236379 (133.7 MiB) xenbr0 Link encap:Ethernet HWaddr 00:15:17:2D:8A:91 inet addr:10.1.80.20 Bcast:10.1.80.255 Mask:255.255.255.0

181 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 As shown in the next example, type “brctl show” to confirm that the desired interface is using the appropriate Xen bridge. # brctl show bridge name bridge id STP enabled interfaces xenbr0 8000.0015172d8a91 no bond0 The output for “brctl show” lists the bridge name, bridge ID, spanning-tree protocol (STP) configuration, and the configured interfaces. The next example shows a vm.cfg file from a guest that is configured to use xenbr0. bootloader = '/usr/bin/pygrub' disk = ['file:/OVS/running_pool/v52x8611g1 /System.img,xvda,w', 'file:/OVS/running_pool/v52x8611g1 /oracle11g_x86_asm.img,xvdb,w', ] memory = '2048' name = 'v52x8611g1' on_crash = 'restart' on_reboot = 'restart' uuid = 'f8725f79-c6c8-26d4-a51e-1f32cf010c84' vcpus = 2 vif = ['bridge=xenbr0,mac=00:16:3E:56:20:63,type=netfront'] vif_other_config = [] Tip: All Oracle VM servers using bonded interfaces should modify two default O2CB heartbeat configurations to allow a bond fail-over event to occur without the Oracle VM server fencing from the pool. Edit the /etc/sysconfig/o2cb file using your favorite text editor and change "O2CB_HEARTBEAT_THRESHOLD=" to "O2CB_HEARTBEAT_THRESHOLD= 61" and "O2CB_IDLE_TIMEOUT_MS=" to "O2CB_IDLE_TIMEOUT_MS= 60000". The next example shows the suggested modifications made to a /etc/sysconfig/o2cb file. # vi /etc/sysconfig/o2cb # # This is a configuration file for automatic startup of the O2CB # driver. It is generated by running /etc/init.d/o2cb configure. # Please use that method to modify this file # # O2CB_ENABELED: 'true' means to load the driver on boot. O2CB_ENABLED=true # O2CB_BOOTCLUSTER: If not empty, the name of a cluster to start. O2CB_BOOTCLUSTER=rmvsx # O2CB_HEARTBEAT_THRESHOLD: Iterations before a node is considered dead. O2CB_HEARTBEAT_THRESHOLD= 61 # O2CB_IDLE_TIMEOUT_MS: Time in ms before a network connection is considered dead. O2CB_IDLE_TIMEOUT_MS= 60000 # O2CB_KEEPALIVE_DELAY_MS: Max time in ms before a keepalive packet is sent O2CB_KEEPALIVE_DELAY_MS=

802.3AD NIC Bonding and Xen Bridges with 802.1Q Configuration
This section will examine how to configure NIC bonding with VLANning using multiple Xen bridge on an Oracle VM server with two NICs. To be able to use 802.1Q VLANning, your switches will need to support VLAN trunking and be correctly configured. The example configuration uses a single bond with multiple Xen bridges for dom0 and guests. Each Oracle VM server in a pool should have the identical networking configuration. Figure 8 shows an example 802.3AD NIC bonding and Xen bridges with 802.1Q VLANning.

182 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

The example configuration will have an 802.3AD bond with two NICs with one interface using four Xen bridges. After the bonding interface is configured, dom0 will have the following interfaces: eth0: the first physical NIC eth1: the second physical NIC bond0: bonding interface for eth0 and eth1 bond0:1 with vlan1 for dom0 bond0:2 with vlan2 bond0:3 with vlan3 bond0:4 with vlan4 The example configuration offers HA for both dom0 and the guest Xen bridges if the Oracle VM server loses network connectivity. Note: Please note that the :XX part of the interface name, that is, bond0:1, bond0:2, bond0:3, bond0:4, is referencing VLAN 1, 2, 3, and 4, respectively. Step 1: Disable the default xend network configuration. From dom0 type the following commands while logged in as root to disable the xend network configuration: # cd /etc/xen/scripts/ # ./network-bridges stop Step 2: Create a file in the /etc/xen/scripts directory named “network-bridge-ovs” that contains the lines: #!/bin/sh /bin/true For example, from dom0 while logged in as root type “vi /etc/xen/scripts/network-bridge-ovs” to create a file named networkbridge-ovs, then enter the text in the next example. #vi /etc/xen/scripts/networkbridge-ovs #!/bin/sh /bin/true :wq! Step 3: Make the etc/xen/scripts/network-bridge-ovs file executable by typing the text in the next example. # chmod 755 /etc/xen/scripts/networkbridge-ovs Step 4: Edit the /etc/xen/xend-config.sxp file and change the line“network-script network-bridges” to “network-script networkbridge-ovs”. Step 5: Edit /etc/modprobe.conf and load and configure the bonding modules. For example: alias eth0 <DriverName> alias eth1 <DriverName> alias bond0 bonding options bonding mode=1 miimon=100 downdelay=200 updelay=200 Change the <DriverName> to the correct driver for your NICs, for example, tg3, e1000, or the like.

183 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Note: Please review Bonding with LACP does not work and consider testing the "xmit_hash_policy=layer2+3" setting in the above modprobe.conf file. Adding the "xmit_hash_policy=layer2+3" setting in the modprobe.conf file may help spread the load between the NICs in the bond and provide better throughput.

To list the NIC drivers, use one of the following commands from dom0 while logged in as root. # dmesg | grep 'Ethernet driver' # grep 'Ethernet driver' /var/log/dmesg # grep ethX /etc/modules.conf Step 6: Configure the network devices and the Xen bridge. To configure the eth0 and eth1 interfaces, edit the /etc/sysconfig/network-scripts/ifcfg-eth0 file as follows: DEVICE=eth0 BOOTPROTO=none ONBOOT=yes MASTER=bond0 SLAVE=yes USERCTL=no HWADDR=<YOUR MAC ADDRESS> For /etc/sysconfig/network-scripts/ifcfg-eth1 DEVICE=eth1 BOOTPROTO=none ONBOOT=yes MASTER=bond0 SLAVE=yes USERCTL=no HWADDR=<YOUR MAC ADDRESS> The HWADDR parameter is the pointer to the NIC's MAC address. The SLAVE parameter defines the network card as a slave of a bond-device, in this case, bond0. The MASTER parameter points to the actual bonding device that this network interface will be part of. Next, create the bonding device bond0 by creating a file named /etc/sysconfig/network-scripts/ifcfg-bond0 containing the following lines: DEVICE=bond0 ONBOOT=yes USERCTL=no The next example shows how to create and save a file named /etc/sysconfig/network-scripts/ifcfg-bond0 with the bond-device parameters. # vi /etc/sysconfig/network-scripts/ifcfg-bond0 DEVICE=bond0 ONBOOT=yes USERCTL=no :wq! Next, create the interfaces by making a file for each interface named /etc/sysconfig/network-scripts/ifcfg-<INTERFACE NAME>. The following examples show four interface files. Note: Any descriptive name can be used for the interface and Xen bridge. You can create as many interfaces and Xen bridges as needed for your environment. The next example is the file for the bond0:1 interface that uses vlan1. DEVICE=bond0:1 ONBOOT=yes BOOTPROTO=none VLAN=yes TYPE=Ethernet BRIDGE=vlan1 The next example is the file for the bond0:2 interface that uses vlan2. DEVICE=bond0:2 ONBOOT=yes BOOTPROTO=none VLAN=yes TYPE=Ethernet BRIDGE=vlan2

184 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

The next example is the file for the bond0:3 interface that uses vlan3. DEVICE=bond0:3 ONBOOT=yes BOOTPROTO=none VLAN=yes TYPE=Ethernet BRIDGE=vlan3 The next example is the file for the bond0:4 interface that uses vlan4. DEVICE=bond0:4 ONBOOT=yes BOOTPROTO=none VLAN=yes TYPE=Ethernet BRIDGE=vlan4 Next, create the Xen bridges by making a file for each Xen bridge, named /etc/sysconfig/network-scripts/ifcfg-<XEN BRIDGE NAME>. The following examples show four Xen bridge files. The next example is the file for the vlan1 that is used by bond0:1. DEVICE=vlan1 BOOTPROTO=static ONBOOT=yes IPADDR=<YOUR IP ADDRESS FOR DOM0> NETMASK=<YOUR NETMASK FOR DOM0> An IP address is assigned to xenbr11 using the IPADDR and NETMASK sections. The NETWORK and BROADCAST parameters are deprecated and are automatically calculated with ipcalc. The next example is the file for the vlan2 that is used by bond0:2. DEVICE=vlan2 BOOTPROTO=none ONBOOT=yes Type=Bridge The next example is the file for the vlan3 that is used by bond0:3. DEVICE=vlan3 BOOTPROTO=none ONBOOT=yes Type=Bridge The next example is the file for the vlan4 that is used by bond0:4. DEVICE=vlan4 BOOTPROTO=none ONBOOT=yes Type=Bridge The next example shows how to create and save a file named ifcfg-vlan2 that contains the Xen bridge parameters. # vi /etc/sysconfig/network-scripts/ifcfg-vlan2 DEVICE=vlan2 BOOTPROTO=none ONBOOT=yes Type=Bridge :wq! Step 5: Restart the network and xend services from dom0 by typing “service network restart” followed by “service xend restart” as shown in the next example. # service network restart # service xend restart Once xend and the network service have been restarted, the new configuration will be ready for use. In Oracle VM Manager, each guest can be configured to use any of the Xen bridges. To validate the configurations, from dom0 type “ifconfig” to list the interfaces and the Xen bridges, as shown in the next example.

185 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

bond0 Link encap:Ethernet HWaddr 00:15:17:2D:8A:91 UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 RX packets:11770170 errors:0 dropped:0 overruns:0 frame:0 TX packets:10814791 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:1057328427 (1008.3 MiB) TX bytes:1040946196 (992.7 MiB) bond0:1 Link encap:Ethernet HWaddr 00:15:17:2D:8A:91 UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 RX packets:10710127 errors:0 dropped:0 overruns:0 frame:0 TX packets:5126322 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:788159801 (751.6 MiB) TX bytes:585997143 (558.8 MiB) bond0:2 Link encap:Ethernet HWaddr 00:15:17:2D:8A:91 UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 RX packets:335757 errors:0 dropped:0 overruns:0 frame:0 TX packets:186954 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:33593433 (32.0 MiB) TX bytes:69169196 (65.9 MiB) bond0:3 Link encap:Ethernet HWaddr 00:15:17:2D:8A:91 UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 RX packets:183889 errors:0 dropped:0 overruns:0 frame:0 TX packets:805 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:9773662 (9.3 MiB) TX bytes:53429 (52.1 KiB) bond0:4 Link encap:Ethernet HWaddr 00:15:17:2D:8A:91 UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1 RX packets:183889 errors:0 dropped:0 overruns:0 frame:0 TX packets:805 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:9773662 (9.3 MiB) TX bytes:53429 (52.1 KiB) eth0 Link encap:Ethernet HWaddr 00:15:17:2D:8A:91 UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1 RX packets:321110 errors:0 dropped:0 overruns:0 frame:0 TX packets:12551 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:41226443 (39.3 MiB) TX bytes:1203300 (1.1 MiB) eth1 Link encap:Ethernet HWaddr 00:15:17:2D:8A:90 BROADCAST MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 b) TX bytes:0 (0.0 b) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:808396 errors:0 dropped:0 overruns:0 frame:0 TX packets:808396 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:140236379 (133.7 MiB) TX bytes:140236379 (133.7 MiB) vlan2 Link encap:Ethernet HWaddr 00:15:17:2D:8A:91 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:10710118 errors:0 dropped:0 overruns:0 frame:0 TX packets:5126341 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:788082919 (751.5 MiB) TX bytes:586000638 (558.8 MiB) vlan1 Link encap:Ethernet HWaddr 00:15:17:2D:8A:91 inet addr:10.1.80.20 Bcast:10.1.80.255 Mask:255.255.255.0

186 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 vlan2 Link encap:Ethernet HWaddr 00:15:17:2D:8A:91 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:187198 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:9782976 (9.3 MiB) TX bytes:0 (0.0 b) vlan3 Link encap:Ethernet HWaddr 00:15:17:2D:8A:91 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:183316 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:9658438 (9.2 MiB) TX bytes:0 (0.0 b) vlan4 Link encap:Ethernet HWaddr 00:15:17:2D:8A:91 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:183316 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:9658438 (9.2 MiB) TX bytes:0 (0.0 b) As shown in the next example, type “brctl show” to confirm that the desired interfaces are using the correct Xen bridges. # brctl show bridge name bridge id STP enabled interfaces vlan1 8000.0015172d8a91 no bond0:1 vlan2 8000.0015172d8a91 no bond0:2 vlan3 8000.0015172d8a91 no bond0:3 vlan4 8000.0015172d8a91 no bond0:4 The output for “brctl show” lists the bridge name, bridge ID, spanning-tree protocol (STP) configuration and the configured interfaces. Tip: All Oracle VM servers using bonded interfaces should modify two default O2CB heartbeat configurations to allow a bond fail-over event to occur without the Oracle VM server fencing from the pool. Edit the /etc/sysconfig/o2cb file using your favorite text editor and change "O2CB_HEARTBEAT_THRESHOLD=" to "O2CB_HEARTBEAT_THRESHOLD= 61" and "O2CB_IDLE_TIMEOUT_MS=" to "O2CB_IDLE_TIMEOUT_MS= 60000". The next example shows the suggested modifications made to a /etc/sysconfig/o2cb file. # vi /etc/sysconfig/o2cb # # This is a configuration file for automatic startup of the O2CB # driver. It is generated by running /etc/init.d/o2cb configure. # Please use that method to modify this file # # O2CB_ENABELED: 'true' means to load the driver on boot. O2CB_ENABLED=true # O2CB_BOOTCLUSTER: If not empty, the name of a cluster to start. O2CB_BOOTCLUSTER=rmvsx # O2CB_HEARTBEAT_THRESHOLD: Iterations before a node is considered dead. O2CB_HEARTBEAT_THRESHOLD= 61 # O2CB_IDLE_TIMEOUT_MS: Time in ms before a network connection is considered dead. O2CB_IDLE_TIMEOUT_MS= 60000 # O2CB_KEEPALIVE_DELAY_MS: Max time in ms before a keepalive packet is sent O2CB_KEEPALIVE_DELAY_MS= Watch this space Oracle OpenWorld 2011 Call for Papers - Submit Today! Temporary Post Used For Theme Detection (409fc5e9-d7ee-4e1a-b645-8b82f256f434 - 3bfe001a-32de-4114-a6b4-4005b770f6d7) Debugging Oracle VM 2.2 & Oracle VM Manager 2.2 errors using the many log files New VDI Training Course now available! HIMSS 2011 and New Press Release Oracle Desktop Virtualization at HIMSS 2011 Oracle VM VirtualBox 4.0.4 Released! Two Virtualization Webinars This Week We're Hiring! - Server and Desktop Virtualization Product Management

Chapter 8: Virtualizing Oracle Database 10g/11g Oracle Database 10g/11g Virtualization
Copyright © 2009 - 2011 Roddy Rodstein. All rights reserved.

187 of 243

18.04.2011 16:35

The Underground Oracle VM Manual
Show a printer friendly version of this book with its sub pages

http://itnewscast.com/book/export/html/844

This chapter will review how to rapidly deploy a virtualized Oracle 10g/11g Database using Oracle VM Templates with Oracle VM Manager. Note: As of this writing, the Oracle 10g/11g Database Oracle VM Templates will not boot on an Oracle VM server with more than 60 G or RAM. Oracle VM templates using Oracle Linux 5.3 and below ship with a known kernel bug that will not allow the Oracle VM template to boot on an Oracle VM server with more than 60 G or RAM. If your Oracle VM server has more than 60 G of RAM and an Oracle VM template will not boot, edit the grub.conf file on the Oracle VM server to add the memory parameter to kernel line that loads the Xen hypervisor, i.e. kernel /xen-64bit.gz dom0_mem=1024M mem=50G After adding the mem=50G entry, reboot the Oracle VM server. With the addition of the mem=50G entry, the Oracle VM template will be able to boot on the Oracle VM server. Once the Oracle VM template is running, patch the kernel to resolve the kernel bug, and then remove the the mem=50G entry on the Oracle VM server.

Oracle VM Templates Overview
Oracle VM Templates are self-contained and pre-configured virtual machines with key Oracle technologies that can be downloaded and rapidly deployed in to production or test environments. Each Oracle VM Template is packaged using Oracle best practices, which eliminates operating system and application installation and configuration costs, reduces risk and dramatically shortens deployment time. Oracle VM Templates are available from the Linux Oracle eDelivery portal. There is no other vendor in the virtualization market that can offer this level of in-depth knowledge, integration, management and pre-packaging of applications with a production ready operating system. The use of Oracle VM Templates for the deployment of applications in Oracle VM guest virtual machines eliminates the need for a user to install and configure the operating system or applications. The virtual machines created using Oracle VM Templates can be started either from the Oracle VM Manager or the Oracle VM Management Pack (an Oracle Enterprise Manager plug-in). Oracle VM Templates include a free download and free trial license with the option to purchase a product license. Oracle VM Templates do not have time limits or feature limitations, e.g. Oracle VM Templates are full featured and do not have expiration dates. Oracle VM Templates can be quickly transitioned from evaluation into production by purchasing Oracle technology licenses. List 1 shows Oracle’s current Oracle VM templates collection:

Oracle Virtualization and Management
Oracle VM Manager 2.2 and 2.1.5 Oracle VM Template Builder 2.1 Oracle VM Server 2.2.1 and 2.1.5 SDK Sun Ray Software 5 Oracle Secure Global Desktop Oracle Enterprise Manager Grid Control 10g R5 (10.2.0.5) Oracle Enterprise Manager Grid Control 11g R1 (10.2.0.5)

Oracle Applications
E-Business Suite 12.1.3 E-Business Suite 12.1.1 E-Business Suite 12.X Sparse Middle Tier JD Edwards EnterpriseOne 9.0 Update 1 with ESUs and JD Edwards EnterpriseOne Tools 8.98 Update 3 JD Edwards EnterpriseOne 9.0 Update 1 and JD Edwards EnterpriseOneTools 8.98 Update 2 PeopleSoft ELM 9.1 Bundle #2 with PeopleTools 8.50.09 PeopleSoft FSCM 9.1 Bundle #4 (includes Maintenance Pack 2) with PeopleTools 8.50.10 PeopleSoft CRM 9.1 Bundle #2 with PeopleTools 8.50.09 PeopleSoft Portal Solutions 9.1 and PeopleTools 8.50.09 PeopleSoft HCM 9.1 and PeopleTools 8.50.02 Siebel CRM SIA 8.1.1 Siebel CRM SIA 8.0

Oracle Middleware
Oracle Oracle Oracle Oracle Oracle Oracle WebLogic Server on JRockit Virtual Edition 11g R1 (10.3.2) WebLogic Server 10g Release 3 Business Intelligence Enterprise Edition 10.1.3.4 Application Server 10g Release 3 WebCenter Identity Management 10g Release 2 Fusion Middleware Service Oriented Architecture (SOA) 10.1.3.4 and 10.1.3.3

Oracle Database & Oracle Real Application Clusters (RAC)
Oracle Real Application Clusters (RAC) 11g Release 2 Oracle Real Application Clusters (RAC) 11g Release 1 Oracle Database 11g and Oracle Database 10g Release 2

Oracle Operating Systems
Oracle Linux 4 and 5

188 of 243

18.04.2011 16:35

The Underground Oracle VM Manual
Oracle Solaris 10

http://itnewscast.com/book/export/html/844

3rd Party Software
Zeus Traffic Manager Watch this space Oracle OpenWorld 2011 Call for Papers - Submit Today! Temporary Post Used For Theme Detection (409fc5e9-d7ee-4e1a-b645-8b82f256f434 - 3bfe001a-32de-4114-a6b4-4005b770f6d7) Debugging Oracle VM 2.2 & Oracle VM Manager 2.2 errors using the many log files New VDI Training Course now available! HIMSS 2011 and New Press Release Oracle Desktop Virtualization at HIMSS 2011 Oracle VM VirtualBox 4.0.4 Released! Two Virtualization Webinars This Week We're Hiring! - Server and Desktop Virtualization Product Management

How to Download Oracle VM Templates
This chapter covers how to download Oracle VM templates. Note: The Oracle VM Templates are only available from the Linux Oracle eDelivery site at http://edelivery.oracle.com/linux. The Oracle VM templates are not available from the root http://edelivery.oracle.com/ eDelivery URL.

How to Download Oracle VM Templates
a) Enter http://edelivery.oracle.com/linux in to a Web browser. Click the Continue button to proceed to the Registration form. Figure 1

b) Complete the Registration form, accept the license agreement and the export agreement. Next, click the Continue button to proceed to the Search page. Figure 2

189 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

c) From the Media Pack Search page you can select the a) Oracle VM Templates b) Enterprise Linux c) Oracle VM search options. Figure 3

Select the Oracle VM Templates search option from the Select a Product Pack drop down list. Next, select ether x86 32 bit or the x86 64 option from the Platform drop down list. Click the Go button to proceed to the Oracle VM Templates Media Pack Search page. Figure 4

190 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

d) From the Media Pack Search page, locate and click on the desired Oracle VM Template link. Figure 5

191 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

d) Review the Oracle VM Templates on the desired <Oracle VM Template Name> Media Pack Download pages and select and download the zip files. Figure 6 shows the Oracle VM Template for Oracle Database Media Pack Download page. To download the Oracle Database Media Pack, select and download both files.

192 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

e) Next, click the Readme button and review the Readme for the Oracle VM Template template. f) Once the Oracle VM template files have been downloaded, copy the zip files to your Oracle VM server's /OVS/running_pool directory or the /OVS/seed_pool directory. Next, unzip and untar the files in the directory. Note: Use the running_pool directory if you would like to deploy the Oracle VM template as a running virtual machine. Use the seed_pool directory if you would like to deploy the Oracle VM template as a reusable template that can be re-deployed an unlimited number of times. Watch this space Oracle OpenWorld 2011 Call for Papers - Submit Today! Temporary Post Used For Theme Detection (409fc5e9-d7ee-4e1a-b645-8b82f256f434 - 3bfe001a-32de-4114-a6b4-4005b770f6d7) Debugging Oracle VM 2.2 & Oracle VM Manager 2.2 errors using the many log files New VDI Training Course now available! HIMSS 2011 and New Press Release Oracle Desktop Virtualization at HIMSS 2011 Oracle VM VirtualBox 4.0.4 Released! Two Virtualization Webinars This Week We're Hiring! - Server and Desktop Virtualization Product Management

How to Import the Oracle Database 11g Template with Oracle VM Manager
This chapter will review how to import the Oracle Database 11g template using Oracle VM Manager. a) Access the Oracle VM Manager GUI by typing the Oracle VM Manager URL in your browser. Enter your username and password in to the Username and Password text boxes. Then, press the Login button to authenticate to access the Oracle VM Manager GUI.
Figure 1

b) From the Virtual Machines home page click the Resources tab. From the Resources tab click the Virtual Machine Image link. From the Virtual Machine Images page click the Import Button. Note: The example assumes that the Oracle Database 11g template has been unzipped and untarred in the /OVS/running_pool directory. If the Oracle database 11g template was unzipped and untarred in the /OVS/seed_pool directory, click the Virtual Machine Templates tab to import the template. Figure 2

193 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

d) From the Source page select the Select from Server Pool (Discover and register) radio button and click Next to proceed. Note: The Select from Server Pool (Discover and register) requires that the Oracle Database 11g template has been staged, i.e. unzipped

and untarred in the /OVS/running_pool directory.
Figure 3

d) From the General Information page complete the form for the Oracle VM 11g Template's properties. Once all the data is complete click Next to proceed. Server Pool Name: Required Sharing: Required Virtual Machine Image Name: Enable High Availability Required Operating System: Oracle Enterprise Linux 5 (either x86 or x86_64) Virtual Machine System Username: Virtual Machine System Password: Required Console Password: Required Confirm Console Password: Description: Figure 4

e) Click the Confirm button to proceed. Figure 5

194 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

f) From the Virtual Machine Images page click the Refresh button. The status will change from Importing to Pending. Next, click the Approve button. Figure 6

g) From the View Imported Virtual Machine page click the Approve button. Figure 7

h) After the Virtual Machine Image is approved click the Virtual Machines tab. From the Virtual Machines page you will see the Oracle Database 11g template in the list of Virtual Machines in the Powered Off status. Figure 8

195 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Watch this space Oracle OpenWorld 2011 Call for Papers - Submit Today! Temporary Post Used For Theme Detection (409fc5e9-d7ee-4e1a-b645-8b82f256f434 - 3bfe001a-32de-4114-a6b4-4005b770f6d7) Debugging Oracle VM 2.2 & Oracle VM Manager 2.2 errors using the many log files New VDI Training Course now available! HIMSS 2011 and New Press Release Oracle Desktop Virtualization at HIMSS 2011 Oracle VM VirtualBox 4.0.4 Released! Two Virtualization Webinars This Week We're Hiring! - Server and Desktop Virtualization Product Management

How to Start, Configure and Deploy the Oracle Database 11g Template using Oracle VM Manager
This chapter show how to start, configure and how to deploy the Oracle Database 11g template. The Virtual Machines page is the Oracle VM virtual infrastructure dashboard.
The dashboard offers visibility in to the virtual machine's status and properties, i.e. memory allocation, status, owner, group name, server name, server pool name. For example, click the the virtual machine's +Show option to list the virtual machine properties dashboard. The Select and feature allow you to Power On, access the virtual machine's Console, Power Off, and Configure a virtual machine's properties, i.e. resource allocations and policies. The More Actions drop down list offers the following virtual machine administrative features: Deploy, Live Migration, Clone, Save as Template, Pause, Unpause, Suspend, Resume, Delete and Reset. a) From the Virtual Machine page start your virtual machine by pressing the Power On button. The virtual machine status will change from Powered Off to Initializing to Running. Figure 1

196 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

b) Once the status is Running click the Console button to access the virtual machine's console. Note: If you receive any Java Warnings, click the Always trust content from the publisher check box and click Run. c) Enter the VNC Authentication Password in the Password text box and click the OK button to access the virtual machine's console. Figure 2

d) When a virtual machine created from Oracle Database template boots up for the first time, the boot up process gathers database configuration information from the user and configures the database automatically, please follow the prompts in the Oracle VM Manager virtual machine VNC console to complete the process. 1) System configuration User will be asked if a static IP address should be configured. Answer no “y” to use DHCP . Configuring network for Oracle Database. Current network is using DHCP Using static IP address is recommended. . Use DHCP? y/n [n] Configuring static IP . Starting network... 2) Oracle database configuration (ACCEPT THE DEFAULTS, JUST PRESS ENTER) Starting Oracle database reconfiguration. Do you want to relink binaries? (y/n) [n] Specify the HTTP port that will be used for Oracle Application Express [8080]: Specify a port that will be used for the database listener [1521]: Specify a password to be used for database accounts. Note that the same password will be used for SYS, SYSTEM and ADMIN for APEX. Oracle recommends the use of different passwords for each database account. This can be done after initial configuration: Confirm the password: Do you want Oracle Database to be started on boot (y/n) [y]: Running orainstRoot.sh and root.sh... Changing permissions of /u01/app/oraInventory to 770. Changing groupname of /u01/app/oraInventory to dba. The execution of the script is complete Running Oracle 11g root.sh script... The following environment variables are set as: ORACLE_OWNER= oracle

197 of 243

18.04.2011 16:35

The Underground Oracle VM Manual
ORACLE_HOME= /u01/app/oracle/product/11.1.0/db_1 Enter the full pathname of the local bin directory: [/usr/local/bin]: Copying dbhome to /usr/local/bin ... Copying oraenv to /usr/local/bin ... Copying coraenv to /usr/local/bin ... Creating /etc/oratab file... Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root.sh script. Now product-specific root actions will be performed. Finished product-specific root actions. Starting CSS. Starting Oracle Net Listener...Done ASM instance started ORACLE instance started. Configuration Completed Successfully. To access the Oracle Application Express go to "http://myhost.mydomain.com:8080/apex"

http://itnewscast.com/book/export/html/844

d) The Oracle Database 11g Template is configured, on the network and 100% operational. The virtual machine is accessible via ssh using the IP address select during the configuration process with the default user name root and the default root password ovsroot. The virtual machine's console can be accessed using the console password that was selected during the configuration process using the console button in Oracle VM Manager.
Notes:

1. The system image contains a minimal install of Oracle Linux. Only basic RPM packages are installed. 1. The default root password is 'ovsroot'. 2. A database instance 'orcl' is created in the templates. 1. The database user is named “oracle” and belongs to groups 'orainstall' and 'dba'. The password is 'oracle'. 2. The oracle-validated package is installed. This package verifies and sets system parameters based on Oracle validated configuration recommendations for Oracle Enterprise Linux. 3. The database storage is managed by Automatic Storage Management (ASM). Two ASM disks 'VOL1' and 'VOL2' are created, and the default ASM group name is 'DATA'.

Watch this space Oracle OpenWorld 2011 Call for Papers - Submit Today! Temporary Post Used For Theme Detection (409fc5e9-d7ee-4e1a-b645-8b82f256f434 - 3bfe001a-32de-4114-a6b4-4005b770f6d7) Debugging Oracle VM 2.2 & Oracle VM Manager 2.2 errors using the many log files New VDI Training Course now available! HIMSS 2011 and New Press Release Oracle Desktop Virtualization at HIMSS 2011 Oracle VM VirtualBox 4.0.4 Released! Two Virtualization Webinars This Week We're Hiring! - Server and Desktop Virtualization Product Management

Chapter 9: Virtualizing Oracle Enterprise Manager 10g
Copyright © 2009 - 2011 Roddy Rodstein. All rights reserved. Show a printer friendly version of this book with its sub pages

The Oracle Enterprise Manager 10g Oracle VM template allows customers to quickly deploy Oracle Enterprise Manager 10g/11g using Oracle VM Manager. The Oracle Enterprise Manager 10g template is comprised of a virtual machine assembly with two pre-configured virtual machines. Both virtual machines use Oracle Enterprise Linux 4 update 5. The first virtual machine hosts the Oracle Management Service (OMS) and an optional YUM repository. The second virtual machine hosts the Oracle Management Repository (OMR). Each virtual machine was created with Oracle best practices, which reduce deployment risk and dramatically shortens deployment timelines. Oracle VM templates are freely available from http://edelivery.oracle.com/linux. The Oracle Enterprise Manager 10g Oracle VM template is downloaded as two sets of compressed files (14.81 GB total) from http://edelivery.oracle.com/linux. The first set of four files (5.69 GB) is named “Oracle VM Enterprise Manager Grid Control Template for Oracle VM” and the second set of five files (9.12 GB) is named “Yum Repository of Oracle Enterprise Linux 4 and 5 for use with Enterprise Manager Grid Control Template”. The second set of files for the Yum repository is optional. Figure 1 shows how the Oracle Enterprise Manager 10g Oracle VM template is deployed in to an Oracle VM server farm.

198 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

System Requirements
A minimum of one physical servers is required for the Oracle VM Enterprise Manager Grid Control Template.

Oracle VM Server:
Oracle VM Server requires one bare-metal server capable of running at least three virtual machines simultaneously. The recommended minimum hardware requirements for an Oracle VM Server are: 4 CPUs 4 GB RAM 80 GB disk space without the Yum repository 100 GB disk space with the Yum repository

The Oracle Management Repository Virtual Machine:
2 Virtual CPUs 2 GB RAM 5.2 GB disk space in /OVS/seed_pool/ (virtual machine template storage) 33 GB disk space in /OVS/running_pool/ (virtual machine storage)

The Oracle Management Service and the Yum repository Virtual Machine:
2 Virtual CPUs (virtual machine) 2 GB RAM (virtual machine) 16 GB disk space in /OVS/seed_pool/ (virtual machine template storage) 72 GB disk space in /OVS/running_pool/ (virtual machine storage)

Oracle Enterprise Manager 10g Oracle VM Template Installation Steps
There are a total of eight steps to deploy the Oracle Enterprise Manager 10g Oracle VM template. Table 1 lists each step with a brief explanation including an estimated time to completion.

Step

Description

Estimated Time to Completion 2 to x hours depending on your Internet connection speed. Up to 30 minutes depending on system resources.

1. Download the Template.

Download the Oracle VM Enterprise Manager Grid Control Template from http://edelivery.oracle.com /linuxand place the zip files in the /OVS/seed_pool/ directory. Uncompress the zip files in the /OVS/seed_pool/ directory, then concatenate each set of files and add the yum.img file to the OMS vm.cfg file.

2. Unzip and concatenate the file sets and add the yum.img file to the OMS vm.cfg file.

199 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

3. Create the OMR and OMS virtual machine templates. 4. Create the OMR and OMS virtual machines. 5. Start and configure the OMR and OMS virtual machines.

With Oracle VM Manager create the OMR and OMS virtual machine templates.

Up to 15 minutes.

With Oracle VM Manager create the OMR and OMS virtual machines with the OMR and OMS templates. With Oracle VM Manager start the OMR virtual machine, access the console via Oracle VM Manager to enter the setup options. After the OMR virtual machine is configured, start the OMS virtual machine to enter the setup options. Note: The OMR virtual machine must be started first.

Up to 6 hours depending on system resources. Up to 1 hour for each virtual machine.

6. Install OMA on the OMS virtual machine.

Login to the OMS virtual machine as the root and execute the “/installagent.sh” script to install the Oracle Management Agent (OMA) version 10.2.0.4 x86. Login to the Enterprise Manager Grid Control portal using the sysman account and the password selected in step 5. With Oracle VM Manager use the Deploy action to change the ownership of the OMR and OMS virtual machines from the My Workspace private group to your desired group. Note: Using the Deploy action will create a copy of each virtual machine, which doubles the storage requirements.

Up to 15 minutes depending on system resources. 1 Minute.

7. Log into the Enterprise Manager Grid Control portal. 8. Change the ownership of the OMR and OMS virtual machines.

Up to 6 hours depending on system resources.

Watch this space Oracle OpenWorld 2011 Call for Papers - Submit Today! Temporary Post Used For Theme Detection (409fc5e9-d7ee-4e1a-b645-8b82f256f434 - 3bfe001a-32de-4114-a6b4-4005b770f6d7) Debugging Oracle VM 2.2 & Oracle VM Manager 2.2 errors using the many log files New VDI Training Course now available! HIMSS 2011 and New Press Release Oracle Desktop Virtualization at HIMSS 2011 Oracle VM VirtualBox 4.0.4 Released! Two Virtualization Webinars This Week We're Hiring! - Server and Desktop Virtualization Product Management

Step 1: Download the Template
Copyright © 2009 - 2011 Roddy Rodstein. All rights reserved. Show a printer friendly version of this book with its sub pages Step 1 will require a visit to http://edelivery.oracle.com/linux to register, search for and to download the Oracle VM Enterprise Manager Grid Control Template. 1. Point a web browser to http://edelivery.oracle.com/linux as show in Figure 2 click the Continue button to proceed.

200 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

2. From the Registration page enter a Full Name, Company Name, Email Address, select a Country from the drop down box, accept the Agreement Terms by selecting the Agreement Terms check box, accept the Export Restrictions by selecting the Export Restrictions check box and click Continue to proceed. Figure 3 shows the Registration page.

3. From the Media Pack Search page, select Oracle VM Templates from the Select a Product Pack drop down menu. Then select x86 32 bit from the Platform drop down menu and click the Go button to proceed as shown in Figure 4.

201 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

4. As shown in Figure 5, from the Media Pack Search results page select the Oracle VM Enterprise Manager Grid Control Template for Oracle VM radio button and click the Continue button to proceed to the Oracle VM Grid Control Template 1.0 Media Pack for x86 (32 bit) download page.

5. From the Oracle VM Grid Control Template 1.0 Media Pack for x86 (32 bit) download page click the Download button next to each part of compressed files (14.81 GB total) to start the download. Figure 6 shows the Oracle VM Grid Control Template 1.0 Media Pack for x86 (32 bit) download page.

202 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Watch this space Oracle OpenWorld 2011 Call for Papers - Submit Today! Temporary Post Used For Theme Detection (409fc5e9-d7ee-4e1a-b645-8b82f256f434 - 3bfe001a-32de-4114-a6b4-4005b770f6d7) Debugging Oracle VM 2.2 & Oracle VM Manager 2.2 errors using the many log files New VDI Training Course now available! HIMSS 2011 and New Press Release Oracle Desktop Virtualization at HIMSS 2011 Oracle VM VirtualBox 4.0.4 Released! Two Virtualization Webinars This Week We're Hiring! - Server and Desktop Virtualization Product Management

Step 2: Unzip and Concatenate Both Sets of Files
Copyright © 2009 - 2011 Roddy Rodstein. All rights reserved. Show a printer friendly version of this book with its sub pages In Step 1 we downloaded two sets of files, the first set of files has both the OMR and OMS file set and the second file set is the Yum repository. List 1 shows the compressed 5.69 GB OMR/OMS file set. V13510-01_1of4.zip V13510-01_2of4.zip V13510-01_3of4.zip V13510-01_4of4.zip List 2 shows the compressed 9.12 GB Yum repository file set. V13511-01_1of5.zip V13511-01_2of5.zip V13511-01_3of5.zip V13511-01_4of5.zip V13511-01_5of5.zip

203 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

In Step 2 we will run through the procedure to unzip and concatenate both files sets in the /OVS/seed_pool directory as well as to add the yum.img file in the OMS vm.cfg file. Note: The procedure is identical with a local Oracle VM OVS repository or a centralized Oracle VM OVS repository. 1. Place the zip template files in the /OVS/seed_pool directory as shown in the example: /OVS/seed_pool/V13510-01_1of4.zip /OVS/seed_pool/V13510-01_2of4.zip /OVS/seed_pool/V13510-01_3of4.zip /OVS/seed_pool/V13510-01_4of4.zip /OVS/seed_pool/V13511-01_1of5.zip /OVS/seed_pool/V13511-01_2of5.zip /OVS/seed_pool/V13511-01_3of5.zip /OVS/seed_pool/V13511-01_4of5.zip /OVS/seed_pool/V13511-01_5of5.zip 2. From the Oracle VM Server console login as root and unzip the files using the unzip command as shown in the following example. # unzip 'V*.zip' Archive: V13510-01_1of4.zip inflating: GC_Container.tar.gz.1of4 Archive: V13510-01_2of4.zip inflating: GC_Container.tar.gz.2of4 Archive: V13510-01_3of4.zip inflating: GC_Container.tar.gz.3of4 Archive: V13510-01_4of4.zip inflating: GC_Container.tar.gz.4of4 Archive: V13511-01_1of5.zip inflating: yum.tar.gz.1of5 Archive: V13511-01_2of5.zip inflating: yum.tar.gz.2of5 Archive: V13511-01_3of5.zip inflating: yum.tar.gz.3of5 Archive: V13511-01_4of5.zip inflating: yum.tar.gz.4of5 Archive: V13511-01_5of5.zip inflating: yum.tar.gz.5of5 Unzipping all of the file sets will generate the following tar files: /OVS/seed_pool/GC_Container.tar.gz.1of4 /OVS/seed_pool/GC_Container.tar.gz.2of4 /OVS/seed_pool/GC_Container.tar.gz.3of4 /OVS/seed_pool/GC_Container.tar.gz.4of4 /OVS/seed_pool/yum.tar.gz.1of5 /OVS/seed_pool/yum.tar.gz.2of5 /OVS/seed_pool/yum.tar.gz.3of5 /OVS/seed_pool/yum.tar.gz.4of5 /OVS/seed_pool/yum.tar.gz.5of5 3. Next we will concatenate the OMR/OMS file set. The next example shows how to concatenate the OMR/OMS file set. # cat GC_Container.tar.gz.1of4 GC_Container.tar.gz.2of4 GC_Container.tar.gz.3of4 GC_Container.tar.gz.4of4 | tar xzvf GC_DB_TEMPLATE/ GC_DB_TEMPLATE/vm.cfg GC_DB_TEMPLATE/README GC_DB_TEMPLATE/system.img GC_DB_TEMPLATE/db.img GC_TEMPLATE/ GC_TEMPLATE/vm.cfg GC_TEMPLATE/README GC_TEMPLATE/gc.img GC_TEMPLATE/system.img README The GC_Container contains the Oracle VM images for: Oracle Enterprise 4 Update 5 for x86 Oracle Enterprise Manager Grid Control Release 10.2.0.4 for Linux x86 Oracle Enterprise Manager Grid Control Release 10.2.0.4 database repository for Linux x86 4. Next we will concatenate the Yum repository file set. The next example shows how to concatenate the Yum repository file set.

204 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

# cat yum.tar.gz.1of5 yum.tar.gz.2of5 yum.tar.gz.3of5 yum.tar.gz.4of5 yum.tar.gz.5of5 | tar xzvf README yum.img The yum.img file contains a Yum repository populated with the full Linux distribution RPMs for: Oracle Oracle Oracle Oracle Enterprise Enterprise Enterprise Enterprise Linux Linux Linux Linux Release Release Release Release 4 4 5 5 Update Update Update Update 6 6 1 1 for for for for x86 x86_64 x86 x86_64

5. The final step is to add the yum.img virtual disk to the OMS virtual machine. We will modify the /OVS/seed_pool/GC_TEMPLATE /vm.cfg file to include the Yum repository virtual disk. The next example shows the default unmodified OMS vm.cfg file. # OracleVM XEN config file name = "EMGC" memory = "2048" disk = [ 'file:/OVS/seed_pool/GC_TEMPLATE/system.img,hda,w', 'file:/OVS/seed_pool/GC_TEMPLATE/gc.img,hdb,w'] vif = [ '', ] bootloader="/usr/bin/pygrub" vcpus=2 vfb = [ 'type=vnc,vncunused=1,vnclisten=0.0.0.0' ] on_reboot = 'restart' on_crash = 'restart' Use your favorite text editor for example vi to add “'file:/OVS/seed_pool/GC_ TEMPLATE/yum.img,hdd,w'” to the end of the disk attribute in the vm.cfg file as shown in the following example. # OracleVM XEN config file name = "EMGC" memory = "2048" disk = [ 'file:/OVS/seed_pool/GC_TEMPLATE/system.img,hda,w', 'file:/OVS/seed_pool/GC_TEMPLATE/gc.img,hdb,w', 'file:/OVS /seed_pool/GC_ TEMPLATE/yum.img,hdd,w'] vif = [ '', ] bootloader="/usr/bin/pygrub" vcpus=2 vfb = [ 'type=vnc,vncunused=1,vnclisten=0.0.0.0' ] on_reboot = 'restart' on_crash = 'restart'

Watch this space Oracle OpenWorld 2011 Call for Papers - Submit Today! Temporary Post Used For Theme Detection (409fc5e9-d7ee-4e1a-b645-8b82f256f434 - 3bfe001a-32de-4114-a6b4-4005b770f6d7) Debugging Oracle VM 2.2 & Oracle VM Manager 2.2 errors using the many log files New VDI Training Course now available! HIMSS 2011 and New Press Release Oracle Desktop Virtualization at HIMSS 2011 Oracle VM VirtualBox 4.0.4 Released! Two Virtualization Webinars This Week We're Hiring! - Server and Desktop Virtualization Product Management

Step 3: Create the OMR and OMS Virtual Machine Templates
Copyright © 2009 - 2011 Roddy Rodstein. All rights reserved. Show a printer friendly version of this book with its sub pages In Step 2 we walked through the procedure to unzip and concatenate both files sets and we added an entry for the yum.img file in the OMS vm.cfg file. In Step 3 we will use Oracle VM Manager to create an OMR and OMS virtual machine template. 1. Login to Oracle VM Manager. Enter the desired user name and password and click the Login button to proceed as shown in Figure 7.

205 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

2. From the Virtual Machine home page click the Resources tab to access the Virtual Machine Template page. From the Virtual Machine Template page click the Import button to start the importation process. Figure 8 shows the Virtual Machine Template page.

3. From the Source page select the Select from Server Pool (Discover and register) radio button and click Next as shown in Figure 9.

206 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

4. From the General Information page we need to enter the following data: a. Select the desired server pool from the Server Pool Name drop down list. b. Select the CG_DB_Template from the Virtual Machine Template Name drop down list. c. Enable High Availability check box if desired, HA will need to be setup in advance to enable HA. d. Select Oracle Enterprise Linux 4 from the Operating System drop down list. e. Enter root in the Virtual Machine User Name text box. f. Enter ovsroot in the Virtual Machine Password text box. g. Enter a Description for the virtual machine (optional) h. Click Next to proceed to the Confirmation page. Figure 10 shows the General Information page.

5. From the Confirm Information page click the Confirm button to proceed. Figure 11 shows the Confirm Information page.

6. We have successfully created the CG_DB_Template virtual machine template, which is in a pending status. Templates that are in the pending status are not available to Oracle VM Manager users until they have been approved. Next we will approve the CG_DB_Template by selecting the CG_DB_Template and clicking the Approve button as shown in Figure 12.

207 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

7. From the Virtual Machine Templates page click the Approve button to approve the template as shown in Figure 13.

8. The CG_DB_Template has been successfully approved and its Status has changed to Active as shown in Figure 14.

208 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Next we will repeat the proceeding procedure for the GC_TEMPLATE. 1. From the Virtual Machine home page click the Resources Tab to access the Virtual Machine Template page. From the Virtual Machine Template page click the Import button to start the importation process. Figure 15 shows the Virtual Machine Template page.

2. From the Source page select the Select from Server Pool (Discover and register) radio button and click Next as shown in Figure 16 to proceed.

209 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

3. From the General Information page we need to enter the following data: a. Select the desired server pool from the Server Pool Name drop down list. b. Select the CG_Template from the Virtual Machine Template Name drop down list. c. Select the Enable High Availability check box if desired. HA will need to be setup in advance to enable HA. d. Select Oracle Enterprise Linux 4 from the Operating System drop down list. e. Enter root in the Virtual Machine User Name text box. f. Enter ovsroot in the Virtual Machine Password text box. g. Enter a Description for the virtual machine (optional). h. Click Next to proceed to the Confirmation page. Figure 17 shows the General Information page.

4. From the Confirm Information page click the Confirm button to proceed. Figure 18 shows the Confirm Information page.

210 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

5. We have successfully created the CG_DB_Template virtual machine template, which is in the pending status. Templates that are in the pending status are not available to Oracle VM Manager users until they have been approved. Next we will approve the CG_DB_Template by selecting the CG_DB_Template radio button and clicking the Approve button as shown in Figure 19.

6. From the View Virtual Machine Template page click the Approve button to approve the template as shown in Figure 20.

211 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

7. The CG_Template has been successfully approved and is now in the Active State as shown in Figure 21.

Watch this space Oracle OpenWorld 2011 Call for Papers - Submit Today! Temporary Post Used For Theme Detection (409fc5e9-d7ee-4e1a-b645-8b82f256f434 - 3bfe001a-32de-4114-a6b4-4005b770f6d7) Debugging Oracle VM 2.2 & Oracle VM Manager 2.2 errors using the many log files New VDI Training Course now available! HIMSS 2011 and New Press Release Oracle Desktop Virtualization at HIMSS 2011 Oracle VM VirtualBox 4.0.4 Released! Two Virtualization Webinars This Week We're Hiring! - Server and Desktop Virtualization Product Management

Step 4: Create the OMR and OMS Virtual Machines
212 of 243 18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Copyright © 2009 - 2011 Roddy Rodstein. All rights reserved. Show a printer friendly version of this book with its sub pages In Step 3 we used Oracle VM Manager to create an OMR and OMS template. Next in Step 4 we will use Oracle VM Manager to create the OMR and OMS virtual machines. 1. From the Virtual Machines page click one of the two Create Virtual Machine buttons to proceed. Figure 22 shows the Virtual Machines page with two Create Virtual Machine buttons.

2. From the Creation Method page select the Create virtual machine based on virtual machine template radio button, then click Next to proceed as shown in Figure 23.

3. From the Server Pool page select the desired Server Pool radio button and accept the default Preferred Server setting. Click Next to proceed as shown in Figure 24.

213 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

4. From the Source page as shown in Figure 25 select the CG_DB_TEMPLATE radio button and click Next to proceed.

5. From the Virtual Machine Information page enter the following data: a. Virtual Machine Name b. Console Password c. Confirm Console Password d. Select the Enable High Availability check box if desired. HA will need to be configured in advance to enable HA. e. Select the desired Network Interface Card and Bridge. i. The Network Interface Card can be deleted by selecting the Delete radio button. ii. The Add Row radio button allows us to create up to three Network Interface Cards iii. The Bridge drop down list will allow us to select any configured bridges. f. Click Next to proceed. Figure 26 shows the Virtual Machine Information page.

6. As shown in Figure 27, from the Confirm Information page click Confirm to proceed.

214 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

7. The web browser will automatically redirect to the Virtual Machine page. From the Virtual Machine page validate the status of the new virtual machine, as Creating. When the creation process is completed the status will change to Powered Off. Figure 28 shows the new virtual machine status as Creating.

8. From the Virtual Machines page click one of the two Create Virtual Machine buttons to proceed. Figure 29 shows the Virtual Machines page with two Create Virtual Machine buttons.

215 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

9. From the Creation Method page select the Create virtual machine based on virtual machine template radio button, then click Next to proceed as shown in Figure 30.

10. From the Server Pool page select the desired Server Pool radio button and accept the default Auto Preferred Server option. Click Next to proceed as shown in Figure 31.

11. From the Source page as shown in Figure 32, select the CG_TEMPLATE radio button and click Next to proceed.

216 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

12. From the Virtual Machine Information page enter the following data: a. Virtual Machine Name b. Console Password c. Confirm Console Password d. Select the Enable High Availability check box if desired. HA will need to be configured in advance to enable HA. e. Select the desired Network Interface Card and Bridge. i. The Network Interface Card can be deleted by selecting the Delete radio button. ii. The Add Row radio button allows us to create up to three Network Interface Cards iii. The Bridge drop down list will allow us to select any configured bridges. f. Click Next to proceed. Figure 33 shows the Virtual Machine Information page.

13. As shown in Figure 34, from the Confirm Information page click the Confirm button to proceed.

217 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

14. The web browser will automatically redirect to the Virtual Machine page. From the Virtual Machine page validate the status of the new virtual machine, as Creating. When the creation process is completed the status will change to Powered Off. Figure 35 shows the new virtual machine status as Creating.

15. When the creation process is completed the status will change to Powered Off as shown in Figure 36.

218 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Watch this space Oracle OpenWorld 2011 Call for Papers - Submit Today! Temporary Post Used For Theme Detection (409fc5e9-d7ee-4e1a-b645-8b82f256f434 - 3bfe001a-32de-4114-a6b4-4005b770f6d7) Debugging Oracle VM 2.2 & Oracle VM Manager 2.2 errors using the many log files New VDI Training Course now available! HIMSS 2011 and New Press Release Oracle Desktop Virtualization at HIMSS 2011 Oracle VM VirtualBox 4.0.4 Released! Two Virtualization Webinars This Week We're Hiring! - Server and Desktop Virtualization Product Management

Step 5: Start the OMR and OMS Virtual Machines and Enter Setup Values
Copyright © 2009 - 2011 Roddy Rodstein. All rights reserved. Show a printer friendly version of this book with its sub pages In Step 4 we used Oracle VM Manager to create the OMR and OMS virtual machines from the templates we created in Step 3. In Step 5 we will used Oracle VM Manager to start the virtual machine that was created with the GC_DB_TEMPLATE, access the virtual machine’s console and enter values for a number of setup options. Once we have successfully entered the setup values for the OMR virtual machine we will start the OMS virtual machine (created from the GC_TEMPLATE template), we will access the virtual machine’s console and will enter values for a number of setup options. Note: Its necessary to start and configure the OMR virtual machine first followed by the OMS virtual machine. List 3 and 4 show the default user names and password for the OMR and OMS virtual machines and the virtual machine console password. List 3 shows the default user name and passwords for the virtual machines. User Name: root Password: ovsroot User Name: oracle Password: oracle List 4 shows the default password used to access the virtual machine console from Oracle VM Manager. Password: oracle

Prerequisites
Before the OMR and OMS virtual machines are started ensure that your DNS server has entries for the OMR and OMS virtual machines and that you have selected and documented the following setup options: Select Select Select Select Select Select a static IP address the network mask a hostname (FQDN) the default gateway the DNS server IP address a single password for the SYS, SYSTEM, SYSMAN and DBSNMP database accounts

*The password must be at least 6 characters in length and contain at least one numeral character. Setup Tips: After the virtual machine is started, if you do not enter a password for the database accounts, the setup will fail and you will need to restart the virtual machine. After the virtual machine is started, if the setup hangs towards at the end of the setup process, allow up to 20 minutes for the process to automatically complete. Once the setup is completed you will be presented with the logon screen. After the virtual machine is started, if the setup is not completed, the next time the virtual machine starts the setup process will start over from the beginning. The following section will run through the steps to start and enter the setup data for the OMR virtual machine. 1. Login to Oracle VM Manager and select the OMR virtual machine and click the Power On button as shown in Figure 37. The Status will change from Powered Off to Initializing.

219 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

2. Next click the Refresh button until the status changes to Running. Once the status is Running click the Console button to access the OMR virtual machine console as shown in Figure 38.

3. Enter the default password oracle and click the OK button to proceed. Figure 39 shows the OMR virtual machine console where the setup options are entered.

220 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

4. Enter the IP address and press Enter. 5. Enter the netmask address and press Enter. 6. Enter the hostname (FQDN) and press Enter. 7. Enter the default gateway and press Enter. 8. Enter the DNS server IP address and press Enter. 9. Enter the full pathname of the local bin directory: [/usr/local/bin]: If you want to use the default directory, leave this field empty and press Enter. 10. Enter a password for database accounts SYS, SYSTEM, SYSMAN and DBSNMP. The password must be at least 6 characters in length that contains at least one numeral character. Enter the password and press Enter. 11. Once presented with the following banner press any key on the keyboard to complete the setup process. ====================================================================== Press any key to continue... ---------------------------------------------------------------------The following section will run through the steps to start and enter the setup data for the OMS virtual machine. 1. Login to Oracle VM Manager and select the OMS virtual machine and click the Power On button as shown in Figure 40. The Status will change from Powered Off to Initializing.

2. Next click the Refresh button until the status changes to Running. Once the status is Running click the Console button to access the OMS virtual machine console as shown in Figure 41.

221 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Enter the default password oracle and click the OK button to proceed. Figure 42 shows the OMS virtual machine console where the setup options are entered.

3. Enter the IP address and press Enter. 4. Enter the netmask address and press Enter. 5. Enter the hostname (FQDN) and press Enter. 6. Enter the default gateway and press Enter. 7. Enter the DNS server IP address and press Enter. 8. Enter the full pathname of the local bin directory: [/usr/local/bin]: Enter the pathname of the local bin directory. If you want to use the default directory, leave this field empty and press Enter. 9. Enter repository database hostname and press Enter. 10. Enter the password you created for the OMR repository SYSMAN user and press Enter. 11. Enter new password for OMA Secure Registration and press Enter. 12. Enter the full pathname of the local bin directory: [/usr/local/bin]: If you want to use the default directory, leave this field empty and press Enter. 13. If any of the dbhome, oraenv, coraenv files already exist, you are prompted to overwrite them. At each prompt, enter y and press Enter to overwrite the file. The file "dbhome" already exists in /usr/local/bin. Overwrite it? (y/n) [n]: y

222 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

The file "oraenv" already exists in /usr/local/bin. Overwrite it? (y/n) [n]: y The file "coraenv" already exists in /usr/local/bin. Overwrite it? (y/n) [n]: y 14. Once presented with the following banner press any key on the keyboard to complete the setup process. ====================================================================== Press any key to continue... Watch this space Oracle OpenWorld 2011 Call for Papers - Submit Today! Temporary Post Used For Theme Detection (409fc5e9-d7ee-4e1a-b645-8b82f256f434 - 3bfe001a-32de-4114-a6b4-4005b770f6d7) Debugging Oracle VM 2.2 & Oracle VM Manager 2.2 errors using the many log files New VDI Training Course now available! HIMSS 2011 and New Press Release Oracle Desktop Virtualization at HIMSS 2011 Oracle VM VirtualBox 4.0.4 Released! Two Virtualization Webinars This Week We're Hiring! - Server and Desktop Virtualization Product Management

Step 6: Installing the OMA Client on the OMR Virtual Machine
Copyright © 2009 - 2011 Roddy Rodstein. All rights reserved. Show a printer friendly version of this book with its sub pages In Step 5 we used Oracle VM Manager to start both the OMR and OMS virtual machines, access the virtual machine’s console and enter values for a number of setup options. Next in Step 6 we will install OMA using the installagent.sh script. The following procedure will walk through the installation of the Oracle Management Agent (OMA) on the OMR virtual machine. 1. Log into the OMR virtual machine via the Oracle VM Manager console or via an SSH client as root using the default ovsroot password. 2. Run the OMA installation script named installagent.sh which is located in the /u01/scripts directory, e.g. #cd /u01/scripts #./installagent.sh 3. Enter the Agent Registration password and press Enter. 4. Enter the OMR SYSMAN password and press Enter. 5. Enter the full pathname of the local bin directory: [/usr/local/bin]: If you want to use the default directory, leave this field empty and press Enter. 6. If any of the dbhome, oraenv, coraenv files already exist, you are prompted to overwrite them. At each prompt, enter y and press Enter to overwrite the file. The file "dbhome" already exists in /usr/local/bin. Overwrite it? (y/n) [n]: y The file "oraenv" already exists in /usr/local/bin. Overwrite it? (y/n) [n]: y The file "coraenv" already exists in /usr/local/bin. Overwrite it? (y/n) [n]: y 7. Once presented with the following banner press any key on the keyboard to complete the setup process. ====================================================================== Press any key to continue... The next example shows a complete OMA install using the /u01/scripts/installagent.sh script. login as: root root@192.168.4.13's password: ovsroot Last login: Tue Oct 21 16:13:19 2008 # /u01/scripts/installagent.sh ================================================================== Start installing Management Agent 10.2.0.4 X86 ================================================================== Extracting the source files... Provide Management Service hostname that the Management Agent willl communicate with. Enter OMS hostname:v45oms10g1 Provide the Agent Registration password so that the Management Agent can communicate with Secure Management Service. Enter Agent Registration Password: Cloning agent... Starting Oracle Universal Installer... No pre-requisite checks found in oraparam.ini, no system pre-requisite checks will be executed. Preparing to launch Oracle Universal Installer from /tmp/OraInstall2008-10-21_05-44-30PM. Please wait ...Oracle Universal Installer,

223 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Version 10.2.0.4.0 Production Copyright (C) 1999, 2007, Oracle. All rights reserved. You can find a log of this install session at: /u01/app/oraInventory/logs/cloneActions2008-10-21_05-44-30PM.log .................................................................................................... 100% Done.

Installation in progress (Tuesday, October 21, 2008 5:44:40 PM EDT) ..................................................................... Install successful Linking in progress (Tuesday, October 21, 2008 5:44:46 PM EDT) . 70% Done. Link successful Setup in progress (Tuesday, October 21, 2008 5:45:17 PM EDT) .................... 100% Done. Setup successful

69% Done.

End of install phases.(Tuesday, October 21, 2008 5:45:24 PM EDT) Starting to execute configuration assistants The following configuration assistants have not been run. This can happen for following reasons - either root.sh is to be run before config or Oracle Universal Installer was invoked with the -noConfig option. -------------------------------------The "/u01/app/oracle/product/agent10g/cfgtoollogs/configToolFailedCommands" script contains all commands that failed, were skipped or were cancelled. This file may be used to run these configuration assistants outside of OUI. Note that you may have to update this script with passwords (if any) before executing the same. The "/u01/app/oracle/product/agent10g/cfgtoollogs/configToolAllCommands" script contains all commands to be executed by the configuration assistants. This file may be used to run the configuration assistants outside of OUI. Note that you may have to update this script with passwords (if any) before executing the same. -------------------------------------The cloning of cloneagent10g was successful. Please check '/u01/app/oraInventory/logs/cloneActions2008-10-21_05-44-30PM.log' for more details. Running /u01/app/oracle/product/agent10g/root.sh... Running Oracle10 root.sh script... The following environment variables are set as: ORACLE_OWNER= oracle ORACLE_HOME= /u01/app/oracle/product/agent10g Enter the full pathname of the local bin directory: [/usr/local/bin]: The file "dbhome" already exists in /usr/local/bin. Overwrite it? (y/n) [n]: y Copying dbhome to /usr/local/bin ... The file "oraenv" already exists in /usr/local/bin. Overwrite it? (y/n) [n]: y Copying oraenv to /usr/local/bin ... The file "coraenv" already exists in /usr/local/bin. Overwrite it? (y/n) [n]: y Copying coraenv to /usr/local/bin ... Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root.sh script. Now product-specific root actions will be performed. Finished product-specific root actions. Enter password of database account SYSMAN: Oracle Enterprise Manager 10g Release 4 Grid Control 10.2.0.4.0. Copyright (c) 1996, 2007 Oracle Corporation. All rights reserved. Agent is already stopped... Done. Securing agent... Started. Requesting an HTTPS Upload URL from the OMS... Done. Requesting an Oracle Wallet and Agent Key from the OMS... Done. Check if HTTPS Upload URL is accessible from the agent... Done. Configuring Agent for HTTPS in CENTRAL_AGENT mode... Done. EMD_URL set in /u01/app/oracle/product/agent10g/sysman/config/emd.properties Securing agent... Successful. Stopping the agent using /u01/app/oracle/product/agent10g/bin/emctl stop agent

224 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Oracle Enterprise Manager 10g Release 4 Grid Control 10.2.0.4.0. Copyright (c) 1996, 2007 Oracle Corporation. All rights reserved. Agent is Not Running Running agentca using /u01/app/oracle/product/agent10g/oui/bin/runConfig.sh ORACLE_HOME=/u01/app/oracle/product/agent10g ACTION=Configure MODE=Perform RESPONSE_FILE=/u01/app/oracle/product/agent10g/response_file RERUN=TRUE INV_PTR_LOC=/etc/oraInst.loc COMPONENT_XML={oracle.sysman.top.agent.10_2_0_1_0.xml} Perform - mode is starting for action: Configure

Perform - mode finished for action: Configure You can see the log file: /u01/app/oracle/product/agent10g/cfgtoollogs/oui/configActions2008-10-21_05-46-19-PM.log Oracle Enterprise Manager 10g Release 4 Grid Control 10.2.0.4.0. Copyright (c) 1996, 2007 Oracle Corporation. All rights reserved. Stopping agent ... stopped. Stopping crond: [ OK ] Starting crond: [ OK ] Oracle Enterprise Manager 10g Release 4 Grid Control 10.2.0.4.0. Copyright (c) 1996, 2007 Oracle Corporation. All rights reserved. Starting agent ....... started. Oracle Enterprise Manager 10g Release 4 Grid Control 10.2.0.4.0. Copyright (c) 1996, 2007 Oracle Corporation. All rights reserved. --------------------------------------------------------------EMD upload completed successfully =================================================================================== Management Agent 10.2.0.4 X86 cloning completed successfully. The following env variables are set for user oracle: ORACLE_BASE = /u01/app AGENT_HOME = /u01/app/oracle/product/agent10g =================================================================================== Press any key to continue... # Watch this space Oracle OpenWorld 2011 Call for Papers - Submit Today! Temporary Post Used For Theme Detection (409fc5e9-d7ee-4e1a-b645-8b82f256f434 - 3bfe001a-32de-4114-a6b4-4005b770f6d7) Debugging Oracle VM 2.2 & Oracle VM Manager 2.2 errors using the many log files New VDI Training Course now available! HIMSS 2011 and New Press Release Oracle Desktop Virtualization at HIMSS 2011 Oracle VM VirtualBox 4.0.4 Released! Two Virtualization Webinars This Week We're Hiring! - Server and Desktop Virtualization Product Management

Step 7: Access the Enterprise Manager Grid Control Portal
Copyright © 2009 - 2011 Roddy Rodstein. All rights reserved. Show a printer friendly version of this book with its sub pages In Step 6 we installed OMA on the OMS virtual machine using the installagent.sh script. Next we will access the Enterprise Manager Grid Control portal using the sysman account and the password that was selected during Step 5. Once logged into the Enterprise Manager Grid Control portal please refer to http://download.oracle.com/docs/cd/B16240_01 /welcome.html for the Enterprise Manager Grid Control configuration and setup documentation. 1. Point a web browser to the FQDN of the OMS virtual machine i.e. http://FQDN:4889/em/ to access the Enterprise Manager Grid Control portal. From the Login page enter sysman as the User Name and the password that was selected during Step 5. Then press the Login button to proceed as shown in Figure 43.

225 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

2. Figure 44 shows the Enterprise Manager Home page.

Watch this space Oracle OpenWorld 2011 Call for Papers - Submit Today! Temporary Post Used For Theme Detection (409fc5e9-d7ee-4e1a-b645-8b82f256f434 - 3bfe001a-32de-4114-a6b4-4005b770f6d7) Debugging Oracle VM 2.2 & Oracle VM Manager 2.2 errors using the many log files New VDI Training Course now available! HIMSS 2011 and New Press Release Oracle Desktop Virtualization at HIMSS 2011 Oracle VM VirtualBox 4.0.4 Released! Two Virtualization Webinars This Week We're Hiring! - Server and Desktop Virtualization Product Management

Step 8: Change the Ownership of the OMR and OMS Virtual Machines from the My Workspace Group
Copyright © 2009 - 2011 Roddy Rodstein. All rights reserved. Show a printer friendly version of this book with its sub pages In Step 7 we accessed the Enterprise Manager Grid Control portal using the sysman account and the password that was selected during Step 5. Next we will change the ownership of the OMR and OMS virtual machines from the default My Workspace group to the operations group. Note: You will need to have created groups in Oracle VM Manager to complete this step. By default when a virtual machine is created in Oracle VM Manager, it’s owned by the administrator that created the virtual machine. By default when a virtual machine is created it will be placed in the My Workspace group, which makes the virtual machine visible exclusively to the virtual machine owner. For example, both the OMR and OMS virtual machines from the above examples are only visible in Oracle VM Manager by the administrator that created the virtual machines. To allow other Oracle VM Manager users to view and manage the virtual machine the Deploy action is used to change the ownership from the My Workspace group to a different group. To use the Deploy action the virtual machine must be powered off.

226 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

The next section will walk through the procedure to change the ownership of the OMR virtual machine. 1. From Oracle VM Manager ensure that the OMR virtual machine is Powered Off and then select the OMR virtual machine. Once the OMR virtual machine is selected click the More Actions drop down menu and select the Deploy action. Then click the Go button to start the Deploy action as shown in Figure 45.

2. From the Virtual Machine Information page enter the Virtual Machine Name in the text box, select the desired group from the Group Name drop down list and select the desired server pool name from the Server Pool Name drop down list as shown in Figure 46. Click Next to proceed.

3. From the Confirm Information page click the Confirm button to start the deploy action as shown in Figure 47.

4. As shown in Figure 48 the status of the My Workspace virtual machine changed from Powered Off to Deploying and the deployed virtual machine status is Creating.

227 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

When the Deploy action completes, each virtual machine’s status changes to Powered Off. The deploy action results in two identical virtual machines with different group membership. The original virtual machine remains in the My Workspace group and the Deployed virtual machine will be in the group selected from the Virtual Machine Information page. Repeat the process for the OMS virtual machine to change the group membership. Watch this space Oracle OpenWorld 2011 Call for Papers - Submit Today! Temporary Post Used For Theme Detection (409fc5e9-d7ee-4e1a-b645-8b82f256f434 - 3bfe001a-32de-4114-a6b4-4005b770f6d7) Debugging Oracle VM 2.2 & Oracle VM Manager 2.2 errors using the many log files New VDI Training Course now available! HIMSS 2011 and New Press Release Oracle Desktop Virtualization at HIMSS 2011 Oracle VM VirtualBox 4.0.4 Released! Two Virtualization Webinars This Week We're Hiring! - Server and Desktop Virtualization Product Management

Oracle VM Centralized Logging
Oracle VM for x86 Troubleshooting Made Easier
Author Martin Foster Editor Roddy Rodstein By default Oracle VM 2.x logs all events locally. Logging events locally makes troubleshooting Oracle VM server pool issues a challenge, because different log information is being echoed to different local log files. In this chapter, we will walk through a centralized logging configuration for Oracle VM that makes troubleshooting Oracle VM much easier when compared to the default local Oracle VM logging configuration. As of Oracle VM 2.2, the Oracle VM agent's logging functionality is customizable by using the Python Logger class configuration file. The Oracle VM Manager application runs on OC4J, a JSP container that has log4j style configuration capabilities. Both log4j and python's logger do support logging to syslog. Change Log Revision Change Description 1 Draft Release Updated By Roddy Rodstein Date 03/28/11

Table of Contents Change Log Upgrade the Oracle VM Server and Manager Local Syslog Daemon Oracle VM Server Syslog Configuration ...Example /etc/ovs-agent/logger_server.ini file How to Configure Rsyslog on Oracle VM Server ...Example /etc/rsyslog.conf file ...Example /etc/sysconfig/rsyslog file How to Configure the Central Log Host ...Example /etc/sysconfig/rsyslog file The Oracle VM Manager Syslog Configuration ...OC4J Logging

228 of 243

18.04.2011 16:35

The Underground Oracle VM Manual
...Oracle VM Manager Logging Configuration Files …...Example /etc/rsyslog.conf file ...How to Make Rsyslog Read the oc4j.log file ...Oracle VM Manager /etc/rsyslog.conf and Adding imfile Support …...Example /etc/rsyslog.conf file ...Empty the Existing Oracle VM Manager oc4j.log ...Configure /etc/rsyslog.conf to Forward local2 Events Configuring the Central Log Host ...Example central log host /etc/rsyslog.conf file How to Configure the Oracle VM Server's Logrotation

http://itnewscast.com/book/export/html/844

Upgrade the Oracle VM Server and Manager Local Syslog Daemon
The first step to configure centralized logging for Oracle VM is to upgrade the local syslog daemon to the Oracle Linux 5.5 rsyslog package on the Oracle VM servers and on the Oracle VM Manager host. Note: The default Oracle VM 2.x server configuration does not have rsyslog. The next three steps show how to install and configure rsyslog on an Oracle VM 2.x server and on an Oracle VM Manager x86 or x86-x64 host: 1. Download and install the Oracle Linux 5.5 rsyslog rpm (3.22 at time of writing) using wget and the rpm programs. Oracle VM Server: The Oracle VM server will always use the i386 RPM regardless of the hardware platform, i.e. both x86 or x86-x64 servers both use a x86 dom0 and will use the i386 RPM package. Oracle VM Manager: Depending on the hardware/OS platform for your Oracle VM Manager host, use the i386 RPM package for x86 or the x86_64 package x64. The next two examples show how to download and install the rsyslog rpm package for the i386 and x86-64 platforms using wget and the rpm programs. I386 # wget http://public-yum.oracle.com/repo/EnterpriseLinux/EL5/5/base/i386/rsyslog-3.22.1-3.el5.i386.rpm # rpm -Uvh rsyslog-3.22.1-3.el5.i386.rpm x86-64 # wget http://public-yum.oracle.com/repo/EnterpriseLinux/EL5/5/base/x86_64/rsyslog-3.22.1-3.el5.x86_64.rpm # rpm -Uvh rsyslog-3.22.1-3.el5.x86_64.rpm 2. In the next example, we use the syslog configuration file for rsyslog. We also disable syslog and enable rsyslog: # # # # # cp /etc/syslog.conf /etc/rsyslog.conf chkconfig syslog off chkconfig rsyslog on service syslog stop service rsyslog start 3. Next, check /var/log/messages to validate that rsyslog has started. For example, type “tail /var/log/messages”

Oracle VM Server Syslog Configuration
The /etc/ovs-agent/logger_server.ini file stores the logger class configuration and is read when the Oracle VM agent starts. Note: For more information please refer to: http://docs.python.org/library/logging.html#sysloghandler The next list shows the changes that will be made to the ovs-agent Python logger: 1. 2. 3. 4. 5. 6. 7. Maintain the various handlers that Oracle uses, for consistency with Oracle support. Propagate all handers to the parent (root) handler; all information logged by the ovs-agent will be available at this handler. Set the log level to NOTSET, which is everything (more than DEBUG). Forward logs to local syslog over udp/514 (default syslog port). Set all loggers to "propagate=1", so they forward up logs to their parent handlers. By default performance and macip logging doesn't propagate up. We need "propagate=1" for centralization. Write to unix socket /dev/log, which must be created by rsyslog!

Note: We don't use the localhost 514/udp destination because this will create a message from a hostname of localhost or 127.0.0.1, which is of no use for centralized logging. By writing to the socket, the syslog daemon appends its hostname, which is necessary for

229 of 243

18.04.2011 16:35

The Underground Oracle VM Manual
centralized logging. 8. Send all to syslog by altering the root handler 9. Create a separate formatter that matches the syslog "name: message" style.

http://itnewscast.com/book/export/html/844

Example /etc/ovs-agent/logger_server.ini file
The following example shows a logger_server.ini file that meets the above prerequisites: # cat /etc/ovs-agent/logger_server.ini [loggers] keys=root,performance,operation,query [logger_root] handlers=root level=NOTSET [logger_operation] qualname=ovs.operation handlers=operation level=DEBUG propagate=1 [logger_performance] qualname=ovs.performance handlers=performance level=DEBUG # default propagate is 0 propagate=1 [logger_query] qualname=ovs.query handlers=query level=DEBUG propagate=1 [logger_macip] qualname=ovs.macip handlers=macip level=DEBUG # default propagate is 0 propagate=1 ;---------------------------------------------------------------------[handlers] keys=root,performance,operation,query,macip [handler_root] class=handlers.RotatingFileHandler ;append to log file, and file size is 1M with 3 archive files args=("%(log.dir)s/ovs_root.log", "a", 1024*1024, 3) formatter=ovs level=DEBUG # Syslog handler # - log to local syslog daemon, which can forward to central loghost # - using unix socket, which must match the socket created by the syslog daemon # - could use UDP to localhost, which loses the originating host information # (get messages tagged with an IP that makes no sense centrally, like 127.0.0.1 # for the localhost config) # - SysLogHandler unix socket: args=('/dev/log', handlers.SysLogHandler.LOG_LOCAL3) # - SysLogHandler to 514/udp: (('localhost', handlers.SYSLOG_UDP_PORT), handlers.SysLogHandler.LOG_LOCAL3) # - if the level is set on the syslog handler, all messages sent through that handler # inherit this level. Preferable to set level on a per-logger basis # - if level inherited by the loggers, then a simple formatter can be used that best matches # the expectation of syslog parsers. see formatters section for details. eg: # format=%(name)s: %(message)s # # address: /dev/log unix socket

230 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

# facility: LOG_LOCAL3 # level: do not set at the handler level # ref: http://docs.python.org/library/logging.html#sysloghandler # class=handlers.SysLogHandler #args=(('localhost', handlers.SYSLOG_UDP_PORT), handlers.SysLogHandler.LOG_LOCAL3) args=('/dev/log', handlers.SysLogHandler.LOG_LOCAL3) formatter=syslog [handler_operation] class=handlers.RotatingFileHandler ;append to log file, and file size is 1M with 3 archive files args=("%(log.dir)s/ovs_operation.log", "a", 1024*1024, 3) formatter=ovs level=DEBUG [handler_performance] class=handlers.RotatingFileHandler ;append to log file, and file size is 1M with 3 archive files args=("%(log.dir)s/ovs_performance.log", "a", 1024*1024, 3) formatter=ovs level=DEBUG [handler_query] class=handlers.RotatingFileHandler ;append to log file, and file size is 1M with 3 archive files args=("%(log.dir)s/ovs_query.log", "a", 1024*1024, 3) formatter=ovs level=DEBUG [handler_macip] class=handlers.RotatingFileHandler ;append to log file, and file size is 1M with 2 archive files args=("%(log.dir)s/ovs_macip.log", "a", 1024*1024, 2) formatter=ovs level=DEBUG ;---------------------------------------------------------------------[formatters] keys=ovs,syslog [formatter_ovs] class=logging.Formatter format=%(asctime)s %(levelname)s=> %(message)s datefmt="%Y-%m-%d %H:%M:%S" # formatter for syslog # - no datefmt required, syslog tags with event time # - if level is not specified by the handler, then we do not need to add # it to the formatter # - formatter options # with level in syslog message field: format=%(name)s: %(levelname)s %(message)s # closest to traditional syslog: format=%(name)s: %(message)s # [formatter_syslog] class=logging.Formatter format=%(name)s: %(message)s

How to Configure Rsyslog on Oracle VM Server
The next list shows the steps to configure rsyslog on an Oracle VM server as root: 1. Create a UNIX syslog socket at /dev/log. 2. Eliminate the local3 channel from /var/log/messages to keep the ovs-agent logs in one place and not to spread the ovs-agent logs to multiple logfiles. 3. Route all local3 facility messages to a file, /var/log/ovs-agent/all.log. 4. Route all local3 facility messages to a central log host, over port 601/tcp. The following /etc/rsyslog.conf file example contains rsyslog specific configuration elements. The "@@" remote forwarding terminology means "use TCP", whereas "@" means "use UDP". Enter your system details in the following sections:

231 of 243

18.04.2011 16:35

The Underground Oracle VM Manual
<MAIL PROXY HOST NAME GOES HERE> <FQDN OF CENTRAL LOG HOST>

http://itnewscast.com/book/export/html/844

Example /etc/rsyslog.conf file
# cat /etc/rsyslog.conf # ----------------------------- Queues (required for forwarding) ----------------------------$WorkDirectory /var/spool/rsyslog $ActionQueueType LinkedList # use asynchronous processing $ActionQueueFileName srvrfwd # set file name, also enables disk mode $ActionResumeRetryCount -1 # infinite retries on insert failure $ActionQueueSaveOnShutdown on # save in-memory data if rsyslog shuts down # ----------------------------- Modules & Functions ----------------------------# module: kernel logs, klogd replacement $ModLoad imklog # module: MARK messages (usually disable) $ModLoad immark # module: input udp # NOTE: documetation says InputUDPServerRun, that's wrong $ModLoad imudp $UDPServerAddress * $UDPServerRun 514 # module: input tcp (works fine) $ModLoad imtcp $InputTCPServerRun 601 $InputTCPServerRun 4096 # module: input socket $ModLoad imuxsock $InputUnixListenSocketHostName /dev/log $InputUnixListenSocketIgnoreMsgTimestamp on # ----------------------------- Templates ----------------------------# Use traditional timestamp format # eg: "Jan 1 15:00:01 " $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat # RFC5424 format # also known as RSYSLOG_SyslogProtocol23Format, and draft-internet-ietf-syslog-protocol-23 (now RFC5424) # the "1" in "<%PRI%>1 " denotes syslog protocol version 1, as per the RFC # eg: <21>1 2011-01-01T16:09:05+00:00 <MAIL PROXY HOST NAME GOES HERE> perdition 14185 - $template RFC5424FMT,"<%PRI%>1 %TIMESTAMP:::date-rfc3339% %HOSTNAME% %APP-NAME% %PROCID% %MSGID% %STRUCTURED-DATA% %msg%\n" # ----------------------------- Destinations ----------------------------# Log all kernel messages to the console. # Logging much else clutters up the screen. kern.* /var/log/kernel # Log anything (except mail) of level info or higher. # Don't log private authentication messages! *.info;mail.none;authpriv.none;cron.none;local3.none /var/log/messages # The authpriv file has restricted access. authpriv.* /var/log/secure # MAIL: Log locally & forward a copy to syslog1-syd.internal for further analysis mail.* -/var/log/mail/mail.log;RFC5424FMT mail.* @@syslog1-<FQDN OF CENTRAL LOG HOST>:601 # Log cron stuff cron.* /var/log/cron

232 of 243

18.04.2011 16:35

The Underground Oracle VM Manual
# Everybody gets emergency messages #*.emerg * # Save news errors of level crit and higher in a special file. #uucp,news.crit /var/log/spooler # Save boot messages also to boot.log local7.* /var/log/boot.log # All, save mail *.debug;mail.none;authpriv.none;cron.none -/var/log/debug # Oracle VM: Log locally & forward a copy to syslog1-syd.internal for further analysis local3.* -/var/log/ovs-agent/all.log;RFC5424FMT local3.* @@syslog1-<FQDN OF CENTRAL LOG HOST>:601 Example /etc/sysconfig/rsyslog file The next example shows a rsyslog file.

http://itnewscast.com/book/export/html/844

# cat /etc/sysconfig/rsyslog # Options to syslogd # -m 0 disables 'MARK' messages. # -rPortNumber Enables logging from remote machines. The listener will listen to the specified port. # -x disables DNS lookups on messages received with -r # See syslogd(8) for more details SYSLOGD_OPTIONS="-c3 " # Options to klogd # -2 prints all kernel oops messages twice; once for klogd to decode, and # once for processing with 'ksymoops' # -x disables all klogd processing of oops messages entirely # See klogd(8) for more details KLOGD_OPTIONS="-x" Next, enable the changes by reloading rsyslog by typing, “service rsyslog reload”. After rsyslog has been reloaded, there should be an empty file named all.log in the /var/log/ovs-agent/ directory. Next, reload the Oracle VM agent by typing “service ovs-agent stop --disable-nowayout; service ovs-agent start” as shown in the next example. # service ovs-agent stop --disable-nowayout; service ovs-agent start OVSAgentServer shutdown... OVSAgentServer forced stop. OVSAgentServer is now starting... OVSAgentServer started. After the Oracle VM agent has been reloaded, messages will appear in the local /var/log/ovs-agent/all.log file, as shown in the next example. <158>1 2011-01-25T01:03:03.245088+00:00 virtual5-syd <158>1 2011-01-25T01:03:08.633781+00:00 virtual5-syd success. <158>1 2011-01-25T01:03:08.658913+00:00 virtual5-syd success. <158>1 2011-01-25T01:03:10.863599+00:00 virtual5-syd remove shutdown mode flag. <158>1 2011-01-25T01:03:14.944380+00:00 virtual5-syd sys_vmstat_info:cpu_idle=0.99;mem_free=427 … root - - - OVSAgentServer start serving! ovs.operation - - - clusterm_init_sr: ovs.operation - - - cluster_init_sr: ovs.operation - - - ha_set_shutdown_mode: ovs.performance - - -

Note: If you see "localhost.localdomain" or "127.0.0.1" instead of the hostname in the all.log file, use the unix socket instead of udp in configuring syslog for the Oracle VM agent.

How to Configure the Central Log Host
The next list shows the configuration steps to configure the central log host. 1. Receive remote syslog over port 601/tcp 2. Write local3 messages to /var/log/ovs-agent/all.log 3. Keep local3 message from other major logfiles

233 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Note: For the /etc/rsyslog.conf file, please see the Centralized Log Host full rsyslog.conf section.

Example /etc/sysconfig/rsyslog file
The next example shows the /etc/sysconfig/rsyslog file with the above prerequisites. # cat /etc/sysconfig/rsyslog # Options to syslogd # -m 0 disables 'MARK' messages. # -rPortNumber Enables logging from remote machines. The listener will listen to the specified port. # -x disables DNS lookups on messages received with -r # See syslogd(8) for more details #SYSLOGD_OPTIONS="-c3 -x -m 0 -r514 -t601,4096" # syslog compat SYSLOGD_OPTIONS="-c3 " # rsyslog # Options to klogd # -2 prints all kernel oops messages twice; once for klogd to decode, and # once for processing with 'ksymoops' # -x disables all klogd processing of oops messages entirely # See klogd(8) for more details KLOGD_OPTIONS="-x" Next, reload the rsyslog service by typing, “service rsyslog reload”. After the rsyslog service has been reloaded, the central log host will receive messages to /var/log/ovs-agent/all.log. Notes: The rsyslog configuration must allow inbound 601/tcp. Check the central log host firewall to ensure that inbound 601/tcp is enabled. The central log host's /var/log/ovs-agent directory must exist. If SElinux is in use, it must have the "user_u:object_r:var_log_t" context

The Oracle VM Manager Syslog Configuration
OC4J uses java.util.logging, which cannot directly log to syslog. To log Oracle VM Manager events centrally, we will use rsyslog's "imfile" module to read in the existing oc4j logfile and send it to the central log host line by line.

OC4J Logging
The Oracle VM Manager application is a J2EE Web application, running in an OC4J container. Assuming OC4J is installed to /opt/oc4j, then the default logging is configured via the “/opt/oc4j/j2ee/home/config/j2ee-logging.xml” file.

Oracle VM Manager Logging Configuration Files
The deployment descriptors for Oracle VM Manager application are installed by default to: /opt/ovs-manager-2.2/bin/default-server.xml (the non-ssl server) /opt/ovs-manager-2.2/bin/secure-server.xml (the ssl server) By default the deployment descriptors both point to the j2ee-logging.xml configuration file. So we only need to edit the j2ee-logging.xml configuration file.

How to Make Rsyslog Read the oc4j.log file
The default logfile for OC4J is in the /var/log/ovm-manager/oc4j.log file. Our configuration will read the /var/log/ovm-manager/oc4j.log file to the local2/debug priority with the rsyslog imfile module. Note: The configuration will result in the lose of the event severity.

Oracle VM Manager /etc/rsyslog.conf and Adding imfile Support
Next, we will add the “input file” module to /etc/rsyslog.conf. This configuration will write the state file to “$WorkDirectory”. Note: The full imfile module documentation can be found here: http://www.rsyslog.com/doc/imfile.html

234 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Example /etc/rsyslog.conf file
# cat /etc/rsyslog.conf # module: input file # - emits each line of the given file to syslog, for apps that don't do syslog # - keeps track of position, file rotation $ModLoad imfile $InputFileName /var/log/ovm-manager/oc4j.log $InputFileTag OVM: $InputFileStateFile state-oc4j $InputFileSeverity debug $InputFileFacility local2 $InputRunFileMonitor

Empty the Existing Oracle VM Manager oc4j.log
When the imfile module starts, it will read every line in the existing oc4j.log file, which is too much information for the imfile module to start. Empty the oc4j.log file by typing “cp /dev/null /var/log/ovm-manager/oc4j.log”.

Configure /etc/rsyslog.conf to Forward local2 Events
Next, add the following lines to the /etc/rsyslog.conf file to forward local2 events to the central log host. Enter your central log host FQDN in the <CENTRAL LOG HOST> section. # Oracle VM Manager: Log locally & forward a copy to syslog1-syd.internal for further analysis # - Oracle VM Manager to local2, not logging to file here as the input is a file local2.* @@<CENTRAL LOG HOST>:601

Configuring the Central Log Host
On the central log host add the following line to /etc/rsyslog.conf to receive local2 events. # Oracle VM: Log locally # - Oracle VM Manager on local2 # - Oracle VM agent on local3 local2.* -/var/log/ovm/ovm-managers.log;RFC5424FMT local3.* -/var/log/ovm/ovs-agents.log;RFC5424FMT Next, restart the rsyslog service on the central log host and on Oracle VM Manager. After the rsyslog service is restarted, log host should see inbound messages: <151>1 2011-01-25T13:12:01+00:00 machines status succeeded. <151>1 2011-01-25T13:12:01+00:00 master status succeeded. <151>1 2011-01-25T13:12:01+00:00 version succeeded. <151>1 2011-01-25T13:12:01+00:00 vmmanager1-syd OVM - - - INFO: ServerPoolSync.run: sync virtual vmmanager1-syd OVM - - - INFO: ServerPoolSync.run: sync cluster vmmanager1-syd OVM - - - INFO: ServerPoolSync.run: sync agent vmmanager1-syd OVM - - - INFO: master server is:10.200.28.202

Example central log host /etc/rsyslog.conf file
The next example shows the complete central log host rsyslog.conf file. # cat /etc/rsyslog.conf # ----------------------------- Queues (required for forwarding) ----------------------------$WorkDirectory /var/spool/rsyslog $ActionQueueType LinkedList # use asynchronous processing $ActionQueueFileName srvrfwd # set file name, also enables disk mode $ActionResumeRetryCount -1 # infinite retries on insert failure $ActionQueueSaveOnShutdown on # save in-memory data if rsyslog shuts down # ----------------------------- Modules & Functions ----------------------------# module: kernel logs, klogd replacement $ModLoad imklog # module: MARK messages (usually disable) $ModLoad immark

235 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

# module: input udp # NOTE: documentation says InputUDPServerRun, that's wrong $ModLoad imudp $UDPServerAddress * $UDPServerRun 514 # module: input tcp (works fine) $ModLoad imtcp $InputTCPServerRun 601 $InputTCPServerRun 4096 # ----------------------------- Templates ----------------------------# Use traditional timestamp format # eg: "Jan 1 15:00:01 " $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat # RFC5424 format # also known as RSYSLOG_SyslogProtocol23Format, and draft-internet-ietf-syslog-protocol-23 (now RFC5424) # the "1" in "<%PRI%>1 " denotes syslog protocol version 1, as per the RFC # eg: <21>1 2011-01-01T16:09:05+00:00 <MAIL PROXY HOST NAME> perdition 14185 - $template RFC5424FMT,"<%PRI%>1 %TIMESTAMP:::date-rfc3339% %HOSTNAME% %APP-NAME% %PROCID% %MSGID% %STRUCTURED-DATA% %msg%\n" # ----------------------------- Destinations ----------------------------# Log all kernel messages to the console. # Logging much else clutters up the screen. #kern.* /dev/console kern.* /var/log/kernel # Log anything (except mail) of level info or higher. # Don't log private authentication messages! *.info;mail.none;authpriv.none;cron.none;local3.none /var/log/messages # The authpriv file has restricted access. authpriv.* /var/log/secure # MAIL: Log locally & forward a copy to sysdev1-syd3 for further analysis # - RFC5424 parseable output mail.* -/var/log/mail/mail.log;RFC5424FMT mail.* @@<SMTP HOST>:601 # Log cron stuff cron.* /var/log/cron # Everybody gets emergency messages #*.emerg * # Save news errors of level crit and higher in a special file. #uucp,news.crit /var/log/spooler # Save boot messages also to boot.log local7.* /var/log/boot.log # All, save mail *.debug;mail.none;authpriv.none;cron.none -/var/log/debug # Oracle VM: Log locally # - OVM manager on local2 # - OVS agent on local3 local2.* -/var/log/ovm/ovm-managers.log;RFC5424FMT local3.* -/var/log/ovm/ovs-agents.log;RFC5424FMT

How to Configure the Oracle VM Server's Logrotation
The next example show the recommended Oracle VM server and the central log host configurations by creating an executable file named ovs-agent in the /etc/logrotate.d/ directory. # cat /etc/logrotate.d/ovs-agent /var/log/ovs-agent/all.log { create 0644 root root

236 of 243

18.04.2011 16:35

The Underground Oracle VM Manual
missing ok # keep for 14 days rotate 14 # dateext tags with date, max 1 rotation/day dateext # compression parameters compress compresscmd /usr/bin/bzip2 compressext .bz2 compressoptions -9 # notify syslog or rsyslog postrotate /bin/kill -HUP `cat /var/run/syslogd.pid 2> /dev/null` 2> /dev/null || true /bin/kill -HUP `cat /var/run/rsyslogd.pid 2> /dev/null` 2> /dev/null || true endscript }

http://itnewscast.com/book/export/html/844

The above configuration can be tested verbosely by typing “logrotate -v -f /etc/logrotate.d/ovs-agent”. Watch this space Oracle OpenWorld 2011 Call for Papers - Submit Today! Temporary Post Used For Theme Detection (409fc5e9-d7ee-4e1a-b645-8b82f256f434 - 3bfe001a-32de-4114-a6b4-4005b770f6d7) Debugging Oracle VM 2.2 & Oracle VM Manager 2.2 errors using the many log files New VDI Training Course now available! HIMSS 2011 and New Press Release Oracle Desktop Virtualization at HIMSS 2011 Oracle VM VirtualBox 4.0.4 Released! Two Virtualization Webinars This Week We're Hiring! - Server and Desktop Virtualization Product Management

Virtualization Policy Project Introduction
The Virtualization Security Policy Project Second Edition
Author: Roddy Rodstein, CISSP Copyright © 2008 - 2011 Roddy Rodstein. All rights reserved. Limits of Liability and Disclaimer of Warranty This publication contains information protected by copyright. This publication may not be duplicated in any way without the express written consent of the publisher, except in the form of brief excerpts or quotations for the purpose of review. The information contained herein is for the personal use of the reader and may not be incorporated in any commercial programs, other books, databases, or any kind of software without the written consent of the publisher. Making copies of this book or any portion, for any purpose other than your own, is a violation of United States copyright laws. Warning and Disclaimer Every effort has been made to make this publication as complete and as accurate as possible, but no warranty or fitness is implied. The information provided is on an "as is" basis. The authors and the publisher shall have neither liability nor responsibility to any person or entity with respect to any loss or damages arising from the information contained in this publication. The information found in this publication was gathered from many different sources in the computing world. It is provided for informational purposes only. Use common sense in applying these concepts and tips. Screen shots may vary from environment to environment. Please verify correctness and applicability in a test environment first and then deploy to your production environment(s).

Introduction
I would like to introduce the Virtualization Security Policy Project Second Edition. The Virtualization Security Policy Project Second Edition consists of an Enterprise Architecture Introduction, a Hardware and Software Sunset Policy, a Server Virtualization Policy, and a Server Virtualization Standards. The Enterprise Architecture Introduction provides the framework for the Virtualization Security Policy Project and runs through a brief introduction of Enterprise Architecture to illustrate how your virtualization technologies fit within your Enterprise Architecture. The Hardware and Software Sunset Policy define an organization’s hardware and software sunset policy. The Server Virtualization Policy and Server Virtualization Standards define an organization’s server virtualization requirements and

237 of 243

18.04.2011 16:35

The Underground Oracle VM Manual
minimum standards.

http://itnewscast.com/book/export/html/844

You can use the policies and standards as is or modify them to meet your specific business requirements. I welcome your comments about the Virtualization Security Policy Project. Please let me know if you would like to see additional policies added to the Virtualization Security Policy Project to help support your virtual infrastructure.

Enterprise Architecture Introduction
The Virtualization Policy Project begins with a high level overview of Enterprise Architecture (EA) and concludes with example IT policies. Because Enterprise Architecture is a field unto itself, a detailed review of its principles, processes, and approach is beyond the scope of this publication. The goal of this publication is to explain how virtualization technologies fit within an Enterprise Architecture. The purpose of Enterprise Architecture is to establish an Enterprise wide blueprint used to achieve business objectives while maximizing the business value of information technology. An Enterprise Architecture is a “blueprint” that describes an organization’s strategic direction, security and regulatory requirements, information technology portfolio and their inter-dependencies, baseline and target architectures, and the processes to implement technologies. In business terms, Enterprise Architecture is accomplished by efficiently achieving an organization’s mission with minimal investment in information technology; and in technical terms, by optimizing business operations, effective information technology planning, information technology budgeting, information technology acquisition, human resource utilization, and the implementation of security controls. After the goals and stakeholders of an Enterprise Architecture project have been identified, a framework is selected to help implement and support the Enterprise Architecture through its entire life cycle. There are a number of frameworks, such as Cobit, ISO/IEC 17799, ITIL, and many others that represents a variety of methodologies and toolsets to fulfill the requirements of any size or type of organization. Frameworks provide methodologies, standards, guidelines, and formats that can be used as is or adapted to meet specific requirements. Because organizations have different missions and business objectives, no single framework will be right for each situation. Organizations typically select a framework or a mixture of frameworks to meet their requirements. Enterprise Architecture has well defined principles and processes, along withan approach that generates a comprehensive layered policy infrastructure used to communicate management’s goals, principles, instructions, procedures, and response to laws and regulatory mandates. A policy infrastructure consists of tier 1, tier 2 and tier 3 policies that encompass people, systems, data, and information. A policy infrastructure consists of policies, standards, procedures, baselines, and guidelines. Tier 1 policies are at the top layer of the policy infrastructure and address broad organizational issues, vision and direction. Most organizations will develop and support up to a dozen tier 1 policies. An example tier 1 policy is an Employee Practices Policy or a Conflict of Interest Policy. Global in scope, Tier 1 policies are high level documents that define vision and direction. Tier 2 policies are topic specific, and tier 3 policies are application or system specific. Standards are tier 2 policies that describe system design concepts, implementation steps, and detailed configurations. Procedures are tier 2 & 3 policies that provide step by step compulsory measures, communicating best practices in using information systems and organizational data/information. Baselines are tier 3 policies that are application or system specific and describe step by step instructions to implement technologies. Guidelines are tier 3 documents, offering application, system, or procedural specific best practices. Guidelines are recommendations unlike policies, standards, procedures, and baselines, which are compulsory. Figure 1 shows the organization of Enterprise Architecture’s layered policy infrastructure.

Figure 1 represents Enterprise Architecture’s layered policy infrastructure, starting with tier 1 policies which address broad organizational issues, vision, and direction. The next layer, General Organizational Policy, consists of tier 1 policies in which management makes security statements, explains roles and responsibilities, and defines which assets are considered valuable. The following layer, Practical Implementation Policies, contains tier 2 and 3 policies which are topic, application, and system specific and are used to enforce upper layer policies. The lower layer consists of tier 2 and 3 policies which are topic and technology specific and are used to enforce higher layer policies. Figure 2 shows the work flow of a policy infrastructure.

238 of 243

18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

A policy infrastructure contains confidential information relating to running a business and the publication, distribution and storage of that information should be strictly monitored. Many organizations leverage the human resource department and secured intranet solutions to distribute and control access to policies. An Enterprise Architecture groups together infrastructure components within topic specific domains. An example of Enterprise Architecture domains are infrastructure, applications, network, and security. After an organization has defined its Enterprise Architecture domains, all infrastructure components are grouped within their corresponding domain and reviewed individually and as a single cohesive unit. Layered policies are developed for each domain and each individual technology within a domain. Table 1 shows the Enterprise Architecture domain structure that will be used throughout this publication. The example encompasses five domains split between two high level groups; infrastructure and applications. The five domains are platform, network, software, data / information, and security. Enterprise Architecture Scope Infrastructure Platform Network Security An organization’s mission and business objectives drive its Enterprise Architecture domain structure. As we have all learned, there is no ‘one size fits all’ with information technology, and Enterprise Architecture is no exception. Enterprise Architecture is flexible and can be molded to suit any organization’s mission and business objectives. Table 2 shows a variation of the above Enterprise Architecture domain structure. Enterprise Architecture Scope Platform Network Software Data/Information Security Access Domain Integration Domain Privacy Domain Project Management Domain Systems Management Domain Each of the domains within an Enterprise Architecture will have its corresponding layered policy infrastructure, with tier 1 & 2 policies, tier 2 & 3 standards, procedures, baselines, and guidelines. Applications Software Data/Information

Watch this space Oracle OpenWorld 2011 Call for Papers - Submit Today! Temporary Post Used For Theme Detection (409fc5e9-d7ee-4e1a-b645-8b82f256f434 - 3bfe001a-32de-4114-a6b4-4005b770f6d7) Debugging Oracle VM 2.2 & Oracle VM Manager 2.2 errors using the many log files New VDI Training Course now available! HIMSS 2011 and New Press Release Oracle Desktop Virtualization at HIMSS 2011 Oracle VM VirtualBox 4.0.4 Released!

239 of 243

18.04.2011 16:35

The Underground Oracle VM Manual
Two Virtualization Webinars This Week We're Hiring! - Server and Desktop Virtualization Product Management

http://itnewscast.com/book/export/html/844

Hardware and Software Sunset Policy
The following example Hardware and Software Sunset Policy defines an organization’s hardware and software sunset policy. This policy is intended for informational purposes only.

Hardware and Software Sunset Policy
Purpose The purpose of this policy is to establish hardware and software sunset requirements. In an ongoing effort to meet business requirements, reduce IT costs and provide reliable, high-quality IT services, <Company Name> periodically sunsets (retires), old hardware and software. Once sunsetted, active support and all business services for the product are discontinued. Sunsetting older versions of hardware and software allows <Company Name> to focus resources on enhancing IT services, and providing support for more current, secure and stable products. In most cases, replacement costs for products identified for sunset are generally less than the expenses of continued support and maintenance. The Sunset policy will result in better customer service and reduced costs. This policy provides controls to ensure that Enterprise issues are considered along with business objectives when sunsetting hardware and software. Scope The scope of this policy encompasses server, desktop and network hardware platforms, operating systems and application software. Policy Products that have reached the end of their life cycle and are no longer supported by a vendor will be given a sunset date. The sunset date is when the product is scheduled to be removed from production. The sunset date will be set far enough in advance to give <Company Name> at least a budget cycle to fund and plan for the replacement. When a particular version of hardware or software is scheduled to be sunsetted, <Company Name> will provide the affected users with advance notice via email. A Sunset list will be used to track all hardware and software sunset dates. In order to keep the sunset list up to date, <Company Name> will update the sunset list quarterly with hardware and software for review. Department managers with staff that use products on the sunset list will be notified quarterly via email regarding the sunset review process and sunset dates. If you are currently using application software that has been designated sunset and would like to extent support, you will need to acquire a version that meets the current minimum standards as defined in <Company Name> Software Standards. If you are currently using hardware that has been designated sunset, any technical issues with the unit will trigger a replacement process with a unit that meets the current minimum standards as defined in <Company Name> Hardware Standards. List 1 shows the sunset categories: Hardware four years or older. Operating systems that have reached their sunset date or are no longer supported by the vendor. Proprietary application software that is no longer supported by the vendor. Open Source application software that is no longer supported by the community. Application software that does not support <Company Name> centralized authentication and authorization system. Review Cycle This policy is subject to annual review. Compliance All information technology investments shall conform to existing policies in order to ensure the integrity and interoperability of computing platforms. Any employee found to have violated this policy may be subject to disciplinary action, up to and including termination of employment. Related Policies Platform Infrastructure Standard Xen.org @ OSCON Xen.org GSoC 2011 Update KVM Forum 2011 scheduled for August 15-16 Libvirt 0.9.0 Released Virt-Manager 0.8.7 Released: New UI Features Proxmox VE 1.8 Released: Supports qemu-kvm 0.14 Xen 4.1 releases Xen hack-a-tron day 1 Xen.org accepted for GSoC 2011 Xen.org spring clean

Server Virtualization Policy
The following example Server Virtualization Policy defines an organization’s server virtualization requirements and minimum standards. This policy is intended for informational purposes only.

Server Virtualization Policy
240 of 243 18.04.2011 16:35

The Underground Oracle VM Manual

http://itnewscast.com/book/export/html/844

Purpose The purpose of this policy is to establish server virtualization requirements that define the acquisition, use, and management of server virtualization technologies. This policy provides controls that ensure that Enterprise issues are considered along with business objectives when making server virtualization related decisions. Platform Architecture policies, standards and guidelines will be used to acquire, design, implement and manage all server virtualization technologies. Scope The scope of this policy encompasses all new and existing workloads. Responsibilities The CEO and CIO ensure that policies are followed in order to establish contracts, review procurement requests and to develop and manage services. Policy <Company Name>’ legacy IT practice was to dedicate one physical server to a single workload. The result of <Company Name>’ legacy IT practice was excessive server underutilization, an ever-expanding data center footprint and excessive data center power consumption. Server virtualization software allows the consolidation of new and existing workloads onto high capacity x86 servers. Consolidating workloads onto high capacity x86 servers allows <Company Name> to reduce the x86 server inventory, which in turn decreases the data center footprint and data center power consumption. <Company Name> will migrate all new and existing workloads from physical servers to virtual machines. All workloads that cannot be migrated to a virtual machine will be subject to <Company Name>’ Hardware and Software Sunset Policy. Server Virtualization Software Requirements: Support industry-wide open-standards Embedded security Single centralized management console Support industry standard management tools Support industry standard backup and recovery tools Interoperate with other platform technologies Support industry standard x86 hardware Support industry standard storage Support unmodified guest operating systems Migrate running guests without interruption Add disks to a running guest Snapshot running guests Revert to a previous snapshots on a running guest Automatically detect a hardware failure and restart guests on another physical server Functionality to configure role based access for the administrative console Support LDAP for authentication and authorization for administrative console Encrypt all intra host and administrative console traffic Integrated graphical CPU, memory, disk and network performance monitoring, alerting, and historical reporting for hosts and guests. Retain performance data for up to one (1) year Functionality to manage host CPU, memory, storage and network resource allocation Functionality to manage guest CPU, memory, disk and network resource allocation Functionality to create, stop, start, pause, migrate, clone and provision guests Functionality to automatically load balance guests across multiple hosts Consolidated logging for hosts and guests that log date and time of all administrative user actions Functionality to convert x86 physical servers to virtual machines Encrypted remote administrative console access Review Cycle This policy is subject to annual review. Compliance All information technology investments shall conform to existing policies in order to ensure the integrity and interoperability of computing platforms. Any employee found to have violated this policy may be subject to disciplinary action, up to and including termination of employment. Related Policies Platform Infrastructure Standard Server Virtualization Standard Server Virtualization Guidelines Hardware and Software Sunset Policy Xen.org @ OSCON Xen.org GSoC 2011 Update KVM Forum 2011 scheduled for August 15-16

241 of 243

18.04.2011 16:35

The Underground Oracle VM Manual
Libvirt 0.9.0 Released Virt-Manager 0.8.7 Released: New UI Features Proxmox VE 1.8 Released: Supports qemu-kvm 0.14 Xen 4.1 releases Xen hack-a-tron day 1 Xen.org accepted for GSoC 2011 Xen.org spring clean

http://itnewscast.com/book/export/html/844

Server Virtualization Standards
Standards are used to provide the uniform use of technologies to drive consistency and reproducibility, lower operational costs, and enable faster deployments of technologies and functions. The following example Server Virtualization Standards policy shows how an organization defines their Server Virtualization Standards.

Server Virtualization Standards
Purpose The purpose of these standards is to define Enterprise wide server virtualization requirements in order to provide opportunities to meet strategic and tactical Information Technology objectives. These standards define a template and a set of requirements used to implement and support server virtualization software. Platform Architecture policies, standards and guidelines will be used to acquire, design, implement and manage all server virtualization software. Scope The scope of this policy encompasses all server virtualization software. Responsibilities The CEO and CIO ensure that policies are followed in order to establish contracts, review procurement requests and to develop and manage services. Standards: Table 1 shows the Technology Classifications used to define <Company Name>’ technology life cycle. Current Contain Technologies that are approved and recommended for use. Technologies that are being phased out over the next 3 to 5 years. Technologies that will be phased out and replaced. A discontinuance date has been set. Technologies that may become current. Research technologies should be used only for test environments for evaluation purposes.

Retire Research

Current Table 2 lists the current server virtualization technologies. Products Oracle VM 2.2 Platforms Hypervisor whithout a host operating system. Servers will be x86-64 platform. Hypervisor whithout a host operating system. Servers will be x86-64 platform. Workstations running Linux or Windows. Category Production, Test, and Development Technology Classification Current

VMware ESXi 3.5

Production, Test, and Development

Current

VirtualBox 3.0.

Test and Development only

Current

Contain Table 3 lists the contain technologies that are being phased out over the next 3 to 5 years. Products Platforms Category Technology Classification

242 of 243

18.04.2011 16:35

The Underground Oracle VM Manual
VMware ESX Server 3.x Hypervisor whithout a host operating system. Servers will be x86-64 platform. Servers and Workstations running Linux or Windows. Workstations running Linux or Windows. All categories Contain

http://itnewscast.com/book/export/html/844

VMware Server 1.x

All categories

Contain

VMware Workstation 5.x

All categories

Contain

Retire Table 4 lists the retire technologies which are being phased out. Referance the Hardware and Software Sunset Policy for the software retirment plan. Products VMware ESX Server versions prior to v. 2.5 Platforms Hypervisor whithout a host operating system. Servers will be x86-64 platform. Servers running Linux or Windows. Servers running Linux or Windows. Workstations running Linux or Windows. Category All categories Technology Classification Retire on 12/31/08

VMware GSX Server versions prior to v. 3 VMware Server versions prior to v. 2 VMware Workstation versions prior to v.5

All categories

Retire on 12/31/08

All categories

Retire on 12/31/08

All categories

Retire on 12/31/08

Reaearch Table 5 lists the research technologies. Research technologies could become current standards. The use of research technologies is restricted to test environments. Products KVM Microsoft Hyper-V Platforms Linux only. Does not need a host OS. Category Test only Test only Technology Classification Research Research

Review Cycle This policy is subject to annual review. Compliance All information technology investments shall conform to existing policies in order to ensure the integrity and interoperability of computing platforms. Any employee found to have violated this policy may be subject to disciplinary action, up to and including termination of employment. Related Policies Platform Infrastructure Standard Server Virtualization Policy Server Virtualization Guidelines Hardware and Software Sunset Policy Xen.org @ OSCON Xen.org GSoC 2011 Update KVM Forum 2011 scheduled for August 15-16 Libvirt 0.9.0 Released Virt-Manager 0.8.7 Released: New UI Features Proxmox VE 1.8 Released: Supports qemu-kvm 0.14 Xen 4.1 releases Xen hack-a-tron day 1 Xen.org accepted for GSoC 2011 Xen.org spring clean

243 of 243

18.04.2011 16:35

Sign up to vote on this title
UsefulNot useful