P. 1


|Views: 142|Likes:
Published by muthuramrr

More info:

Published by: muthuramrr on May 19, 2011
Copyright:Attribution Non-commercial


Read on Scribd mobile: iPhone, iPad and Android.
download as PDF, TXT or read online from Scribd
See more
See less






  • Session 1
  • Session 2
  • Session 3
  • Session 4

Xen Hypervisor Deployment, Management, and Cloud Computing Tools

Todd Deshane and Patrick F. Wilbur Clarkson University

Copyright Notice
Copyright 2010, Todd Deshane and Patrick F. Wilbur. Last modified: September 30, 2010 11:02 PM EST. The Xen panda logo is property of Xen.org . == LICENSE: Todd Deshane and Patrick F. Wilbur Department of Mathematics and Computer Science Clarkson University Potsdam, NY USA Current slides available at: http://cosi.clarkson.edu/docs/installingxen/ These slides and content are released under the Creative Commons Attribution-Share Alike 3.0 Unported license, available online at http://creativecommons.org/licenses/by-sa/3.0/ You may share (copy, distribute, and transmit) this work, and remix (adapt) this work, as long as you attribute this work to the author and share adapted works under the same or similar license by leaving this entire notice in place (including the original author's name/contact information/URL and this license notice).

usability. Wilbur is a Ph. policy. and systems architecture.About Us Todd Deshane is a Ph. and usability.D. http://pdub.net Patrick F.net . His research interests include information technology. security. His research interests include security. graduate of Clarkson University. http://todddeshane. virtualization.D student at Clarkson University.

and at other various locations across the United States. Wilbur.Acknowledgments This 2010 Xen Training / Tutorial. USENIX LISA. Portions of this work are inspired by Jeremy Fitzhardinge's Pieces of Xen slides. Variations of this work have been presented numerous times at the USENIX Annual Technical Conference. is derived from the 2009 Xen Training / Tutorial as updated by Zach Shepherd and Jeanna Matthews from the original version written by Zach Shepherd and Wenjin Hu. . Wilbur. by Todd Deshane and Patrick F. originally derived from materials written by Todd Deshane and Patrick F.

Overview Session 1: Xen Introduction and Installing Xen Session 2: Guest Creation and Management Session 3: Xen in the Datacenter Session 4: Xen in the Cloud .

Session 1 Xen Introduction and Installing Xen .

Xen Overview .

Xen and the Art of Virtualization Xen is a virtualization system supporting both paravirtualization and hardware-assisted full virtualization Name from neXt gENeration virtualization Initially created by University of Cambridge Computer Laboratory Open source (licensed under GPLv2) .

there are one or more virtual machines (domains) running A virtual machine is an encapsulated operating system which can run applications as a physical machine The management virtual machine (Domain0) is responsible for interacting with the hypervisor Other virtual machines are called guests .Xen Virtualization Basics A physical machine runs a program to manage virtual machines (Virtual Machine Monitor or hypervisor) On the physical machine.

exploitation of multiple cores Running heterogeneous environments on one machine: different operating systems.Ways to Use Virtualization Fully utilize hardware resources: consolidation of workloads on single machine. backup/disaster recovery. portability . different libraries Isolation: separate workloads that have different requirement and/or to avoid attacks on one from compromising another Manageability: rapid deployment and provisioning.

Types of Virtualization Emulation: Fully-emulate the underlying hardware architecture Full virtualization: Simulate the base hardware architecture Paravirtualization: Abstract the base architecture OS-level virtualization: Shared kernel (and architecture). separate user spaces .

OS kernel Guest contains grub and kernel Normal device drivers Can run Windows Guest doesn't "know" it's a VM. normal. so hardware manages it .Virtualization in Xen Paravirtualization: Uses a modified Linux kernel Guest loads Dom0's pygrub or Dom0's kernel Front-end and back-end virtual device model Cannot run Windows Guest "knows" it's a VM and talks with the hypervisor Hardware-assisted full virtualization: Uses the same.

Reasons to Use Xen Paravirtualization (PV): High performance (claim to fame) High scalability Uses a modified operating system Hardware-assisted full virtualization (HVM): Co-evolution of hardware and software on x86 architecture Uses an unmodified operating system .

Reasons to Use Xen Xen is powered by a growing and active community and a diverse range of products and services Xen offers high performance and an isolating architecture .

Xen Architecture .

privileged abstraction layer between the hardware and operating systems Defines the virtual machine that guest domains see instead of physical hardware: Grants portions of physical resources to each guest Exports simplified devices to guests Enforces isolation among guests .Xen: Hypervisor Role Thin.

Xen: Domain0 (Dom0) Role Creates and manages guest VMs xm (Xen management tool) A client application to send commands to xend Interacts with the Xen hypervisor xend (Xen daemon) Daemon to communicate with the hypervisor Supplies device and I/O services: Runs (backend) device drivers Provides domain storage .

Normal Linux Boot Process BIOS Master Boot Record (MBR) GRUB Kernel Module Linux .

The Xen Boot Process GRUB starts Kernel Hypervisor starts Module Domain0 starts Daemon Xend starts xm Guest domain starts .


Installing the Open-source Xen Hypervisor .

Installing Xen from a Package root:~>yum install xen .

netbsd.org/doc/en/xen-guide.Installing Xen from a Package OpenSUSE: Install with YaST http://www.org/ports/xen/howto.xml NetBSD: Xen package support as of BSD 4.com/general/how-to-install-configure-xen-virtualization-inopensuse-110/ Gentoo: Install with portage http://www.susegeek.html .gentoo.0 http://www.

xensource.0 . including from source: http://wiki.Installing Xen from Source Reasons to use the latest Xen version: Performance optimization. BSD) Ability to patch/customize Xen4 installation instructions. cutting-edge features Security and bug fixes Support for additional Dom0 OSes (Linux.com/xenwiki/Xen4. Solaris.

snapshots and cloning Primary graphics card GPU passthru for high-performance 3D graphics and hardware-accelerated video TMEM allows improved utilization of unused (for example page cache) PV guest memory Memory page sharing and page-to-disc for HVM guests Copy-on-Write sharing of memory pages between VMs .Installing Xen from Source New in Xen4: blktap2 for VHD image support.

SR-IOV functionality On-line resize of guest disks without reboot/shutdown Remus Fault Tolerance: live transactional synchronization of VM state between physical servers RAS features: physical cpu/memory hotplug .Installing Xen from Source Also new in Xen4: Netchannel2 for improved networking acceleration. smart NICs. multi-queue support.

18.0) kernel /boot/xen-3.24-23-generic .img-2.8-xen Sample Normal Linux GRUB Configuration: title Ubuntu kernel /boot/vmlinuz-2.6.gz module /boot/vmlinuz-2.GRUB Configuration Sample Xen GRUB Configuration: title Xen 3.24-23 root (hd0.6.6.24-23-generic root=/dev/sda1 initrd /boot/initrd.8-xen root=/dev/sda1 module /boot/initrd.img-2.4 root (hd0.6.

Xend Configuration Xen daemon's configuration in /etc/xen/xend-config.sxp : Configure Xen to listen for remote connections Set max/min Dom0 CPU and memory resources Set up the virtual network: Bridging Routing NAT Configure live migration (enable and set relocation port) .

0.1') (vncpasswd '') .Sample Xend Configuration # (logfile /var/log/xen/xend.0.log) # (loglevel DEBUG) # (xend-http-server no) (xend-unix-server yes) (xend-unix-path /var/lib/xend/xend-socket) (network-script network-bridge) (vif-script vif-bridge) (dom0-min-mem 256) (dom0-cpus 0) # (xend-relocation-server no) # (xend-relocation-port 8002) # (vnc-listen '127.

Network Modes Bridging mode: Guest domains are transparently on the same network as Dom0 Routing mode: Guest domains sit behind Dom0 and packets are relayed to the network by Dom0 NAT mode: Guest domains hide behind Dom0 using Dom0's IP for external traffic .

Bridge Mode .

Routing Mode .

NAT Mode .

Restart Xen daemon: /etc/init. Set network-script and vif-script in xend config.Network Configuration 1.d/xend restart service xend restart . 2.

Bridge Mode Configuration Default network mode for Xen Xen uses bridge-utils in Dom0 to provide virtual network bridging .

Routing Mode Configuration Modify xend configuration: (network-script network-route) (vif-script vif-route) Xen uses iptables in Dom0 to perform software routing .

NAT Mode Configuration Modify xend configuration: (network-script network-nat) (vif-script vif-nat) Xen uses iptables in Dom0 to perform NAT .

everything's fine 1 .everything's not so fine 4 .you did not run as root xend .d/xend start echo $? 0 ./etc/init.


xm list (No Guests Running) .

Session 2 Guest Creation and Management .

Guest Configuration .

Local Storage Raw File: Use a filesystem within a single file Takes advantage of loopback devices Partition: Use a partition on a logical partition Can be physical partition or on an LVM volume Partitioned File: Less common Treats a raw file as a disk (instead of single partition) .

Install the operating system (needs PV drivers): debootstrap hardy /mnt/tmp or cp -a /* /mnt/tmp .img /mnt/tmp 4. Format: mkfs. mount -o loop /path/to/new/image.img bs=1024k count=1024 2. Allocate storage: dd if=/dev/zero of=/path/to/image.Local Storage: Raw File for PV 1.ext3 -F /path/to/image.img 3. Mount the storage: mkdir /mnt/tmp.

/etc/fstab .Local Storage: Raw File for PV 5. Modify various files in guest filesystem and unmount: e. Create the guest configuration file for Xen to use . /etc/ifconfig 6.g. /etc/hostname .

Allocate storage: dd if=/dev/zero of=/path/to/image.Local Storage: Raw File for HVM 1.img bs=1024k count=1024 2. Create the guest configuration file 3. Install the operating system .

Guest Storage Configuration Options
Array of disk specifications:
'real dev in dom0, virtual dev in domU, Access (r, w)'

SCSI (sd) and IDE (hd) examples:
disk = [ 'phy:sda, sda, w', 'phy:/dev/cdrom, cdrom:hdc, r' ] disk = [ 'tap:aio:hdb1, hdb1, w', 'phy:/dev/LV/disk1, sda1, w' ]

Xen virtual device example:
disk = [ 'tap:aio:hdb1, xvdb1, w', 'phy:/dev/LV/disk1,xvda1, w' ]

General Guest Configuration Options
(For both PV and HVM guests) name
The name of the guest (defaults to configuration filename)

The number of virtual CPUs (defaults to 1)

The amount of memory (in MB) (defaults to 128)

Guest Network Configuration
Array of virtual interface network parameters specify 'MAC Address, IP Address,' for each interface Examples:
vif = [ ' ' ] # Default bridge, random MAC address vif = [ 'mac=00:16:3e:36:a1:e9, ip=, bridge=xenbr0' ]

g.0.1) . 192.g.10) NAT mode networking (if chosen in xend config): Set DomU's gateway (in guest OS's network configuration) to Dom0's internal IP (e.Guest Network Configuration Bridge mode networking (default in xend config): Set vif statement in the DomU's configuration file Routing mode networking (if chosen in xend config): Set DomU's gateway (in guest OS's network configuration) to Dom0's external IP (e.32. 10.0.0.

Guest Creation .

HVM-specific Configuration Options kernel The location of the HVM loader builder Domain build function ("hvm" for an unmodified kernel) device_model Location of the device emulation tool (e. hard drive) vnc Enable VNC utility for the guest to display .g. "qemu_dm") boot The boot order (CD-ROM.

'phy:/dev/cdrom. bridge=eth0' ] vnc = 1 # (or sdl = 1) .r' ] vif = [ 'type=ioemu.w'.Sample HVM Guest Configuration vcpus = 1 memory = 512 kernel = "/usr/lib64/xen/boot/hvmloader" builder = "hvm" device_model = "/usr/lib64/xen/bin/qemu-dm" boot = "cd" disk = [ 'tap:aio:/xen/images/hvm. ioemu:hdc:cdrom.disk. ioemu:hda.

iso as first boot device 3.Installing HVM Guest OSes (CD/. with CD/. Create HVM config. Allocate disk image for the VM 2. Change boot order in guest configuration file. reboot . Follow normal installation process of guest OS 5.cfg 4. Boot the guest: xm create /path/to/guest.iso) 1.

g.PV-specific Configuration Options kernel Location of the Xen-modified kernel in Dom0's filespace ramdisk Location of the initial RAM disk image in Dom0's filespace or: bootloader The location of the bootloader (e. pygrub) .

PV-specific Configuration Options root The partition to use as root inside the guest extra The parameters appended to the kernel command line (as would be normally set at the end of a kernel line) vfb Virtual framebuffer for PV guest to use in addition to console .

vncunused=1'] .18.8-xen" disk = [ 'phy:hda1.Sample PV Guest Configuration vcpus = 1 memory = 64 kernel = "/boot/vmlinuz-2.img-2.xvda1.6.18.8-xen" extra = 'xencons=tty' ramdisk = "/boot/initrd.6.w' ] root = "/dev/xvda1" vif = [ '' ] vfb = ['type=vnc.

Allocate disk image for the guest VM 2. Boot the guest: xm create /path/to/guest.cfg . Unmount image and create PV guest configuration 4. Mount and populate disk image with distro tools: Stacklet Bundler virt-install virt-manager (discussed further later) vmbuilder debootstrap The tool that comes with your favorite distro 3.Installing PV Guest OSes 1.

com Advantages: Simple to download and extract the images Available with different distribution OSes and pre-installed applications .com http://jumpbox.com http://rpath.Pre-built Guest Images Sources: http://stacklet.

P2V : Physical Machine to a VM Conversion of a physical machine into a virtual machine Scenarios: Virtualizing existing infrastructure Supporting legacy applications System administration benefits of virtualization Available Tools: Use existing backup tools to create a file backup P2V LiveCD XenServer conversion tool Various third-party tools .

xxx Better graphics: SDL or VNC Install vncviewer package Enable the vnc or sdl option in guest config file .xxx.xxx.xxx.Guest Access Methods The simplest way: console xm console domU_name A better way: SSH directly to DomU ssh user@xxx.xxx Simple graphics: SSH with X11 forwarding to DomU ssh -X user@xxx.xxx.

xm create .

xm console .

xm list .

xentop .


Session 3 Xen in the Datacenter .

Network Storage .

Network Storage Options ATA over Ethernet (AoE): Export block devices over the network Lightweight Ethernet layer protocol No built-in security Internet Small Computer System Interface (iSCSI): Exports block devices over the network Network layer protocol Scales with network bandwidth Client and user-level security Network File System (NFS): Exports file system over the network Network layer protocol Known performance issues as root file system .

Network Storage Options Network Block Device (NBD): Exports block devices over the network Network layer protocol Scales with network bandwidth Not recommended as root file system Distributed Replicated Block Device (DRBD): Exports and shares block devices over the network Integration with Heartbeat No additional storage server necessary .

vbladed 1 1 eth0 /path/to/image.1.img (for files) 3.xvda1..Using AoE 1. Export a guest image from the storage server: vbladed 1 1 eth0 /dev/ (for partitions) . Point the guest configuration to the image: disk = ['phy:etherd/e1..w'] Notes: Remember that AoE provides no security Never use the same shelf/slot for two images . Install required packages: Install vblade on the storage server Install aoe-tools and the aoe module in the Domain0 2.

d/drbd.xvda. Install required packages: Ubuntu/Debian: drbd8-utils and drbd8-module Red Hat/CentOS: drbd and drbd-km 2.Using DRBD 1.drbd.conf : options drbd disable_sendpage=1 3.org/users-guide/ch-xen. Configure DRBD: Mostly beyond the scope of this presentation Disable sendpage in /etc/modprobe.w' ] Documentation: http://www.html . Point the guest configuration to the image: disk = [ 'drbd:resource.

Management Tools and Integration .

Guest Management Tools
Simplify: Creation of guest images Manipulation of guest domains Generation of guest configuration files Monitoring resource usage by guests Popular tools: Convirt
Open-source Third-party product and support

Open-source Web-based tool

Virtual Machine Manager
Open-source Desktop tool

Designed for full datacenter management Allows for managing the complete lifecycle of Xen (and KVM) guests and hosts Open-source with commercial support

Convirt: Manage VMs

Convirt: Manage Domain0s .

Convirt: Provision VMs Based on Templates .

Convirt: Manage VM Configuration .

Zentific Open source web-based management tool Allows for managing and provisioning Xen guests .

Zentific: Main Screen .

Zentific: VM Status Panel .

Zentific: VM Configuration .

Zentific: Web-based Guest Console .

Open source with Red Hat support . resource allocation.Virtual Machine Manager Graphical user interface for managing virtual machines Allows for Xen guest performance monitoring. and domain creation.

Virt-manager: Virtual Machine Manager .

Virt-manager: Creating a PV CentOS Guest by URL .

Virt-manager: Configuring a PV Guest

Virt-manager: Installing PV guest

Virt-manager: Accessing a PV CentOS Guest

Virt-manager: Configuring an HVM Guest .

Installing a Windows HVM Guest from CD-ROM .

Virt-manager: Windows HVM Guest Running in Xen .

Xen Integration and Compatibility libvirt: Provides a uniform interface with different virtualization technologies Mainline Virtualization API (pv_ops): Provides a common paravirtualization interface in mainstream Linux kernel for increased performance and capabilities Open Virtual Machine Format (OVF): Defines a set of metadata tags that can be used to deploy virtual environment across multiple virtualization platforms Xen API (XAPI) .

Advanced Networking .

. Run network bridge script for each physical interface: /etc/xen/scripts/network-bridge start vifnum=0 netdev=eth1 bridge=xenbr1 2.. Configure the DomU's vif option for each bridge: vif = ['bridge=xenbr1'..] .Multiple Dom0 Network Interfaces Motivation: Segregate DomUs over different networks Procedure: 1.

Multiple DomU Network Interfaces Motivation: Allow a DomU to connect to different virtual bridges Procedure: Modify DomU configuration file: vif = ['bridge=xenbr0'.] ... . 'bridge=xenbr1'.

but allow them to communicate with one another Procedure: 1. Create a dummy bridge in Dom0 in network configuration or with brctl 2. Configure DomUs to connect to that dummy bridge: vif = ['bridge = dummy0'] .DomU Network Isolation Motivation: Isolate DomUs from external network.

.DomU Network Rate Limiting Motivation: Rate limiting for DomU network usage for better performance isolation Procedure: Configure DomU's vif option with rate parameter : vif = ['.. rate=50Kb/s'] ..


Performance and Scalability .

Measuring Performance CPU: xm top / xentop Buffer: xentrace Disk I/O: xenmon Hardware Events: xenoprof .

Memory and Scalability Using memory overcommitment. allowing greater scalability Risk poor performance due to swapping . a VM is made available for use by other VMs Reduces wasted resources. more memory can be allocated than is on the system Memory allocated to. but unused by.


Session 4 Xen in the Cloud .

Guest Relocation .

Guest Relocation Cold Relocation Warm Migration Live Migration .

Cold Relocation Motivation: Moving guest between hosts without shared storage or with different architectures or hypervisor versions Process: 1. Shut down a guest on the source host 2. Start the guest on the destination host . Move the guest from one Domain0's file system to another's by manually copying the guest's disk image and configuration files 3.

Cold Relocation Benefits: Hardware maintenance with less downtime Shared storage not required Domain0s can be different Multiple copies and duplications Limitation: More manual process Service should be down during copy .

Resumes guest's execution on destination host . Transfers guest's state across network to a new host 3.Warm Migration Motivation: Move a guest between hosts when uptime is not critical Command: xm migrate Result: 1. Pauses a guest's execution 2.

the guest is not externally accessible Requires shared storage Network connections to and from guest are interrupted and will probably timeout .Warm Migration Benefits: Guest and processes remains running Less data transfer than live migration Limitations: For a short time.

and power management Command: xm migrate --live Result: 1. Re-routes network connections. Begins transferring guest's state to new host 2. Repeatedly copies dirtied guest memory (due to continued execution) until complete 3. and guest continues executing with execution and network uninterrupted . hardware maintenance.Live Migration Motivation: Load balancing.

Live Migration Benefits: No downtime Network connections to and from guest often remain active and uninterrupted Guest and its services remain available Limitations: Requires shared storage Hosts must be on the same layer 2 network Sufficient spare resources needed on target machine Hosts must be similar .


Xen Cloud Platform (XCP) .

and is free XCP promises to contain cutting-edge features that will drive future developments of Citrix XenServer . standardscompliant virtual switch) XCP was originally derived from Citrix XenServer (a free enterprise product). is open-source.Xen Cloud Platform (XCP) Xen Cloud Platform (XCP) is turnkey virtualization solution that provides out-of-the-box virtualization/cloud computing XCP includes: Open-source Xen hypervisor Enterprise-level XenAPI (XAPI) management tool stack Support for Open vSwitch (open-source.

XCP Features Fully-signed Windows PV drivers Single Root I/O Virtualization (SR-IOV) support Heterogeneous machine resource pool support Installation by templates for many different guest OSes .

XCP XenAPI Management Tool Stack VM lifecycle: live snapshots. disaster recovery Flexible storage. notification Upgrade and patching capabilities Real-time performance monitoring and alerting . checkpoint. auto configuration. networking. and power management Event tracking: progress. migration Resource pools: live relocation.

XCP's xsconsole (SSH or Local) .

XCP's XAPI Viewer

XCP Management Tools

Open-source clone of Citrix XenCenter Manages both Citrix XenServer and Xen Cloud Platform Freely available: http://www.openxenmanager.com

Multi-guest Status .OpenXenManager: Multi-host.

OpenXenManager: Host and Guest Consoles .

net/projects/xenwebmanager/ .XenWebManager Web-based utility that shares OpenXenManager's codebase Open-source and freely available: http://sourceforge.

XenWebManager: Guest Creation with Templates .

XenWebManager: Guest Creation with Templates .

XenWebManager: Guest Creation with Templates .

XenWebManager: Guest Creation with Templates .

XenWebManager: Guest Creation with Templates .

XenWebManager: Guest Creation with Templates .

XenWebManager: Guest Creation with Templates .

open-source management for both Citrix XenServer and Xen Cloud Platform VNC guest console via web browser Freely available as software or a virtual appliance: http://www.Xen VNC Proxy (XVP) Web-based.xvpsource.org .

XVP: Host Pool and Guest Status .

XVP: Manage Guests .

XVP: Two-server Pool Running Virtual Appliances .

XVP: Select Server to Boot VM .

XVP: Web-based Guest Consoles .

XCCS is totally web based so any computer or smart phone with a web browser can be used with it!" Open-source and freely available as software/appliance: http://www.xencloudcontrol.Xen Cloud Control System (XCCS) "XCCS is a lightweight front end package for the excellent Xen Cloud Platform cloud computing system.com .

Xen Cloud Control System (XCCS) .

Xen Cloud Control System (XCCS) .

Xen Cloud Control System (XCCS) .

Xen Cloud Control System (XCCS) .

Xen Cloud Control System (XCCS) .

Xen Cloud Control System (XCCS) .

Xen Cloud Control System (XCCS) .

Xen Cloud Control System (XCCS) .

Xen Cloud Control System (XCCS) .

Closing Remarks .

html Abstracts.xen. slides.xensource.xensource. and videos from Xen Summits .com/xenwiki/ Xen Blog: http://blog.org http://wiki.org/community/ Xen Wiki: http://wiki.com/xenwiki/XenCommonProblems Books: The Definitive Guide to the Xen Hypervisor Running Xen: A Hands-On Guide to the Art of Virtualization Discussion: http://www.xen.Useful Resources and References Community: Xen Mailing List: http://www.org/community/xenpapers.xen.

You're Reading a Free Preview

/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->