You are on page 1of 44

M ASARYKOVA UNIVERZITA FAKULTA INFORMATIKY

}w¡¢£¤¥¦§¨!"#$%&123456789@ACDEFGHIPQRS`ye|
B ACHELOR THESIS

Virtual machine management software

Ondrej Famˇ ra e

Brno, Spring 2011

Declaration
I hereby declare, that this paper is my original authorial work, which I have worked out by my own. All sources, references and literature used or excerpted during elaboration of this work are properly cited and listed in complete reference to the due source.

Ondrej Famˇ ra e

Advisor: RNDr. Jan Kasprzak ii

Acknowledgement
I would like to thank my supervisor for support and feedback. Also I would like to thank Z.F. for grammar check and moral support.

iii

Abstract
The goal of the work is to introduce concept of virtualization management software (VMS), compare existing solutions in this area and introduce architecture of libvirt, virtualization API library. The practical output of this thesis is to design VMS for use at Faculty of Informatics at Masaryk University.

iv

libvirt v . qemu. kvm.Keywords virtualization.

. . . . . . . . . 4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2. . . . . . . .1. . . . . . . . . . . . . . 3. . . .1.4 Control Groups (cgroups) . . . .1 VM lifecycle management . . . . . . . . . . . 2. . . . .5 User management . . . .2. . . . . . . . . . . . . . . . . . . .4 OS-level virtualization/containers virtualization . . . . . . . . . . . . . . . . . . 2. . . . .1. . . . . . . . . . . . . . . . . . . . . . .2 Amazons Elastic Compute Cloud (Amazon EC2) Libvirt virtualization API . . . . . . . . . . . .3. . . . . . . . . . . . . . .2. . . .3. . . . . . . . . . . . . .8 VM access control . . 3. . . . . . . . . . . . 3. . . . . . . . . . . . . Virtualization management software . 4. . . . 3. . . . . . . . . 3. . . .1 Architecture . . . .3 Linux-specific technologies used in virtualization . . . . . . . . . . . 4. . . . . . .1. . . . . .1. . . . . .1.2 libvirt . .3. . . . . .2 VM configuration management . . . . . . . . . . . . 3 5 5 6 6 6 7 7 8 8 8 9 9 9 10 10 10 11 11 12 12 12 13 13 14 14 15 15 16 17 17 18 19 19 20 21 21 1 3 4 . . . . . . . . . . . . . . .3. . . . . . . . . 2. . . 3. . . . . . . 2. . . . . . . . . . . . . . 3. . . 3. . . 3. . . .1 Full hardware emulation . . .3. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3. . . . . . . . . . . . . . . . . . . . . . . . . .1. . . .1. . .2 Built-in VMS . . . . . . . . . . . . .4. .4. . . . . . . . . . . 2. 2. . . . .3 Networking management . . . .2 Types of virtualization . . . . . . . . . . . .2. . . . . . . 2. . . . 2. . . . . . . . 4. . . . . . . . .1. . . . . .1 Architecture of VMS .3 Storage management . . . . 3. . . .5 VirtIO paravirtualized I/O drivers . . . . . . . . . . . . 3. . . . .2 Storage management . . . . . . . . 3. . . . . . . . . . . . .6 VM migration . . . . . About Virtualization . . . .1 Guest configuration management . . . . . . . . . . . . . . . . . . .1 Ganeti . . . . . . . 3. .3 sVirt . . . . .2. . . . .1 VirtualBox .3 Paravirtualization . . .2 Kernel Samepage Merging (KSM) . .1 OpenNebula . . . . . . . . . . . . . . . . . . . . . .7 VM provisioning .1 Huge pages . . . . . . . . . . . . 3. . . . . . . . . . 2. . . . . . . . . . .1 Common terminology . . . . .2 Hardware assisted/accelerated emulation . . . . . 3. . 2.3. . . . . . . . . . . . . . . . . . . . . 3. . . . .4 Cloud computing . . . . .4 Virtual network management . . . . . . . . . . . 3. . . . . . . . . . . . . . . . . . . . . . .Contents 1 2 Introduction .3 Stand-alone VMS . .1. . . . .1.2. . . . . . . . 2. . . . . .

.1 Networking . 5 VMS for Faculty of Informatics at Masaryk University (FI MU) 5. . 7. . . . .1 PV090 UNIX . . . .3 Limiting resource usage . . . . . . .2. .1 PV175/PV176 MS Windows Systems Management I. . . . . . . . . . . . . . . . . . . . . . 6. . . . . . . . . . . . . . . . . .1.2. . . 5. .3 Stats and graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1 Virtualization server Adikia . . . . . . . . . 7. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6. . . . . . . . . . . . . . . . . . . . . . . . . . .2 State of virtualization deployment . . . . . 6. .1 Future work . . . . . .2. . . . 8. . . . . . 6. .4 Special requirements . . . . . .5 Web application vs. . . . . . . . . . 7. . . . . . . . . . . . . . 5. .1 Faculty Administration (fadmin) . . . . . . . . 6 fadmin/virt .2. . . . . 5. . . . . . . . . . . . 5. 7.3 KSM . . .2 API bindings for other languages . . 7. . . . . . . . .2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3 VM hibernation script . . . . . . . . . . 6. . . . . . . . . . . . . . . .Screenshots . 5. . . . . . . . . . . . . . . . . . . . . .3 General requirements . . . . . . . . 8 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2 Memory . . . . . . . . . .4. . . . .3. 5.3. . . 7. . . . . . . . . . . . . Desktop application . . . . . . . . . .4. . . . . . . . . . . . . . . . . . . . . . .1 CPU . . 5. . . . . . . . . . . . . . . . . . . . . . . . . . . . .2 LaSArIS lab . . 5. . . . .2. . .3 Software . . . . . . . . . . . . . .2 Current deployment . . . . . . .3 Generic servers for research projects . . . . . . . . . . . . ./II. . . . 5. . . . . . . . . . . . . . .2 VM IP discovery script . . . . . . . . . .1. . . . . . .1 one-click VM creation . . . . . . . . . .4 Access and user management . . . . . . . . . . . . . . . . . 6. . . .4. . . . 7. . 22 22 23 23 23 24 24 24 25 25 25 25 26 27 27 28 28 28 29 29 30 30 31 31 31 31 32 32 32 33 33 34 35 35 37 2 . . .2 Custom scripts . . . . . 6. . . . . A Appendix . . . . . . . . . . . . . . . . . . . . . . . . .1 VM serial console script . . . . . . .2 Disks . . . . . . 6. . . . . . . . . . . . . . . . . . . . . . . . . .1. . . . . . . . . . .1 Database design .2 LaSArIS lab . . . . . . .4 Front-end . . .Seminar of System Management . . . .4. 7 Use case . . . . . . . . . . 7. . 7.3. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1. . . . . . . . . . . 4. . . . . . . . . . .

In order to simplify the management of growing number of VMs. virtualization management software (VMS) appeared. aiming to show how this is provided by libvirt. It provides a good building block for developing VMS on the top of it. Chapter 5 introduces concept of VMS for Faculty of Informatics at Masaryk University (FI MU). Based on the analysis we define general as well as some specific requirements for universal VMS at FI MU. As the potential platform for VMS. not only they offer VM management. Architecture of guest configuration management. the overview of architecturally interesting VMS is provided. After that current state of penetration of virtualization and VMS at FI MU is analysed. In chapter 4 VMS called libvirt is introduced. These techniques are discussed in chapter 2. With increasing popularity of virtualization. access and user management is discussed here. interactive access to a mainframe computer.3. [3] Since then several new techniques of virtualization has been developed to improved VMs’ performance. The concept of VM was in existence since 1960s when it was first developed by IBM to provide concurrent. Faculty Administration (fadmin) system is briefly introduced. but they go beyond that and provide management of activities tightly connected to virtualization. question of VM administration emerges.1 Introduction Virtualization is a technique that allows simultaneous execution of applications on single physical computer providing them with isolated environment called virtual machine (VM). General architecture of these systems is discussed in chapter 3 Later in the chapter. storage management. such as network and storage management. network management. Improvements specific to the virtualization on Linux systems are discussed in section 2. 3 .

1. VMS for FI MU. In conclusion (8) we summarize the achieved goals and suggest possible future improvements. Screenshots of created application can be found in appendix A. 4 . Here the architecture of system built on libvirt library as well as custom additions that adds functionality missing in libvirt is showed. is introduced in chapter 6. I NTRODUCTION The output of the work. In chapter 7 we take look at deployment of the work in real world environment.

executes the same programs.1 Common terminology Host computer is a real physical machine that runs hypervisor and posses hardware resources we want to use for virtualization.[6] It creates the same data. also called virtual machine (VM). Hypervisor also called virtual machine monitor (VMM) is a program that creates virtual environment and schedules resources of host for VMs.2 About Virtualization The primary goal of virtualization is to separate software from the available hardware resources in order to allocate them optimally to individual isolated systems[6]. This way virtualization allows the placement of specific virtual machines. 2. Compared to non-virtualized systems it adds another layer between running system and hardware. each containing only the necessary services for their task. 5 . Guest computer. Emulation in computing is referred to as functionally simulating a system by another. is an isolated virtual environment within host Operating system (OS). This improves security of these services as it brings another layer of isolation. achieves same results as original system.

Limitation of this approach is. However.2 Types of virtualization Depending on hardware capabilities of virtualization host and guest operating system (OS) compatibility we have several virtualization types.2. which allocates resources to hypervisor and then it allocates them to VMs. Till this point guest OS could run in emulated environment without modifications. 2. type 2 hypervisor runs within OS. with this compatibility comes the price of poor performance. Compared to the non-virtualized application.2 Hardware assisted/accelerated emulation If we have hardware that supports virtualization we can offload some work from software emulation to hardware to get better performance. 2. A BOUT V IRTUALIZATION There are two types of hypervisors: type 1 hypervisor runs directly on hardware and allocates resources as requested by guests. Sorting them from the slowest one but the most compatible with various guest OSs and broad range off processor architectures. that not all hardware provides virtualization support. This way we can run applications that were intended for hardware different from one we have. because every instruction sent to emulated hardware must be processed by software running on the host.2.2. 6 . The advantage of this approach is ability to emulate virtually any kind of hardware. performance is significantly worse. so some of it still needs to be emulated for guest by the software. to the fastest one but restricted to run only on host processor architecture and to use only host kernel calls. 2.1 Full hardware emulation This type uses only the software emulation to simulate guest hardware.

2. Performance of this solution is native-like. which is still considered to be a virtualization. The example of this kind of virtualization is Linux command chroot. 7 . it may be a good idea to improve their performance to make the aware of this environment. is that guest OS must be modified to be able to run and to communicate with hypervisor. This technique is called paravirtualization. Linux systems only need two parts. OS-level virtualization isolates guest in its own file system namespace and provides it with access to running host kernel. The basic idea is to ensure co-operation between guests and host system. but guest can only use kernel calls of currently running host kernel.4 OS-level virtualization/containers virtualization The fastest technique. is deployed only by isolating guest on file system level.3 Paravirtualization Because fully emulated VMs might not be aware of the virtual environment they run in.2. that arises here. The only problem.2. which creates functional system. in order to schedule resources of host more effectively among other guests. A BOUT V IRTUALIZATION 2.2. which creates file system isolation for guest. kernel and root file system.

8 . Since virtualization is a "memory-hungry". but in vast majority of nowadays kernels there is also support for larger pages called huge pages. these pages can be 2MB or 4MB in size.1 Huge pages Linux uses by default 4KB memory pages. Most of these duplicities can be suppressed by using copy-on-write memory pages 2 but this requires the system to recognize which memory pages are the same. called KSM. A BOUT V IRTUALIZATION 2. Depending on the architecture.3 Linux-specific technologies used in virtualization 2. it can benefit from this. The main advantage of using huge pages is that it can lower the number of cache misses in processor’s TLB (Translation Look-aside Buffer) .3. 1. This content is copied in case it need to be changed in one of pages. To mitigate memory page duplicities the technique. copy-on-write memory page doesn’t take up any space in memory. Performance improvement while using huge pages with KVM1 is around 20 %.[7] 2. Kernel virtual machine 2. Once these are found.3. they are converted into copy-on-write pages freeing up the memory.2. what makes "memoryhungry" applications run faster.2 Kernel Samepage Merging (KSM) Even while running Linux without virtualization. is deployed. This technique periodically scans memory searching for duplicate memory pages. in page table it only refers to memory page holding the same content. there is probability of allocating the memory page containing duplicate content.

which doesn’t provide paravirtualization. It can also be used for fair processor scheduling among guests if used with CFQ6 Linux I/O scheduler.[2] In virtualization it’s mainly used for limiting and accounting of basic resources (CPU. 3 It deploys Mandatory Access Control (MAC) to isolate guests from host and others guests. 5.3. (this requirement comes out from the fact that they are paravirtualized). 3. into hierarchical groups with specialized behaviour.3 sVirt sVirt is pluggable security framework for libvirt.4 Control Groups (cgroups) Control Groups provide a mechanism for aggregating/partitioning sets of tasks. [4] It allows use of paravirtualized drivers within hypervisor.5 VirtIO paravirtualized I/O drivers VirtIO is an API for paravirtualized drivers. 4. due to flaw in hypervisor. The second limitation of these drivers is that they have to be supported by the host system as well. On supported guest OSs it brings better performance comparing to their emulated counterparts but reducing compatibility with guest OSs. RAM.3. A BOUT V IRTUALIZATION Because for virtualization it’s not anything rare to have multiple guests running OS or applications that stores the same data in memory. 2. network). this can lower memory usage of guests on host.3.2. 2. and all their future children. libvirt is VM management system which will be covered later in chapter 4 Security Enhanced Linux Simplified MAC Kernel Completely Fair Queuing 9 . 2. 6. This solution is designed to work with SELinux4 and SMACK5 security drivers. This adds another level of isolation in case the guest would manage outbreak from virtual environment. disk.

modified and started VM as well as information about VM’s uptime before its crash. 1. This includes who.dumping their active memory to file on hosts for later restoration and shutting them effectively down. 3.3 Virtualization management software Since hypervisors provides virtual environment for guests.1.1 VM lifecycle management The fundamental property of VMS is ability to keep the track of VM’s life. VMS should also provide feature for pausing or hibernating VMs . is used. Some of the features usually required from VMS include: • • • • • • • • VM lifecycle management VM configuration management management of storage used by VMs management of virtual networks user management VM migration from one host to another VM provisioning VM access control Interface provided by VMS can be in form of: • • • command line interface (CLI). when and how created. they are not usually accessed directly.1 Architecture of VMS 3. which provides interface that makes management of guests easier. The main goal remains the same. Graphical user interface 10 . to abstract work with hypervisor and related utilities. Instead virtualization management software (VMS). application with GUI1 or application programming interface (API).

which populates empty virtual disk with selected data. this subsystem provides basic services such as: • • • resizing the virtual disk.3.2 VM configuration management As the VM is something that exists only virtually. or snapshoting which allows to create a snapshot of the virtual disk in selected moment and adds possibility to return its state to this point. While in case of less abstract one.1. the configuration of VM can be variably abstracted. which knows where the virtual disk is and passes this information to VM. 11 . This is provided by VMS. Storing the configuration is only the question of implementation and it usually includes using files and databases to store it. as it may consists only of command line arguments that needs to be passed to emulator in order to start VM. space allocation on host (in case of file-based virtual disks).1. a partition on physical disk on host or a physical disk attached to the host. Possible virtual disk’s locations on host are: • • • a file (on host’s or network file system). In case of more abstract configuration it brings better interoperability across different VMS and hypervisors. there needs to be a place where its configuration is stored. Depending on VM’s storage type. It also can provide features such as disk provisioning. V IRTUALIZATION MANAGEMENT SOFTWARE 3. virtual disk compression or encryption provided by host. This subsystem keep track of disks and their attachment to VMs. in order to provide optimization in case of simultaneous access by multiple VMs to the disk by disabling caching. 3. it can be more optimized for selected hypervisor.3 Storage management VM is not aware of virtual disk’s location on host. Depending on VMS.

1.3. Lightweight Directory Access Protocol 12 . 3. VMS can provide two types of migration: • • offline migration which is done while VM is not running during migration and online migration that transfers VM without stopping it. Other approaches enables host to use virtual networks to have control over traffic that flows through it.1. In second case this can be very difficult because the VM’s memory and storage may be in unpredictably changing state that makes online migration sometimes impossible.4 Virtual network management VM can be connected to physical network in different ways. it leaves host minimum control over the network card.5 User management For multi-user environment deployment the user management is the most important feature. direct access to host’s physical network (using routing or network bridge on host) or exclusive access to network card of the host. NAT-ed access to host’s physical network. V IRTUALIZATION MANAGEMENT SOFTWARE 3. depending on host’s level of control over this connection. host or host’s physical network. 2. These are: • • • • host-only access which allows the VM’s network to access only the host.1. Virtual networks provides ability to interconnect VMs with themselves.6 VM migration Process of moving VM from one host to another we call migration. 3. While the last option grants VM the same access to network as the host have with nearly the same performance as host. Virtual network management stores information about these virtual networks and also keep track of their lifecycle. as it provides way to limit access to actions provided by VMS by users and groups. ISO images) connected to VM. This process provides temporarily or permanently transferring the VM configuration along with storage (such as virtual disks. Authentication of users can be performed by VMS or it can be provided by external mechanism such as LDAP2 or Kerberos.

V IRTUALIZATION MANAGEMENT SOFTWARE 3.1. This rapidly speeds up deployment of VMs serving the common tasks such as web or database servers. VMS deploys access restrictions depending on used technology they can vary from simple file system access restrictions (for example in case of connecting to local socket on which serial console of guest resides). VMS can provision template of VM’s hardware configuration or even disk template containing beforehand prepared OS and applications. serial console to VM on host or over network.3. direct connection to VM’s OS over network (for example using ssh). Also it can provide the option to upload user’s disk or VM configuration to system. The common way is to use password to protected access to VM. Remote Desktop Protocol 13 . 3.7 VM provisioning To simplify creation of VMs.8 VM access control Several ways how to access VM include access using: • • • graphical console to VM on host or over network.1. to certificate based authentication using RDP3 . 3.

Storage management is rather simple one as it only keeps track of disk and ISO images4 that are used by VMs. 4. Mac.2. Among this tools we can find GUI named VirtualBox Manager and CLI utility named VBoxManage for advanced users. relies on host system none built-in VirtualBox (VB) is hypervisor bundled with set of tools acting as VMS in one package.2 Built-in VMS The model in which hypervisor is inseparable part of VMS is called Built-in VMS.1 VirtualBox Supported platforms: Supported hypervisors: used version: Management of Storage: Network: Users: VM provisioning: VM migration: Linux. Usual deployment of this model is used on workstations as virtualization solution for single user or as part of bigger VMS where it servers the role of node. ISO image is an "image" of an entire CD. The hypervisor provides support for VM migration. The entire contents of a disc can be perfectly represented in a single ISO file. Windows built-in VirtualBox hypervisor 4. Its limitation is that it often can manage only the built-in hypervisor. The GUI compared to CLI contain only subset of features that VB provides. Network management is quite simple and limits VMs to use 4 network cards at max. V IRTUALIZATION MANAGEMENT SOFTWARE 3. Advantage of this approach is that it can fully exploit all the feature of hypervisor. however it involves use of CLI and so it provides solution to advanced user only. which it calls teleportation.6 OSE simple.[1] 14 . or BD. Access control is based on "all or nothing" principle and it does not provide user management. DVD.0. 3. built-in built-in none.3.

2 built-in built-in built-in.3. V IRTUALIZATION MANAGEMENT SOFTWARE 3. cluster = set of machines that cooperate in order to bring better performance or redundancy 6. This allows easy migration of VM’s between hosts.org/ 15 .3 Stand-alone VMS Opposed to built-in VMS they come without bundled hypervisor and serves purpose of • • an alternative VMS for built-in VMS or universal VMS capable to manage more than one hypervisor.org/projects/ganeti-webmgr 7. limited built-in built-in Ganeti represents cluster-based 5 VMS that can manage Xen and KVM hypervisor. 3. 5. Distributed Replicated Block Device 9. across cluster.1 Ganeti Supported platforms: Supported hypervisors: used version: Management of Storage: Network: Users: VM provisioning: VM migration: Linux Xen.4.osuosl. Redundant Array of Independent Disks (in mirroring mode) 8.3. http://www. this abstraction can be limiting for some hypervisor-specific features which may break portability of VM’s configuration. Ability to manage multiple hypervisors usually incorporates abstraction of VM’s configuration in VMS. called nodes.drbd. http://code. Storage management provides as interesting feature ability to setup virtual disk on network RAID17 disk facilitating DRBD89 technology. Primary goal is deployment on 1-40 hosts with easy setup for redundancy of VMs. KVM 2. It is accessed through CLI or API that provides management by other applications such as Ganeti Web Manager6 . However.

. KVM. To demonstrate all capabilities it also provides CLI utility that can be used for access to this API.3. . LXC. Mac. Main aim of libvirt is to provide building block for other VMS.2 libvirt Supported platforms: Supported hypervisors: used version: Management of Storage: Network: Users: VM provisioning: VM migration: Linux. More about libvirt can be found in chapter 4.2 built-in built-in none built-in built-in Libvirt is an API for wide range hypervisors that tries to provide unified management of VMs on different hosts using different hypervisors.3.8.. V IRTUALIZATION MANAGEMENT SOFTWARE 3. VirtualBox. VMware. 16 . 0. Windows Xen.

KVM. web GUI is mostly deployed.3. To access cloud-based VMS. Before continuing we establish some additional terminology: instance is name for VM running inside cloud. This infrastructure is then called a cloud. 17 . Compared to Amazon EC2 it provides only VMS without infrastructure on which it will run. which can help us in times we need more hardware than we have and we don’t want to run all our instances in public cloud. [5] It builds on the top of other VMS and provide cloud infrastructure for private. Similarly it uses web based GUI to manage the whole infrastructure. Main advantage of these VMS is that they are aimed to provide easy deployment of virtualization.4. storage and user management.1 OpenNebula Supported platforms: Supported hypervisors: Linux Xen. private cloud means that we provide hardware for cloud infrastructure. hybrid and public clouds. They have already built-in network. It aims not only to provide access to hypervisors but also to manage whole infrastructure around hosts lowering the maintenance cost of virtualization. public cloud means that whole infrastructure runs on hardware over which we have no control. extensible. and comprehensive management layer to automate and orchestrate the operation of virtualized data centers by leveraging and integrating existing deployed solutions for networking. V IRTUALIZATION MANAGEMENT SOFTWARE 3. hybrid cloud is mix between public and private cloud where we use own hardware but we also can use public infrastructure. storage. flexible. VMware OpenNebula aims to provide a open. virtualization.4 Cloud computing Cloud computing represents VMS that tries to simplify whole concept about VMs in order to allow less technically skilled user to use virtualization on large scale. monitoring or user management. 3.

V IRTUALIZATION MANAGEMENT SOFTWARE 3. Configuration of VMs is defined by templates and cannot be fully customized. 18 . Mac. but because it is limited to be used only on infrastructure provided by Amazon it may not be solution for everybody. Pricing the usage of this service is based on resources consumed by running instances. This solution is example of how the cloud-based VMS can look-like a worklike. Windows Xen Amazon EC2 is paid public cloud that provides not only VMs web GUI and API to access VMs but also underlying infrastructure.4.3.2 Amazons Elastic Compute Cloud (Amazon EC2) Supported platforms: Supported hypervisors: Linux.

Goal is to provide one way how to do something in libvirt. Drivers implement same API for communicating with libvirt through which they provide access to hypervisor they represent. Libvirt is intended to be a building block for higher level management tools and for applications focusing on virtualization of a single node (the only exception being domain migration between node capabilities which involves more than one node). so when the external application calls libvirt function that starts the VM. 4.1 consists of public API for external applications and driver API that includes drivers for communication with various hypervisors. [6] In our terminology node is a host computer. and domain is a guest computer. possibly remote.4 Libvirt virtualization API The goal of libvirt is to provide a common and stable layer sufficient to securely manage domains on a node.1: libvirt architecture diagram Libvirt as shown in figure 4. this function will be called 19 .1 Architecture Figure 4.

The only exception is when application starts to communicate with libvirt. it firstly uses a special URI1 which identifies which driver to use. Figure 4. 1. or one provided by remote libvirt as shown in figure 4. It can be persistent or transient. Besides VM configuration management it provides also virtual network and storage management.4. To ease initial understanding of how libvirt works it provides virsh. that is shipped with libvirt. While persistent guest configurations are defined in libvirt permanently until user decides to undefine them from it.2. Extensible Markup Language 20 . Driver can refer not only to hypervisor running locally but also to remote hypervisor either running on its own. CLI management utility. L IBVIRT VIRTUALIZATION API same way no matter what underlying driver is used to access hypervisor. virsh implements features that libvirt API provides in simple shell environment allowing user to interact with libvirt API directly. transient configurations are defined only for time the guest is running or until the host on which they are defined is restarted.1 Guest configuration management Guests configuration is described by XML2 files.2: libvirt remote architecture diagram 4. Uniform Resource Identifier 2.1.

1. a physical disk. or used for creating further pools. L IBVIRT VIRTUALIZATION API 4.3 we can see physical diagram of all possible virtual network configurations on host.3 Networking management Configuration of virtual networks is represented by XML files. Following terminology applies: volume is a single storage volume which can be assigned to a guest.[6] Pool can be used to manage: • • • • • a directory on hosts containing files that represents virtual disks. Dividing by connectivity of these networks to physical network connected to hosts we differentiate the consecutively: • • Bridged networking: Guest A is connected to the same physical network as the host is connected using bridge on host. routed networking: similar to NAT-ed networking this can provide access to physical network but without need to use NAT.1. LVM3 group. Configuration of storage pools as well as volume definitions are represented by libvirt in XML format. Guest C can so communicate only with network VNET1. • • 3. Internet Small Computer System Interface 5.2 Storage management Concept of storage volumes and storage pools is used to provide management of VM’s storage. Logical Volume Manager 4. In figure 4.[6] pool provides a means for taking a chunk of storage and carving it up into volumes. NAT-ed networking: Guest B is connected with interface eth0 to virtual network VNET0 which is forwarded by host to physical network using NAT5 isolated networking: Guest C is connected to virtual network VNET1 which isn’t forwarded by host to any physical network directly. Network address translation 21 . iSCSI4 target or host adapter.1. 4.4.

cpan.http://libvirt. http://www.http://search.org/python.html OCaml . 4.html Ruby .org/php Python .org/collaborate/workgroups/ networking/bridge 22 . Beyond this there is in the time of writing no user management provided by libvirt. it provides support for C and C++ directly and for some other languages it provides API bindings: • • • • • • • C# .1.http://libvirt.org/java. There are two sockets from which one provides read only and second full access to libvirt.4.org/csharp.org/ocaml/ Perl .linuxfoundation.http://libvirt.2 API bindings for other languages To develop own VMS on top of libvirt.4 Access and user management The main way how to communicate with libvirt is using sockets that libvirt creates.http://libvirt.org/ruby/ 6.html Java . L IBVIRT VIRTUALIZATION API Virtual networks in libvirt on Linux are created by using Linux Ethernet bridge 6 which is protocol independent.org/dist/Sys-Virt/ PHP .http://libvirt. 4.http://libvirt.

It’s purpose is to provide framework into which other applications that require user authentication at FI MU can be integrated.1: VMs deployment 1. We have also deeply analyzed the two examples of their deployment in production. except passwords of users.1 Faculty Administration (fadmin) Fadmin is web based administration system that also provides user management at FI MU.fi. PV175. The application can be found at https://fadmin. 5. However usability of these systems was not very good. very limited number of configuration options for VMs. underestimated propagation of services they provide. are stored in fast database which enables easy development of new applications. As main problems we have identified: • • • • lack of documentation on how to use these systems.2 State of virtualization deployment At FI MU in time when we were starting with development of our software1 already existed some VMS. All data. overall complexity of management system.muni. PV176 1 105/160 10/20 LaSArIS lab 2 7/10 4/5 virtualization hosts total VMs (active/inactive) VMs not used for teaching (project VMs) (active/inactive) Figure 5. around summer 2010 23 .5 VMS for Faculty of Informatics at Masaryk University (FI MU) 5.cz/auth/.

/II. Despite effort of administrators the management of VMs is sometimes clumsy and lacks variety of options such as serial console for VMs or access to graphical console other than through MS ActiveX plug-in. Additionally this approach provides easier access to functions such as remote machine reset in case of guest OS freeze or other problem which could be done using VMS.muni. Solution was optimized for MS Windows platform and was not very suitable for generic guest OS hosting.2. All these problems could be solved by use of VMs. On one server we found web server running along with hypervisor because of performance issues in virtual environment.2. Deployed VMS was Microsoft System Center Virtual Machine Manager 2008 (MSCVMM) which manages guests running on Microsoft’s (MS) Hyper-V hypervisor. VMS was accessible only through web and was operated only by administrators of lab.https://is. VMS FOR FACULTY OF I NFORMATICS AT M ASARYK U NIVERSITY (FI MU) 5.5. MS Windows Systems Management I. PV175 . Many of these servers were even operated outside designed server rooms where they can be easily physically compromised or can suffer from instability due environment.cz/predmet/FI/PV176 24 .2. Primary goal of MSCVMM is to provide management of VMs for students attending this courses2 . 2.2 LaSArIS lab At Lab of Software Architectures and Information Systems (LaSArIS) we found virtualized infrastructure using two lab servers running VMware Server hypervisor. This web cannot be placed elsewhere because it requires special (sometimes obscure) software which possibly could cause troubles along with other webs on existing web servers.cz/predmet/FI/PV175 PV175 ./II.3 Generic servers for research projects During research we noticed non-neglectable number of physical servers which primary purpose was to host web of project and results of research. 5.1 PV175/PV176 MS Windows Systems Management I.muni. Aim of this solution was to provide ability to create VMs on demand as they were needed for research at lab. On demand VMs were provided for staff and other students conducting research. 5.https://is. Another problem was need to have administrator privileges for some of these webs to get setup and maintain.

25 . ability to use various processor architecture including x86.4 Special requirements Additionally some special requirements raised.2) inside VM. VMs owned by group of users not only one user. VMS FOR FACULTY OF I NFORMATICS AT M ASARYK U NIVERSITY (FI MU) 5. easy to use intuitive user interface.cz/predmet/FI/PV090 5.4.3 General requirements From previous analysis we created list of following requirements: • • • • easy access to VM’s graphical console. 3. 5.1 PV090 UNIX .2. Main requirements were: • • • custom network topology accessible only by students attending the seminar. Advanced RISC(Reduced instruction set computing) Machine 4. plans to start using VMs for teaching.Seminar of System Management PV090 course4 . nested virtualization support5 make use of own hardware for hosting VMs.4. serial console and life cycle control.muni. 5. Ability to use assisted virtualization (2.2 LaSArIS lab Main goal was simplification of access to VMs and ability to provide free capacities for other members of lab to run own VMs without assistance of administrators.5. which is similar to PV175 and PV176 courses. https://is. x86_64 and ARM 3 . 5.

5 Web application vs. VMS FOR FACULTY OF I NFORMATICS AT M ASARYK U NIVERSITY (FI MU) 5. the only reasonable way was to deploy web based applications. Desktop application To provide this application to widest range of users without limiting them on used OS.5. Along with universal access we gained easier maintainability of application. Compared to desktop based ones which provides in general better performance in this case was neglectful compared to ability to access the application easily from wide range of devices. 26 .

6. VM hardware configuration is stored in several tables which has to be joined in order to provide enough information to create VM configuration.6 fadmin/virt Our implementation of management system is divided into two parts: • • libvirt as back-end for VM management on virtualization servers fadmin as front-end for VM management in database Libvirt cannot store VM configurations in database and it leaves us two options where to store these data and how to retrieve them back. While user management is handled by fadmin. because data are stored in fast database and retrieving them from it is much quicker than querying virtualization server and parsing its response. Second option although provides better performance. To provide integration with fadmin’s user management some tables are interconnected with fadmin’s core tables. However this decision brings consistency problem in which data on virtualization server and in database can be different. either by • • storing all information in libvirt on virtualization server and every time they are needed we retrieve them from virtualization server or storing all information in database and use ability to store information on virtualization servers only as cache but not as primary source. storage pools and other generic system and performance data. Separate tables for disks and network adapters are used. because they are stored only once. These are stored in tables separately from fadmin’s core tables. list of virtualization servers. 27 . our application takes care of limiting resources that users may use for their VMs. Keeping in mind that only right data are in database and not on virtualization server we can ignore this problem and implement cache-like behaviour of virtualization server. Other tables are used to store logs. The first option provides simpler design and easy to achieve consistency of data.1 Database design VM configurations are stored in database provided by fadmin along with other virtualization related information. This enables us to add multiple instances of same type of hardware without database layout modification.

Libvirt creates names of devices that holds access to serial console of VM unpredictably. 6. but after creation of such device it stores information about it’s location into configuration file of VM. • Both script can be found on attached CD in directory scripts/serial_console/. list of these VM’s is stored in file created by first script 3.6.1 VM serial console script Version of libvirt we were using1 was not able to provide reasonably secure and easy to setup way to access serial console of running VM over network we created own solution.2. To address these problems we divided solution into these two sub-problems: • script that waits for newly started VM to create its temporary configuration from which we can read out path to serial console of guest on host.2 Custom scripts To overcome some missing functionality in libvirt we have implemented this functionality using single purpose custom scripts. Our goal was to provide serial console access over ssh which is both secure and comfortable for users (as they could login using same login and password as they use on other machines at FI MU). Dynamic Host Configuration Protocol 28 . Script is deployed • on main DHCP server for FI’s physical network to provide access to DHCP leases for VMs that are connected to network directly and 1. Some of them may render useless in future as the libvirt will implement their functionality.2 VM IP discovery script This script tries to guess the IP address of VM by parsing DHCP3 server’s leases file on machine on which it is used and provides mapping of MAC addresses to IP addresses if it finds a match. As another part of solution we provided users with ssh access to virtualization server. 6.change permissions on this file to user whose VM it is and write down this info to the file and script that after successful login of user through ssh to virtualization server will jail user in restricted environment which will provide only our interface to access user’s serial consoles that are active2 .2 2. but for security reasons we did not want them to have shell access on this server.2. 0.8. FADMIN / VIRT 6.

FADMIN / VIRT • on all virtualization servers to provide information about IP address of VMs behind NAT or in isolated networks which are served by DHCP server running on virtualization hosts. Sometimes guests OS may not survive resume from hibernation as great change in time of guests hardware clock occurs. we setup script to periodically check if such VMs exists. If VM’s lifetime reaches zero or negative values periodically running script will request VM’s hibernation.6. Out Of Memory 29 . As not fully implemented feature that should protect virtualization servers from free memory exhaustion we deploy cgroups.3 VM hibernation script To protect virtualization servers from being overwhelmed by inactive VMs that are only consuming resources. Detection of these machines is based on value of VM’s lifetime provided by database. but this happens only occasionally 6. This technique is called overcommiting and it wouldn’t be safe if we hadn’t used KSM that lowers memory usage by deduplicating memory pages. Script relies on fact that VM uses DHCP to configure its IP address. Next time the VM is started. dumping it’s memory to disk and effectively shut it down to free up resources of virtualization server.3 Limiting resource usage Ensuring virtualization servers stability we limit amount of memory that VMs can use to total of 110 % of host memory. This values is by default 7 days from VM’s start and can be indefinitely times extended to 7 days from the time this was requested. It can be found on attached CD in directory scripts/dhcp_leases_parser/. 4. When memory limit of group of processing that are used for virtualization is reached OOM4 killer tries to kill process inside this group instead of some other processes that may be important for hosts OS to function properly. 6. By hibernation we mean pausing VM. the dumped memory loads back and VM execution is restored from the point when this happened.2.

5. VM details page with details about selected VM and ability to change them and disk manager page which lists user’s virtual disks and shows whether they are attached to VMs. https://fadmin. It allows creation of VM by just typing it’s name and optionally description to form on main page and then just hit submit button to create it.muni. FADMIN / VIRT 6. VM is instantly ready to be started with default configuration or to be modified by user.cz/auth/virth/help.1 one-click VM creation For users the most interesting feature we have implemented is single-click VM creation.mpl 30 .4 Front-end To make users feel familiar with interface we chose fadmin as it is already known by users.6.fi. Simplification of whole interface was a bit harder but in the end it resulted in only three page design consisting of • • • main page with list of user’s VMs and form to create a new VM. 6. Use of this interface is well documented in online help 5 which is referred from main page of application.4. Screenshots of pages can be found in Appendix A.

1 Networking Adikia is connected to the network using 2 network cards: • • eth0 provides access for management and administration of server. 1. eth1 provides networks for VMs and acts as default route for traffic from Adikia. This way we separated management traffic (backups. named Adikia.7 Use case To test usability and stability of our application we introduced it to PhD students and staff of FI and granting them access to use it.1. around 2 TB of raw disk space and 2 Gigabit network cards. Serial Attached SCSI(Small Computer System Interface) 31 . rotations per minute 2.1 Virtualization server Adikia As virtualization server we have used abandoned server. We extended resource limits for those interested in further testing. Disks were arranged in RAID5 configuration to provide more capacity rather than more performance. 4x 300GB disks and 4x 136GB disks.2 Disks Disks spins at 15000 RPM1 and are attaches using SAS2 interface. 32 GB RAM. with 4 dual-core Xeon processors. 7. 7. Currently used disks are: • • • 2x 36GB disks. VM migrations) from traffic of virtual machines and also put all VMs to network which is specificaly assigned to VMs at FI.1. 7.

cpu usage and KSM effectiveness with MRTG 3 .3 as hypervisor and libvirt 0.3 Stats and graphs Since summer of 2010 we monitor Adikia’s memory usage. We enabled KSM.12.ch/mrtg/ 32 .2 for VM management. 7.2 Current deployment In time of writing the work around 50 VMs were defined in database by around 30 users and 14 VMs were running on Adikia.oetiker. Multi Router Traffic Grapher .1. which deduplicates the same memory pages and in order to enhanced security of both host and guests we configured SElinux with sVirt extension in enforcing mode. 7.http://oss. Most of these machines run much longer than the standard lifetime of the VM and are meant as research machines for students and staff. To access serial consols of running VMs we allowed users to connect to ssh server on host with restriction to run only application for connection to these consols. U SE CASE 7.8.3 Software • • • Fedora 12 as an operating system (OS). 3. QEMU/KVM 0.7.

2: Memory in bytes(light blue .active memory) Memory graph (7. This high CPU usage by KSM was lowered in newer kernels. The graph also shows that VMs utilize at average same amount of CPU as the systems tasks.3. green .2 Memory Figure 7.3.7.non-idle CPU) Looking at CPU usage graph (7.system CPU.inactive memory. U SE CASE 7.1: CPU usage in % (orange .2) shows us how much of memory pages were active (used) and how much of it was inactive (unused). 33 . While memory overcommit was set to 110 % of total memory there was still at least 5-8 GB of inactive memory that could be used.1) we can observe constant usage of CPU by system which is caused by KSM daemon scanning the memory. red .3. 7.1 CPU Figure 7.3.

Figure 7.3 KSM Effectiveness of KSM was monitored in two graphs.7.4: dark khaki . First one (7. but only shows how much memory was duplicit. U SE CASE 7. Figure 7.3. orange .shared pages 34 .3. green .unshared memory pages) On second graph (7.sharing of pages.3.3) we have amount of shared memory and amount of memory that these pages would take if they weren’t converted to copyon-write pages by KSM.3) shows amount of unique memory pages used by VMs compared to shared memory pages. Difference between these two values represents memory savings brought by KSM.3: KSM memory usage in bytes (light blue .shared memory pages. This graphs doesn’t show how much memory we saved using KSM.

8. users are apparently interested in our solution. We have also implemented VMS for users at FI MU hoping it will be useful and that it will unify process of implementing VMs at FI MU. As we showed in Use case chapter (7). that brings new features in every new version. Since the beginning of the application’s development many new and interesting ideas appeared and we would like to implement them in future releases. World of software moves very fast and so does the development of libvirt API. that can benefit from advantages of virtualization. that provides easy access for regular users. we will continue our work on finishing it and bringing it into final release as soon as possible.1 Future work While fadmin interface is still not in final state of its development. At some point we would like to supersede VMS of Windows administration seminar and migrate users to our interface.8 Conclusion We have provided an introduction to the world of virtualization on Linux systems aiming to show virtualization management software (VMS). We would also like to catch up with it and implement some of its new features. 35 .

Technical report. [2] Paul Menage. Ignacio M. Rev. http://fedoraproject. SIGOPS Oper. Germany. IEEE Internet Computing. 13:14–22. 2005. Books on Demand GmbH. Ruben S. September 2009. [4] Rusty Russell.Bibliography [1] Tim Fisher. Montero. Norderstedt. [3] Susanta Nanda and Tzi cker Chiueh. 2009. Kvm huge page backed memory. [6] Robert Warnke and Thomas Ritzau. qemu-kvm & libvirt.kernel. July 2008. Virtual infrastructure management in private and hybrid clouds. A survey of virtualization technologies. Llorente. [5] Borja Sotomayor.com/od/termsi/ g/isofile. http://www.about. Cgroups.org/doc/ Documentation/cgroups/cgroups. Syst. 2010. May 2011.. 42:95–103.txt. http://pcsupport. May 2011. 36 . Iso file. virtio: towards a de-facto standard for virtual i/o devices. and Ian Foster.org/wiki/Features/KVM_Huge_ Page_Backed_Memory.htm. [7] Chris Wright and John Cooper.

Screenshots Figure A.A Appendix .1: Disk manager page 37 .

2: Main page of application 38 .A.S CREENSHOTS Figure A. A PPENDIX .

A PPENDIX .A.3: Page with VM details and configuration 39 .S CREENSHOTS Figure A.