You are on page 1of 11


What is Private Cloud?

A System in which Infrastructure is dedicatedly allocated to an individual or an enterprise and provides cloud based services: Dynamic Provision, Elastic Resources, Chargeback and Automation. A Private cloud provides shared hosting environment for multiple applications and environments owned or managed by an Enterprise or individual. Its a preferred platform for most enterprises who fear of security, compliance regulations and ownership of data problems with Public cloud and yet need the elastic power of computing to be available ondemand.

What is open source technology?

Open source software and standards are not just beneficial, but highly essential for a heterogeneous, shared and scalable environment such as the cloud. open source software (which indicates the source code software is open to all) or free software (which means it is free for users to run, copy, distribute, study, change and improve) which have specific definitions, cloud computing is a more loosely defined term that can refer either to infrastructure, services or applications.

Open source has built the cloud

Sharing, the freedom to mix and match, choice, and many other characteristics of cloud computing demonstrate a likeness to the principles of open source software. More than just the similarities, there is also a sheer necessity for open source software and open standards in clouds that comprise heterogeneous, and often proprietary, infrastructure. The availability of source code, the freedom to modify and redistribute, the flexibility and constant evolution, and other open philosophies greatly favour the cloud schema of things.

Why built private cloud using open source technology?

1. Open source lies at the foundation of many of the earliest cloud implementations. 2. When we think of the services we consume on the cloud, from Facebook and Google to Amazon, none would have been affordable or scalable using a traditional licensing model. 3. The fact that most of the public clouds, including Amazons Elastic Compute Cloud (EC2), run on Linux-based platforms is just one facet of the story. In fact, that is now taken so much for granted that the focus has shifted to tools and platforms that enable the building of private or hybrid clouds, the integration of legacy infrastructure with the cloud. 4. A rather large number of such open source tools are now available, right from platforms and development tools, to management dashboards and automated migration tools for applications. Existing open source platforms are also fast adapting to the needs of cloud computing and include features such as intelligent workload management and cloud-enabled scalability considerations, to help massive horizontal scalability at all the layers of the technology stack.

Softwares and tools available to implement our project..

Large choice of open source applications for the cloud includes Red Hat, Traffic Server, Puppet, Zoho, Cloudera, Enomaly and Joyent. Among these Ubuntu, Red Hat, and almost every other open source platform now has a stable cloud offering. Eucalyptus is a modular platform that is capable of working with a variety of interfaces, including Amazons EC2 and Simple Storage Service (S3) services. Eucalyptus works with various distros, including Red Hat Enterprise Linux (RHEL), CentOS, SUSE Linux Enterprise Server (SLES), OpenSUSE, Debian and Fedora. It can also host MS Windows images. It is capable of working with many virtualisation technologies such as VMware, Xen and KVM hypervisors, in order to implement the abstraction demanded by a cloud environment. Though, Ubuntu prefers KVM for the visualization support and has added to the default packages. But limitation of KVM to run only virtualization extensions (Intel VT or AMD-V) processors. So XEN is most preferable for virtualization. XEN is the most popular Open Source Virtualization software that allows multiple OS to run on the same computer hardware concurrently, thereby improving the effective usage and efficiency of the underlying hardware. It benefits the enterprises with the power of consolidation, increased utilization and rapid provisioning.

Setting up Eucalyptus Cloud on Xen :In any Eucalyptus Cloud Installation, there are 2 top-level components: Cloud Controller (CLC) and Walrus. These 2 components manage the various clusters, where cluster is a set of physical machines that host the Virtual Instances. In each cluster, there are 2 components that interact with the high level components: Cluster Controller (CC) and Storage Controller (SC). CC and SC are cluster level components. Each cluster is composed of various Nodes, or physical machines. Each Node will run a Node Controller (NC) that will control the hypervisor for managing the Virtual Instances. For this setup, we have implemented a Single-Cluster Installation, where all the components except NC are colocated on one machine. As per Eucalyptus documentation, this co-located system is called: front-end. So in a gist, we have 1 physical machine which hosts CLC, Walrus, CC, SC, and 5 other machines that hosts NC each. The Node Controller uses Xen as a hypervisor. The NC service runs on Domain-0 kernel in the Xen Setup.

Hardwares and softwares required.. Hardware:1 Admin Machine with the config : Intel Core 2 Duo Processor 1.8 GHz, with 1GB RAM,160 GB HDD, and 5 Node Controllers each with the config : Intel Core 2 Duo Processor 1.8 GHz (VT enabled) with 2 GB RAM, 160 GB HDD.

Software:Eucalyptus 1.6.1 running on Ubuntu 9.04 (Jaunty), Xen 3.3 hypervisor. Eucalyptus has 3 major packages. Cluster Controller: provides support for the virtual network overlay Cloud Controller: includes the front-end services and the Walrus storage system. Node Controller: that interacts with XEN to manage individual VMs. The basic architecture how client, cloud controller, cluster controller and node controllers communicate with each other in a eucalyptus setup. > eucalyptus-nc | | Client > eucalyptus-cloud > eucalyptus-cc > eucalyptus-nc | | > eucalyptus-nc Packages provided by Ubuntu are: eucalyptus-cloud, eucalyptus-cc and eucalyptus-nc.

Implementation method..

Front end setup:

Front end runs both eucalyptus-cloud and eucalyptus-cc. So I install both packages on this machine. Follow the steps for installing the front end:

Step 1: Prepare a raw Ubuntu OS system.

Install Ubuntu 9.04 on a machine. Select SSH server to be installed on the machine by default, so that we can connect to machine remotely.

Step 2: Install postfix.

Postfix is a mail server. Eucalyptus send mails to users that are added by admin on eucalyptus web interface. So we install postfix that eucalyptus will use as mail server. Steps to install postfix and to setup configurations and testing successful Postfix installation command# apt-get install postfix Set configuration command# sudo dpkg-reconfigure postfix Set following parameters with given values. General type of mail configuration: Internet Site NONE doesnt appear to be requested in current config System mail name: Root and postmaster mail recipient: <admin_user_name> (Optional) Other destinations for mail:,,, localhost Force synchronous updates on mail queue?: No Local networks: Yes doesnt appear to be requested in current config Mialbox size limit (bytes): 0 Local address extension character: + Internet protocols to use: all Test postfix installation command# EHLO localhost mail from: root@localhost rcpt to: data subject: your sub (press enter twice) Type your message and finish with . then Enter Finish with Quit

Check your mail (if received your mail is working) Step 3: Install Cloud and Cluster controller Install the packages provided by Ubuntu, all the dependencies will be installed automatically by package manager. command# apt-get install eucalyptus-cloud eucalyptus-cc

Step 4: Install DHCP server

Install the DHCP server which is used by eucalyptus for assigning the IPs to instances. DHCP server should not be in running state and set the DHCP server not to run when system is booted up again. command# apt-get install dhcp3-server command# /etc/init.d/dhcdbd stop command# update-rc.d -f dhcdbd remove Set the DHCP server path in the eucalyptus config file /etc/eucalyptus/eucalyptus.conf VNET_DHCPDAEMON=/usr/sbin/dhcpd3

Step 5: Eucalyptus configuration

Set the static mode for eucalyptus so that eucalyptus uses IPs from the pool of IPs provided in the config file and use the machine addresses provided along with IPs. Configuration file for eucalyptus is /etc/eucalyptus/eucalyptus.conf open /etc/eucalyptus/eucalyptus.conf and set the following parameters VNET_INTERFACE=peth0 VNET_BRIDGE=eth0 Locate kvm and replace to xen locate VNET_MODE=SYSTEM and disable it locate VNET_MODE=STATIC and enable it Add machine and IPs for instances to be used and change the netowork setting to current network settings VNET_MODE=STATIC VNET_SUBNET= VNET_NETMASK= VNET_BROADCAST= VNET_ROUTER= VNET_DNS= VNET_MACMAP=AA:DD:11:CE:FF:ED= AA:DD:11:CE:FF:EE= AA:DD:11:CE:FF:EF=

Step 6: Other configurations

Edit : /usr/share/eucalyptus/gen_libvirt_xml Locate all sda to xvda Edit : /etc/xen/xend-config.sxp (xend-http-server yes) gateway = netmask = broadcast =

Step 7 Reboot the system to take effect Step 8 Cloud configuarions

Once eucalyptus front end has been installed, we can use the eucalyptus web interface and setup cluster and set other parameters. Eucalyptus cloud web URL: https://<front end ip>:8443/ Under configuration tab add cluster name Set host to localhost Create kernel and ramdisk images and set in eucalyptus site. (Procedure to create images will be described later in the page) Restart the eucalyptus daemon to take effects

Back end setup

On the systems where we want to run instances, we will install node controller and XEN 3.0. Since I used the XEN as virtualization package, I install the node controller package eucalyptus-ncmanually. I installed required dependencies using the package manager apt-get, but the eucalyptus-nc was installed by donwloading and manually installing bypassing dependencies list. Step 1: Hardware compatibility eucalyptus-nc requires libvirt-bin package. Libvirt is a C toolkit to interact with the virtualization capabilities of recent versions of Linux (and other OSes). More info can be found at: To install libvirt-bin package, certain flags should be supported by CPU. command# egrep (vmx|svm) /proc/cpuinfo [if no print], libvirt-bin cant be installed Stop the apparmor for Ubuntu System command# /etc/init.d/apparmor stop command# update-rc.d -f apparmor remove

Step 2: Install XEN package Use the XEN virtualization package instead of Ubuntus default KVM, I install the eucalyptus-nc dependencies manually. command# apt-get install iproute iptables module-init-tools python2.5 python2.6 command# apt-get install xen-utils command# apt-get install ubuntu-xen-server Step 3: Install libvirt-bin package eucalyptus-nc requires libvirt-bin package. Libvirt is a C toolkit to interact with the virtualization capabilities of recent versions of Linux (and other OSes). More info can be found at: command# apt-get install adduser bridge-utils dnsmasq-base iptables libsasl2-2 libxen3 logrotate command# apt-get install netcat-openbsd libavahi-client3 libavahi-common3 libc6 libdbus-1-3 command# apt-get install libgcrypt11 libgnutls26 libhal1 libpolkit-dbus2 libpolkit2 libreadline5 libselinux1 command# apt-get install libtasn1-3 libvirt0 libxml2 zlib1g policykit Download the libvirt-bin package from URL: I used i386 package, Install lib-virt command# dpkg -i libvirt-bin_0.6.1-0ubuntu5_i386.deb command# sudo adduser $USER libvirtd Step 4: Install eucalyptus node package command# apt-get install apache2 eucalyptus-common eucalyptus-gl libapache2-modaxis2c command# apt-get install libaxis2c0 librampart0 dhcp3-server vlan aoetools command# apt-get install libc6 libcurl3-gnutls libvirt0 zlib1g Download eucalyptus-nc package from Ubuntu package repository. eucalyptus-nc package URL: Download the i386 package from URL: command# dpkg -i eucalyptus<version> Step 5: Grub loader for Xen Kernel Setup grub menus and load xen images to boot into XEN kernel. Download the modules and image from URLs: wget

wget command# sudo dpkg -i linux-modules-2.6.26-2-xen-686_2.6.26-15_i386.deb command# sudo dpkg -i linux-image-2.6.26-2-xen-686_2.6.26-15_i386.deb Step 6: Eucalyptus Configuerations Stop the eucalyptus daemon and set the configuration depending upon your installation. command# /etc/init.d/eucalytpus-nc stop Edit : /etc/sysctl.conf (uncomment net.ipv4.ip_forward=1) Edit : /etc/eucalyptus/eucalyptus.conf (set) VNET_BRIDGE=eth0 VNET_INTERFACE=peth0 HYPERVISOR=xen VNET_MODE=STATIC Edit : /usr/share/eucalyptus/gen_libvirt_xml locate sda s and replace to xvda s for static ips Edit : /etc/xen-tools/xen-tools.conf gateway broadcast netmask (xend-http-server yes) (xend-unix-server yes) command# /etc/init.d/eucalyptus-nc start

Cloud integration
Now, we have front controller and node controllers ready. We have to setup now the communication between cluster controller and node controllers. We will add the nodes list to the cloud, so cloud know about the available node. command# sudo euca_conf -addnode <node ip> <node ip> <node ip> This will add nodes to cloud config file and replicates the permission and certificate files to the nodes to be able to have password-less communication between nodes and cloud. To check the cloud installation, and node communication, you can run euca-describeavailability-zones verbose command and check if all the nodes are listed in output.

ec2 or euca2ools setup

Euca2ools setup Install dependencies command# apt-get install curl libopenssl-ruby ruby Download the appropriate tarball from

Steps command# tar zxvf euca2ools-1.0-*.tar.gz command# cd euca2ools-1.0-* (Replace this with the actual directory name) command# sudo -s command# echo deb file://${PWD} ./ >> /etc/apt/sources.list command# apt-get update command# apt-get install euca2ools WARNING: The following packages cannot be authenticated! Install these packages without verification [y/N]? y ec2 tools Setup Eucalyptus supports ec2-ami-tools-1.3-26357 and ec2-api-tools-1.3-30349 versions of ec2 tools. Install dependencies Install jdk1.6.0_12 java version. Steps Download ec2-ami-tools-1.3-26357 and ec2-api-tools-1.3-30349. unzip these archives Copy the lib and bin folders under ~/.euca folder. Export parameters export EC2_HOME=~/.euca export JAVA_HOME=<directory>/jdk1.6.0_12 export PATH=$PATH:<directory>/jdk1.6.0_12/bin export PATH=$PATH:~/.euca/bin Set URL and credentials Download file from cloud site at https://<front end ip>:8443/.

Unzip this archive under ~/.euca folder and export the eucarc file. command# source ~/.eucarc

Create guest machine

I create my image using vmbuilder. More info on vmbuilder can on: be found

Steps Select a directory where all files will be stored. I used /usr/akash/cloud Create a part file under directory. The content of part is following: root 400 /mnt/ephemeral 0 /dev/sda2 swap 1 /dev/sda3 Create firstboot file, which is run first time, instance boots up. apt-get -y install openssh-server Create the image. command# vmbuilder xen ubuntu firstboot ./firstboot mirror= addpkg ssh rootsize=4096 arch i386 verbose debug It creates the image file that i used for creating emi. You can found the image under ubuntu-xen folder. This image can be tested, if its compatible to xen. Use xm create command. command# xm create /usr/akash/cloud/ubuntu-xen/xen.cfg Check the status of emi using xm list command. command# xm list If you see your emi ID in the output, your image can be run with XEN.

Creating EMIs
Now, cloud is setup, we should create images that can be run using eucalyptus. We will create emis for kernel, ramdisk and instance. kernel emi Steps
command# mkdir kernel command# ec2-bundle-image -i vmlinuz-2.6.28-13-server -d ./kernel kernel true command# ec2-upload-bundle -b eucaKernel -m ./kernel/vmlinuz-2.6.28-13-server.manifest.xml command# EKI=`ec2-register eucaKernel/vmlinuz-2.6.28-13-server.manifest.xml | awk {print $2}` command# echo $EKI

ramdisk emi
Steps command# mkdir ramdisk command# ec2-bundle-image -i initrd.img-2.6.28-13-server -d ./ramdisk ramdisk true command# ec2-upload-bundle -b eucaRamdisk -m ./ramdisk/initrd.img-2.6.28-13server.manifest.xml command# ERI=`ec2-register eucaRamdisk/initrd.img-2.6.28-13-server.manifest.xml | awk {print $2}` command# echo $ERI

instance emi
Steps command# mkdir image command# ec2-bundle-image -i root.img -d ./image command# ec2-upload-bundle -b plabImage -m ./image/root.img.manifest.xml command# EMI=`ec2-register plabImage/root.img.manifest.xml | awk {print $2}` command# echo $EMI

Run instance
Add key pair command# ec2-add-keypair mykey > ~/.euca/mykey.priv command# chmod 0600 ~/.euca/mykey.priv command# ec2-run-instances emi-73FD0EAD -k mykey kernel eki-8D701398 ramdisk eriE47F14E8 instance-type m1.xlarge