Eucalyptus: Setting up a private infrastructure cloud
There are a few Infrastructure-as-a-Service offerings that available to download and use. Eucalyptus and OpenNebula are two such offerings. I ended up installing and experimenting with both Eucalyptus and OpenNebula. In this blog post, I’ll detail my experience of installing and setting up Eucalyptus 1.6.2 on CentOS. For the sake of keeping things simple but still practical enough, we will have:

1 front-end machine. This will house the Cloud Controller (CLC) and Walrus. Since we intend to keep things fairly simple we will limit ourselves to a single cluster and setup the Cluster Controller (CC) and Storage Controller (SC) on this same machine. In my case, this machine has one network interface (NIC) with an IP address of

2 machines (Nodes) that will serve as hosts running Xen hypervisor for the virtual machines i.e. each machine will have a Node Controller (NC) installed. In my case, each machine has a single NIC and the IP addresses are and respectively.

Before we install Eucalyptus we need to first prep these machines.

Note: For the rest of this document, run the commands as root user.
This document is organized as below. Feel free to skip any sections if you have already implemented the steps in that section. Prepare the machines Download Eucalyptus Install Eucalyptus on the Front-end Install Eucalyptus on the Nodes Run Eucalyptus Register Eucalyptus components First-time Configuration Test your Eucalyptus install

Prep work On the front-end machine, we first install Java and Ant. You can download Sun JDK from here and Ant from here . I’m using JDK version 1.6u20 (jdk-6u20-linux-i586-rpm.bin) and Ant version 1.8.0 (apache-ant-1.8.0-bin.tar.gz).

ntpdate pool. 2. install it as follows: view plaincopy to clipboardprint? 1. 3.6. yum update 2. install Ant under /opt directory as follows: view plaincopy to clipboardprint? 1.0_19″ 1. 4.0 latest Next.8. . 2. 3.bin You can confirm that java is on the PATH by running the following command: view plaincopy to clipboardprint? 1. xen-libs. cd /opt mkdir ant cd ant tar zxvf ~/apache-ant-1.8. yum install dhcp xen-libs httpd bridge-utils ntp 3. sharing) Next. bridge-utils. we need to add an environment variable ANT_HOME that points to/opt/ant/latest and append the $ANT_HOME/bin to the PATH environment variable. yum list dhcp xen-libs httpd bridge-utils .ntp.6.Once you have downloaded Sun JDK to a directory./jdk-6u20-linux-i586-rpm.2-b04. java -version You should output similar to: java Java(TM) SE Runtime version Environment (build ―1. You can do this as follows: view plaincopy to clipboardprint? 1. 5. chmod +x I have the following versions installed: view plaincopy to clipboardprint? 1. mixed mode.0-bin.0_19-b04) Java HotSpot(TM) Client VM (build 16.tar. ntp) and synchronize the system clock on the front-end machine. httpd.bin 2. 4. cd /etc cp profile profile. Add this to the /etc/profile file as follows: view plaincopy to clipboardprint? 1.gz ln -s apache-ant-1.ORIG echo "export ANT_HOME=/opt/ant/latest" >> profile echo "export PATH=\$PATH:\$ANT_HOME/bin" >> profile Next we need to install a few dependencies (dhcp.

0.ipv4.5-21.3-94. sed -i "s/net.Loaded Loading * * * * Installed bridge-utils.i386 Available dhcp.3-31.3-105. cd /etc 2. cp config config. yum install xen ntp mirrors.4 3. run the following command: view plaincopy to clipboardprint? 1.el5 base mirror speeds plugins: from cached addons: base: extras: updates: 1.ubiquityservers. cp sysctl. sed -i "s/SELINUX=permissive/SELINUX=disabled/" config Let’s reboot the front-end machine at this point.ipv4. ntpdate pool. sysctl -p /etc/sysctl.i386 xen-libs.i386 httpd.2.i386 .ip_forward = 1/" sysctl. cd /etc/selinux 2.conf Next.0.3-43. let’s start with disabling SELinux on the front -end machine as follows: view plaincopy to clipboardprint? 1. we need to configure firewall rules to permit the various Eucalyptus communicate with each other.ecvps.1-2 12:3.i386 3.i386 dhcp. We start by installing xen hypervisor and also synchronize the system clock on each Node as follows: view plaincopy to clipboardprint? mirror.conf To change this value immediately without rebooting.ORIG 3.el5_4.el5. Since we are planning on using security groups in Eucalyptus. we need to prep the two Nodes.centos.1 2.el5_4.2.ip_forward = 0/net.3 Packages installed installed installed installed Packages base base We also allow the front-end machine to forward IP packets as follows: view plaincopy to clipboardprint? 1. yum update 2. Next.el5 2.conf.i386 xen-libs.5-23.ORIG 3.0.fdcservers.ntp.conf sysctl.centos fastestmirror hostfile mirror.

img NOTICE: have partition.1.conf) under/boot/grub.15.3-94.ubiquityservers. (hd0.arc. We do this by editing the GRUB configuration file (grub.georgetown.0)/grub/ hpc. If grub.sxp xend-config.sxp file and then restart xen daemon as follows: view plaincopy to clipboardprint? 1. then edit menu.conf of a symlink to grub.sxp sed -i "s/#(xend-address localhost)/(xend-address localhost)/" xend-config.xpm.6.1.lst which should be a file instead In my case.I have the following versions of xen installed: view plaincopy to clipboardprint? 1.el5_4.0) root=/dev/sda3 /initrd-version.conf is: # # # # # # # # #boot=/dev/sda default=1 timeout=5 splashimage=( mirror.el5xen) (hd0. 5.0.0.gz hiddenmenu title root kernel CentOS (2. cd /etc/xen cp Packages installed Packages Once we have Xen installed we need to configure it to allow for the hypervisor to be controlled via HTTP from localhost. 2.i386 Available xen.6.fastserv.3 mirror speeds addons: base: extras: updates: plugins: from cached fastestmirror hostfile mirror. grub.15.sxp /etc/init.gz-2. yum list xen Loaded Loading * * * * Installed xen.d/xend restart Next we need to make sure the correct kernel with xen enabled is started at mirror. this file that eg.ORIG sed -i "s/#(xend-http-server no)/(xend-http-server yes)/" xend-config.conf is not available.18-164. /boot/grub/grub.el5 kernel initrd Note all that you kernel do You and not have initrd to a root /vmlinuz-version ro rerun /boot paths are grub after making relative changes This to to means /boot/.i386 3. generated by anaconda . 4. 3.18-164.sxp. We can do this by editing /etc/xen/xend-config.3-105.0) /xen.el5 base 3.

cd /boot/grub 2.el5) (hd0.conf grub.18-164.6.15. we disable SELinux on the Node machines as follows: view plaincopy to clipboardprint? 1.el5.ORIG 3. The first title is 0. We can do this as follows: view plaincopy to clipboardprint? 1.el5xen.el5xen CentOS /vmlinuz-2.conf Next.2-centos-i386.6.el5.1.18-164. sed -i "s/SELINUX=permissive/SELINUX=disabled/" config Let’s reboot both Node machines at this point. Since we wanttitle CentOS (2. cp grub.el5) (hd0.6. sed -i "default=1/default=0/" grub.6. eucalyptus-1.15.6.ORIG 3.18164. Install Eucalyptus on the front-end Once you have downloaded Eucalyptus (in my case.18-164.1.0) ro root=LABEL=/ /initrd-2.gz file contains all Eucalyptus components though.img (2.15. The aforementioned tar.tar.1.15.conf.6.6.18-164.6. cd /etc/selinux 2. cp config config.18-164.15.gz) on the frontend.el5xen) to be the default kernel we would set default to 0.0) root=LABEL=/ initrd /initrd-2.img (2. Note: Different Eucalyptus components need to be installed on the front-end and each of the Node machines.6. Download Eucalyptus You could choose to install Eucalyptus via yum if needed which is easier that You can downloaded Eucalyptus from here.el5 ro ro root=LABEL=/ /initrd-2.1.img The default line is the line we want to change.18-164.el5 CentOS /vmlinuz-2. We are not ready to proceed with the installation of Eucalyptus. Therefore download it once on the front-end and then copy this file over to each of the Node machines. I picked the 32-bit CentOS 5 rpms that come bundled in a gzip compressed tar file.18- untar it to root’s home folder /root.1. view plaincopy to clipboardprint? .module module title root kernel initrd title root kernel /vmlinuz-2.18-164.

Cluster Controller. Next..6. the Eucalyptus components will start up fine when the time comes to run them.0 is needed…. and a few other dependencies on the front-end machine as follows: view plaincopy to clipboardprint? 1.i386.rpm 3..i386.i586.6. But we have installed Sun Java instead.gz cd eucalyptus-1. Note: The steps in this section need to be performed on each Node (in my case on each of the two Nodes).gz 2.i386.tar.i386. Adding –nodeps will get us past this error message.rpm eucalyptus-cc-1.rf.2-centos-i386.21.rpm groovy-1.gz file on each Node.i386. tar zxvf eucalyptus-1.2-centos-i386 We are ready to install. cd eucalyptus-1.i386. rpm -Uvh eucalyptus-1. cd eucalyptus-1.6.01.i386. Don’t worry.―.rpm Now let’s move on the installing Eucalyptus components on the Node.6.6.0. Walrus.rpm eucalyptus-common-java-1.rpm eucalyptus-cloud-1.i386.6.rpm euca-axis2c- Let’s start by installing the 3rd-party dependency RPMs included in the eucalyptus1.el4.2-centos-i386 cd eucalyptus-1.2-rpm-deps-i386 directory.el4.6.6. run the above rpm -Uvh… with –nodeps.i386.2-1.0-1.rpm eucalyptus-sc- vblade-14-1mdv2008.2-rpm-deps-i386 rpm -Uvh aoetools-21-1.6.21. Next. Note the above ―rpm -Uvh…‖ command might fail with an error about ―Failed dependencies…java-sdk > 1. 3.6. rpm -Uvh aoetools-21-1.el5. To get past this error.rpm vtun-3.6.el5.6.02-3. Install all the rpms in this directory as follows: view plaincopy to clipboardprint? 1.noarch. The error is because it is trying to look for Openjdk during installation.1.tar.2-1. Storage Controller.1.rpm euca-rampartc-1.6. let’s install the Cloud Controller.i386.rpm euca-axis2c-1. cd . cd .6. Install Eucalyptus on the Nodes First.rpm euca-rampartc-1.i386. copy (or download) the Eucalyptus tar.i386. 2.0-1.5-1. we install the Node Controller (and a couple of dependencies) on each Node as follows: view plaincopy to clipboardprint? .2-centos-i386. Let’s begin by installing a few 3rd-party dependency RPMs.rpm 5.rpm eucalyptus-gl1. tar zxvf eucalyptus-1.i386.2-1. view plaincopy to clipboardprint? 1.01.2-1.2-rpm-deps-i386 2.i386.rpm lzo2-2. 4. Untar it to root’s home folder /root.i386.3.rpm eucalyptus-walrus-1.

7 4.9 5. su eucalyptus -c "virsh list" The output of the above command should look something like: Id ———————————0 Domain-0 running Name State Note: If you don’t have libvirt installed/running on the Nodes.4 pidfile 500 500 30840 31612 0.2-1.i386.2-1.9 1137232 1103636 4824 40556 ? ? S S May13 May13 500 30587 6.i386.rpm eucalyptus-gl-1.1 4.conf //etc/eucalyptus/httpd-cc.3 0.conf //etc/eucalyptus/httpd-cc.rpm eucalyptus-nc1.3 0.d/eucalyptus-cloud start Run ps command to confirm Eucalyptus is running on the front-end: view plaincopy to clipboardprint? 1.3 //var/run/eucalyptus/eucalyptus-cloud.conf //etc/eucalyptus/httpd-cc.6.7 7.conf .rpm Next. you could install it: yum install libvirt That’s it with the installation! Running Eucalyptus You are now ready to start Eucalyptus up.conf //etc/eucalyptus/httpd-cc.6.3 937236 403344 ? Sl May13 569:24 eucalyptus-cloud –remote-dns –disable-iscsi -h / -u eucalyptus – //etc/eucalyptus/httpd-cc.conf //etc/eucalyptus/httpd-cc.0 0.2-1.d/eucalyptus-cc start 2.1 0. confirm that the user eucalyptus can connect with the hypervisor throughlibvirt.1.conf //etc/eucalyptus/httpd-cc. SSH to the front-end machine and start the Cluster Controller and Cloud Controller as follows: view plaincopy to clipboardprint? 1. ps auxww | grep euca root 500 500 500 500 500 30499 30500 30501 30502 30503 30504 0.5 39. /etc/init.0 9840 1103496 1103496 1136720 1103484 1103568 1480 48552 48704 75808 40232 51416 ? ? ? ? ? ? Ss S S R S S May13 May13 May13 May13 May13 May13 0:00 33:14 33:26 33:22 32:56 32:55 -f -f 12:53 32:50 /usr/sbin/httpd /usr/sbin/httpd -f -f /usr/sbin/httpd /usr/sbin/httpd /usr/sbin/httpd /usr/sbin/httpd /usr/sbin/httpd /usr/sbin/httpd -f -f -f -f -f -f -L -L //etc/eucalyptus/httpd-cc.3 0.0 0.3 0. view plaincopy to clipboardprint? 1.conf console-log console-log root 30586 0.i386.6.0 1852 224 ? Ss May13 0:00 eucalyptus-cloud –remote-dns –disable-iscsi -h / -u eucalyptus –pidfile //var/run/eucalyptus/ 0.3 3.3 0. /etc/init. rpm -Uvh eucalyptus-1.

euca_conf --register-walrus 192.500 31676 0. 192.0. 192. rosh-cluster1 – is the cluster name that I used.114) and run the following commands: view plaincopy to clipboardprint? 1.114 where. run the following command: view plaincopy to clipboardprint? 1.conf file on the front-end reflects the node IP addresses added via the above euca_conf – register-nodes command. Replace the above IP addresses with the IP addresses of your Nodes.3 4.3 9856 1488 ? Ss May13 0:00 /usr/sbin/httpd -f //etc/eucalyptus/httpd-nc.0. Add additional Nodes separated with a space.168.168.conf 500 20639 0. On the front-end machine.168.168.168. Walrus.conf Next.conf 500 31678 0.d/eucalyptus-nc start Confirm eucalyptus is running on the Nodes: view plaincopy to clipboardprint? 1. In my case: . ps auxww | grep euca root 20637 0. 3.0. you will need to register them so that they can talk to each other. euca_conf --register-nodes "192.114 – is the IP address of my front-end machine which has CLC. Replace it with your own cluster name.3 3.conf Registering Eucalyptus components Now that you have started all components. euca_conf --register-cluster rosh-cluster1 192. /etc/init.168.0.0. CC and SC installed/running.5.168.7 – are the 2 Nodes in my case.0 0.19 192. SSH to each Node and start Node Controller as follows: view plaincopy to clipboardprint? 1.0. euca_conf --register-sc rosh-cluster1 192.114 2. Next. 192. Replace this with the IP address of your front-end machine in all the above commands.7 1103636 48628 ? S May13 32:54 /usr/sbin/httpd -f //etc/eucalyptus/httpd-cc.9 10.2 80688 50904 ? Sl May13 78:15 /usr/sbin/httpd -f //etc/eucalyptus/httpd-nc. SSH to the front-end machine (in my case.5.7" where.19.9 1103568 40400 ? S May13 33:09 /usr/sbin/httpd -f //etc/eucalyptus/httpd-cc.0. You can verify that the nodes are registered by verifying that value of the NODESelement in the eucalyptus. we need to register the 2 Nodes.

114:8773/services/Eucalyptus Before you run any client‖ to a directory. The EC2_URL in this case will point to your front-end machine.168.168. Run the following command from under . .euca folder and unzip the contents of this file in this folder.eucarc file that exports some variables.0.0. browse to https://<front-end-ip-address>:8443. Using a browser.114:8443. I left these unchanged for now. grep NODES /etc/eucalyptus/eucalyptus. set the admin email address. you will be asked to change the password. You will get a warning page stating that the ―site’s security certificate is not trusted―. Clusters.168. On the ―Configuration‖ web page you will see Cloud Configuration. Save the ―euca2 -adminx509.view plaincopy to clipboardprint? 1. browse to ―Credentials‖ web page and click the ―Download Credentials‖ zip file.eucarc EUCA_KEY_DIR=$(dirname export export … … $(readlink -f ${BASH_SOURCE})) S3_URL=http://192. Enter the relevant details and hit ―Submit‖.7″ We are done with registering the Eucalyptus components. Next.114.168. Accept the certificate and you will be prompted for a user_id/password. These should all be pre-populated. Since Eucalyptus is using a selfsigned certificate which is not verified by a third-party that the browser trusts.168.168. In my case. etc. Create a . Once you have logged in for the first time. Enter admin for both. You will need these credentials when you use client tools such as euca2ools to manage virtual machines. you will need to source this file.euca folder: view plaincopy to clipboardprint? 1. shows you this warning. you will find a . https://192. unzip euca2-admin-x509. You could make changes to the configurations if you wish. etc. First-time Configuration We are now ready to perform some quick configuration.conf NODES=‖ 192. cat . Walrus Configuration. view plaincopy to clipboardprint? 1. images.19 192.114:8773/services/Walrus EC2_URL= Once you have unzipped the contents. 192. etc.0. In my case.0.

114 ram 128 256 512 1024 disk 2 5 10 20 AVAILABILITYZONE |.xlarge 0000 / 0000 4 2048 20 where. 3. Unzip the downloaded ec2-api-tools.eucarc cd ec2-api-tools-1. give yourself a pat on the back! You are now ready to bundle images and create instances from those images on your own private infrastructure cloud! .large m1.114) If you see something like the Next source the . unzip ec2-api-tools.168. view plaincopy to clipboardprint? 1.Testing our Eucalyptus install To keep things simple and quickly test our Eucalyptus installation.res.eucarc file under and run the ec2-describe-availability-zones command provided by the ec2-api-tools.euca folder that you created in the ―First-time Configuration‖ section. it corresponds to the Cluster Controller running on my front-end machine (192. downloadAmazon EC2 API Tools.euca folder run the following commands: view plaincopy to clipboardprint? 1.euca source .0.apache.xml. cd .168.xlarge Xalan: rosh-cluster1 free 0004 0004 0002 0000 / / / / / max 0004 0004 0002 0000 2 2 cpu 1 1 org. From under .3-46266/bin ec2-describe-availability-zones verbose You should see output similar to the following: [Deprecated] AVAILABILITYZONE AVAILABILITYZONE AVAILABILITYZONE AVAILABILITYZONE AVAILABILITYZONE AVAILABILITYZONE |||||vm types m1.0. 4. rosh-cluster1 – is the cluster I registered using euca_conf and in my to under the .small c1.XMLErrorResources_en_US 192.medium m1. 2.

Sign up to vote on this title
UsefulNot useful