Server Virtualization Explained

You have probably heard about lots of distinct types of server virtualization; full, bare metal, para-virtualization, guest OS, OS assisted,
hardware assisted, hosted, OS level, kernel level, shared kernel, hardware emulation, hardware virtualization, hypervisor based, containers or
native virtualization. Confusing, right?

Fear not my faithful readers; the whole purpose of this blog is exactly to explain these things so that everyone can have a clear view over
issues usually restricted to a bunch of geeks. But keep in mind that some of these terms are popularized by certain vendors and do not have
a common industry-wide acceptance. Plus, many of the terms are used rather loosely and interchangeably (which is why they are so
confusing).

Although others classify the current virtualization techniques in a different way, I will use the following criteria:

1. Full Virtualization;
2. Para-Virtualization;
3. Operating System-level Virtualization;
4. Hardware assisted virtualization.

On the following exciting chapters I will explain these techniques, one by one, but before that I believe it would be useful to give you a quick
introduction to some underlying concepts.

Server Virtualization History

Virtualization was first developed in the 1960s to partition large mainframes for better hardware utilization. In 1964, IBM developed a Virtual
Machine Monitor to run their various operating systems on their mainframes providing a way to logically partition these big computers into
separate virtual machines. These partitions allowed mainframes to multitask; run multiple applications and processes at the same time.
Hardware was too expensive to leave underutilized so it was designed for partitioning as a way to fully leverage the investment.

However with the advent of cheap commodity hardware, virtualization was no longer popular and was viewed as a relic of an era where
computing resources were scarce. This was reflected in design of x86 architectures which no longer provided enough support to implement
virtualization efficiently. Virtualization was effectively abandoned during the 1980s and 1990s when client-server applications and inexpensive
x86 servers and desktops led to distributed computing. The broad adoption of Windows and the emergence of Linux as server operating
systems in the 1990s established x86 servers as the industry standard.

With the cost of hardware going down and complexities of software increasing, a large number of administrators started putting one
application per server. This provided them isolation, where one application did not interfere with any other applications. However, over some
time this IT management attitude started resulting into a problem called server sprawl. There were too many servers with average utilization
between 5% and 15%. In addition to the cost of the hardware, there are also power and cooling requirements for all these servers and this
caused the old problem of utilization of hardware resources to surface again.

The growth in x86 server and desktop deployments led to new IT infrastructure and operational challenges, namely:

• Low Infrastructure Utilization;

• Increasing Physical Infrastructure Costs;
• Increasing IT Management Costs;
• Insufficient Failover and Disaster Protection;

Virtualization Terminology

Guests and hosts

A virtual machine (VM) typically has two components: the host server and the guest virtual machine. The host server is the underlying
hardware that provides computing resources, such as processing power, memory, disk and network I/O, and so on. The guest virtual machine
is a completely separate and independent instance of an operating system and application software. Guests are the virtual workloads that
reside on a host server and share that server's computing resources.

Virtual Machine Monitor

System virtual machines are capable of virtualizing a full set of hardware resources, including a processor (or processors), memory and
storage resources and peripheral devices. A Virtual Machine Monitor (VMM, also called a hypervisor) is the piece of software that provides
the abstraction of a virtual machine. There are three properties of interest when analyzing the environment created by a VMM:

• Fidelity/Equivalence;
• Safety/Resource Control;
• Performance/Efficiency
In the terminology of Popek and Goldberg, a VMM must present all three properties.

Popek and Goldberg Criteria

The Popek and Goldberg criteria are a set of conditions sufficient for a computer architecture to support system virtualization efficiently. They
were introduced by Gerald Popek and Robert Goldberg in their 1974 article "Formal Requirements for Virtualizable Third Generation
Architectures". The paper establishes three essential characteristics for system software to be considered a VMM:

• Equivalence/Fidelity: The software running under the VMM should exhibit a behavior essentially identical to that demonstrated
when running directly on equivalent hardware, barring timing effects;

• Efficiency/Performance: A vast majority of machine instructions must be executed by the hardware without VMM intervention;

• Resource Control/Safety: The VMM must be in complete control of the virtualized resources.

Even though these requirements are derived under simplifying assumptions, they still represent a convenient way of determining whether a
computer architecture supports efficient virtualization and provide guidelines for the design of virtualized computer architectures.

CPU Protection Levels

The x86 architecture offers a range of protection levels, also known as rings, in which code can execute and which operating systems and
applications use to manage access to the computer hardware. Ring 0 has the highest level privilege and it is in this ring that the operating
system kernel normally runs. Code executing in ring 0 is said to be running in system space, kernel mode or supervisor mode. All other code
such as applications running on the operating system operates in less privileged rings, typically ring 3.

Keep in mind that the CPU privilege level has nothing to do with operating system users. Whether you’re root, Administrator, guest, or a
regular user, it does not matter. All user code runs in ring 3 and all kernel code runs in ring 0, regardless of the OS user on whose behalf the
code operates.
The term "ring" (as it applies to x86 architecture machines and Windows) appears to refer to the original 80386 architecture reference
manual's drawing of the four levels of protection - as concentric circles of operations. Thus, Ring 3 was the outermost ring and provided the
most protection, allowing only the execution of instructions that could not affect overall processor state. Ring 0 was the innermost ring and
allowed total control of the processor.

Problems in x86 Virtualization

Unlike mainframes, the x86 architecture was not designed to support full virtualization because in these operating systems, designed to run
directly on the bare-metal hardware, certain sensitive instructions expect the OS to be directly interacting with the hardware, making it
difficult to realize ‘true’ virtualization (as defined by the Popek and Goldberg criteria).

Virtualizing the x86 architecture requires placing a virtualization layer under the operating system (which expects to be in the most privileged
Ring 0) to create and manage the virtual machines that deliver shared resources. Further complicating the situation, some sensitive
instructions can’t effectively be virtualized as they have different semantics when they are not executed in Ring 0. The difficulty in trapping
and translating these sensitive and privileged instruction requests at runtime was the challenge that originally made x86 architecture
virtualization look impossible.

VMware was the first company to find a way around this obstacle, by way of a method called the Binary Translation in which the privileged
instructions were trapped by a software interpreter layer known as the Virtual Machine Monitor (VMM), and converted to ‘safe’ instructions
that could be virtualized.

In 1998, VMware figured out how to virtualize the x86 platform, developing binary translation techniques that allow the VMM to run in Ring 0
for isolation and performance, while moving the operating system to a user level ring with greater privilege than applications in Ring 3 but
less privilege than the virtual machine monitor in Ring 0. This combination of binary translation and direct execution on the processor allows
multiple guest OSes to run in full isolation on the same computer.

Fuente: http://networksandservers.blogspot.com.ar/2011/11/server-virtualization-explained.html
Full Virtualization Explained

This is probably the most common and most easily explained kind of server virtualization. When IT departments were struggling to get
results with machines at full capacity, it made sense to assign one physical server to every IT function taking advantage of cheap hardware A
typical enterprise would have one box for SQL, one for the Apache server and another physical box for the Exchange server. Now, each of
those machines could be using only 5% of its full processing potential. This is where hardware emulators come into play in an effort to
consolidate those servers.

A hardware emulator presents a simulated hardware interface to guest operating systems. In hardware emulation, the virtualization software
(usually referred to as a hypervisor) actually creates an artificial hardware device with everything it needs to run an operating system and
presents an emulated hardware environment that guest operating systems operate upon. This emulated hardware environment is typically
referred to as a Virtual Machine Monitor or VMM.

Hardware emulation supports actual guest operating systems; the applications running in each guest operating system are running in truly
isolated operating environments. This way, we can have multiple servers running on a single box, each completely independent of the other.
The VMM provides the guest OS with a complete emulation of the underlying hardware and for this reason, this kind of virtualization is also
referred to as Full Virtualization.

Thus, full virtualization provides a complete simulation of the underlying hardware and is a technique used to provide support for unmodified
guest operating systems. This means that all software, operating systems and applications, which can run natively on the hardware can also
be run in the virtual machine.

The term unmodified refers to operating system kernels which have not been altered to run on a hypervisor and therefore still execute
privileged operations as though running in ring 0 of the CPU. Full virtualization uses the hypervisor to coordinate the CPU of the server and
the host machine's system resources in order to manage the running of guest operating systems without any modification. In this scenario,
the hypervisor provides CPU emulation to handle and modify privileged and protected CPU operations made by unmodified guest operating
system kernels.

The guest operating system makes system calls to the emulated hardware. These calls, which would actually interact with underlying
hardware, are intercepted by the virtualization hypervisor which maps them onto the real underlying hardware. The hypervisor provides
complete independence and autonomy of each virtual server to other virtual servers running on the same physical machine. Each guest server
has its own operating system and it may even happen that one guest server is running Linux and the other is running Windows.

The hypervisor also monitors and controls the physical server resources, allocating what is needed to each operating system and making sure
that the guest operating systems (the virtual machines) cannot disrupt each other. With full virtualization, the guest OS is not aware it is being
virtualized and thus requires no modification.

Type 1 Hypervisor

Also known as Native or Bare-Metal Virtualization, this is a technique where the abstraction layer sits directly on the hardware and all the
other blocks reside on top of it. The Type 1 hypervisor runs directly on the hardware of the host system in ring 0. The task of this hypervisor is
to handle resource and memory allocation for the virtual machines in addition to providing interfaces for higher level administration and
monitoring tools.The operating systems run on another level above the hypervisor.

Clearly, with the hypervisor occupying ring 0 of the CPU, the kernels for any guest operating systems running on the system must run in less
privileged CPU rings. The Type 1 Hypervisor contains functionalities like CPU scheduling and Memory Management and, even though there is
no Host OS, usually one of the Virtual Machines has certain privileged status (Control/Admin/Parent VM).

Unfortunately, most operating system kernels are written explicitly to run in ring 0 for the simple reason that they need to perform tasks that
are only available in that ring, such as the ability to execute privileged CPU instructions and directly manipulate memory.
Also, depending on the architecture, the hypervisor may either contain the drivers for the hardware resources (referred to as a Monolithic
Hypervisor) or the drivers may be retained at the Guest OS level itself (in which case it can be called a Microkernelized Hypervisor).

Since it has low level direct access to the hardware, a Type 1 hypervisor is more efficient than a hosted application and delivers greater
performance because it uses fewer resources (no separate CPU cycles or memory footprint as in the case of a full-fledged Host OS).

The disadvantage of this model is that there is dependency on the hypervisor for the drivers (at least in case of the Monolithic Hypervisor).
Besides, most implementations of the bare-metal approach require specific virtualization support at the hardware level (“Hardware-assisted”,
to be discussed in a future post).

Examples of this architecture are Microsoft Hyper-V and VMware ESX Server.

Note: Microsoft Hyper-V (released in June 2008) exemplifies a Type 1 product that can be mistaken for a Type 2. Both the free stand-alone
version and the version that is part of the commercial Windows Server 2008 product use a virtualized Windows Server 2008 parent partition
to manage the Type 1 Hyper-V hypervisor. In both cases the Hyper-V hypervisor loads prior to the management operating system, and any
virtual environments created run directly on the hypervisor, not via the management operating system.

Type 2 Hypervisor

Guest OS Virtualization is perhaps the easiest concept to understand. In this scenario the physical host computer system runs a standard
unmodified operating system such as Windows, Linux, Unix or MacOS X, and the virtualization layer runs on top of that OS being in effect a
hosted application. In this architecture, the VMM provides each virtual machine with all the services of the physical system, including a virtual
BIOS, virtual devices and virtual memory. This has the effect of making the guest system think it is running directly on the system hardware,
rather than in a virtual machine within an application.

The OS itself provides the abstraction layer (known as Type 2 Hypervisor), such that it allows other OSes to reside within; thus creating virtual
machines. This architecture then can be called as Hosted Virtualization (since an OS is ‘hosting’ it), as depicted below.
The virtualization layer in the above figure contains the software needed for hosting and managing the virtual machines but its exact
functionality can vary based on the different architectures from different vendors. In this approach, the Guest OS relies on the underlying
Host OS for access to the hardware resources.

The multiple layers of abstraction between the guest operating systems and the underlying host hardware are not conducive to high levels of
virtual machine performance. This technique does, however, have the advantage that no changes are necessary to either host or guest
operating systems and no special CPU hardware virtualization support is required.

Examples of a Type 2 Hypervisor are Oracle's VirtualBox, VMware Server, VMware Workstation and Microsoft Virtual PC.

Embedded Hypervisor

In Kernel Level Virtualization the host operating system runs on a specially modified kernel which contains extensions designed to manage
and control multiple virtual machines each containing a guest operating system.

The virtualization layer is embedded into an operating system kernel and each guest runs its own kernel, although restrictions apply in that
the guest operating systems must have been compiled for the same hardware as the kernel in which they are running.
The real benefit with this approach is that the hypervisor code is dramatically leaner than either Type 1 or Type 2. With the hypervisor
embedded into the Linux kernel, the guest operating systems benefit from excellent disk and network I/O performance.

Examples of kernel level virtualization technologies include User Mode Linux (UML) and Kernel-based Virtual Machine (KVM).

Full Virtualization Advantages

• This approach to virtualization means that applications run in a truly isolated guest OS, with one or more of these guest OSs
running simultaneously on the same hardware. Not only does this method support multiple OSes, it can support dissimilar OSes,
differing in minor ways (for example, version and patch level) or in major ways (for example, completely different OSes like
Windows and Linux);

• The guest OS is not aware it is being virtualized and requires no modification. Full virtualization is the only option that requires no
hardware assist or operating system assist to virtualize sensitive and privileged instructions. The hypervisor translates all operating
system instructions on the fly and caches the results for future use, while user level instructions run unmodified at native speed;

• The VMM provides a standardized hardware environment that the guest OS resides on and interacts with. Because the guest OS
and the VMM form a consistent package, that package can be migrated from one machine to another, even though the physical
machines the packages run upon may differ;

• Full virtualization offers the best isolation and security for virtual machines, and simplifies migration and portability as the same
guest OS instance can run virtualized or on native hardware.

Full Virtualization Limitations

• The virtualization software hurts performance, which is to say that applications often run somewhat slower on virtualized systems
than if they were run on unvirtualized systems. The hypervisor needs data processing, which means that part of the computing
power of a physical server and related resources should be reserved for the hypervisor program. While the VMMs appears to solve
the entire problem with regard to virtualized machines, it does bring in some level of performance degradation, caused by the
extra processing (in terms of the ‘instruction translation’) that the hypervisor has to do. This can have a negative impact on overall
server performance and slow down the applications;

• The hypervisor must contain the interfaces to the resources of the machine; these interfaces are referred to as device drivers.
Because hardware emulation uses software to trick the guest OS into communicating with simulated non-existent hardware, this
approach has created some driver compatibility problems. The issue is that the hypervisor contains the device drivers and it might
be difficult for new device drivers to be installed by users (unlike on your typical PC). Consequently, if a machine has hardware
resources the hypervisor has no driver for, the virtualization software can’t be run on that machine. This can cause problems,
especially for organizations that want to take advantage of new hardware developments.

Fuente: http://networksandservers.blogspot.com.ar/2011/11/full-virtualization-explained.html
Paravirtualization Explained

“Para“ is an English affix of Greek origin that means "beside," "with," or "alongside.” Paravirtualization is another approach to server
virtualization where, rather than emulate a complete hardware environment, paravirtualization acts as a thin layer, which ensures that all of
the guest operating systems share the system resources and work well together.

Under paravirtualization, the kernel of the guest operating system is modified specifically to run on the hypervisor. This typically involves
replacing any privileged operations that will only run in ring 0 of the CPU, with calls to the hypervisor (known as hypercalls). The hypervisor in
turn performs the task on behalf of the guest kernel and also provides hypercall interfaces for other critical kernel operations such as memory
management, interrupt handling and time keeping.

Paravirtualization tries to fix the full virtualization problems by allowing the guest operating systems to gain direct access to the underlying
hardware, thus it refers to communication between the guest OS and the hypervisor to improve performance and efficiency. Since it involves
modifications to the OS, paravirtualization is also sometimes referred to as OS-Assisted Virtualization.

Paravirtualization, where the guest OS "knows" it is being virtualized, is different from full virtualization, where the unmodified OS does not
know it is virtualized and sensitive OS calls are trapped and translated using binary translation.
Paravirtualization Advantages

There are two advantages to this approach:

• The ability of the guest kernel to communicate directly with the hypervisor results in greater performance levels. Total
virtualization, you’ll recall, inserts an entire hardware emulation layer between the guest operating system and the physical
hardware. By contrast, paravirtualization’s thin software layer acts more like an air traffic controller for virtualized servers, allowing
one guest OS access to the physical resources of the hardware while stopping all other guest OSs from accessing the same
resources at the same time. This method is generally much more efficient than traditional hardware emulation virtualization The
value proposition of paravirtualization is in lower virtualization overhead, but the performance advantage over full virtualization
can vary greatly depending on the workload;

• The second advantage of the paravirtualization approach compared to full virtualization is that paravirtualization does not limit
you to the device drivers contained in the virtualization software; in fact, paravirtualization does not include any device drivers at
all. Instead, it uses the device drivers contained in one of the guest operating systems, referred to as the privileged guest. Without
going into too much detail about this architecture here, suffice to say that this is a benefit, since it enables organizations to take
advantage of all the capabilities of the hardware in the server, rather than being limited to hardware for which drivers are available
in the virtualization software as in total virtualization.

Paravirtualization Limitations

• Paravirtualization requires the guest operating systems to be modified in order to interact with the paravirtualization interfaces.
This typically limits support to open source operating systems such as Linux, which may be freely altered and proprietary
operating systems where the owners have agreed to make the necessary code modifications to target a specific hypervisor. As
paravirtualization cannot support unmodified operating systems (e.g. the Windows family), its compatibility and portability are
poor;

• Paravirtualization can also introduce significant support and maintainability issues in production environments as it requires deep
OS kernel modifications.

The open source Xen project is an example of paravirtualization that virtualizes the processor and memory using a modified Linux kernel and
virtualizes the I/O using custom guest OS device drivers.

Conclusion: Paravirtualization is not ‘Full Virtualization’ since it involves modifications to the Guest OS. Therefore, while it does bring about a
significant improvement in terms of virtualization overhead, it has its limitations when it comes to compatibility and portability. However,
recent developments have allowed Xen to run unmodified OSes using hardware assistance techniques. I’ll go into that in a future article.

Fuente: http://networksandservers.blogspot.com.ar/2011/11/para-is-english-affix-of-greek-origin.html
Operating System-Level Virtualization Explained

This kind of server virtualization is a technique where the kernel of an operating system allows for multiple isolated user-space instances.
These instances run on top of an existing host operating system and provide a set of libraries that applications interact with, giving them the
illusion that they are running on a machine dedicated to its use. The instances are known as Containers, Virtual Private Servers or Virtual
Environments.

Operating system level virtualization is achieved by the host system running a single OS kernel and through its control of guest operating
system functionality. Under this shared kernel virtualization the virtual guest systems each have their own root file system but share the kernel
of the host operating system.

This is a special case of Hosted Virtualization where the hypervisor (the container) has very limited functionality, relying on the Host OS for
CPU scheduling and memory management. If you prefer, this method, which uses virtualization at the OS level, doesn’t even include the use
of a real hypervisor; instead, the virtualization capabilities are part of the hosting operating system, which performs all the functions of the
hypervisor.

In other words, OS level virtualization is based on the creation of isolated containers or partitions on a single physical server, and the use of
OS instances for each guest application environment to work independently of the other partitions. With this technique, you install the
virtualization software layer on top of the operating system and all of the guest systems run on top of this layer using the same operating
system as the host OS but with each guest having its own resources and running in complete isolation from the other guests.

It can be argued that this is not virtualization in the strict sense of the term; rather a technique that just allows for machine consolidation.

Examples of OS level virtualization are OpenVZ, Virtuozzo, or Solaris Zones.

Advantages of OS Level Virtualization

• This form of virtualization usually imposes little or no overhead thereby ensuring most of the machine’s resources are available to
the applications running in the containers;

• This is a cost-effective and efficient solution for creating similar guests, but it’s only practical for certain situations. It is an ideal
approach for web hosting companies which have multiple virtual web servers running on a single box or blade. This is very
convenient because patches or modifications can be made to the host server and they will instantly be applied to all of the
containers. Another example would be a company that has to manage multiple SQL databases or any other scenario where many
similar or identical servers need to be hosted or managed within the same datacenter.

Limitations of OS Level Virtualization

This approach typically limits operating system choice. Containerization usually means that every guest OS must be identical or similar to the
host in terms of version number and patch level. For example, with Linux as the host operating system, only Linux distributions can be run as
the guest operating systems. This can cause problems if you want to run different applications in the containers, since applications are often
certified for only a certain OS version and patch level. It is not possible for a Linux guest system designed for the 3.0.9 version of the kernel to
share a 3.1.1 version kernel.

Fuente: http://networksandservers.blogspot.com.ar/2011/11/this-kind-of-server-virtualization-is.html
Hardware-Assisted Virtualization Explained

Hardware-assisted virtualization was first introduced on the IBM System/370 in 1972, for use with VM/370, the first virtual machine operating
system. Virtualization was forgotten in the late 1970s but the proliferation of x86 servers rekindled interest in virtualization driven for the
need for server consolidation; virtualization allowed a single server to replace multiple underutilized dedicated servers.

However, the x86 architecture did not meet the Popek and Goldberg Criteria to achieve the so called “classical virtualization″. To compensate
for these limitations, virtualization of the x86 architecture has been accomplished through two methods: full virtualization or
paravirtualization. Both create the illusion of physical hardware to achieve the goal of operating system independence from the hardware but
present some trade-offs in performance and complexity.

Thus, Intel and AMD have introduced their new virtualization technologies, a handful of new instructions and — crucially — a new privilege
level. The hypervisor can now run at "Ring -1"; so the guest operating systems can run in Ring 0.

Hardware virtualization leverages virtualization features built into the latest generations of CPUs from both Intel and AMD. These
technologies, known as Intel VT and AMD-V respectively, provide extensions necessary to run unmodified virtual machines without the
overheads inherent in full virtualization CPU emulation. In very simplistic terms these new processors provide an additional privilege mode
below ring 0 in which the hypervisor can operate essentially leaving ring 0 available for unmodified guest operating systems.

With hardware-assisted virtualization, the VMM can efficiently virtualize the entire x86 instruction set by handling these sensitive instructions
using a classic trap-and-emulate model in hardware, as opposed to software. The hypervisors that support this technology can load at Ring -
1 and guest OSes can access the CPU at Ring 0, just as they normally would when running on a physical host. This allows for guest OSes to be
virtualized without any modifications.

As depicted in the figure, privileged and sensitive calls are set to automatically trap to the hypervisor, removing the need for either binary
translation or paravirtualization.

Advantages of Hardware-Assisted Virtualization

Hardware-assisted virtualization changes the access to the operating system itself. x86 operating systems are designed to have direct access
to system resources to run. With software virtualization the VMM emulates the required hardware to the operating system. With hardware-
assisted virtualization the operating system has direct access to resources without any emulation or modification and this improves overall
performance.

This means OS kernels no longer need to be tweaked (as in paravirtualization) and can run as-is. At the same time, the hypervisor also does
not need to be involved in the inefficient Binary Translation of the sensitive instructions. Thus not only does hardware-assisted virtualization
satisfy the Popek and Goldberg criteria (of full virtualization) it also provides improved performance because the privileged instructions are
now trapped and emulated in the hardware directly.

Limitations of Hardware-Assisted Virtualization

Hardware-assisted virtualization requires explicit support in the host CPU, which is not available on all processors. A “pure” hardware-assisted
virtualization approach, using entirely unmodified guest operating systems, involves many VM traps, and thus high CPU overheads, limiting
scalability and the efficiency of server consolidation. This performance hit can be mitigated by the use of paravirtualized drivers; the
combination has been called “hybrid virtualization”.

http://networksandservers.blogspot.com.ar/2011/11/hardware-assisted-virtualization-was.html