See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/326466968

Data Center Security and Virtualization Report.1

Preprint · January 2018

DOI: 10.13140/RG.2.2.21222.93762

CITATIONS READS

0 3,197

1 author:

Shishir K C
Touro College
1 PUBLICATION   0 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Data Center Security and Virtualization View project

All content following this page was uploaded by Shishir K C on 18 July 2018.

The user has requested enhancement of the downloaded file.

 Capstone Research Project MSIS 695

DATA CENTER SECURITY & VIRTUALIZATION

Submitted By

Shishir K C

M.S. In Data Communication

Supervisor

Dr. Shmuel Fink

In partial fulfillment of the requirements

For the Degree of M.S. In Data Communication

Touro College Graduate School of Technology

New York, NY

Spring 2018

Acknowledgement:

I would like to express my sincere gratitude and appreciation to Dr. Shmuel Fink who helped

me complete this research paper. Writing this paper was not possible without his constant

guidance and support. I am also very thankful to Touro College and faculty members for

providing knowledge and this platform to reflect what I have learned here in this paper.

Data Center Security & Virtualization | Shishir K C 1

Abstract:

It is a matter of common knowledge that internet is not secure. Many instances have shown

that there are people in this huge interconnection of networks that want to, with various

intentions, steal others information, disrupt service of a general service provider, and attack

into systems to gain access or to bring them down. Network security has turned out to be a

fundamental element of every organization to ensure secure internet connectivity and

protection against data breach. While many organizations have turned towards Data Center

service providers to save their time and effort on the acquisition, installation, management

and security of hardware, servers and other devices, Data Centers themselves are not secure

from goons on the internet. This is high time for Data Center to prove their trustworthiness to

customers by not only securing their data but also by providing them isolation from other

customers that share the same infrastructure and by providing uninterrupted service with a

minimum amount of downtime. To secure Data Centers networks and prevent data breaches,

different vendors and Data Center professionals have suggested various solutions out of

which some have been discussed in this paper. Moreover, as Data Center technology has been

developing to fully adapt to automation through software abstraction, virtualization has

become an inseparable part of it. This paper explores virtualization of Data Center and

incorporation of security in virtualized systems.

Data Center Security & Virtualization | Shishir K C 2

Table of Contents

Acknowledgement: ........................................................................................................... 1

Abstract: .......................................................................................................................... 2

Introduction: .................................................................................................................... 4

Data Center Evolution: .................................................................................................... 6

Background: .................................................................................................................... 7

Problem Domain: ............................................................................................................. 8

Threats: .......................................................................................................................... 11

Infrastructure Security Solutions and Threat mitigation: ............................................. 14

Physical Security: .................................................................................................................... 14
Network Security: ................................................................................................................... 15
Logical Cage Model using Trusted Virtual Domain (TVD): ....................................................... 15
Integrated security solution using Juniper SRX service gateway: ............................................... 16
Security based on Virtualization Architecture and Infrastructure: ............................................... 18
Collaborative Network Security: ................................................................................................. 20
Network security with VMware NSX: ........................................................................................ 20

Virtualization ................................................................................................................. 22
Server Virtualization .............................................................................................................. 22
Network virtualization ............................................................................................................ 23
Storage Virtualization ............................................................................................................. 25

Data Center virtualization to integrate security: ........................................................... 27

Conclusion ...................................................................................................................... 28

Bibliography .................................................................................................................... 29

Appendix ........................................................................................................................ 31

Data Center Security & Virtualization | Shishir K C 3

Introduction:

As human life is becoming more digital and more virtual, they tend to produce a huge amount

of data every second. The tremendous amount of data that are produced every second require

a dynamic, secure and persistent storage facility. Gone are the days when computers and its

parts would fill up large rooms with special requirements like power, processing, networking

and storage facilities. Since its emergence in 2008, cloud computing has made computing as a

utility a possibility (Fernandes, et. al. , 2014). Cloud environment provides a virtualization

technique that offers an organized way of dispatching resources on the go. Hence,

organizations these days prefer storing data and using compute service from cloud

infrastructures like Data Centers.

NIST defines cloud computing as a model for empowering an appropriate, readily available

on-demand network access to shared pool of manageable resources such as networks, servers,

storage, application and services that can be easily used or released (Mell & Grance, 2011).

The cloud computing has five essential characteristics, four deployment models, and three

service models. The cloud computing must provide on-demand self-service computing

capabilities and should be available over the broad network to help the customer get access to

pooled resources. The resources should be available to be provisioned or released as the

customer’s wish and must be automatically controlled and optimized (Mell & Grance, 2011).

Based on the level of abstraction of computing resources, the cloud computing service can be

classified into following service models:

Software as a Service (SaaS): It provides the customers capability to use cloud service

provider's applications that are running on cloud infrastructure (Hu et. al., 2017).

Platform as a Service (PaaS): The customers are provided with the capability to deploy

applications created using programming languages, libraries, services, and tools onto the

cloud that is supported by cloud service provider (Hu et. al., 2017).

Data Center Security & Virtualization | Shishir K C 4

Infrastructure as a Service (IaaS): Here, customers are capable to provision computing

resources like processing, network, storage and can also deploy operating systems and other

applications (Hu et. al., 2017).

The cloud computing infrastructure deployment model can be either public where resources

are shared between multiple mutually untrusted tenants or can be private where the resources

are solely used and reserved for only one tenant for their exclusive use (Huang, et. al., 2015).

This paper focuses mainly on the public data center that provides IaaS cloud service and

seeks to find out security challenges in modern data centers and their virtualization.

A cloud data center is built considering geological, environmental, political, governmental

and energy-saving aspects. They keep in mind factors such as location, temperature,

earthquake probability, and humidity. There is tiers level that defines data center quality,

from lowest level being 1 to the highest level being 4 % (Fernandes, Soares, Gomes, Freire,

& Inacio, 2014). A data center tier is determined by the set of components that it is composed

of and the attributes it offers to the customers like supply system, physical infrastructure,

cooling system, and expected uptime level (Hu et. al., 2017). Every cloud service provider

aims to provide cloud uptime as high as 99.99. A well-designed data center architecture

speaks for the cooperation between its components which includes utility system, security

system, IT-infrastructure, monitoring system, and control system (Hu et. al., 2017). A data

center must be well secured physically to prevent unauthorized access or break-ins. Only

people with security clearance to operate management must be allowed access to data center

resources like computing servers, storage servers, and network devices. In addition to the

physical security, perimeter security must be in place to safeguard the network of cloud

computing environment and to analyze the traffic (Fernandes, et. al., 2014).

Data Center Security & Virtualization | Shishir K C 5

Data Center Evolution:

Computer and IT have evolved a long way through. In its earlier days, the computers were

made up of electrical switches and mechanical relays. Later a transistorized, integrated circuit

based microprocessor revolutionized the computer technology which led to the

manufacturing of Intel's 8086 chip. With computer technology, the capability to store data

was being developed. IBM released first disk-based storage in 1956 which could store 3.75

Mb of data and weighed over a ton. The microprocessor/x86 architecture and disk-based

storage medium laid the foundation for a modern data center. This architecture, however, had

inefficiency when deployed at scale. The servers only used a fraction of the computing power

available and storage utilization also had the same issue. To address this issue, disks were

then started to be pooled and were made available via the network. The utilization was

increased and disk management overhead of storage was decreased. Now, the arrays of disks

were connected on a network referred to a Storage Area Network. The SAN made use of

network protocol called Fibre Channel Protocol which was suitable for delivering storage

because of its lossless and high-speed nature (Lowe, et. al., 2016).

With evolution in the industry and many organizations adopting shared storage model, the

architecture's value continued increasing. Several features like storage snapshot, replication,

and data reduction were added to the management platform. It proved to become efficient and

faster way for backup and recovery. In addition to that storage systems also contained data

replication from one storage array to another. By mid-2000s, the shared storage performance

increased significantly and manufacturers also kept improving physical disk, networking

protocols and file systems governing the storage array. During this time, shared storage

arrays offered more agility and flexibility luring organizations to implement boot from SAN

model. Although this decreased Data center costs, the CPU and memory resources were still

Data Center Security & Virtualization | Shishir K C 6

configured high above the actual utilization of application that the server was built for but

this problem was later addressed by use of hypervisors (Lowe, et. al., 2016).

Background:

In recent years, network security has become an important aspect of data center security with

various types of attacks evolving that target user data and compromise data center resources.

The data center service provider must ensure that customer’s data are safe and its security

properties are not compromised by attackers. The security properties of data include

confidentiality, integrity, and availability. Confidentiality ensures the authorized access to

data and includes information like existential information and data access pattern. Service

providers also need to make sure that the data is unchanged so that it is consistent and its

integrity is maintained. Availability defines accessibility to data to the customer whenever

they require (Huang, et. al., 2015).

While all data centers are secured with physical security measures like locks, alarms,

cameras, and guards, there are various ways that data center information security can be

carried out. Information security in the data center is based on its architecture. A secure data

center architecture includes implementation of these essential building components: a)

Physical Protection; b) Server Protection; c) Data Protection; d) Protection of Application and

Platforms; e) Secure Encryption and Key Management system and f) Network Security (Hu

et. al., 2017).

To identify security subjects that data center for cloud covers, following subsections must be

understood.

Virtualization elements: Virtualization is basically the process of abstracting computing

services, operating systems, and applications from underlying hardware. VM image is

basically a file that contains a copy of storage and memory content, virtualized OS with

multiple applications running on it. A VM does not have direct access to physical resources.

Data Center Security & Virtualization | Shishir K C 7

VMMs, which are also known as hypervisors, mediate allocation of virtual hardware

resources like CPUs, network adapter, memory, a hard disk for each VM (Fernandes, et. al. ,

2014).

Multi-tenancy: It is the capability to run multiple instances on a same shared platform. It is a

feature where one or more users, called tenants, access instances while sharing the same

platform. In IaaS, instances refer to VMs and multi-tenancy sharing platform refers to

hypervisors or VMMs (Fernandes, et. al. , 2014).

Data outsourcing: Data outsourcing is being widely adopted to IT industry. It is basically the

process of transferring the responsibility and delegating duties of storage, computing and

security to the third party that is managed in a data center (Fernandes, et. al. , 2014).

Data storage security and virtualization: As data centers hold a large amount of data for

processing, applying common techniques of cryptography, integrity checking and

authentication mechanism is impracticable (Fernandes, et. al. , 2014).

Trust: Since customer's data are in an infrastructure whose location is unknown and is

handled by the third party, there comes trust issue between customer and service provider. As

the high valued data is in an infrastructure, whose security management is in hand of

someone else, the question of trust arises. The service providers must be able to prove

trustworthiness to the customer by providing uninterrupted secure access to infrastructure

under all conditions. The trustworthiness can only be built by combining reliability with

security (Fernandes, et. al. , 2014).

Problem Domain:

Data centers contain many computing and a mass-memory device that provide computing and

storage capabilities to users. All the devices in these data centers are interconnected through

high-speed LAN switches with low latency (often 10 GB Ethernet). The capacity of storage

arrays that are organized into SAN (Storage Area Network), depends on the size and number

Data Center Security & Virtualization | Shishir K C 8

of disk device they contain, while the processing capacity depends on the number of cores

and processors. The computing, storage and LAN devices are power hungry elements.

Continuous power supply to the data center infrastructure is a must to deliver uninterrupted

service to users. Recently, there has been an increase in energy-related DoS attacks that target

energy efficiency and power-management features (Palmieri, et. al., 2015). Different types of

DDOS attacks have emerged that have in ways affected data center operations. Firewalls are

commonly deployed security measure in the data center, but most of the firewalls experience

issues during DDoS attacks which is a matter of serious concern (Arbor Networks, 2016).

According to Datacenter Dynamics, DDoS attacks are the biggest threats to the data center as

shown in a survey taken by 220 service providers, network operators and data center

professionals (Jones, 2014).

For a 5000 square-foot data center with about 1000 servers, a computing capacity exhaustion

energy attack can increase daily consumption of energy by 480kW. The attackers can cause

huge financial loss by forcing high energy consumption rates over the maximum cost hours.

The overconsumption of power can also cause SLA violations and cause a power outage.

The overloading of CPU causes chip temperature to increase to a high level which affects the

rate of failures in electronic components and reduces component lifespan (Palmieri, et. al.,

2015).

As much as it is cost-effective and beneficial, multi-tenancy and virtualized approach

increase co-location and attack surface. While virtualization can be considered as a primary

defense point, any misconfiguration can make it a point of entrance for attackers. It has been

found that network security methods like firewalls and VLANs turn out to be less effective in

a virtualized environment (Fernandes, et. al., 2014). Since, each tenant on the same server

may have different security requirement, setting up rules for a device is no longer practical.

Data Center Security & Virtualization | Shishir K C 9

So, meeting diverse security requirement for multiple tenants and enforcing it is a major issue

(Chen, et al., 2014).

Moreover, Virtual machines allow to keep a snapshot of the status of the machine at a certain

time and access it later, which with time may not be kept up-to-date meaning when an older

image is rolled back during disaster recovery the image is outdated. This leads to

vulnerabilities in the system that may be exploited by attackers until a patch cycle detects it

and patches are applied. The companies must regularly update software and apply security

patches. On a setup where multiple VMs are connected over a virtual switch creating a virtual

network, the attacks originating from one VM and targeting VM on the same server may not

be detected by traditional network security services like IDS or Data loss prevention. This is

because the traffic does not have to pass through the physical network. There are two

scenarios that pose a security threat to a virtual system; one infecting virtual machines (the

crisis malware) where an attacker who compromises host server can infiltrate virtual

machines. The other being escaping virtual environments where a malware from virtual

machine escapes and infects the host server. This is also called guest-to-host infection.

(Wueest, Threats to virtual environment, 2014).

In addition to the DDoS attacks, it has been found that many attacks on virtualized data

center are targeted towards gaining access of hypervisor; like attacks on hypervisor through

the host OS and attack on hypervisor through a guest OS. Exploiting the vulnerabilities of the

host OS on which the hypervisor runs, the attacker can compromise hypervisor which

attacker can use for any malicious activities on any of the hosted hypervisor. An attacker can

also use guest OS to gain unauthorized access to other VM or the hypervisor, which is called

escapes or jailbreak attack. If an attacker finds out how VM's virtual resources map to

physical resources, the attacker can conduct a direct attack on physical resources (Dhawale,

2014).

Data Center Security & Virtualization | Shishir K C 10

Threats:

A Data Center is susceptible to numerous threats. Customer’s applications running on the

servers can be vulnerable to SQL injection and cross-site scripting. While different types of

DDoS attacks are emerging, it is important to not only secure data center perimeter but also

to develop secure coding practice while developing applications that run on data center

servers. Some of the threats to a data center and cloud infrastructures are explained below.

Network Bandwidth Exhaustion DoS:

This type of attack targets network connectivity and aims to exhaust available bandwidth on

Internet connection interface. Basically, these attacks are performed by generating a bulk of

service requests or packets (ICMP or UDP ECHO) directed towards target system so that the

victim generates corresponding reply packet. SYN flood is one of the dangerous network

attacks. It overloads victim with initial TCP connection attempts in a large quantity. The

victim then allocates a buffer for new TCP connection and replies with SYN-ACK. The

attacker refrains from completing three-way handshake, so it does not respond to SYN-ACK

leaving half-open connections at the victim site (Palmieri, et. al., 2015).

Processing power exhaustion DoS:

Like network exhaustion attack, a large amount of CPU intensive request can be used to

attack computing resources. A continuous and randomized HTTP, HTTPS request flood can

be used to exhaust victim Web Server communication channel. If HTTPS or any kind of SSL

enabled services are targeted, the CPU may be overloaded by cryptographic operations. In a

technique known as coercive parsing, the processing of a large number of namespace

declarations, oversize prefix names or namespace URLs, and very deeply nested XML

structure/tags can be used to exhaust CPU and memory of a target system. Exploiting

algorithmic deficiencies in data structures, protocols and tools are one of the recent

processing power attacks. Technologies like PHP, ASP.NET, JAVA, PYTHON and Google’s

Data Center Security & Virtualization | Shishir K C 11

open source JavaScript engine V8 are vulnerable DoS attacks that target hash table structure

they use. The exploitation of these hashing algorithms can cause increased workload to

complete collapse due to total CPU capacity exhaustion in target host (Palmieri, et. al., 2015).

Disk hardware solicitation DoS:

Network file systems like NFS, CIFS, AFS, SAMBA or file servers like FTP, FSP, RPC tools

can provide interfaces for DoS attacks that can overwhelm drive hardware with a huge

amount of randomized read/write request on different files, frustrating buffer cache or disk

scheduling algorithms. This may introduce a considerable amount of burden on a mechanical

component of magnetic drives that can lead to reduced performance, effectiveness and

lifetime. Attacks on solid state drives that solicit their NAND-based memorization hardware

cells with a huge amount of write operations can reduce their performance and lifespan

(Palmieri, et. al., 2015).

Energy-related DoS:

New kind of DoS attacks that have emerged that targets hardware component on server

equipment that experiences maximum energy demand gaps between idle and busy

operational state. The attack aims to generate maximum possible workload on target

component by keeping it 100% busy, such that it can never enter lower power consumption

state. This forces the component to operate continuously at its maximum frequency, voltage,

and temperature. Since server’s energy demand is directly related to HVAC (Heating,

ventilation, and air conditioning) and other operating features, the electric power absorbed is

maximized which increases data center operational cost. Overloading server’s hard disks with

millions of read-write operations forcing them to constantly operate at maximum rate can be

another way of consuming systems power (Palmieri, et. al., 2015).

Even security tools like anti-virus or anti-malware can be exploited for energy attacks. Since

anti-virus operation is CPU and I/O intensive, it causes a long period of CPU and disk

Data Center Security & Virtualization | Shishir K C 12

overload. Delivering a massive amount of malicious content from different origins can trigger

anti-virus which ultimately can waste a huge amount of CPU power (Palmieri, et. al., 2015).

Threats from shared resources and multi-tenancy:

Tenants in the data center network share resources like Hard disk, RAM, CPU cache, GPU

and other elements that typically are not designed to support multi-tenancy requirement. This

often results in data loss and leakage through incidents like side-channel timing attacks. The

side-channel timing attacks leak cryptographic keys across virtual systems. Any vulnerability

found in a core component, like a hypervisor, can compromise an entire virtual system and

cloud infrastructure (Wueest, et. al. , 2015)

Storage enumeration attack:

Data centers provide storage service to a customer with access to storage resources through

user's account that has unique domain designated to each of them. These storage spaces are

called containers, blobs or buckets. The structure of the domain names assigned to storage

account allows an attacker to lunch dictionary attack using a list of commonly used words to

find the valid domain prefixes. The attacker may also guess folder names correctly and gain

access to them if they do not have read access restrictions. The authors conducted a research

that identified 51 open directories out of 16,000 domains which returned them 11,000

accessible files. It was found in the research that most administrators did not verify the

container’s permissions which made their backup files publicly accessible (Wueest, et. al. ,

2015).

Link swap attacks:

Once the target’s resource URL is known, the attackers can launch attacks from that location.

For instance, when a URL expires, it is publicly available and an attacker may reuse that

resource’s old URL to register their malicious server. If an application connects to the

resource using old URL, they will be redirected to attacker’s malicious server. The attacker

Data Center Security & Virtualization | Shishir K C 13

may then steal user’s credentials or infect victim’s computer with malware (Wueest, et. al. ,

2015).

Infrastructure Security Solutions and Threat mitigation:

Data Center infrastructure includes facility, network, hardware, and operational software that

supports provisioning of processing and storage resources. Data centers must comply with

security best practices and various compliance standards. Infrastructure security comprises of

both Physical Security and Network Security.

Physical Security:

A Data Center must be physically as well as environmentally secured making innovative and

comprehensive use of architectural and engineering approach. The physical access to data

center must be controlled and should be monitored by security personnel at the perimeter and

through video surveillance. Some of the environmental factors that need to be taken the

utmost care of are listed below.

Fire Detection and Suppression: Data centers must be equipped with automatic fire

detection and suppression system. All data center environments, chiller rooms, generator

equipment rooms, mechanical and electrical infrastructure spaces should have smoke

detection sensors (Amazon Web Services, 2016).

Power: Electrical power systems must be redundant and maintainable with Uninterruptible

Power Supply Units providing back-up in the event of power disruption (Amazon Web

Services, 2016).

Climate and Temperature: A constant operating temperature must be maintained for servers

and hardware to operate at normal condition. This prevents overheating of hardware and

lowers the probability of service outage (Amazon Web Services, 2016).

Data Center Security & Virtualization | Shishir K C 14

Network Security:

As Data centers provide computing and storage service to multiple tenants, various issues

arise in multi-tenancy because of resource sharing. To address security and isolation concern

of customers, a logical cage model in the data center can be created using trusted virtual

domain.

Logical Cage Model using Trusted Virtual Domain (TVD):

The basic concept of this technique is creating a logical cage model in the data center by

combining various VMs of any specific customer that are spread across multiple physical

resources into a Trusted Virtual Domain (TVD) or a virtual zone. A domain-based security

policy in the virtualized data center must be enforced that simulates physical separation of

data center customers. This technique aims to logically separate networks, storage, VM and

users and virtual devices of one tenant from the other. Here, domain isolation is achieved by

implementing security policies within a domain, independently of another domain that may

co-exist and share the same infrastructure. Below are the components that help enforce those

security policies (Cabuk, et al., 2010).

High-level policy model: Here security policy is based on TVD that isolates resources of one

TVDs from resources of another TVD to enforce domain policy. This security model includes

two policies defining security objectives. An Inter-TVD policy defines how information can

be exchanged with another TVD. If no information sharing between TVDs is permitted, then

a resource cannot be shared between them. The Intra-TVD policy allows customers to define

security objectives within their own domain (Cabuk, et al., 2010).

Security objectives and policy enforcement points: For all shared resources in the TVD

infrastructure, policies are enforced that emphasizes the isolation at the boundary of each

TVD. If information flow is allowed between two TVDs, resources can be shared between

members of different TVDs. A TVD can permit certain resource on one host to provide

Data Center Security & Virtualization | Shishir K C 15

service to another domain. To restrict the flow of inbound and outbound information, each

TVD defines rules and then underlying policy-enforcement infrastructure ensures only

resources trusted by all TVDs are shared. Furthermore, security within a virtual domain can

be attained by making all the resources satisfy membership requirements to be admitted to a

TVD and to retain membership (Cabuk, et al., 2010).

Policy refinements for protected resources: To enforce security, the goal of creating policies

is to transform them into data center configurations and security mechanism that is specific to

each resource like VLAN configuration. For all the policies defined in the High-level model,

a policy refinement model should fine-grain it for specific resources because as policy

translation moves to the lower level of abstraction, it will require additional information. A

Network security policy across TVD outlines isolation and flow control between TVDs. A

Inter-TVD storage security defines storage policies administering the usage and security of its

storage. A single policy can be used across all the storage volumes (Cabuk, et al., 2010).

Unified policy enforcement for virtual data centers: The TVD infrastructure consists of

management and enforcement layer. Each TVD can be identified by unique TVD master that

arranges TVD deployment and configuration and can be implemented as a central entity. A

TVD proxy helps to translate high-level policies into host configurations and security

services. A Virtual Networking (VNET) infrastructure enables the use of virtual switches,

Ethernet encapsulation, VLAN tagging, and VPNs to group VMs that belong to same TVD.

Each tenant's TVD has separate virtual network ensuring isolation by connecting the VMs at

Ethernet level (Cabuk, et al., 2010).

Integrated security solution using Juniper SRX service gateway:

Traffic among physical workloads can be segmented and isolated by creating security zones

in data center network. These security zones are the collection of interfaces that share similar

security requirement that define a security boundary. Security policies can be assigned to

Data Center Security & Virtualization | Shishir K C 16

security zones to control traffic flows between different security zones. A zone-based firewall

provides packet processing, security policy management and reporting services (Juniper

Networks, 2013).

The firewall can be deployed in data center core that restricts traffic flow between two or

more hosts in a stateful manner. The firewall should also provide services like Network

Address Translation (NAT), VPN or Intrusion Prevention System (IPS). It must be in the path

of traffic that is supposed to be filtered, so while using a traditional firewall, it needs to be

configured as a default gateway (Juniper Networks, 2009). The firewall can be deployed in

two ways:

Inline Firewall Deployment: Firewalls should be a part of network service tier attached to

the core network infrastructure as shown in Enterprise Data Center Reference Architecture

(Appendix A). A firewall can be deployed physically inline between aggregation and core

layer for all the data traffic. This deployment method confirms that all traffic between access

layer and the core layer is protected by the firewall. This deployment method lowers

flexibility of bypassing the firewall for access to core layer traffic, however, it needs fewer

ports on aggregation devices (Juniper Networks, 2009).

One-Arm Firewall Deployment: Sometimes referred to as firewall on a stick, this approach

provides administrators with the flexibility to decide what traffic gets filtered and what

bypasses the firewall using routing policy configuration. This configuration allows Layer 3

termination at the firewall and inter VLAN traffic can be routed through the firewall. The

advantage of using this approach is that all the traffic to or from such VLANs is protected by

the firewall. For legacy applications that do not support firewalls or for the applications that

do not require firewall protection, the VLANs that are hosting these applications can be

terminated at aggregation devices so the communication among those VLANs can bypass the

firewall (Juniper Networks, 2009).

Data Center Security & Virtualization | Shishir K C 17

Cloud service providers also need to meet their security service-level agreements for their

customer by providing proper isolation among multiple tenants that share common

infrastructure. They need to make sure that customer's communication and access to their

VM is limited to the authorized entity and not accessible to any other customer sharing the

same resource. Juniper's security suite of SRX series isolates customer traffic flow by

wrapping customer VMs within security zones defined by customer's security policy. This

ensures that customers traffic flow only to correct zone (Juniper Networks, 2013).

Authentication Mechanism: Two-factor authentication is recommended to use for increased

security. Hardcoding access tokens in public places should not be practiced in public places

such as an application's source code. Any leaked credentials should immediately be revoked.

It is best practice to keep user privileges at the minimum to keep damage of a breach at a

minimum (Wueest, et. al. , 2015).

Security based on Virtualization Architecture and Infrastructure:

This approach aims to employ security measures on virtualization components and

infrastructure either by securing VM components like Hypervisor or by creating secure

gateways in virtualization infrastructure.

Hypervisor Security: Here, a traditional security measure is implemented on a hypervisor. If

the hypervisor is compromised, all the VMs created and controlled by it are also

compromised. The VMs are as strong enough as the security of the hypervisor. One way to

guarantee secure access to hypervisor can be the use of a hardware token possessed by an

administrator to launch the hypervisor. Access control, automatic updating, networking, and

introspection on guest OS are traditional ways to protect hypervisor from unauthorized

access. These security elements are usually implemented in software and can easily be

updated to keep security features up to date on the hypervisor (Dhawale, 2014).

Data Center Security & Virtualization | Shishir K C 18

Guest OS Security: In a virtual environment, every component must be secure. Since each

Guest OS running in VM act as a real OS running on a physical machine, all the OSs must be

configured with security measures. The communication between guest OS and the hypervisor

must be secure and abstraction that is provided by hypervisor must be implemented. It is a

good practice to use guest OS monitoring to detect and quarantine infected OS (Dhawale,

2014).

Image management Security: It ensures the security of VM image storage, transportation,

and management in a virtualized data center. To achieve image management security; strong

storage encryption and strong network security must be in place so that sensitive information

does not leak form images and to ensure their secure transportation. Since VM images can be

created easily and quickly which is also called as VM sprawl, it can create the unnecessary

distribution of same VM image. This can be mitigated using access control on the image

management facility (Dhawale, 2014).

Security on Virtual Layer: Virtual layer security is achieved by securing communication

between VMs and hypervisor in a virtual network. Virtual Private Networks are commonly

created to take advantage of a virtualization infrastructure. The virtual nature of network

makes features like monitoring, access control, integrity, encryption, authentication, and

transportation of VMs can be easily implemented in the network.

Security on Physical Layer: Here, physical security measures are emphasized to protect the

virtual environment. Host-based intrusion and detection system is one of the significant

features of this area. It ensures that physical layer will not be compromised. The physical

layer security also depends on the structure of data center and physical interconnection of

hypervisors (Dhawale, 2014).

Data Center Security & Virtualization | Shishir K C 19

Collaborative Network Security:

In this approach, in the Data Center Network, Security Center and peer-UTMs are deployed

(Appendix B). The security center interacts with peer-UTM and the peer-UTM is managed by

the commands from security center. Each peer UTM manages a virtual domain of a tenant in

Data Center. Here, Security Center issues rules and peer-UTMs reports events. For antivirus

modules, security center imports and issues virus signature database and synchronizes between

peer-UTMs. Similarly, Firewall rules are imported and updated by security center and the peer-

UTMs choose to implement rules updated by security center (Chen, et al., 2014).

The security center centrally manages security rules and collects feedback from deployments

and stores the collected data into a security log. Security rules are incrementally downloaded

and the new rules are available to be downloaded in a package to peer-UTMs. Protocol control

modules enforce UDP protocol rules. The UTMs regularly get firewall rules that contain

content inspection rule for UDP and blacklist for content filtering. The firewall module and

UDO content filtering module block specified network flow and send back the log to Security

Center (Chen, et al., 2014).

Network security with VMware NSX:

A logical firewall in VMware NSX provides the security mechanism for the dynamic virtual

data center and offers two components.

Distributed Firewall: It is a hypervisor kernel-embedded firewall and offers visibility and

control over virtualized workloads and networks. VXLAN, security groups, user group

identity form Active Directory and access control policies can be created based on VMware

vCenter objects. To provide consistent access control even after virtual machine gets

vMotined, firewall rules are implemented at the vNIC level of each virtual machine. In the

distributed firewall, Layer 3 packets are processed for existing state first. If the state match is

Data Center Security & Virtualization | Shishir K C 20

found, then the packet is processed through a set of rules until another match is found

(VMware, Inc, 2017).

Edge Firewall: This firewall monitors North-South traffic offering perimeter security

functionality including firewall, NAT, site-to-site IPsec and SSL VPN functionality

(VMware, Inc, 2017).

NSX also integrates security functions that are provided by other vendors so it stands out to

be more effective and efficient solution for data center architecture and security. NSX's

Service composer tool can be used to implement third-party Firewalls, anti-malware,

vulnerability management, data loss protection, intrusion detection and intrusion prevention

platform. In addition to that, network service gateway conduits physical and virtual

environments and offers load balancing, application delivery and WAN optimization services

(Chen, et al., 2014).

Dell s6000 data center switching gateway for NSX offers programmability, automation, and

scalability. S6000 is a high-performance gateway for NSX that helps to connect physical

workload that is accessed by VLANs to the logical network through layer 2 network services.

Palo Alto PA-5000 offers a powerful firewall that provides identification, control function

and defense option to network threats (Chen, et al., 2014).

Other security best practice that can ensure data center security is:

Encryption key management: A proper key management must be implemented and

whenever possible, all the data should be encrypted (Wueest, et. al. , 2015).

Logging: The key to investigating breaches and investigating issues is the good event-

logging and monitoring. All the data logged must be combined and associated with a security

information and event management tool (Wueest, et. al. , 2015).

Data Center Security & Virtualization | Shishir K C 21

Virtualization

The core-aggregation-access layered data center architecture has widely been accepted. In

this architecture, each layer offers different networking functionalities for distinct traffic

policies. The core layer provides forwarding power to the data center ingress and egress

traffic. The aggregation layer acts as a meeting point for server IP subnets, usually their

default gateway. In addition to that, it also forwards server-to-server traffic between multiple

pairs of access switches. The access layer contains Ethernet switches that are attached to

servers. In this section, taking this Data center architecture as a reference, different

techniques to virtualize network, storage and servers will be accessed (Santana, 2014).

Virtualization, in its simplest form, is emulating underlying IT resources that provide benefits

to its users that were unavailable in its physical form. For purpose of this report,

virtualization solution is attached with three basic areas of Data Center infrastructure: Server,

Network, and Storage.

Server Virtualization

Server virtualization is basically the use of a software application i.e. hypervisor, to divide

one physical server into multiple isolated virtual machines (VMs). The hypervisor, also

denoted as virtual machine manager (VMM), allows multiple guest operating systems to run

on a host server. The guest operating systems are provided with a virtual operating platform

by the hypervisor. Each of these Virtual Machines run their own guest operating system and

their specific applications (Jin, et. al. , 2013).

With the release of VMware ESX and VMware GSX in 2001, the underutilization of

hardware resources was practically reduced. These are based on two leading virtualization

models. The first model is the one where a hypervisor runs directly on the host's hardware

thus controlling the underlying hardware and managing the guest operating system (VMware

ESX). The other being the one where a hypervisor runs as a module within the operating

Data Center Security & Virtualization | Shishir K C 22

system environment. The hypervisor layer, in this case, is a second software level while the

guest operating system runs at the third level above the hardware (VMware GSX) (Jin, et. al.

, 2013).

Based on both VMware ESX and VMware GSX, the virtual machines can be defined as

software computers running on an operating system that is comprised of configuration

manageable files and specifications like emulated hardware definition file, virtual disk data,

and VM BIOS. A virtual machine state can be cloned and saved on a different host merely by

a file copy operation. The server virtualization infrastructure enables data center features

through a software layer with numerous functional benefits (Santana, 2014). For example, it

provides High Availability for virtual machines where a VM running on a failed host can be

accessed by another host from the cluster and can be reinitiated automatically. A virtual

machine state can be cloned and saved on a different host merely by a file copy

operation.VM provisioning can also be simplified using VM templates which offer ease of

VM creation as they include resource setting, installed applications, and common

configuration such as DNS servers. Moreover, the most astounding feature of server

virtualization can no doubt be elected as online virtual machine migration (VMware

vMotion). vMotion allows VMs to be transferred from one host to another without any

interruption to applications. Alternatively, a feature called Fault Tolerance also supports

stateful redundancy for VMs in a virtualized infrastructure (Santana, 2014).

Network virtualization

Network virtualization is essentially the creation of logically isolated network partitions that

are placed on top of a shared physical infrastructure. These network partitions should act as

dedicated networks that provide security, service levels, routing decisions and an independent

set of policies (Santana, 2014). Network virtualization offers network consolidation for

increased forwarding capacity and maximized use of ports and the implementation of distinct

Data Center Security & Virtualization | Shishir K C 23

security zones and multitenancy. Network virtualization provides an opportunity to transform

physical connection and devices into simpler logical entities thus improving resource

utilization and reducing design complexities. It also allows single physical router to have

multiple routing tables. Furthermore, virtualization can be achieved by allowing multiple

virtual networks per physical connection. This report focuses mainly on packet forwarding,

load-balancing, and efficient network resource sharing aspects of network virtualization. The

network partitioning can be achieved using either VLAN and Virtual Routing and

Forwarding (VRF) instances, or virtual context for server load balancer, or Virtual device

context (VDC) for data center switches (Santana, 2014).

In data center network, VLANs are deployed for segregating traffic in distinct environments,

so when two servers are in different VLANs, they are not supposed to communicate with

each other unless they are allowed by another network element such as a router or a firewall.

In VLAN, as a virtualization technique, the administrator must ensure proper configuration of

native VLANs, reserved VLANs, resource sharing, and management plane. Virtualization

can also be achieved by using VRF which to create multiple virtual networks into a single

network device. It basically creates virtual routing element that can be logically provisioned

within existing equipment and natively virtualize data and control plane as they provide

partitioning of forwarding and routing tables within a networking device (Cisco Systems,

2008). The VRF can import and export routes from routing peer that are unaware of this

virtualization technique. Through VLANs and VRFs, network virtual partitions can be built

easily (Santana, 2014).

Virtualizing the network service like application load balancer is also trending in modern data

centers. Depending on the size of the data center, the number of load balancers used can be

bigger or relatively smaller. Cisco created the concept of virtual context and implemented it

in Application Control Engine (ACE) product. The ACE virtual context is fundamentally an

Data Center Security & Virtualization | Shishir K C 24

abstraction of an independent load balancer with its own configuration, administrators,

policies, and interfaces. It allows for the creation of multiple load balancers. ACE virtual

context is capable of significantly increasing efficiency in application roll out as its creation

does not depend on an acquisition and physical installation. Resource allocation for ACE

virtual context is a tremendously powerful tool to increase data center hardware utilization.

Thus, it allows for the creation of a completely tailored load balancer for any specific

application environment performance and its swift deployment. In addition to that, virtual

context performance parameter can be changed anytime to avoid unnecessary hardware

migration and configuration (Santana, 2014). In a data center network, by making necessary

VLAN changes, virtual context can be easily inserted and deployed in its networking

environment to load-balance servers or other devices (Santana, 2014).

Virtual Device Context (VDC) are virtual Ethernet switches that can help to achieve a higher

level of data center network consolidation and isolation. VDCs are essentially a logical

partition of a physical switch and are used to virtualize the device itself by presenting a

physical switch as multiple logical devices. VDC also allows the creation of a unique and

independent set of VLANs and VRFs, also called virtualization nesting. A separate

management domain within each VDC can manage VDC itself, thus enabling management

plane itself to be virtualized (Cisco Systems, 2012).

Storage Virtualization

Several virtualization techniques for storage are used in Data center storage task. LUN

translation and RAID are commonly used modern storage system abstraction technique.

Some of the virtualization solutions are outlined below.

Disk Array Virtualization: Disk Array Virtualization can be achieved by grouping several

disks or by partitioning a single disk. A physical disk array can further be subdivided into

logical devices through partitioning with allocated resources like disks, cache, memory, and

Data Center Security & Virtualization | Shishir K C 25

ports. Hence, they can be used as a pool of resources to create exclusive LUNs or file systems

to different departments or customers from each virtual array partition. This virtualization

technique protects data access between partitions and controls hardware resource between

each one of them, thus, encouraging storage consolidation and resource optimization in

multitenant data centers (Santana, 2014). Storage virtualization also lets multiple physical

arrays to act as a single system, offering data redundancy and management consolidation.

LUN virtualization: In virtualizing LUN, a virtualizer element is placed between host and its

associated target disk array. The virtualizer then creates a vLUN that proxies server I/O

operations and hides specialized data block processes that occur in the pool of disk arrays

under its dominion (Santana, 2014) (see Appendix C).

Implementation of virtualizer offers numerous LUN features like storage resource pooling,

thin provisioning, data online migration and LUN extension. In storage resource pooling, a

physical group of devices is treated like a single repository of data block resources. Here, a

vLUN is generated using smaller LUNs that are distributed across different physical arrays.

Thin-provisioning on another hand helps reduce the waste of block storage resources by

bringing the concept of oversubscription to data storage. In addition to that, vLUNs can also

enable data online migration between different arrays or types of disk drives. Using this

virtualization feature, non-disruptive LUN resizing and storage tiering can be achieved

(Santana, 2014). Furthermore, LUN extension allows mirroring between dissimilar storage

resources. Here, both virtualizers need to declare the same vLUN that enables write

operations on both sites. This virtualization feature is exceptionally worthwhile for geo-

cluster switchover and server migrations between data center locations (Santana, 2014).

Virtualizing file system: Virtual file system is primarily an abstraction layer that permits

client applications using diverse file-sharing protocols to use a unified pool of storage

Data Center Security & Virtualization | Shishir K C 26

resources. This system is built on a global file directory structure that is contained on

dedicated file metadata servers (Santana, 2014) (see Appendix D).

Data Center virtualization to integrate security:

Device Virtualization: The aggregation layer in the core-access-aggregation architecture,

provides a building block for installing firewall services. The internal firewalls provide a line

of defense for data center assets as most of the requests for the data center will be generated

from the internal network. Using virtual context feature, a Cisco ASA firewall can be

implemented with multiple contexts allowing it to be divided into multiple logical firewalls,

each capable of supporting different interfaces and policies. The firewalls are recommended

to allow a minimum of following protocols: HTTP, HTTPS, SMTP, DNS, FTP, routing

protocols, unified communications, VOIP, video protocols, multicast, ICMP and a host of

others. However, depending on the organizational need, its security policies, and the type of

application deployed, the firewall policy may differ. The configuration of Authentication,

Authorization, and Accounting (AAA) for role-based access control and use of Network

Time Protocol (NTP) is a good practice (Cisco Systems, 2009).

Application Control Engine (ACE) for web applications firewall: Cisco ACE can be used to

scale web application firewall (WAF) appliances. When WAFs are configured as a server

farm, the Cisco ACE can distribute connection to the WAF pool. The ACE also can store

server certificate locally, allowing it to proxy SSL connection for client’s requests and then

forward client’s request to the server in clear text. In this case, the ACE can terminate HTTPS

request and decrypt traffic before forwarding it to WAF farm. Subsequently, the WAF farm

and IPS can view the clear text for inspection. The Cisco ACE WAF protects web

applications from attacks like cross-site scripting (XSS), SQL and command injection,

privilege escalation, cross0site request forgeries (CSRF), buffer overflows, cookie tampering

and Denial-of-Service (DoS) attacks (Cisco Systems, 2009).

Data Center Security & Virtualization | Shishir K C 27

Server Virtualization: VMware ESX version 4 and newer versions support Cisco Nexus

1000V, which is a virtual switching platform. It provides physical access switch abilities at

virtual switching footprint. It is comprised of two components: Virtual Supervisor Module

(VSM) and Virtual Ethernet Module (VEM). Networking and policy configuration are

executed at VSM and applied on the ports of each VEM. Virtual machine ports and how they

are connected to physical ports are all mapped in VEM. VSM can communicate with vCenter

through VMware API. Security policies on the Cisco Nexus 1000V are defined through a

feature called port profiles. These profiles allow to configure network and security feature

under a single profile and propagate to multiple interfaces. Multiple profiles can also be

defined and assigned to different interfaces. These port profiles can then be applied to

specific VMs as a port group in VMware vCenter. This feature helps in network security

policy to still be defined and applied to virtual switch as physical by network and security

administrators. The port profile feature allows server administrator the ease of only choosing

port groups and assigning it to VMs (Cisco Systems, 2009).

Conclusion

It is evident that Data Centers have become an essential part of enterprise business solution.

With all the threats and vulnerabilities in data centers, hackers and attackers are always on a

search of an opportunity to break into the infrastructure. Data Center Service providers are

coming up with various techniques and solutions to minimize the threats. With improving

security, Data centers are moving towards complete virtualization of its infrastructure; from

servers to storage and to the networks. Service providers are considering full automation of

data center through software by abstraction and configuration of underlying hardware into

deliverable service. Hence, it is safe to predict that Software Defined Data Centers are the

future of virtualization and cloud computing.

Data Center Security & Virtualization | Shishir K C 28

Bibliography

Amazon Web Services. (2016). Amazon Web Services: Overview of Security Processes.
Amazon Web Services.
Arbor Networks. (2016). Worldwide Infrastructure Security Reprot. Burlington: Arbor
Networks, Inc.
Cabuk, S., Dalton, C. I., Eriksson, K., Kuhlmann, D., Ramasamy, H. V., Ramunno, G., . . .
Stuble, C. (2010). Towards automated security policy enforcement in multi-tenant
virtual data centers. Journal of Computer Security, 89-121.
Chen, Z., Dong, W., Li, H., Zhang, P., Chen, X., & Cao, J. (2014). Collaborative Network
Security in Multi-tenant Data Center for Cloud Computing. Tsinghua Science and
Technology, 19(1), 82-94.
Cisco Systems. (2008). Virtual Route Forwarding Design Guide for VRF-Aware Cisco Unified
Communications Manager Express. San Jose: Cisco Systems.
Cisco Systems. (2009). Security and Virtualizations in the Data Center. San Jose: Cisco
Systems, Inc.
Cisco Systems. (2012). Technical Overview of Virtual Device Context. Cisco Systems. San
Jose: Cisco Systems.
Dhawale, P. S. (2014). Virtualization security in Data Center & Cloud. International Journal of
Scientific & Engineering Research, 5(1), 1299-1306.
Fernandes, D. A., Soares, L. F., Gomes, J. V., Freire, M. M., & Inacio, P. R. (2014). Security
Issues in cloud environments: a survey. International Journal of Information Security,
113-170.
Hu, Z., Gnatyuk, S., Gnatyuk, V., & Bondarovets, S. (2017). Anomaly Detection System in
Secure Cloud Computing Environment. International Journal of Computer Network
and Information Security , 10-21.
Huang, W., Ganjali, A., Kim, B. H., Oh, S., & Lie, D. (2015). The State of Public Infrastructure-
as-a-Service Cloud Security. ACM Computing Surveys, 47(4).
Jin, Y., Wen, Y., Chen, Q., & Zhu, Z. (2013). An Empirical Investigation of the Impact of Server
Virtualization on Energy Efficiency for Green Data Center.
Jones, P. (2014, January 29). DataCenter Dynamics. Retrieved March 11, 2018, from
DataCenterDynamics: http://www.datacenterdynamics.com/content-tracks/servers-
storage/ddos-attacks-against-data-centers-hit-peak/84778.fullarticle
Juniper Networks. (2009). Integrating firewall service in the data center network
architecture using SRX series services gateway. Juniper Networks. Juniper Networks .
Juniper Networks. (2013). An integrated security solution for the virtual data center and
cloud. Juniper Networks. Juniper Networks .

Data Center Security & Virtualization | Shishir K C 29

Lowe, S. D., Green, J., Davis, D., & Kirchner, H. (2016). Building a modern data center
principle sna strategies of design. Atlantis Computing. Bluffton, SC: ActualTech
Media.
Mell, P., & Grance, T. (2011). The NIST definition of cloud computing. National Institute of
Standards and Technology, U.S. Department of Commerce. Gaithersburg: National
Institute of Standard and Technology.
Palmieri, F., Ricciardi, S., Fiore, U., Ficco, M., & Castiglione, A. (2015). Energy-oriented denial
of service attacks: an emerging menace for large cloud infrastructures. Journal of
Supercomputing, 1620-1641.
Santana, G. A. (2014). Data Center Virtualization Fundamentals (2nd Edition ed.).
Indianapolis, Indiana: Cisco Press.
VMware, Inc. (2017). NSX Administration Guide. VMware, Inc. Palo Alto: VMware, Inc.
Wueest, C. (2014). Threats to virtual environment. Symantec. Mountain View, CA: Symantec.
Wueest, C., Barcena, M. B., & O'Brien, L. (2015). Mistakes in the Iaas cloud could put your
data at risk. Symantec.

Data Center Security & Virtualization | Shishir K C 30

Appendix

Appendix A:

Figure 1: Enterprise Data Center Reference Architecture

Data Center Security & Virtualization | Shishir K C 31



Figure 2: Inline Firewall Deployment

Figure 3: One-arm Firewall Deployment

Data Center Security & Virtualization | Shishir K C 32

Appendix B:

Figure 4: Collaborative Network Security Solution

Data Center Security & Virtualization | Shishir K C 33

Appendix C:

Figure 5: Virtual LUN Appliances

Data Center Security & Virtualization | Shishir K C 34

 Appendix D

Figure 6: File System Virtualization

Data Center Security & Virtualization | Shishir K C 35

View publication stats