Openstack Object Storage Whitepaper

The
OpenStack
TM
Object Storage system

Deploying and managing a scalable, open-source cloud

storage system with the SwiftStack Platform

By SwiftStack, Inc. contact@swiftstack.com
The OpenStack Object Storage system
February 2012
Contents
Introduction............................................................................................................... 4
Introducing Swift ....................................................................................................... 5
Swift Characteristics..............................................................................................................................6
Swift is Scalable .......................................................................................................................................6
Swift is Extremely Durable..................................................................................................................7
Swift is Open Source Software...........................................................................................................7
Swift is Similar to AWS S3 ...................................................................................................................8
Swift is Built on Industry-standard Components ......................................................................8
Swift Can Be Deployed In-House or As-a-Service ......................................................................9
Using Swift............................................................................................................... 10
Swift Commands - The Basics ......................................................................................................... 10
Client Libraries...................................................................................................................................... 11
How Swift Works ..................................................................................................... 12
Building Blocks...................................................................................................................................... 12
Proxy Servers ......................................................................................................................................... 12
The Ring ................................................................................................................................................... 12
Zones: Failure Boundaries ............................................................................................................... 13
Accounts & Containers....................................................................................................................... 14
Partitions ................................................................................................................................................. 14
Replication .............................................................................................................................................. 14
How these are all tied together...................................................................................................... 15
Swift Cluster Architecture ........................................................................................ 17
Access Tier............................................................................................................................................... 17
Storage Nodes........................................................................................................................................ 18
Configuring Networking ........................................................................................... 20
Large-Scale Networking ................................................................................................................... 20
Medium-Scale Networking .............................................................................................................. 21
Management Network ....................................................................................................................... 21
2012 SwiftStack, Inc. All rights reserved.
February 2012
Hardware Recommendations................................................................................... 22
Proxy Nodes ............................................................................................................................................ 22
Storage Nodes........................................................................................................................................ 22
Networking ............................................................................................................................................. 23
Sizing Your Swift Cluster .................................................................................................................. 23
Planning a Deployment ............................................................................................ 24
Datacenter Facility Planning.......................................................................................................... 24
Integrations............................................................................................................................................ 25
Monitoring .............................................................................................................................................. 25
Administration & Maintenance ..................................................................................................... 26
Understanding TCO............................................................................................................................. 26
Rolling Swift Out in Your Organization ..................................................................................... 27
Managing Swift with SwiftStack ............................................................................... 28
Like to Learn More? ................................................................................................. 29

February 2012
Introduction
In this era of connected devices, the demands on storage systems are increasing
exponentially. Users are producing and consuming more data than ever. Social media,
online video, user-uploaded content, gaming, and software-as-a-service applications
are all contributing to the vast need for easily consumable storage systems that can
grow without bounds.
To accommodate these demands, storage systems must be able to handle web-scale
workloads with many concurrent readers and writers to a data-store. Some data is
frequently written and retrieved, such as database files and virtual machine images.
Other data, such as documents, images, and backups are generally written once and
rarely accessed. Web and mobile data assets also need to be accessible over the web
via a URL to support todays web/mobile applications. A one-size-fits-all data storage
solution is therefore neither practical nor economical.
Public cloud storage systems have risen to the task of handling web-scale workloads.
Cloud storage-as-a-services include Amazon.coms Simple Storage Service (S3) and
Rackspaces Cloud Files, which both has grown tremendously in usage. For instance, in
April 2010, Amazon.com reported that 100 billion objects were stored in S3, a 100%
growth from the previous year. In October 2011, Amazon.com reported 566 billion
objects were stored in S3. Technology trend analyzers at BuiltWith.com now tracks
more than 60,000 websites that serve content directly from S3.
However, not every organization will utilize a public storage cloud such as S3 for cost,
regulatory, or control reasons. With the OpenStack Object Storage system, aka Swift,
there is now an open-source option for organizations needing a highly durable object
storage system that is deployed on industry-standard hardware in their own
datacenter. These systems can serve as the core for a private storage cloud or public
storage-as-a-service offerings.
The purpose of this white-paper is to help those who are considering deploying an
object storage system based on OpenStack Swift and complements the official Swift
documentation which is available at http://swift.openstack.org. Not every topic related
to getting Swift up and running in your environment is covered in this white-paper, but
it provides an overview of the key areas to be aware of, including what Swift is, how it
works, how to deploy, manage and monitor Swift with the SwiftStack Platform, how to
use Swift and some general deployment considerations.
February 2012
Introducing Swift
Swift is a multi-tenant, highly scalable and durable object storage system that is
designed to store large amounts of unstructured data at low cost. Highly scalable,
means that it can scale from a few nodes and a handful of drives to thousands of
machines with multiple Petabytes of storage. Swift is designed to be horizontally
scalablethere is no single point-of-failure.
Swift is used by Fortune 500 enterprises, web companies and service providers
worldwide. It is typically used to store unstructured data such as documents, web
content, backups, images and virtual machine snaphots. Originally developed as the
engine behind RackSpace Cloud Files, it was open-sourced under the Apache 2 license
under the OpenStack project in 2010. With now more than 100 companies and
thousands of developers participating in the OpenStack project, the usage of Swift is
increasing rapidly.
Swift is not a traditional file system or a raw block device. Instead, it enables you to
store, retrieve and delete objects (with its associated metadata) in containers
(buckets in Amazon S3 terminology) via a RESTful HTTP API.

.

Developers can either write directly to the Swift API or use one of the many client
libraries that exist for all popular programming languages, such as Java, Python, Ruby
and C#. Amazon S3 and RackSpace Cloud Files users should feel very familiar with Swift.
For users who have not used an object storage system before, it will require a different
approach and mindset than using a traditional filesystem.
Benefits of Swift for developers include:
Data can be directly served over the Internet

RESTful HTTP interface
Access to storage in minutes, not days
One multi-tenant storage system for all your apps
February 2012
Focus on app development, not infrastructure plumbing

A rich ecosystem of tools and libraries

Benefits to IT operations teams include:
Use low-cost, industry-standard servers and disks

Manage more data and use cases with ease
Enable new applications quickly
Highly durable architecture with no single-point of failure
Swift Characteristics
Swift cant be mounted like a folder in your operating system. There is no random
access within a files content and there can be multiple concurrent writers, which
makes it unsuitable for transactional applications such as traditional relational
databases for which a Storage Area Network (SAN) or Networked Attached Storage
(NAS) system may be a better fit. Also, since object storage systems dont provide raw
data blocks that an operating system can form into a filesystem, Swift is unsuitable for
booting an operating system.
The key characteristics and benefits of Swift include:
All objects have a URL

All objects have their own metadata
Developers interact with the object storage system through a RESTful HTTP API
Object data can be located anywhere in the cluster
The cluster scales by adding additional nodes without sacrificing performance,
which allows a more cost-effective linear storage expansion vs. fork-lift upgrades
Data doesnt have to be migrated to an entirely new storage system
New nodes can be added to the cluster without downtime
Failed nodes and disks can be swapped out with no downtime
Runs on industry-standard hardware, such as Dell, HP, Supermicro etc.
Swift is Scalable
To support thousands of concurrent users, todays application architects must take
advantage of the latest in distributed architectures, using distributed noSQL databases
(CouchDB, Cassandra, MongoDB), distributed message / queuing systems (ActiveMQ,
RabbitMQ) and distributed processing systems like Hadoop. To that end, application
architects need their storage system to scale along with their application.
Available space isnt a useful statistic on its own. A key benchmark is the storage
systems concurrency. The ability to handle a great number of simultaneous
February 2012
connections from within a datacenter or across the web is critical to satisfy the needs
of applications that are built for web scale usage.
Swift is designed to have linear growth characteristics. As the system grows in usage
and the number of requests increase, performance doesnt degrade. To scale up, the
system is designed to grow where needed by adding storage nodes to increase
storage capacity, adding proxy nodes as requests increase, and growing network
capacity where choke points are detected.
Swift is Extremely Durable

Built on an architecture similar to Amazon S3, which claims 99.999999999% durability,
Swift is extremely durable. To achieve this level of durability, objects are distributed in
triplicate across the cluster. A write must be confirmed in two of the three locations to
be considered successful. Auditing process run to ensure the integrity of data.
Replicators run to ensure that a sufficient number of copies are in the cluster. In the
event that a device fails, data is replicated throughout the cluster to ensure that three
copies remain.
Another feature is the ability to define failure zones. Failure zones allow a cluster to be
deployed across physical boundaries, each of which could individually fail. For example,
a cluster could be deployed across several nearby data centers, enabling it to survive
multiple datacenter failures.
The servers that handle incoming API requests scale up just like any front-end tier for a
web application. The system uses a shared-nothing approach and employs the same
proven techniques that have been used to provide high availability by many web
applications.
Swift is Open Source Software

Swift is licensed under the permissive Apache 2 open source license. What makes Swift
different from most other open source projects is that the software has already been
stressed tested in a large-scale production deployment at Rackspace before its first
public release. As an open source project, Swift provides the following benefits to its
users:

1. No vendor lock-in - As an open source project, you have the option to work with a
variety of providers or as a DIY project
2. Community support You can access and share tools, best practices and
deployment know-how with other organizations and community members that
are using Swift
3.
February 2012
Large ecosystem With the large number or organizations and developers

participating in the OpenStack project, the development velocity and breadth of
tools, utilities and services for Swift will only increase over time

As the source code is publicly available, it can be reviewed by many more developers
than what is the case for proprietary software. This means that potential bugs also
tend to be more visible and more rapidly corrected than for proprietary software.
In the long term, open generally wins - and Swift might be considered the Linux of
storage.
Swift is Similar to AWS S3

Access to the Swift object storage system is entirely through a REST API, which is
similar to the Amazon.com S3 API and compatible with the Rackspace Cloud Files API.
This means that (a) applications that are currently using S3 can use Swift without major
re-factoring of the application code and (b) applications that like to take advantage of
both private and public cloud storage can do so as the APIs are comparable.
Since Swift is comparable with public cloud services, developers & systems architects
can also take advantage of a rich ecosystem of commercial and open-source tools is
available for these object storage systems. Clients such as Cyberduck, filers like Nasuni
and programming libraries which are available in C#, PHP, Perl, Python, Java, and Ruby
are just some examples.
Swift is Built on Industry-standard Components

If you look under-the-hood, Swift is built using mature, standard components such as
rsync, MD5, sqlite, memcache, xfs and python. Swift runs on off-the-shelf Linux
distributions such as Ubuntu, which is different from most other storage systems,
which run on proprietary or highly-customized operating systems.
From a hardware perspective, Swift is designed ground up to handle failures so that
reliability on the individual component level is less critical. Thus, regular desktop drives
can be used in a Swift cluster rather than more expensive enterprise drives.
Hardware quality and configuration can be chosen to suit the tolerances of the
application and the ability to replace failed equipment.
Since commodity hardware can be used with the system, there is consequently no lock-
in with any particular storage vendor. This means deployments can continually take
advantage of decreasing hardware prices and increasing drive capacity.

February 2012
Swift Can Be Deployed In-House or As-a-Service

For organizations uncomfortable with outsourcing their data storage to a public cloud
storage vendor, Swift can help achieve lower costs and similar performance while
retaining greater control over network access, security, and compliance. Cost is also a
major factor for bringing cloud storage in-house. Public cloud storage costs include per-
GB pricing plus data transit charges, which can become very expensive.
The network latency to public storage service providers may be unacceptable. A
private deployment can provide lower-latency access to storage, as required by many
applications. Also, applications may have large volumes of data in flight, which can't go
over the public Internet.
For the above reasons, organizations can use Swift to build an in-house storage system
that has similar durability/accessibility properties and is compatible with the suites of
tools available for public cloud storage systems.
Swift can also be deployed as a public storage-as-a-service. Operators and service
providers, who want to offer S3-like storage services, can get started by offering Swift
as a value-added service to their customers. With Swift, it is now cost-effective to build
and deploy an object storage cluster for public use.
Swift Is Supported
As an OpenStack project, Swift has the benefit of a rich community, which includes
more than 100 participating companies and 1000+ developers. The following support
options are available for Swift:
Commercial support and tools are available through SwiftStack, which has
experience deploying, running and supporting Swift at scale
Community support is provided through OpenStack community, where best
practices can be shared with other organizations and users that are using Swift
Swifts documentation is publicly available at http://swift.openstack.org
February 2012
Using Swift
Once deployed, all communication with Swift is done over a REST-ful HTTP API.
Application Developers whod like to take advantage of Swift for storing content,
documents, files, images etc. can use one of the many client libraries that exist for all
all popular programming languages, including Java, Python, Ruby, C# and PHP. Existing
backups, data protection and archiving applications which currently support either
Rackspace Cloud Files or Amazon S3 can also use Swift as their storage back-end with
minor modifications.
Swift Commands - The Basics

As Swift has a REST-ful API, all communication with Swift is done over HTTP, using the
HTTP verbs to signal the requested action. A Swift storage URL looks like this:

http://swift.example.com/v1/account/container/object

Swifts URLs have four basic parts. Using the example above, these parts are:

Base: swift.example.com/v1/
Account: An account is determined by the auth server when the account is

created.
Container: Containers are namespaces used to group objects within an account
Object: Objects are where the actual data is stored in swift. Object names may
contain /, so pseudo-nested directories are possible.

To get a list of all containers in an account, use the GET command on the account:
GET http://swift.example.com/v1/account/
To create new containers, use the PUT command with the name of the new container:
PUT http://swift.example.com/v1/account/new_container
To list all object in a container, use the GET command on the container:
GET http://swift.example.com/v1/account/container/

10
February 2012
To create new objects with a PUT on the object:

PUT http://swift.example.com/v1/account/container/new_object
The POST command is used to change metadata on containers and objects.When
planning a Swift deployment, the first step is to define the application workloads and
functional requirements that will determine how your Swift
Client Libraries
Several client libraries for Swift are available, including:

C#/.NET:
http://github.com/rackspace/csharp-cloudfiles
Java:
http://github.com/rackspace/java-cloudfiles
PHP:
http://github.com/rackspace/php-cloudfiles
Python:
http://github.com/rackspace/python-cloudfiles
Ruby:
http://github.com/rackspace/ruby-cloudfiles

In addition, a Ruby library is also available through the fog client:
http://github.com/geemus/fog

The Fuse client can be used to map a filesystem to Swift-
http://github.com/redbo/cloudfuse/

For more information building client libraries for Swift see:
https://github.com/rackspace/docs-cloudfiles

11
February 2012
How Swift Works

Building Blocks
The components that enable Swift to deliver high availability, high durability and high
concurrency are:
Proxy Servers: Handles all incoming API requests.
Rings: Maps logical names of data to locations on particular disks.
Zones: Each Zone isolates data from other Zones. A failure in one Zone doesnt
impact the rest of the cluster because data is replicated across the Zones.
Accounts & Containers: Each Account and Container are individual databases that
are distributed across the cluster. An Account database contains the list of
Containers in that Account. A Container database contains the list of Objects in
that Container.
Objects: The data itself.
Partitions: A Partition stores Objects, Account databases and Container

databases. Its an intermediate bucket that helps manage locations where data
lives in the cluster.
Proxy Servers
The Proxy Servers are the public face of Swift and handle all incoming API requests.
Once a Proxy Server receive a request, it will determine the storage node based on the
URL of the object, e.g. https://swift.example.com/v1/account/container/object. The
Proxy Servers also coordinates responses, handles failures and coordinates
timestamps.
Proxy servers use a shared-nothing architecture and can be scaled as needed based on
projected workloads. A minimum of two Proxy Servers should be deployed for
redundancy. Should one proxy server fail, the others will take over.
The Ring
The Ring maps Partitions to physical locations on disk. When other components need
to perform any operation on an object, container, or account, they need to interact
with the Ring to determine its location in the cluster.
The Ring maintains this mapping using zones, devices, partitions, and replicas. Each
partition in the Ring is replicated three times by default across the cluster, and the
12
February 2012
locations for a partition are stored in the mapping maintained by the Ring. The Ring is
also responsible for determining which devices are used for handoff should a failure
occur.

The Ring maps partitions to physical locations on disk.
Zones: Failure Boundaries

Swift allows zones to be configured to isolate failure boundaries. Each piece of data
resides in multiple zones. At the smallest level, a zone could be a single drive or a
grouping of a few drives. If there were five object storage servers, then each server
would represent its own zone. Larger deployments would have an entire rack (or
multiple racks) of object stores, each representing a zone. The goal of zones is to allow
the cluster to tolerate significant outages of storage servers.
As we learned earlier, everything in Swift is stored, by default, three times. Three zones
may seem sufficient for holding three copies of data, but consider the case when a
zone goes down. There would be no fourth zone into which data may be replicated,
leaving only two copies of all stored data. Therefore, it is recommended that at least
four zones and preferably five zones be deployed.
If a zone goes down, data will be replicated to other zones. Having at least five zones
leaves enough margin to accommodate the occasional Zone failure and enough
capacity to replicate data across the system.
When a disk, node, or zone fails, replica data is distributed to the other zones to ensure
there are three copies of the data
13
February 2012
Accounts & Containers

Each account and container is an individual SQLite database that is distributed across
the cluster. An account database contains the list of containers in that account. A
container database contains the list of objects in that container.
To keep track of object data location, each account in the system has a database that
references all its containers, and each container database references each object.
Partitions
A Partition is a collection of stored data, including Account databases, Container
databases, and objects. Partitions are core to the replication system.
Think of a Partition as a bin moving throughout a fulfillment center warehouse.
Individual orders get thrown into the bin. The system treats that bin as a cohesive
entity as it moves throughout the system. A bin full of things is easier to deal with than
lots of little things. It makes for fewer moving parts throughout the system.
The system replicators and object uploads/downloads operate on Partitions. As the
system scales up, behavior continues to be predictable as the number of Partitions is a
fixed number.
The implementation of a Partition is conceptually simple a partition is just a
directory sitting on a disk with a corresponding hash table of what it contains.
Swift partitions contain all data in the system.
Replication
In order to ensure that there are three copies of the data everywhere, replicators
14
February 2012
continuously examine each Partition. For each local Partition, the replicator compares
it against the replicated copies in the other Zones to see if there are any differences.
How does the replicator know if replication needs to take place? It does this by
examining hashes. A hash file is created for each Partition, which contains hashes of
each directory in the Partition. Each of the three hash files is compared. For a given
Partition, the hash files for each of the Partition's copies are compared. If the hashes
are different, then it is time to replicate and the directory that needs to be replicated is
copied over.
This is where the Partitions come in handy. With fewer things in the system, larger
chunks of data are transferred around (rather than lots of little TCP connections, which
is inefficient) and there are a consistent number of hashes to compare.
The cluster has eventually consistent behavior where the newest data wins.
If a zone goes down, one of the nodes containing a replica notices and proactively
copies data to a handoff location.
How these are all tied together

To describe how these pieces all come together, lets walk through a few scenarios and
introduce the components.
Upload
A client uses the REST API to make a HTTP request to PUT an object into an existing
Container. The cluster receives the request. First, the system must figure out where
the data is going to go. To do this, the Account name, Container name and Object
name are all used to determine the Partition where this object should live.
Then a lookup in the Ring figures out which storage nodes contain the Partitions in
15
February 2012
question.
The data then is sent to each storage node where it is placed in the appropriate
Partition. A quorum is required at least two of the three writes must be successful
before the client is notified that the upload was successful.
Next, the Container database is updated asynchronously to reflect that there is a new
object in it.
Download
A request comes in for an Account/Container/object. Using the same consistent
hashing, the Partition name is generated. A lookup in the Ring reveals which storage
nodes contain that Partition. A request is made to one of the storage nodes to fetch
the object and if that fails, requests are made to the other nodes.

16
February 2012
Swift Cluster Architecture

Access Tier
Large-scale deployments segment off an Access Tier. This tier is the Grand Central
of the Object Storage system. It fields incoming API requests from clients and moves
data in and out of the system. This tier is composed of front-end load balancers, ssl-
terminators, authentication services, and it runs the (distributed) brain of the object
storage system the proxy server processes.
Having the access servers in their own tier enables read/write access to be scaled out
independently of storage capacity. For example, if the cluster is on the public Internet
and requires ssl-termination and has high demand for data access, many access servers
can be provisioned. However, if the cluster is on a private network and it is being used
primarily for archival purposes, fewer access servers are needed.
As this is an HTTP addressable storage service, a load balancer can be incorporated into
17
February 2012
the access tier.

Typically, this tier comprises a collection of 1U servers. These machines use a moderate
amount of RAM and are network I/O intensive. As these systems field each incoming
API request, it is wise to provision them with two high-throughput (10GbE) interfaces.
One interface is used for 'front-end' incoming requests and the other for 'back-end'
access to the object storage nodes to put and fetch data.
Factors to Consider
For most publicly facing deployments as well as private deployments available across a
wide-reaching corporate network, SSL will be used to encrypt traffic to the client. SSL
adds significant processing load to establish sessions between clients; more capacity in
the access layer will need to be provisioned. SSL may not be required for private
deployments on trusted networks.
Storage Nodes

18
February 2012
The next component is the storage servers themselves. Generally, most configurations
should have each of the five Zones with an equal amount of storage capacity. Storage
nodes use a reasonable amount of memory and CPU. Metadata needs to be readily
available to quickly return objects. The object stores run services not only to field
incoming requests from the Access Tier, but to also run replicators, auditors, and
reapers. Object stores can be provisioned with single gigabit or 10 gigabit network
interface depending on expected workload and desired performance.
Currently 2TB or 3TB SATA disks deliver good price/performance value. Desktop-grade
drives can be used where there are responsive remote hands in the datacenter, and
enterprise-grade drives can be used where this is not the case.
Factors to Consider
Desired I/O performance for single-threaded requests should be kept in mind. This
system does not use RAID, so each request for an object is handled by a single disk.
Disk performance impacts single-threaded response rates.
To achieve apparent higher throughput, the object storage system is designed with
concurrent uploads/downloads in mind. The network I/O capacity (1GbE, bonded 1GbE
pair, or 10GbE) should match your desired concurrent throughput needs for reads and
writes.
19
February 2012
Configuring Networking
Below are two examples of deployments at two scales: the larger deployments with a
two-tier networking architecture, and smaller deployments with a single networking
tier.
Note that when a write comes into the proxy server, there is three times the traffic
going to the object stores to write the three replicas. Systems must be designed to
account for the expected read/write traffic.
Large-Scale Networking
Aggregation
A pair of aggregation switches with two links back to the access network / border
network are used to connect to two pools of the Access Tier and to each of the five
Zone switches that connect the Object Stores. All connections to the Access Tier and
the Zones are 10GbE.
Zone Network
Each Zone has a switch to connect itself to the aggregation network. Its possible to use
a single, non-redundant switch as the system is designed to sustain a Zone failure.
20
February 2012
Depending on overall concurrency desired, a deployment can use either a 1GbE or a

10GbE network to the object stores.
Medium-Scale Networking

A single network tier is used for smaller deployments in the range of 300-500TB. Either
1GbE or 10GbE switches can be used for this purpose depending on the throughput the
cluster is expected to sustain. The Access Tier services still contain two interfaces and a
VLAN is created for each front-facing API request and back-end network
connecting the object server Zones.
Management Network
A management network is critical to maintaining the health of the cluster. A separate
1GbE management network is created for IPMI, monitoring, and out-of-band access to
every machine in the cluster. However, it is typically possible to use the higher-
bandwidth connections during provisioning for operating system installation.

21
February 2012
Hardware Recommendations

Swift is designed to store and retrieve whole files via HTTP across a cluster of industry-
standard x86 servers and drives, using replication to ensure data reliability and fault
tolerance. While this model provides great flexibility (and low cost) from a hardware
perspective, it requires some upfront planning, testing and validation to ensure that
the hardware you select is suitable not just for Swift itself, but also for the expected
workload that you are designing your cluster for. Your operations team may also have
opinions on the hardware selection, as they prefer to work with hardware they are
already familiar with.
Proxy Nodes
Proxy nodes use a moderate amount of RAM and are network IO intensive. Typically,
Proxy nodes are 1U systems with a minimum of 12 GB RAM. As these systems field
each incoming API request, it is wise to provision them with two high-throughput
(10GbE) interfaces. One interface is used for 'front-end' incoming requests and the
other for 'back-end' access to the object storage nodes to put and fetch data. For small
Swift deployments, the storage nodes can serve as proxy nodes.
Storage Nodes
Storage nodes are typically high-density 3U or 4U nodes with 16-36 SATA disks each.
These nodes use a reasonable amount of memory and CPU. The storage nodes run
services not only to field incoming requests from the proxy nodes, but also replication,
auditing and other processes to ensure durability. Storage nodes can be provisioned
with single gigabit or 10GbE network interface depending on expected workload and
desired performance.

For storage nodes, we recommend the following specifications:

CPU 64-bit x86 CPU (Intel/AMD), quad-core or greater, running at least 2-2.5GHz

RAM A good rule of thumb is approximately 1 GB of RAM for each TB of Disk. I.e. for
a node with 24 drives, 36-48GB of RAM should be used. The memory is used for the
many processes used field incoming object requests and XFS inode caching.

Drives Either 2TB or 3TB 7200 RPM SATA drives, which deliver good
price/performance value. Desktop-grade drives can be used where there are
responsive remote hands in the data center, and enterprise-grade drives can be used
22
February 2012
where that is not the case. We dont recommend using green drives. Swift is
continuously ensuring data integrity and the power-down functions of green drives
may result in excess wear.

Extreme container update workload consideration Where the application needs to
ingest many millions of files in a single container, it may be necessary to use higher-
performing media (RAID 10 with 15k drives or SSDs) for the container indexes. The data
set is relatively very small in size, so few space is needed on the higher performing
media to store this data.

Controller Cards Swift replicates data across zones so there is no need for data
redundancy to be provided by the controller. Swift therefore uses standard SATA
controller cards without RAID, such as LSI 9211-8i 6Gb/s SAS / SATA HBA. However, if
the controller card requires RAID volumes to be created, set up a RAID 0 group
(without striping) for each drive.

Network Cards Depending on the use case, single gigabit ethernet (1GbE) on each
host may be all that is required. However, it is possible to configure bonded 1GbE or
10GbE if the workload demands it.
Networking
A typical deployment would have a front-facing access network and a back-end
storage network. When designing the network capacity, keep in mind that writes fan-
out in triplicate in the storage network. As there are three copies of each object, an
incoming write is sent to three storage nodes. Therefore network capacity for writes
needs to be considered in proportion to overall workload.
Sizing Your Swift Cluster

Each node and drive that is added to a Swift cluster will not only provide additional
storage capacity, but will also increase the aggregate IO capacity of the cluster as there
are more systems and drives available to serve incoming read requests. When selecting
a specific hardware configuration for your Swift cluster, it is therefore important to
determine which configuration provides the best balance of IO performance, capacity
and cost for a given workload. For instance, customer-facing web applications with a
large number of concurrent users will have a different profile than one that is used
primarily for archiving.

23
February 2012
Planning a Deployment

Benchmarking and Testing

When planning a Swift deployment, the first step is to identity the application
workloads and define the corresponding functional requirements, which will determine
how your Swift environment will be designed, configured and deployed. Benchmarking
goes hand-in-hand with designing the right cluster. Some of the areas to consider when
conducting your benchmarking include:

1. The spread of file sizes
2. Number of concurrent workers
3. The proportion of creation / read / update / delete that is expected

To conduct your benchmarking, a pre-populated cluster with a set of data to mimic an
eventual state should be set up. It is also important to be able to parallelize the
benchmarking workloads as it may require many systems to sustain the aggregate
throughput a Swift cluster can put out.

SwiftStack is available to assist in determining which hardware configuration is most
optimal for your workload. As part of a SwiftStack deployment we provide a
benchmarking test suite to help ensure and tune your OpenStack Swift cluster so it can
handle the workloads youre about to put on it.

A testing plan also needs to be developed which outlines how the environment will be
tested and the overall testing criteria. Some of the areas to address in the testing plan
include:

Swift API calls
Failure scenarios
Swift client libraries that will be used
Datacenter Facility Planning

The datacenter space and power requirements are critical areas to plan for a successful
Swift deployment. They require planning with your datacenter facilities team or
datacenter vendor to deploy a full rack of object stores. Be sure to meet early with this
team to explain your project and plan for:
Power provisioning
24
February 2012
Cooling
Physical space requirements
Networking capacity and planning
Port layouts
Rack layouts
SwiftStack can assist with best practices and datacenter facilities planning advice for a
successful Swift implementation.
Integrations
When getting Swift up and running in your data-center, there are several potential
integrations with 3rd party systems and services to consider, including:

Authentication system, such as LDAP

Operations support systems
Billing systems
External monitoring systems to consume SNMP polling of system information,
and SNMP traps etc.
Content Distribution Networks (CDNs)
Replication to off-site Swift clusters for additional redundancy

Each of these areas can be integrated with your Swift environment but requirements
will differ based on your specific requirements and use-case. While this white-paper
does not cover these areas in any depth, SwiftStack can provide advice and best
practices on how to integrate with 3rd party systems and services.
Monitoring
There are many tools available for application developers and IT operations teams to
measure the health of applications and servers. While many of these tools are helpful,
for a Swift cluster they may be more complex and provide more data than you need. To
ensure that users can quickly measure the overall health of their SwiftStack
environment, the SwiftStack Platform is tracking the key metrics that allow you to
quickly determine the status of your overall Swift cluster and of the individual nodes.

For the overall Swift cluster, the key metrics monitored by the SwiftStack Platform are
node CPU utilization, top 5 least free disks, disk I/O and network I/O. For individual
nodes, the same key metrics are reported, which can be used to tell the overall health
of the node. In addition, external monitoring systems can be configured to consume
SNMP polling of system information, and SNMP traps provided through the SwiftStack
platform.
25
February 2012
Administration & Maintenance

The SwiftStack Platform doesnt eliminate the need for a storage administrator but will
significantly simplify the job of administrating and maintaining a Swift environment.
Some common areas that will need to be administered and maintained include
performance tuning of the overall cluster, adding/removing capacity to the cluster,
identifying and replacing failed hardware, applying software updates etc. While the
SwiftStack Platform automates and simplifies many of these tasks, it will require some
hands-on administration. The automation of many of these tasks in through the
SwiftStack Platfrom is not just a time saver for a storage administrator, but critical to
ensuring system uptime and preventing the loss of data or putting data at risk.
Understanding TCO
Finally, when planning a deployment, understanding the total-cost of ownership for the
cluster is a critical so all direct and indirect costs are included. Costs should include:
Design/Development
Hardware
Hardware Standup
Datacenter Space
Power/Cooling
26
February 2012
Network Access
Ongoing Software Maintenance and Support
Monitoring and Operational Support
Rolling Swift Out in Your Organization

If your organization does not already have experience building applications with an
object storage systems, such as Amazon S3 and Rackspace Cloud Files, it is also critical
to train internal development and product integration teams on how to use and take
advantage of Swift. Because Swift and other object storage systems requires a different
approach to application development from a traditional file system, it is important to
start this process early. SwiftStack provides both training and a virtual training
appliance with Swift, which can be installed and use on a laptop. This enables
developers and IT operations staff to become familiar with how Swift works and how to
take advantage of some of its key benefits vis-a-vi a traditional filesystem based
approach, including:

Self-service provisioning, where developers and users can access storage in
minutes
The ability for end-users to create huge namespaces for applications without
needing to segment storage systems
Access to a wide selection of available tools and libraries that provide integration
convenience.

27
February 2012
Managing Swift with SwiftStack

SwiftStack provides the deployment, management and monitoring plane for Swift,
which:
Drastically simplifies the process of getting Swift up and running in your data-
center
Deploys Swift on nodes and configures the cluster
Enables you to start with one node and add nodes as your data grows
Provides a central management console for your Swift nodes and cluster
Monitors and alerts for issues in nodes, disks and other resources
Enables you to easily expand your cluster and tune for performance
Provides diagnostics for issues, which simplifies support and administration

The SwiftStack Platform incorporates deployment and operational best practices for
Swift and provides a single plane of glass for your entire Swift environment. To get
started with SwiftStack, the first step is to download the SwiftStack ISO consisting of:

Ubuntu
OpenStack Swift
SwiftStack Agents

28
February 2012
After logging in at the SwiftStack Platform, it will guide you through the process of
creating a new cluster, creating accounts and users, installing and provisioning Swift on
cluster nodes, formatting drives, configuring zones and the other tasks required to set
up your Swift environment. Once your Swift environment has been deployed, the
SwiftStack Platform helps provides the on-going administration, management and
monitoring of your Swift environment.

Like to Learn More?

Swift and SwiftStack offers a real alternative to proprietary object storage systems and
is much easier to use than traditional file-system based approaches. Swift is provided
under the Apache 2 open source license, is highly scalable, extremely durable and runs
on industry standard hardware. Swift also has a compelling set of compatible tools
available from third parties and other open source projects. With the SwiftStack
Platform, deployment, on-going management and monitoring can now be done with
ease.
If youd like to learn more about Swift and SwiftStack, contact us at
contact@swiftstack.com.
The SwiftStack Team, February 2012
29

Openstack Object Storage Whitepaper

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Openstack Object Storage Whitepaper

Uploaded by

Copyright:

Available Formats

The

Object Storage system

Deploying and managing a scalable, open-source cloud

By SwiftStack, Inc. contact@swiftstack.com

The OpenStack Object Storage system

The OpenStack Object Storage system

2012 SwiftStack, Inc. All rights reserved.

The OpenStack Object Storage system

2012 SwiftStack, Inc. All rights reserved.

The OpenStack Object Storage system

Data can be directly served over the Internet

2012 SwiftStack, Inc. All rights reserved.

The OpenStack Object Storage system

Focus on app development, not infrastructure plumbing

Use low-cost, industry-standard servers and disks

All objects have a URL

The OpenStack Object Storage system

Swift is Extremely Durable

Swift is Open Source Software

The OpenStack Object Storage system

Large ecosystem With the large number or organizations and developers

Swift is Similar to AWS S3

Swift is Built on Industry-standard Components

The OpenStack Object Storage system

Swift Can Be Deployed In-House or As-a-Service

2012 SwiftStack, Inc. All rights reserved.

The OpenStack Object Storage system

Swift Commands - The Basics

Account: An account is determined by the auth server when the account is

Container: Containers are namespaces used to group objects within an account

The OpenStack Object Storage system

To create new objects with a PUT on the object:

2012 SwiftStack, Inc. All rights reserved.

The OpenStack Object Storage system

How Swift Works

Proxy Servers: Handles all incoming API requests.

Rings: Maps logical names of data to locations on particular disks.

Objects: The data itself.

Partitions: A Partition stores Objects, Account databases and Container

The OpenStack Object Storage system

Zones: Failure Boundaries

The OpenStack Object Storage system

Accounts & Containers

Swift partitions contain all data in the system.

The OpenStack Object Storage system

How these are all tied together

The OpenStack Object Storage system

2012 SwiftStack, Inc. All rights reserved.

The OpenStack Object Storage system

Swift Cluster Architecture

The OpenStack Object Storage system

the access tier.

The OpenStack Object Storage system

2012 SwiftStack, Inc. All rights reserved.

The OpenStack Object Storage system

The OpenStack Object Storage system

Depending on overall concurrency desired, a deployment can use either a 1GbE or a

2012 SwiftStack, Inc. All rights reserved.

The OpenStack Object Storage system

The OpenStack Object Storage system

Sizing Your Swift Cluster

2012 SwiftStack, Inc. All rights reserved.

The OpenStack Object Storage system