SOLARIS™ ZFS AND VERITAS STORAGE FOUNDATION

FILE SYSTEM PERFORMANCE
White Paper June 2007

Sun Microsystems, Inc.

Table of Contents
Executive Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 File System Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Solaris™ ZFS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Simplified Storage Device and File System Administration. . . . . . . . . . . . . . . . . . . 2 Pooled Storage and Integrated Volume Management . . . . . . . . . . . . . . . . . . . . . . 2 Strong Data Integrity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Immense Capacity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Veritas Storage Foundation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 File System Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 Benchmark Tests and Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Hardware and Software Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Filebench . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Filebench Test Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 IOzone File System Benchmark . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 IOzone Test Results. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 For More Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

1

Executive Overview

Sun Microsystems, Inc.

Chapter 1

Executive Overview
The Solaris™ 10 06/06 Operating System (OS) introduces a major new data management technology — the Solaris ZFS file system. Replacing the traditionally separate file system and volume manager functionality found in most operating environments, Solaris ZFS provides immense capacity and performance coupled with a proven data integrity model and simplified administrative interface. The Veritas Storage Foundation — consisting of the Veritas Volume Manager and Veritas File System — provides a set of integrated tools for storage management. This white paper explores the performance characteristics and differences of Solaris ZFS and the Veritas File System through a series of tests using the new Filebench benchmarking framework which reproduces the I/O patterns of applications, as well as the popular IOzone benchmark which tests specific I/O patterns. Figure 1-1 illustrates the differences between Solaris ZFS and the Veritas File System in a number of tests. In many cases, Solaris ZFS performs better at the initial release. In some cases Solaris ZFS does not perform as well — but in almost all cases Solaris ZFS performs differently. These results, as well as supporting testing data described in this document, strive to give organizations sufficient detail on the differences between Solaris ZFS and the Veritas File System so that intelligent decisions can be made about when each technology could or should be used. Indeed, the results presented here can help enterprises reach performance goals, if the goals can be quantified.
2

1.5

1

0.5 Solaris ZFS

0

-0.5

-1 Create Copy Delete Create-alloc Create-sync Create-fsync Create-append Rand-read Rand-read cached Rand-write-sync Rand-write-sync-4thread Seq-read Seq-read cached Seq-write Seq-write dsync Seq-write rand Fileserver Varmail Webproxy Webserver OLTP 2 KB cached OLTP 8 KB nocache OLTP 2 KB cached OLTP 8 KB nocache

Figure 1-1. Filebench testing summary for the Veritas Storage Foundation versus Solaris ZFS

Solaris ZFS decouples the namespace from physical storage in much the same way that virtual memory decouples address spaces from memory banks. there is little pressure to simplify and speed such administrative tasks. or remove it from. As more users handle system administration tasks. logical device creation. Simplified Storage Device and File System Administration In many operating systems. Such a scheme is counterintuitive — file systems are intended to virtualize physical storage. Solaris ZFS storage administration is automated to a greater degree. but is quick and intuitive when needed. Inc. Pooled Storage and Integrated Volume Management The Veritas Storage Foundation products make a one-to-one association between a file system and a particular storage device. Indeed. . rather than perform the constituent steps.2 File System Overview Sun Microsystems. Multiple file systems can share a pool of storage. In contrast. Mistakes are easy to make and can have disastrous consequences. or interrupting file system service. Solaris™ ZFS Solaris ZFS is designed to overcome the limitations of existing file systems and volume managers in UNIX® environments. Administrators can add storage space to. disk partitioning. the file system is assigned to a specific range of blocks on the logical storage device. locking. an existing file system without unmounting. Because these relatively uncommon tasks are only performed by system administrators. such as make a new file system. Chapter 2 File System Overview This chapter provides a brief technical introduction to Solaris ZFS. it can no longer be assumed that users have undergone specialized training. and highlights the features that can impact performance. manual reconfiguration of disk space is virtually unnecessary. More information can be found in the references listed at the end of this document. Administrators simply state an intent. and new file system formatting tend to be detailed and slow operations. Using the volume manager. the Veritas File System and the Veritas Volume Manager. and yet a fixed binding remains between the logical namespace and a logical or physical device. Allocation is moved out of the file system into a storage space allocator that parcels out permanent storage space from a pool of storage devices as file systems make requests.

The Veritas Volume Manager also provides easy-to-use.1 Administrator’s Guide. File system corruption can be caused by disk corruption. • Veritas Volume Manager (VxVM) is a management tool that aims to remove the physical limitations of disk storage so that it can be configured. Unlike many of today’s file systems. system panics. online storage management tools to help reduce planned and unplanned downtime. It also provides online management capabilities that facilitate the creation and maintenance of file systems. metadata allocation. and optimized for performance without interrupting data availability. file systems have trusted the data read in from disk. or software or administrative errors. Veritas Storage Foundation The Veritas Storage Foundation consists of the Veritas Volume Manager and the Veritas File System. and storage capacity is currently doubling approximately every nine to 12 months. Solaris ZFS provides consistent on-disk data and error detection and correction. The maximum size of a Veritas File System is 32TB1. hardware or firmware failures. and does not depend on repair utilities such as fsck to maintain on-disk consistency. or more. 16 exabyte (EB) datasets may begin to emerge in only ten years.3 File System Overview Sun Microsystems. Traditionally. I/O scheduling. . File systems should validate data read in from disk in order to detect downstream data corruption. block allocation. • Designed for UNIX environments that need to support large amounts of data and require high performance and availability. shared. and correct corruption automatically. by writing the correct block back to the disk. However. However. errors can result in corrupted data. one petabyte datasets are plausible today. if possible. March 2005. On the other hand. if the file system does not validate read data. managed. and other routine operations. Validation at the disk block interface level can only catch some causes of file system corruption. Inc. Solaris ZFS uses 128-bit block addresses and incorporates scalable algorithms for directory lookup. Strong Data Integrity The file system mount options in the Veritas Storage Foundation environment often require users to make a trade-off between performance and data integrity. the Veritas File System (VxFS) provides an extent-based. Assuming the rate of growth remains the same. Immense Capacity More information on the Veritas File System can be found in the Veritas Volume Manager 4. The lifetime of a typical file system implementation is measured in decades. intent logging file system. ensuring data consistency while maintaining high performance levels. Such a validation strategy is a key design goal for Solaris ZFS.

412149120 blocks of size 8192. A Veritas File System is constructed on a volume so that files can be stored. A plex consists of one or more subdisks located on one or more physical disks.A physical disk is the basic storage device (media) upon which data is stored. Table 2-1. A volume is a virtual disk device that appears like a physical disk device.com. The pool contains all the disks in the system. A volume consists of one or more plexes contained in a disk group that each hold a copy of the selected data in the volume. More information can be found in the Veritas Volume Manager 4. Veritas Volume Manager uses subdisks to build virtual objects. • Volume. A disk group is a collection of disks that share a common configuration and are managed by Veritas Volume Manager. and is usually assigned to a disk group. called plexes.1 Administrator’s Guide Solaris. When a physical disk is placed under Veritas Volume Manager control. Solaris ZFS presents storage in pools and file systems. March 2005. Each subdisk represents a specific portion of a VM disk. • VM disk. The file system creation procedure for Solaris ZFS and the Veritas File System Solaris ZFS # zpool create -f tank [48 disks] # zfs create tank/fs Veritas File System # /usr/lib/vxvm/bin/vxdisksetup -i c2t16d0 # vxdg init dom -dg c2t16d0 # for i in [list of 48 disks] do /usr/lib/vxvm/bin/vxdisksetup -i $i done # for i in [list of 48 disks] do vxdg -g dom -dg adddisk $i done # vxassist -g dom -dg -p maxsize layout=stripe 6594385920 [get size of volume] # vxassist -g dom -dg make dom -vol 6594385920 layout=stripe [feed volume size back in] # mkfs -F vxfs /dev/vx/rdsk/dom -dg/dom -vol version 6 layout 6594385920 sectors.5 seconds . document number 817-2271-2. Inc. log size 32768 blocks largefiles supported # mount -F vxfs /dev/vx/dsk/dom -dg/dom -vol /mnt Time: 30 minutes Time: 17. • Physical disk. • Subdisk. as well as the time observed for these tasks.4 File System Overview Sun Microsystems. File System Concepts The Veritas File System presents physical storage to the application via several layers of abstraction.sun. In contrast. Table 2-1 lists the activities required to create usable storage using Solaris ZFS and the Veritas File System. a VM disk is assigned to the physical disk. A VM disk is under Veritas Volume Manager control. • Disk group. • Plex. Additional information on Solaris ZFS can be found in the Solaris ZFS Administration Guide located at docs. A VM disk can be divided into one or more subdisks. and consists of contiguous disk blocks. and can contain as many file systems as are needed. • File system.

and parameters used during testing efforts.5 Veritas File System 4. such as Mongo and Postmark. and results obtained using the Filebench tool. such as Web-based commerce and software development. Filebench Test Results The following sections present the tests run. are emulated in Filebench and are supplied as scripts. Fiber channel interface. configuration. Several popular file system benchmarks. configuration used. Table 3-1. operating system and software versions.5 Benchmark Tests and Results Sun Microsystems. Table 3-2 describes the workload.5 GHz UltraSPARC® IV+ processors and 98 GB RAM Operating System Solaris 10 OS Update 2 06/06 (Including Solaris ZFS) Storage Four Sun StorageTek 3500 arrays. these features enable the emulation of complex relational database applications and database benchmarks such as TPC1 and Web applications. Inc. and “f” language constructs to manipulate the I/O environment such as O_SYNC. . Simulations run as several multithreaded processes. 4 x 2 Gb PCI-X 133 MHz Software Veritas Volume Manager 4. and software used to create the platforms tested and perform benchmark testing.1 IOzone 3. Combined with the use of interprocess communication primitives such as POSIX semaphores.64. and benchmarks used for testing efforts. Hardware and Software Configuration Table 3-1 describes the hardware. Chapter 3 Benchmark Tests and Results This chapter describes the hardware platforms. Create and Delete File creation and deletion is a metadata intensive activity which is key to many applications. Integrated statistics provide for microsecond accurate latency and cycle counts per system call.263 Filebench Filebench is a benchmarking framework for simulating the effect of applications on file systems. operating system. Application workloads are configured as definitions in the Filebench “f” scripting language. Hardware and software configuration used for testing efforts Servers Sun Fire E4900 server with 24 1.1 Filebench 1. each with 48 x 72 GB disks. as well as the results observed.

and parameters Personality createfiles deletefiles Workload createfiles deletefiles Variables nfiles 100000. filesize 16k.6 Benchmark Tests and Results Sun Microsystems. CPU time used during create and delete testing deletefiles . Table 3-2. dirwidth 20. filesize 16k. Operations per second obtained during create and delete testing Figure 3-2 shows the CPU time used per operation during testing efforts. nthreads 16 Figure 3-1 shows the number of operations per second obtained during testing efforts. Inc. 14000 12000 Operations Per Second Veritas File System Solaris ZFS 10000 8000 6000 4000 2000 0 createfile deletefiles Figure 3-1. Create and delete workloads. nthreads 16 nfiles 100000. meandirwidth 20. 25000 20000 CPU uSec Per Operation Veritas File System 15000 Solaris ZFS 10000 5000 0 createfiles Figure 3-2. configurations.

Figure 3-3 shows the latency observed during testing efforts. filesize 16k nthreads 16). 45 40 35 30 Latency (ms) 25 20 15 10 5 0 createfiles Figure 3-3. and several parameters (nfiles 100000.7 Benchmark Tests and Results Sun Microsystems. Testing efforts used the following copyfile personality and workload. 25000 20000 Operations Per Second Veritas File System 15000 Solaris ZFS 10000 5000 0 Figure 3-4. Operations per second obtained during copy files testing . Latency observed during create and delete testing Veritas File System Solaris ZFS deletefiles Copy Files The copyfiles test creates two large directory tree structures and measures the rate at which files can be copied from one tree to the other. Inc. dirwidth 20. Figure 3-4 shows the number of operations per second obtained during testing efforts.

Latency observed during copy files testing . Inc. CPU time used during copy files testing Figure 3-6 shows the latency observed during testing efforts. 10000 9000 8000 CPU uSec Per Operation 7000 6000 5000 4000 3000 2000 1000 0 Veritas File System Solaris ZFS Figure 3-5. 18 16 14 12 Latency (ms) Veritas File System Solaris ZFS 10 8 6 4 2 0 Figure 3-6. Figure 3-5 shows the CPU time used per operation during testing efforts.8 Benchmark Tests and Results Sun Microsystems.

9 Benchmark Tests and Results Sun Microsystems. sync. The file sizes are chosen according to a gamma distribution of 1. such as open. Information on I/O can be found in the Solaris OS open(2). 4500 4000 3500 Operations Per Second 3000 2500 2000 1500 1000 500 0 alloc Veritas File System Solaris ZFS sync fsync append Figure 3-7. configuration. Inc. File Creation The file creation test creates a directory tree and populates it with files of specified sizes. sync 1 nthreads 1 filemicro_writefsync filemicro_createrand createallocfsync createallocappend nthreads 1 Figure 3-7 shows the number of operations per second obtained during file creation testing efforts. iosize 1m. count 64 nthreads 1. nthreads 1. fsync and more. File creation workloads. sync(2) and fsync(3C) man pages. Table 3-3 describes the workload. count 1k. Table 3-3. with a mean size of 16 KB.5. and parameters Personality filemicro_create Workload createandalloc createallocsync Variables nfiles 100000. iosize 1m. configurations. The different workloads used are designed to test different types of I/O. and parameters used during file creation testing efforts. Operations per second obtained during file creation testing .

10 Benchmark Tests and Results Sun Microsystems. . 2 KB random reads from a 5 GB file. and parameters used during random read testing efforts. Table 3-4 describes the workload. configuration. Inc. Latency observed during file creation testing Random Reads The random read test performs single-threaded. CPU time used during file creation testing Figure 3-9 shows the latency observed during file creation testing efforts. Figure 3-8 shows the CPU time used per operation during file creation testing efforts. CPU uSec Per Operation 140 120 100 Latency (ms) Veritas File System Solaris ZFS 80 60 40 20 0 alloc sync fsync append Figure 3-9. 160000 140000 120000 100000 80000 60000 40000 20000 0 alloc sync fsync append Veritas File System Solaris ZFS Figure 3-8.

iosize 2k Figure 3-10 shows the number of operations per second obtained during random read testing efforts. Random read workloads. Table 3-4. configurations. iosize 2k randread2kcached cached 1. CPU time used during random read testing 2 KB Cached . Operations per second obtained during random read testing Figure 3-11 shows the CPU time used per operation during random read testing efforts. Veritas File System 1200 Solaris ZFS 1000 CPU uSec Per Operation 800 600 400 200 0 2 KB Figure 3-11. Inc. and parameters Personality filemicro_rread Workload randread2k Variables cached 0.11 Benchmark Tests and Results Sun Microsystems. 6000 Veritas File System 5000 Solaris ZFS Operations Per Second 4000 3000 2000 1000 0 2 KB 2 KB Cached Figure 3-10.

5 3 2. Table 3-5.5 0 2 KB Latency (ms) 2 KB Cached Figure 3-12. Random write workloads. Table 3-5 describes the workload.5 1 0. iosize 2k iosize 2k. sync1 .12 Benchmark Tests and Results Sun Microsystems. Latency observed during random read testing Random Write The random write test consists of multithreaded writes to a single 5 GB file. nthreads 4. Veritas File System Solaris ZFS 4 3. and parameters used during random write testing efforts. configuration.5 2 1. configurations. and parameters Personality filemicro_rwrite Workload randwrite2ksync randwrite2ksync4thread Variables cached 1. Inc. Figure 3-12 shows the latency observed during random read testing efforts.

CPU time used during random write testing 2 KB Sync. Inc. Operations per second obtained during random write testing Figure 3-14 shows the CPU time used per operation during random write testing efforts. 3000 2500 Operations Per Second Veritas File System 2000 Solaris ZFS 1500 1000 500 0 2 KB Sync 2 KB Sync. Figure 3-10 shows the number of operations per second obtained during random write testing efforts. 4500 4000 3500 CPU uSec Per Operation 3000 2500 2000 1500 1000 500 0 Veritas File System Solaris ZFS 2 KB Sync Figure 3-14.13 Benchmark Tests and Results Sun Microsystems. 4 Threads . 4 Threads Figure 3-13.

configurations. Table 3-6 describes the workload. Figure 3-15 shows the latency observed during random write testing efforts. Sequential read workloads. configuration. 4 Threads Figure 3-15. cached 0. and parameters used during sequential read testing efforts. cached 1. Table 3-6. nthreads 1.14 Benchmark Tests and Results Sun Microsystems. Latency observed during random write testing Sequential Read The sequential read test performs a single threaded read operation from a 5 GB file. 2000 1800 1600 1400 Latency (ms) Veritas File System Solaris ZFS 1200 1000 800 600 400 200 0 2 KB Sync 2 KB Sync. Inc. filesize 5g . nthreads 1. and parameters Personality filemicro_seqread Workload seqread32k seqread32kcached Variables iosize 32k. filesize 5g iosize 32k.

Inc.15 Benchmark Tests and Results Sun Microsystems. CPU time used during sequential read testing Solaris ZFS 32 KB Cached . Figure 3-16 shows the number of operations per second obtained during sequential read testing efforts. Veritas File System 90 80 70 CPU uSec Per Operation 60 50 40 30 20 10 0 32 KB Figure 3-17. Veritas File System 25000 Solaris ZFS 20000 Operations Per Second 15000 10000 5000 0 32 KB 32 KB Cached Figure 3-16. Operations per second obtained during sequential read testing Figure 3-17 shows the CPU time used per operation during sequential read testing efforts.

count 128k. sync 0 filemicro_seqwriterand seqread32kcache . Table 3-7. Veritas File System Solaris ZFS 0. cached 0. count 32k. Inc. nthreads 1.06 0. and parameters used during sequential write testing efforts. count 32k.05 Latency (ms) 0.07 0.01 0 32 KB Figure 3-18. configurations. sync 0 iosize 8k. nthreads 1. sync 0 iosize 32k.02 0. nthreads 1.04 0. cached 0. Sequential write workloads. and parameters Personality filemicro_seqwrite Workload seqwrite32k seqwrite32kdsync Variables iosize 32k. Table 3-7 describes the workload.16 Benchmark Tests and Results Sun Microsystems. cached 0. Figure 3-18 shows the latency observed during sequential read testing efforts. Latency observed during sequential read testing 32 KB Cached Sequential Write The sequential write test performs single threaded write operations to a 5 GB file.03 0. configuration.

Figure 3-19 shows the number of operations per second obtained during sequential write testing efforts.17 Benchmark Tests and Results Sun Microsystems. Random Offset Veritas File System Solaris ZFS Figure 3-20. 4000 3500 3000 2500 2000 1500 1000 500 0 32 KB 32 KB dsync 32 KB. Inc. Operations per second obtained during sequential write testing Figure 3-20 shows the CPU time used per operation during sequential write testing efforts. CPU time used during sequential write testing CPU uSec Per Operation . 7000 6000 5000 Veritas File System Solaris ZFS Operations Per Second 4000 3000 2000 1000 0 32 KB 32 KB dsync 32 KB Random Offset Figure 3-19.

and 16 KB of data is appended to the log for every 10 read and write operations. found in UNIX environments. A configurable. /var/mail. • Web server The Web server script performs a mix of open/read/close operations on multiple files in a directory tree. and attribute operations on a file system. hierarchical directory structure is used for the file set. The script uses 100 threads. as well as file append operations that simulates Web server log operation.html . Latency observed during sequential write testing Application Simulation Filebench includes several scripts that emulate the behavior of applications. open/append/delete. reads. the Fileserver workload performs a series of creates. writes. 10 Veritas File System 9 8 7 Latency (ms) Solaris ZFS 6 5 4 3 2 1 0 32 KB 32 KB dsync 32 KB. • Web proxy The Web proxy script performs a mix of create/write/close and open/read/close operations. as well as the deletion of multiple files in a directory tree. Random Offset Figure 3-21.org. and delete operations in a single directory. and file append operations that simulate Web proxy log operation. appends. SPECsfs is a registered trademark of the Standard Performance Evaluation Corporation (SPEC). deletes. Similar to Postmark benchmark workloads — but multithreaded — the Varmail script consists of a multithreaded set of open/read/ close. • Fileserver The Fileserver script emulates file system workloads. Information on the SPECsfs benchmark can be found at http://spec. More information on the Postmark benchmark can be found in Postmark: A New File System Benchmark located at http://netapp.18 Benchmark Tests and Results Sun Microsystems. The script appends 16 KB of data to the Web log for every 10 reads performed. Figure 3-21 shows the latency observed during sequential write testing efforts. Similar to the SPECsfs® benchmark. Inc. • Varmail The Varmail script emulates the Network File System (NFS) mail server.com/tech_library_3022.

and parameters Personality fileserver varmail webproxy webserver Workload fileserver varmail webproxy webserver Variables nfiles 100000. Inc. meandirwidth 20. Operations per second obtained during application emulation testing . nthreads 16. configurations. meaniosize 16k nfiles 100000. meandirwidth 1000000. meaniosize 16k nfiles 100000. Table 3-8. filesize 1k. nthreads 100. nthreads 100 Figure 3-22 shows the number of operations per second obtained during application emulation testing efforts. nthreads 100. filesize 1k.19 Benchmark Tests and Results Sun Microsystems. meaniosize 16k nfiles 100000. meandirwidth 1000000. Application emulation workloads. Table 3-8 describes the workload. filesize 2k. 80000 70000 60000 Operations Per Second Veritas File System Solaris ZFS 50000 40000 30000 20000 10000 0 File Server Varmail Web Proxy Web Server Figure 3-22. meandirwidth 20. filesize 1k. and parameters used during application emulation testing efforts. configuration.

CPU time used during application emulation testing Figure 3-24 shows the latency observed during application emulation testing efforts. 30000 25000 CPU uSec Per Operation Veritas File System 20000 Solaris ZFS 15000 10000 5000 0 File Server Varmail Web Proxy Web Server Figure 3-23. 60 50 Veritas File System 40 Latency (ms) Solaris ZFS 30 20 10 0 File Server Varmail Web Proxy Web Server Figure 3-24. Inc. Latency observed during application emulation testing Online Transaction Processing Database Simulation Filebench includes tests that emulate database operations. The test performs transactions on a file system using the I/O model used in Oracle 9i Database. Figure 3-23 shows the CPU time used per operation during application emulation testing efforts. The .20 Benchmark Tests and Results Sun Microsystems.

workingset 0 large_db_oltp_8k cached 0. usermode 20000. and is sensitive to the latency of moderate (128 KB plus) synchronous write operations that are typical of a database log file. directio 0. configuration. workload tests the performance of small random read and write operations. nshadows 200. Table 3-9. iosize 2k. and parameters Personality oltp Workload Variables large_db_oltp_2k cached 1. usermode 20000. filesize 5g. Inc. _uncached ndbwriters 10. configurations. directio 1. 10 asynchronous write processes. iosize 8k. _uncached ndbwriters 10. directio 0. nshadows 200. directio 1. memperthread 1m. iosize 2k. Intimate shared memory (ISM) is used similarly to Oracle Database. iosize 8k. memperthread 1m. nshadows 200. memperthread 1m. The test launches 200 reader processes. workingset 0 oltp oltp oltp Figure 3-25 shows the number of operations per second obtained during OLTP database emulation testing efforts. usermode 20000. OLTP emulation workloads. _cached ndbwriters 10.21 Benchmark Tests and Results Sun Microsystems. workingset 0 large_db_oltp_8k cached 1. Operations per second obtained during OLTP database emulation testing . memperthread 1m. filesize 5g. and parameters used during online transaction processing (OLTP) database emulation testing efforts. usermode 20000. workingset 0 large_db_oltp_2k cached 0. filesize 5g. filesize 5g. _cached ndbwriters 10. nshadows 200. and a log writer process. Table 3-9 describes the workload. and is critical to I/O efficiency (as_lock optimizations). 12000 10000 Operations Per Second Veritas File System Solaris ZFS 8000 6000 4000 2000 0 2 KB Cached 2 KB Uncached 8 KB Cached 8 KB Uncached Figure 3-25.

CPU time used during OLTP database emulation testing Figure 3-27 shows the latency observed during OLTP database emulation testing efforts. Inc.22 Benchmark Tests and Results Sun Microsystems. Latency observed during OLTP database emulation testing . 300 Veritas File System 250 Solaris ZFS 200 Latency (ms) 150 100 50 0 2 KB Cached 2 KB Uncached 8 KB Cached 8 KB Uncached Figure 3-27. 18000 16000 14000 CPU uSec Per Operation Veritas File System Solaris ZFS 12000 10000 8000 6000 4000 2000 0 2 KB Cached 2 KB Uncached 8 KB Cached 8 KB Uncached Figure 3-26. Figure 3-26 shows the CPU time used per operation during OLTP database emulation testing efforts.

Inc. Typically.23 Benchmark Tests and Results Sun Microsystems. the IOzone file system benchmark generates and measures a variety of file operations. write. iterating over differing file sizes. re-fread. rewrite. initial write performance is lower than that of rewriting a file due to metadata overhead. record re-write. read backwards. iozone -R -a -z -b file. Figure 3-28 shows the results obtained during IOzone write testing. fread. re-fwrite. record sizes and I/O patterns. read strided. It was used to test the following I/O operations: read. IOzone Write The IOzone write test measures the performance of writing a new file. enabling readers to reproduce the results presented. • IOzone provides data in convenient spreadsheet formats for post-processing.wks -g 4G -f testile IOzone Test Results The following sections present the results of the IOzone benchmark testing efforts. IOzone write test results . fwrite. random read. and random write. IOzone File System Benchmark Available on a wide variety of systems and operating systems. IOzone was selected for testing for a variety of reasons. The IOzone command line used during testing efforts included the following arguments and options. re-read. • IOzone tests multiple dimensions of I/O. 500 450 400 Megabytes Per Second • • • • • • Veritas File System Solaris ZFS (RAID 0) Solaris ZFS (RAID Z) 350 300 250 200 150 100 50 0 4 KB • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • 16 KB 64 KB 256 KB 1 MB 4 MB 16 MB Figure 3-28. as well as tools for graphical manipulation of the output. including: • IOzone is freely available.

Figure 3-30 shows the IOzone read results obtained during testing efforts. IOzone rewrite test results IOzone Read The IOzone read test measures the performance of reading an existing file. Inc. rewrite performance is higher than the performance of writing a new file. 1600 • 1400 1200 Megabytes Per Second • • • • • • • • • • 1000 800 600 400 200 0 4 KB 16 KB • • • • • • • • • • • • • • • • • Veritas File System Solaris ZFS (RAID O) Solaris ZFS (RAID Z) • • • • • 64 KB 256 KB • • 1 MB • • 4 MB • • 16 MB Figure 3-29.24 Benchmark Tests and Results Sun Microsystems. 1200 1000 Megabytes Per Second • • • • • • • • • • • • • • • • • • • • • • • • • • • • • 800 Veritas File System Solaris ZFS (RAID 0) Solaris ZFS (RAID Z) 600 • • • • 400 200 • • • • • • • 0 4 KB 16 KB 64 KB 256 KB 1 MB 4 MB 16 MB Figure 3-30. Typically. Writing to a file that already exists requires less work as the metadata already exists. IOzone read test results . IOzone Rewrite The IOzone rewrite test measures the performance of writing a file that already exists. Figure 3-29 details the IOzone rewrite results obtained during testing efforts.

Inc. IOzone re-read test results IOzone Record Rewrite The IOzone record rewrite test measures the performance of writing and re-writing a section of a file. Figure 3-31 shows the results of the IOzone re-read test. 1800 1600 1400 Megabytes Per Second • • • • • • • • 1200 1000 800 600 400 200 0 4 KB 16 KB • • • • • • • • • • • • • • • • • • • • • • Veritas File System Solaris ZFS (RAID 0) Solaris ZFS (RAID Z) • 64 KB 256 KB • 1 MB • • 4 MB • • 16 MB Figure 3-32. IOzone Re-read The IOzone re-read test measures the performance of reading a file that was recently read. Figure 3-32 illustrates the results obtained during testing efforts. 1400 1200 1000 Megabytes Per Second • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • 800 Veritas File System Solaris ZFS (RAID 0) Solaris ZFS (RAID Z) 600 400 • • • • 200 0 4 KB 16 KB 64 KB 256 KB • • 1 MB • • 4 MB • • 16 MB Figure 3-31. Re-read performance can be higher as the file system can maintain a data cache for files read recently. IOzone record rewrite test results .25 Benchmark Tests and Results Sun Microsystems. which can be used to satisfy read requests and improve throughput.

number of disks. and more. seek latencies. such as the size of operating system cache. Inc. Efficient random write performance is important to the operation of transaction processing systems. and more.26 Benchmark Tests and Results Sun Microsystems. . System performance can be impacted by several factors. seek latencies. Figure 3-34 depicts the results of the IOzone random write test. Figure 3-33 illustrates the results obtained during testing efforts. number of disks. such as the size of operating system cache. 1400 • • • Veritas File System Solaris ZFS (RAID 0) Solaris ZFS (RAID Z) 1200 1000 • • • • • • • • Megabytes Per Second 800 • • • • • • • • • • • • • • • • • • • • • • • 600 400 200 0 • 4 KB 16 KB 64 KB 256 KB • 1 MB • • 4 MB • • 16 MB Figure 3-33. IOzone Random Read The IOzone random read test measures the performance of reading a file at random locations. IOzone random read test results IOzone Random Write The IOzone random write test measures the performance of writing a file a random locations. System performance can be impacted by several factors.

such as MSC Nastran and video editing software. Many applications perform backwards reads. Figure 3-35 shows the results obtained during testing efforts. IOzone random write test results IOzone Backward Read The IOzone backward read test measures the performance of reading a file backwards. IOzone backwards read test results • 256 KB • • • • • 4 MB 16 MB . few detect and enhance the performance of reading a file backwards. Inc. While many file systems include special features that speed forward file reading.27 Benchmark Tests and Results Sun Microsystems. 1200 1000 Megabytes Per Second • • • • • • • • • • • • • • • • • • • • • • • • • • • • • 800 Veritas File System Solaris ZFS (RAID 0) Solaris ZFS (RAID Z) 600 • 400 • • • 1 MB 200 0 4 KB 16 KB 64 KB Figure 3-35. 1800 1600 1400 Megabytes Per Second • • • Veritas File System Solaris ZFS (RAID 0) Solaris ZFS (RAID Z) • • • • • • • • • • • • 1200 1000 800 600 400 200 0 • • • • 4 KB • • • • • • • • • • • • • • • 16 KB 64 KB 256 KB • • 1 MB • • 4 MB • • 16 MB Figure 3-34.

Most file systems do not detect such behavior or implement techniques to enhance access performance. Figure 3-37 shows the test results obtained. and repeated the pattern. If an application writes small amounts per transfer. Figure 3-36 depicts the results of the IOzone strided read test.28 Benchmark Tests and Results Sun Microsystems. the buffered and blocked I/O functionality of the fwrite() routine can enhance the performance of the application by reducing the number of operating system calls and increasing the size of transfers. During the test. read for a length of 4 KB. the system read 4 KB. did a seek of 200 KB. Such a pattern is typical behavior for applications accessing a particular region of a data structure that is contained within a file. 1200 1000 Megabytes Per Second • • • • • • • • • • • • • • • • • • • • • • • • • • • • • 800 Veritas File System Solaris ZFS (RAID 0) Solaris ZFS (RAID Z) 600 • 400 • • • • • • 200 0 4 KB 16 KB 64 KB 256 KB Figure 3-36. Note the test writes a new file so metadata overhead is included in the measurement. Inc. and so on. IOzone Strided Read The IOzone strided read test measures the performance of reading a file with strided access behavior. seek 200 KB. . For example. seek 200 KB. the test might make the following types of read requests: Read at offset zero for a length of 4 KB. IOzone strided read test results • • 1 MB • • • • 4 MB 16 MB IOzone fwrite The IOzone fwrite test measures the performance of writing a file using the fwrite() library routine that performs buffered write operations using a buffer within the user’s address space.

IOzone fwrite test results IOzone Re-fwrite The IOzone re-fwrite test performs repetitive rewrites to portions of an existing file using the fwrite() interface. Inc. 450 400 350 Megabytes Per Second • • • • • • • • 4 KB • • • • • • • 300 250 200 150 100 50 0 16 KB 64 KB 256 KB 1 MB • • • • • • • • • • • • • • • • • • • • • • • • • • Veritas File System Solaris ZFS (RAID 0) Solaris ZFS (RAID Z) 4 MB 16 MB Figure 3-37. 900 800 700 Megabytes Per Second • • • • • • • • • • • • • • • • • • • • • 600 500 400 300 200 100 0 4 KB 16 KB • • • • • • • • • • • • • • 64 KB Veritas File System Solaris ZFS (RAID 0) Solaris ZFS (RAID Z) • 256 KB 1 MB 4 MB 16 MB Figure 3-38. Figure 3-38 illustrates the results obtained. IOzone re-fwrite test results .29 Benchmark Tests and Results Sun Microsystems.

the buffered and blocked I/O functionality of the fread() routine can enhance performance by using fewer operating system calls and larger transfer sizes. 800 700 600 Megabytes Per Second • • • • • • • • • • • • • • 500 400 300 200 100 0 4 KB • • • • • • • • • • • • • • • • • • • • • • • • Veritas File System Solaris ZFS (RAID 0) Solaris ZFS (RAID Z) 16 KB 64 KB 256 KB 1 MB 4 MB 16 MB Figure 3-39. Inc. Reading recently read data can result in higher performance as the file system is likely to have the file data stored in a cache. IOzone fread The IOzone fread test measures file read performance using the fread() routine.30 Benchmark Tests and Results Sun Microsystems. except that the file being read was read in the recent past. which performs buffered and blocked read operations using a buffer located in the user’s address space. If applications use very small transfers. IOzone fread test results IOzone Re-fread The Iozone re-fread test is similar to the IOzone fread test. 800 700 600 Megabytes Per Second • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • 500 400 300 200 100 0 4 KB • • • • • • • • • Veritas File System Solaris ZFS (RAID 0) Solaris ZFS (RAID Z) 16 KB 64 KB 256 KB 1 MB 4 MB 16 MB Figure 3-40. IOzone re-fread test results . Figure 3-39 shows the results obtained.

a large memory configuration. Chapter 4 Summary This white paper details the performance characteristics of Solaris ZFS and the Veritas Storage Foundation through Filebench and IOzone benchmark testing. Table 4-1. Related Web sites Description or Title Solaris Operating System Solaris ZFS Solaris ZFS Administration Guide URL sun. and enterprise-class security. can provide customers with high levels of performance for applications and network services. e-mail. the Solaris OS provides users with an attractive price/performance ratio over solutions from other vendors. combined with powerful Sun servers. availability and scalability. • Benchmarks provide general guidance to performance. The following points should be taken into consideration: • The tests were performed on a Sun Fire system incorporating powerful processors. Web server and software development. Sun’s overall enhancements in the Solaris 10 Operating System. and a very wide interface to an array of high-speed disks to ensure tat the fewest possible factors would inhibit file system performance. kernel threading implementation. The test results presented in this document suggest that in application areas such as databases.com (Document Number: 817-2271-2) . Additionally.sun. such as the virtual memory subsystem. • With low acquisition and ownership costs. For the conditions tested. For More Information More information on Solaris ZFS can be found in the references listed in Table 4-1 below. It is possible that the file system differences could be reduced on a less powerful system simply because all file systems could run into bottlenecks in moving data to the disks.31 Summary Sun Microsystems.com/solaris docs. and device drivers. integrated Sun and open source applications. proof-of-concept implementations are invaluable in supporting purchasing decisions for specific configurations and applications. Solaris ZFS can outperform the combination of the Veritas Volume Manager and Veritas File System for many workloads. Proof-of-concepts and real world testing can help evaluate performance for specific applications and services. Solaris ZFS performs better in a side by side comparison with the Veritas Storage Foundation. • A file system performs only as well as the hardware and operating system infrastructure surrounding it. Inc. As such.com/solaris sun.

Veritas File System 4.32 Summary Sun Microsystems. Netapps Tech Report 3022. March 2005.netapp. March 2005.1 Administrator’s Guide Solaris. N13110F. More information on the Veritas Storage Foundation and related topics can be found in the following: Postmark: A New File System Benchmark.com/tech_library/3022. N13073F. 1997. http://www. Inc.1 Administrator’s Guide Solaris. Jeffrey Katcher. .html Veritas Volume Manager 4.

All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International. Sun Microsystems. 4150 Network Circle. in the U. and other countries. in the U. SunWIN #505690 Lit #STWP12872-0 6/07 . Inc. © 2006-2007 Sun Microsystems.S. Sun. CA 95054 USA Phone 1-650-960-1300 or 1-800-555-9SUN (9786) Web sun. Inc.S. Santa Clara. SPECsfs is a registered trademark of the Standard Performance Evaluation Corporation (SPEC). Inc. Inc. the Sun logo. StorageTek. All rights reserved.com © 2007 Sun Microsystems.com Sun Microsystems. and other countries. Inc. UNIX is a registered trademark in the United States and other countries. Inc. and Sun Fire are trademarks or registered trademarks of Sun Microsystems. Solaris. Ltd. exclusively licensed through X/Open Company. Information subject to change without notice.Solaris™ ZFS and Veritas Storage Foundation On the Web sun. Products bearing SPARC trademarks are based upon architecture developed by Sun Microsystems.