Understanding Flash SSD Performance

Douglas Dumitru CTO EasyCo LLC August 16, 2007 DRAFT

Flash based Solid State Drives are quickly becoming popular in a wide variety of applications. Most people think of these solid state devices as just another hard disk, but their performance characteristics can be very different. o start this discussion off, lets see a table of drive performance parameters for a couple of hard disk drives and a couple of Flash SSDs.
Drive Model Description Seek ime rack to rack (ard Drives %estern Digital %D)*++!,,S Seagate S 3/1)*4SS Flash SSDs ranscend S2:;F-11 )-++ #.M /.*0 S! ! 4*9 #.M -.*0 S!S 2:6 -11< ;F ;ard !verage Full Stroke 4-.+ ms 5.- ms *.+ ms8 -.+ ms "atency #ead $F# #ate &uter racks 2* M67sec 'nner racks %rite $F# #ate &uter racks 'nner racks 1+ M67sec8

+.1 ms 2.3 ms +.- ms -.3 ms +.+3ms +.45ms +.4-*ms

1+ M67sec8 2* M67sec

44- M67sec )3 M67sec 5+ M67sec *4 M67sec 12 M67sec

44- M67sec )3 M67sec /- M67sec -2 M67sec 5+ M67sec

Samsung /-: -.*0 M;!=>/-:*!.. .! ! Sandisk S! !*+++ 8 Figure is an estimate /-: -.*0 S! !

&f course there are other hard drives and Flash SSDs on the market, but this mi< is a good starting point to work with.

Read Performance
he write performance of Flash SSDs is very different from the read performance. So we will talk about read and write performance separately. %hen comparing read performance between hard disks and Flash SSDs the specs are often misleading. Flash SSDs have much better access times, but typically are slower in terms of transfer rates. his means that you have to consider the block si?e when evaluating read performance. (ere is a chart that shows the average read performance for these five drives at varying block si?es@

Drive Read Performance

70.00 65.00 60.00 55.00 50.00 45.00 40.00 35.00 30.00 25.00 20.00 15.00 10.00 5.00 0.00 4 8 16 32 65 128 256 512 1024

ran!fer Rate )M*,!ec+

WD 7200 RPM Seagate 15K RPM ran!cend 8" #$ Sam!%ng 32" P& & SanDi!' 32" S& &

ran!fer Si(e )K*+

his is actually a very telling chart. 't shows that Flash SSDs far outperform hard disks unless the transfer si?e is very large. >ven a ;F card outperforms a 4*9 S!S drive for block si?es of -*19 or less. But I Work With Really Large Files ! lot of people underestimate how prevalent small '& operations are. "ets say you have a file server with word processing files on it. he average file si?e might be -*+9. So what will your average '& si?e beA 'n all probability, the average read si?e will be somewhere around 4++9 and the average write si?e will be even lower around *+9. he reason for this is that there are disk blocks in use that are not used for the fileBs data. he file system needs directory entries, free space tables, and other structure to maintain consistency. Some file systems even have backup copies of critical elements. So reading the -*+9 will likely involve /C* small reads before the file is even located. %riting the file is similarly involved e<cept that many file systems will do small writes even if you donBt actually save the file. For e<ample, "inu< will write the ! 'M> D!ccess imeE just because you read the file even if you never update it. !nd this all assumes that your disk is not fragmented and that the file is written to 4++F linear space. Flash SSD Write Performance 't is with write performance that Flash SSDs become problematic. he issue here is the internal structure used within the Flash storage array. his structure includes a collection of bytes called an Gerase block0. %hen you write to a Flash SSD, the drive itself cannot just update the sectors you are changing, but must merge your changes with e<isting data to update a complete erase block. !s Flash SSDs have gotten faster and larger, erase blocks have grown as well. Flash erase blocks used to be 419 in length. How they are 4 Megabyte for small SSDs e<tending up to as large as 5 Megabytes for some models. 't is because of these large erase blocks that Flash SSDs are so slow at random writes. performance of our three e<ample drives is@
Drive ranscend 2: ;F Samsung /-: .! ! SanDisk /-: S! ! #andom %rites7sec 5) -5 4/

he Grandom write0

,our first reaction to this chart should be GthatBs terrible0. ,ou are right. hese numbers are generally accurate for small random block writes. !s the block si?e starts to approach the erase block si?e, you will start to get the stated linear write speed of the drive.

How Writes Im act !"erall Performance 6ecause Flash SSDs have write performance that is so much worse than their read performance, the overall performance mi< with reads and writes can be confusing. 'f you are doing pure reads, a Flash SSD will typically be -+< faster than a hard disk for small random reads. 'f you are doing pure random writes, the same drive might be 4*< slower than a hard disk. 6ut what about a *+C*+ mi<. ,ou might think that the performance would balance out, but you would be wrong. (ere are rough ratios for 59 operations with various read7write percentages Dthis table is for a SanDisk S! !*+++ driveE.
F %rites +F *F 4+F -+F *+F 4++F otal '&.S *5++ -*4/+ 1* -1 4/ .erformance vs 4*9 S!S (ard Drive -+< better 4.-*< better 4.*< worse /< worse 2< worse 41< worse

his shows how even a very small percentage of writes can destroy the overall performance of a Flash SSD. 't is for this reason alone that Flash SSDs, by themselves, are not very effective with random update applications like onCline databases, mail queues, and other environments that involve a lot of small updates.

Im ro"ing Write Performance

here are a couple of techniques that can improve write performance. Hot all of these are available to everyone though. !S Write #aching ,ou can turn write caching on with most operating systems. his will let the system buffer writes. his can make a drive Gappear0 to write faster, but the drive will have to do the actual writes eventually. 'f you are dealing with databases or are concerned about file system corruptions, &S write caching is not considered a good choice. Flash S ecific File Systems Many devices that are designed to use Flash storage use Flash optimi?ed file systems. File Systems like IFFS and ,!FFS are optimi?ed to minimi?e random writes. hey also are designed to minimi?e the possibility of drive wearC out. Fortunately, with current Flash SSDs, the drives high endurance numbers of 4,+++,+++ write7erase cycles combined with large capacity means that drives will last over a decade before wearCout is even an issue. Dri"e Write #aching Some drives have internal memory that will buffer writes. his has the same effect as &S write buffering but is usually protected against power failures with batteries or capacitors that will push writes to flash. 'n general the effectiveness of write caching is limited because write intensive applications tend to overrun the cache pool anyway. ><amples of drive write caching are the S >; MachC2 and the e<as Memory System Flash #amSanC*++. $ulti le #oncurrent %rase Blocks he MachC2 and MS #amSanC*++ also feature internal arrays of drives so there are multiple erase blocks that can be manipulated in parallel. his pushes the sustained write rate of the S >; MachC2 to about 2++ writes7sec, which is a lot better than a Gtraditional0 drive.

Jnfortunately, the cost of these solutions is really high. %here a Gcommodity0 Flash SSD might be K-*7:6, the #amSanC*++ is about K5++7:6 and the S >; MachC2 is e<pected to be over K-++7:6. $assi"e Flash !"er #ommit and #aching he S >; '&.S series of drives go a step further. hey have almost double the internal Flash storage as their stated capacity, lots of memory D*4- M6E, and a #'S; processor. his lets them run 4+,+++ write '&.S, although it is unclear if this is only a *4- byte number. 'f it is, then the 59 rate might be quite a bit lower. Jnfortunately, these drives were recently quoted at K42,+++ for a /-:6 drive or over K*++7:6. Dri"e Block Re$a ing

>asy;oBs patent pending Managed Flash echnology DMF E involves realCtime remapping the data layout of the drive dynamically. his is currently implemented as a software layer inside of the host operating system. his allows the use of commodity drives while achieving random write performance that utili?es about 2+F of the drives available linear write bandwidth. his lets a ranscend ;F card write at ),+++ 59 writes7sec or -2 M67sec. ! single SanDisk S! ! drive writes at over 2,+++ 59 writes7sec.

SSD Write Performance -it. and -it.o%t M$

65.00 60.00 55.00 50.00 45.00 40.00 35.00 30.00 25.00 20.00 15.00 10.00 5.00 0.00 4 8 16 32 65 128 256 512 1024
Seagate 15K RPM ran!cend /ative Write Sam!%ng /ative Write SanDi!' /ative Write ran!cend M$ Write Sam!%ng M$ Write SanDi!' M$ Write

ran!fer Rate )M*,!ec+

ran!fer Si(e )K*+

&he Im act of R'ID 'rrays Flash SSDs have standard disk drive interfaces and are easy to use with hardware or software #!'D. For reads, the #!'D arrays tend to scale just like disk drives. 'f you have an efficient controller or software setup, you should see linear improvements based on the number of drives. &ne e<ample we have tested here is@ 5 Samsung .! ! drives 5 .! ! to S! ! bridge adapters (ighpoint .;'e 5 port raid adapter setup #!'DC* >lectrical tape to hold this mess together #ed (at >nterprise "inu< *.+ 7 <21 7 /-Cbit #andom read performance single thread is about /1++ 59 reads7sec. %ith -+ threads, this jumps to about 45,+++ 59 reads7sec or *1 M67sec. "inear read speed ma<es out at about 45+M67sec. his implies that the .;'e 4< controller is hitting an internal bandwidth limit. he drives should theoretically deliver about -++ M67sec. ,ou should note that 45,+++ random read '&.S would take an array of )+ 4*9 S!S drives. &n writes, the linear write performance of the array is about 2+ M67sec. heoretical would have been 25 M67sec, so this is actually quote good. he random write rate ends up at about /* writes7sec with multiple threads. his shows how #!'DC* and random writes donBt really go together. 'f the array were #!'DC+, you would e<pect a 5 drive random write rate of about 3+ writes7sec. %ith #!'DC4+, you would e<pect a rate of about 5* writes7sec. "arger arrays would scale somewhat with #!'DC*, but would scale linearly with #!'DC+ or #!'DC4+.

'f you take this same array and run it through the >asy;oBs Managed Flash echnology mapping layer, the random write performance goes up to about 4),+++ 59 writes7sec or almost *++< the performance of the bare drives running #!'DC*. his chart shows 5 SanDisk S! ! drives using the MF management layer comparing performance to 5 4*9 S!S hard drives, both running #!'DC*.

R&0D15 Performance 1 SanDi!' SSD -, M$ v! 15K S&S

250 225

ran!fer Rate )M*,!ec+

200 175 150 125 100 75 50 25 0 4 8 16 32 65 128 256 512 1024

42SanDi!' M$ Read 42SanDi!' M$ Write 4215K Read 4215K Write

ran!fer Si(e )K*+

'n order to equal the small block read and write performance of a 5 drive #!'DC* SanDisk SSD array with MF , you would need a raidC4+ array of about 2+ 4*9 S!S drives. he SSDs would draw about / watts of power while the S!S array would need nearly a kilowatt. #onclusion( Flash SSDs are rapidly developing with performance that already eclipses hard disk drives in many aspects. >ach generation of Flash SSDs tend to get denser and faster. Solutions to Flash limitations, including poor random write performance, are finally coming to market. ogether, these insure an increase in adoption of Flash storage solutions in performance critical enterprise servers. ')out the 'uthor( Doug Dumitru is ; & of >asy;o ""; and the lead inventor of the GManaged Flash echnology0 block device random write optimi?ing software. (e has been working with large, centrali?ed database servers for -+ years. (e is coCowner of >asy;o "";, a privately held company with offices in .ennsylvania and ;alifornia. Mr. Dumitru can be reached at dougLeasyco.com or 14+ -/)C-+++ <5/.