Welcome to Scribd. Sign in or start your free trial to enjoy unlimited e-books, audiobooks & documents.Find out more
Standard view
Full view
of .
Look up keyword
Like this
0 of .
Results for:
No results containing your search query
P. 1
An Introduction to Performance Testing

An Introduction to Performance Testing

Ratings: (0)|Views: 11|Likes:
Published by raj_esh_0201

More info:

Published by: raj_esh_0201 on Mar 03, 2010
Copyright:Attribution Non-commercial


Read on Scribd mobile: iPhone, iPad and Android.
download as PDF, TXT or read online from Scribd
See more
See less





with regression testing, it seemsthat everyone today is looking for a per-formance testing tool that will take careof all of their performance testing needs.As it turns out (as with most testing), thetool is the least important piece of theeffort. Performance testing is 90% analy-sis and 10% tool implementation.However, unlike most other testing, per-formance testing is nearly impossible toperform without a tool.Almost all of the performance testingtools on the market today are based uponthe concept of simulating the traffic fromreal users with virtual users. You knowthat you need a performance test tool, buthow many virtual users do you reallyneed to test the performance of yourapplication? What defines a virtual user?What is the correlation of real users tovirtual users? If I can write a test scriptusing one virtual user that will log ontomy system as 100 different users, whatdoes a virtual user really mean? It isimportant to have the answers to thesequestions before you begin performancetesting; the purpose of this article is tohelp you understand why.Why should you even care about what avirtual user is? For one thing, almost allof the performance tools have a pricingstructure based upon the number of virtu-al users that you license. The cost of alicense for 250 virtual users is signifi-cantly less than one for 2,500 virtualusers. So, if you don’t really need 2,500virtual users, why spend the additionalmoney on both the tool and all of thehardware that you will need to supportrunning the additional virtual users?(Expect to need about one PC for each200-250 virtual users).Some companies assume that the rela-tionship of real users to virtual users is1:1. They look at the cost of the tools andthen make the decision not to do any per-formance testing, because they feel thattheir perceived risk by not doing the test-ing will cost them less than buying thetool. This can prove to be a very costlymistake, as many e-business Web siteshave found out (painfully so). In additionto saving money, understanding your realusers and their correlation to virtual userswill allow you to generate more accuratescripts and, in turn, more precise results.One of the pitfalls of performance testingis that - just like with functional testautomation - you can create really badscripts. However, in the case of perform-ance testing, you may think (and have runscripts that show) your Web site can sup-port 1,000 concurrent users, when in real-ity, ten users executing a particular func-tion may cripple your site.The performance of your system shouldbe measured in three areas:Response Time – how long does it takethe system to respond to a request for apiece of information;Throughput – how much activity can thesystem support, often measured in trans-actions or hits/second; and,Round Trip Time – how long does theentire user requested transaction take,including connection and processingtime.Creating and executing scripts using per-formance test tools is extremely easy todo. At most, they take about half a day tomaster the basic record and playback concepts. The hard part is figuring outwhat to script and then what to do withthe copious data that will be generated inorder to reach the right conclusions aboutthe true performance of your system.
Performance TestingOverview
First, let’s take a look at performancetesting in general. Functional testing is afairly easy concept to grasp and theresults are fairly concrete. Most function-al tests pass or fail, as the expected resultis very black and white. Performancetesting on the other hand is neither con-crete nor black and white. What does‘passing’mean for a performance test?Does it mean that with 300 simultaneoususers hitting a site, if the averageresponse time is less than ten seconds,then it passes? What if the maximumresponse time is 90 seconds and the stan-dard deviation is 35 seconds? Does thetest still pass? For some, passing meansthat with n concurrent users on their site,it doesn’t crash. The (first) hard part inembarking upon performance testing is todefine what you want to measure andthen what criteria you will use for deter-mining whether or not the test passes.We sat down with one client to discussexactly what their criteria were; in per-formance testing their site, what was it
An Introduction toPerformance Testing
 By Leslie Segal 
take you to get through the intersectionwhere the accident occurred) is increas-ing, but you aren’t even moving. You arenot doing anything except standing still.It is not until three of the cars in front of you have moved that you can even think about moving. At this point, your traveltime is extremely long and you start tocrawl through the intersection. This alsoexplains why when you are in car number50, by the time you pass through theintersection, you have no idea what hap-pened and why you were stuck in trafficfor so long to begin with.So what does being stuck in traffic haveto do with the performance of your appli-cation? Plenty. They are both governedby basic queuing theory. Your applicationtakes requests; if the requests come infaster than they can be processed, theapplication will start to queue them up.That’s when the properties of queues andqueuing theory take over to explain thebehavior (or non-behavior) of your sys-tem. The principles of queuing theory arebeyond the scope of this article.
Understanding YourSystem and Creating aTransaction/UserProfile
So what does all this have to do with vir-tual users? Alot. Before you can begin tocreate scripts to be executed by the virtu-al users, you need a basic understandingof your system. If you understand thehardware architecture (NTserver, UNIXserver, load balancing, firewalls, separatedatabase server, number of CPUs andmemory, etc.) of your system and theimplementation (use of proxies, indexingin the database, load balancing algo-rithms, etc.) then you can assess whereprobable bottlenecks will occur and startyour performance testing in these areas.For example, if you know that your appli-cation will be asked to perform a lot of queries, but you don’t have a lot of mem-ory in your application server, the firstscripts that you write may just be queriesto the database. On the other hand, if youknow that certain processes in your appli-cation are CPU intensive, you may wantto create scripts that kick off theseprocesses and then try and perform othertasks while the CPU is busy.Before you create the scripts to test yourperformance, you also need to understandhow your users will actually use the sys-tem. What are typical scenarios users per-form? If you already have a live Web site,using a tool like WebTrends
will pro-vide you with useful user profile infor-mation. If you aren’t sure how users willuse your application, then you need tomake some guesses based upon theirbehaviors in similar environments. JohnMusa has written many articles andbooks on the subject of creating opera-tional profiles. Operational profiles areextremely useful in helping to analyze theprobable transactions to your application.If you are also trying to determine themaximum load that your Web site willneed to support, it is often useful to visityour competitors’Web sites. They willoften boast of their traffic (especially if they are selling ad time on their Website). We have found Web sites that boastthat they received 200 million hits in thelast 30 days. That gives you a good start-ing place from which to measure.Assuming a linear distribution of those200 million hits (note that a linear distri-bution is extremely unlikely, but it’s astart), then the 200,000,000/ 30 days =6666667 hits/day = 277778 hits/hour =4630 hits/sec. While the traffic to someweb sites may be indeed be linear over aparticular period of time (8AM-8PM), anexponential, or Poisson distribution maybe more realistic.Creating the scripts involves defining ascenario, or transaction, that your userwill execute. Ascenario may be com-prised of one or more transactions and atransaction may invoke one or more hits.For example, one transaction may bedefined as loading the home page.Actually executing this transaction maygenerate 1-50 hits. (We have seen homepages that load so many GIF files thatthey literally account for 75 hits.) Next,you need to assess how many times thisscenario will be executed over a defined
Figure 1: Expected Performance ResponseFigure 2: Actual Performance Response Approaching a Bottleneck

You're Reading a Free Preview

/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->