This action might not be possible to undo. Are you sure you want to continue?

UF Platform and Product Development Robert Vanderwall

Date

1

It is necessary to combine knowledge born from study with sincere practice in our daily lives. These two must go together. -- Dalai Lama "Theory without practice is empty; Practice without theory is blind." -- Immanual Kant Theory + practice = Expertise

2

Code Camp 2012

Topics

Performance Engineering Arrivals Throughput Queuing models Practical Implications

3

Code Camp 2012

Topics

Performance Engineering Arrivals Throughput Queuing models Practical Implications

4

Code Camp 2012

Performance Engineering

Establish a target

Response time < 1.0 seconds for 90% of users under 1000 user load.

**Model the system
**

Single Incoming queue, N servers.

**Budget the time to each component
**

10 mSec network traversal, 50 mSec data query, etc.

Estimate response using known parameters, queuing theory. Measure the actual system Adjust

5

Code Camp 2012

Topics

Performance Engineering Arrivals Throughput Queuing models Practical Implications

6

Code Camp 2012

Arrivals

MEASURE P XS MEANING Arrival Rate ± Arrivals per second. Service Time ± Time to actually process a request.

7

Code Camp 2012

**Arrivals Uniform arrivals
**

Each 1/P seconds, exactly 1 request arrives. Each request takes exactly XS seconds to complete. Example: P = 1/sec. and XS = 0.5 seconds.

XS

1/P

8

Code Camp 2012

Uniform Arrivals

Increase Arrival rate from P= 1/sec. to P= 1.5/sec. Period goes from 1 sec. to 0.667 sec.

XS

1/P

9

Code Camp 2012

Uniform Arrivals

Continue to increase Arrival rate to P= 2.0/sec. Period goes from 1 sec. to 0.5 sec.

This is the absolute best case

XS

Pnom = XS

1/Pnom

10

Code Camp 2012

Uniform Arrival

Increase Arrival rate beyond P= 2.0/sec. and requests overlap.

** If there is no queue, then requests are lost
**

Dropped

Serviced

11

Serviced Code Camp 2012

**Uniform Arrival Past nominal with queue
**

Add a queue, and requests get queued instead of lost.

Queued

Queued

12

Code Camp 2012

**Uniform Arrival Past nominal with queue
**

Queue will continue to grow without bounds

1 2 3 4 5 6 7 8 9 1011 1213 141516 1 2 3 4 5 6 7 8 9 10 0 1 1 1 2 2 2 2 2 3 average queue size

13

Code Camp 2012

Non-Uniform arrivals

Fortunately, in most real conditions, the arrivals are not uniform. That is, there are bursts of activities and lulls in activities. During the bursts, the queues fill and during the lulls, the queues are emptied.

14

Code Camp 2012

**Typical Arrival Distribution
**

Arrivals often follow a Poisson distribution, not a uniform distribution. P(N) = e-PPN/N! The expected number of arrivals in a time period T is PT.

15

Code Camp 2012

Poisson distribution

Below is a plot for P= 3. For a given interval, there is about a 5% probability that no 0.3 0.224042 requests arrive about a 15% chance that 1 will arrive, 0.2 about 22% chance that 2 will arrive 0.3 0.224042 about 22%(n) p chance for 3. There is an almost 0% chance that 9 or 10 will arrive. 0.2 0.1

pn ( )

4 8.101512 10

0.1

4 8.101512 10

0 0 0

0 0 0

2

4 n

6

8

10 10

10 10

2

4 n

6

8

16

Code Camp 2012

Takeaways

Defined Arrival rates and service time. If arrivals are too quick:

Drop requests Utilization may actually reduce Queues fill and overflow

Arrivals are typically not uniform.

17

Code Camp 2012

Topics

Performance Engineering Arrivals Throughput Queuing models Practical Implications

18

Code Camp 2012

**Canonical Throughput Graph
**

Suppose a system can process a request within S seconds (service time) and N requests arrive each second (arrival rate). When S < 1/N, the arrival rate is low enough that the system can process each request as it arrives.

In a one second period, N requests will arrive and they will all be serviced within that time. This yields a throughput of N requests per second. As N increases, the throughput is equal to the arrival rate (within a multiplicative constant for units), or T = N.

This is depicted in the linear portion of the graph.

19

Code Camp 2012

Canonical Throughput Graph

Throughput

T = N region

T=1/S

User Request (N)

20

Code Camp 2012

**Canonical Throughput Graph
**

If the arrival rate exceeds 1/S, then the system has saturated and the throughput is now constant regardless of increases in N.

21

Code Camp 2012

Canonical Throughput Graph

Throughput

T = constant region

T=1/S

User Request (N)

22

Code Camp 2012

Canonical Throughput Graph

Throughput

User Request (N)

Ideal Throughput graph

23

Code Camp 2012

Realistic Throughput

Several things prevent a real curve from looking like the ideal curve. First, the actual peak throughput will not quite meet the ideal expectations due to many different effects. Second, as the number of requests increases, other effects come into play as well.

Things such as thrashing, resource contention, and bottlenecks also reduce the realized throughput as N increases beyond the saturation point. The end result is the elephant graph as shown in the next graph.

24

Code Camp 2012

More realistic graph

Throughput

³Knee´ of the graph

User Request (N)

25

Code Camp 2012

Other measures

Client Bandwidth, MB/s

7 6 5 4 3 2 1 0 0 10 20 30 40 50 60 70 1.6 1.4 1.2 1 0.8 0.6 0.4 0.2 0 0 10 20 30 40 50 60 70

Client Response Time, s

While these are common, and somewhat useful, they don¶t give the insight that the aggregate bandwidth graph does.

26

Code Camp 2012

Aggregate Throughput

27

Code Camp 2012

Partial graph

**What can we say about this graph?
**

Throughput

User Request (N)

We did not push the system into saturation. i.e., didn¶t find the knee. We need to test with larger values of N. Code Camp 2012

28

Takeaways

Plot the aggregate throughput. This gives you the most insight into how the system behaves. If you don t see a clear knee, continue to increase N. Aggregate throughput = total number of transactions/sec.

29

Code Camp 2012

Topics

Performance Engineering Arrivals Throughput Queuing models Practical Implications

30

Code Camp 2012

**Queuing Theory - Vocabulary
**

MEASURE U P XS Xr XR Xt N MEANING Utilization Arrival Rate Service Time ± Time to actually process a request Resident Time ± Total time in system Response Time Think Time Requests in the system (queued)

31

Code Camp 2012

Three (or 4) Identities

N =Ptr (Little s Law) U = P ts (Utilization) tr = ts/(1-U) (Response) Useful rearrangement: ts = tr/(1+ P tr)

20 18 16 14 12 Tr ( U ) 10 8 6 4 2 0 0 10 20 30 40 50 U 60 70 80 90 100

32

Code Camp 2012

**Queuing Theory - Identities
**

Little¶s Law <N> = P Xr - Average number of requests in the system is the arrival rate multiplied by the resident time. Example: When P is 10 arrivals per second, the average queue size if 5. If the rate increases to 25, what will the queue size be?

**Xr = 0.5 seconds ± N = 25 * 0.5 = 12.5 ± Our queue needs to be at least 13
**

± 5 = 10 Xr

33

Code Camp 2012

**Little s Law: N =Ptr
**

Example: A restaurant can seat 100 people. People take about 1 hour for dinner.

tr = 1 hour, N = 100 people. P = N/ tr = 100p/h.

**If there are 15 people in queue ahead of you, how long is the wait?
**

P = 1.67 p/minute = 1P / 37 seconds. T = N/ P = 9.25 minutes

34

Code Camp 2012

Example 2

We know the current population is approximately 300 million and the life expectancy is about 78.

N = 300M, tr = 78 years

**Can we say anything about the birth rate?
**

P = N/ tr = 300M/78 = 3.8 million births per year.

Actual is 4.2 million.

Not stable, N will grow.

35

Code Camp 2012

Utilization

Suppose it takes 500 mSec (0.5 sec) to service a request and we get 1 request per second. What utilization will we see? U = P ts = 1 * 0.5 = 50% Suppose we get 3 requests per second? U = P ts = 3 * 0.5 = 150% What does this mean?

We ve got more than one server (CPU core) OR We are dropping requests.

36

Code Camp 2012

**Response Time Function
**

tr = ts/(1-U)

20 18 16 14 12 Tr ( U ) 10 8 6 4 2 0 0 10 20 30 40 50 U 60 70 80 90 100

37

Code Camp 2012

Response time

Suppose we have a load of 50 requests per second and the service time is 15 mSec. What is the expected response time? U = P ts = 50 * 0.015 = 0.75 (75%) tr = ts/(1-U) = 0.015/(1 0.75) = 0.060 = 60 mSec. What if we add a second server? U = 37.5% tr = ts/(1-U) = 0.015/(1 0.375) = 0.024 = 24 mSec.

38

Code Camp 2012

Takeaways

With only three equations, you can do a lot of analysis:

N =Ptr (Little s Law) (Utilization) U = P ts tr = ts/(1-U) (Response)

39

Code Camp 2012

Topics

Performance Engineering Arrivals and Little s Law Throughput Queuing models Practical Implications

40

Code Camp 2012

**Simple Client-server example
**

Client Request Server Network Latency - XL Processing time - XS Response Network Latency - XL

Transport Time - XT Response Time - Xr

Xr = XL + XS + XL + XT

NOTE: This is only a model, and hence it is wrong, but still useful.

41

Code Camp 2012

**Not so Simple case Single processor, wide transport pipe.
**

Client Request Processing time - XS Processing time - XS Server

Total Response Time - Xtr

Xtr = XL + XS + XS + XL + XT

42

Code Camp 2012

**Not so Simple case Two processor, wide transport pipe.
**

Client Request Processing time - XS Server

Total Response Time - Xtr

Xtr = XL + XS + XL + XT

43

Code Camp 2012

Multi-server

Suppose we have 4 cores. And, 1 request takes 100 mSec. How long will it take for 2 requests? for 4 requests? for 5 requests? for 8 requests?

44

Code Camp 2012

Multiple cores

We expect that multiple cores will multiply the throughput by the number of cores. If we can get 200 transactions per second with one core, we expect roughly 800 transactions per second with 4 cores. If we don t get that, what do we need to look for?

Another bottleneck!!

45

Code Camp 2012

Bottlenecks

There are essentially only 4 bottlenecks: CPU Memory Disk Network

46

Code Camp 2012

Takeaways

Latency often dominates when there are few arrivals or they are serialized. CPU tends to dominate when there are many arrivals in parallel. To a point, when other resources become bottlenecks.

Memory Disk Network

47

Code Camp 2012

Recap

48

Code Camp 2012

Takeaways

Defined Arrival rates and service time. If arrivals are too quick:

Drop requests Utilization may actually reduce Queues fill and overflow

Arrivals are typically not uniform.

49

Code Camp 2012

Takeaways

Plot the aggregate throughput. This gives you the most insight into how the system behaves. If you don t see a clear knee, continue to increase N.

50

Code Camp 2012

Takeaways

With only three equations, you can do a lot of analysis:

N =Ptr (Little s Law) (Utilization) U = P ts tr = ts/(1-U) (Response)

51

Code Camp 2012

Takeaways

Latency often dominates when there are few arrivals or they are serialized. CPU tends to dominate when there are many arrivals in parallel. To a point, when other resources become bottlenecks.

Memory Disk Network

52

Code Camp 2012

53

Code Camp 2012

Sign up to vote on this title

UsefulNot usefulPerformance Theory You Can Use
Miami Code Camp
Presentation by Robert Vanderwall
Performance Theory that can be applied to Software Development / Engineering and IT infrastructure.
*Poste...

Performance Theory You Can Use

Miami Code Camp

Presentation by Robert Vanderwall

Performance Theory that can be applied to Software Development / Engineering and IT infrastructure.

*Posted with permission by author.

Miami Code Camp

Presentation by Robert Vanderwall

Performance Theory that can be applied to Software Development / Engineering and IT infrastructure.

*Posted with permission by author.

- SILVACO.pdf
- Circuit Maker User Manual
- CPU diag.txt
- Image Install Servce
- Ccmt2013 Cutoff List Round1
- lecture3.pdf
- Puneet
- Pamplet -1
- Opl Tutorial
- Project
- Clase 01. Introduccion. 2x2 ---
- SJ-20120319104909-008-ZXUR 9000 UMTS (V4[1].11.20) Ground Parameter Reference.pdf
- Circular
- Architecture DSP
- Aspen Basic Engineering
- Consolidated CO-PO EE
- 2012 FE Student Course Info
- Beau Randall Moreau - Resume
- Introducing Innovation in Mechanical Engineering Education- The Project of the Engineering for Industry Faculty of the Politecnico Di Milano.
- Sun_server_cmd.doc
- c_me_ee_civil
- Jaguar Land Rover Graduate Brochure
- Solution to Brainteaser No. 5
- Brand Knowledge of Wipro Company
- Otc19115 Rosa Fpso
- Mathematics III November Am Rr220202
- Concept of ME
- HSCC LTD
- Readme
- Corp_ENG_v3
- Performance U Can Use