Welcome to Scribd. Sign in or start your free trial to enjoy unlimited e-books, audiobooks & documents.Find out more
Standard view
Full view
of .
Look up keyword
Like this
0 of .
Results for:
No results containing your search query
P. 1
Buff Goldberg Speeding Up TCP

Buff Goldberg Speeding Up TCP

Ratings: (0)|Views: 7|Likes:

More info:

Published by: Syed Ahtasham ul haq on Dec 27, 2010
Copyright:Attribution Non-commercial


Read on Scribd mobile: iPhone, iPad and Android.
download as DOC, PDF, TXT or read online from Scribd
See more
See less





Web Servers Should Turn Off Nagle to Avoid Unnecessary200 ms Delays
Robert Buff, Arthur Goldberg
Computer Science DepartmentCourant Institute of Mathematical ScienceNew York University{buff, artg}@cs.nyu.eduwww.cs.nyu.edu/cs/faculty/artg
We show that the silly window syndrome (SWS) avoidance algorithms in standard implementations of TCP significantly slow the Web protocols HTTPS and HTTP in certain circumstances. Substantial delaysof several 100 ms may occur on fast Intranet transactions that might otherwise complete in a few tens of milliseconds.We illustrate this performance bug with TCP packet traces from test programs and production Websystems. This bug is easily and reliably avoided by disabling Nagle’s algorithm at the sender on everyconnection.
Current TCP implementations deliver high bandwidth when transmitting large segments. Less attentionhas focused on the response time of TCP transactions that exchange smaller segments. However, thisresponse time is important because widespread applications, like the Web, employ such transactions.
2.1TCP Revie
TCP supports a reliable, full duplex, network transport byte stream. TCP usually packetizes applicationdata into segments that fit into a single IP packet. The largest TCP segment that can be sent on aconnection can hold maximum segment size (MSS) bytes, and is called a MSS segment.HTTPS and HTTP are client/server protocols that use TCP. Typically, a client/server interaction (or transaction) consists of a small request message sent by the client to the server and a response messagesent back. Depending on the implementation, an application message is sent by just one or multiple TCPsocket writes. TCP may map application data arbitrarily to segment boundaries.A TCP receiver acknowledges receipt of data by sending the sender the sequence number of the nextexpected byte. In addition, a receiver manages buffer space by advertising an available ‘window’ beyonddata it has received.
2.1.1The Silly Window Syndrome
As described in RFC 1122 [Braden 89] and Section 13.29 of [Comer 95], early TCP implementationsexhibited a problem known as the silly window syndrome (SWS). In SWS a connection reaches a steadystate in which each acknowledgement advertises a small window and each data segment carries a smallamount of data. SWS occurs, for example, when the receiver repeatedly reads just one byte from aconnection with no advertised window.The TCP standard [Braden 89] requires both senders and receivers to incorporate algorithms that avoidSWS. In brief, a receiver avoids advertising small TCP windows and delays transmittingacknowledgements. A sender implements the Nagle algorithm, which delays transmission of partiallyfilled segments until all previously transmitted data has been acknowledged. For more detail on SWSavoidance, we review the TCP specification.
Submitted to “Protocols for High Speed Networking ‘99”
2.1.2The TCP Specification
The TCP specification describes how the receiver and sender avoid SWS. Section states that atthe receiver 
 A TCP SHOULD implement a delayed ACK, but an ACK should not beexcessively delayed; in particular, the delay MUST be less than 0.5seconds, and in a stream of full-sized segments there SHOULD be an ACK for at least every second segment.
Therefore a receiver can always delay acknowledging a partial segment.Section says that
 A TCP MUST include a SWS avoidance algorithm in the receiver. […] Thereceiver's SWS avoidance algorithm determines when the right windowedge may be advanced; […]
For realistic receive buffers (greater than twice the MSS) window advances are announced in incrementsof MSS.Section, “When to Send Data” says that
 A TCP MUST include a SWS avoidance algorithm in the sender. […] A TCPSHOULD implement the Nagle Algorithm [Nagle 84] to coalesce shortsegments. However, there MUST be a way for an application to disablethe Nagle algorithm on an individual connection. […]The Nagle algorithm is generally as follows: If there isunacknowledged data […] then the sending TCP buffers all user data […]until the outstanding data has been acknowledged or until the TCP cansend a full-sized segment […]
If the receiver delays acknowledgements, and the application writes less than MSS to the socket, andNagle is enabled, then sending TCP delays transmission.The specification also says
To avoid a resulting deadlock, it is necessary to have a timeout toforce transmission of data […].
but in all traces we collected, the delayed acknowledgement appears to timeout before the Naglealgorithm.
2.2HTTPS and HTTP Performance Problems
In several situations HTTPS and HTTP trigger SWS avoidance in both the sender and receiver, therebycreating substantial delays. The application layer situations were the following:
HTTPS / SSL key exchange, new and reused session key:
The server writes two small messagesand blocks waiting for response; the browser reads both messages and responds.
HTTPS / SSL key exchange, reused session key:
Same situation, but with directions reversed. Thebrowser writes two small messages and blocks waiting for response; the server reads both messagesand responds.
HTTP image (GIF), smaller than MSS:
The server sends the HTTP response in two small separatewrites, containing headers and body (image data), respectively.All three cases lead to the same TCP situation. The sender transmits the first message in a separatesegment, then waits for its acknowledgement. It transmits the second message in a separate segmentwhen the acknowledgment arrives. The receiver receives the first segment, but delays theacknowledgment because the segment is partial and the window available to advertise is less than MSS.Eventually, a time-out triggers the acknowledgment, thus causing the sender to send the secondsegment.2
The avoidable delay is determined by the delayed ack time-out. As documented by [Microsoft 97] and our measurements Win32 TCP delays acks by typically 200 ms. Our measurements indicate that Sun’sSolaris delays acks by about 45 ms, on average.
3Related Work
Network designers understood that delayed acknowledgements may slow application communications, asdescribed in [Stevens 98], [Microsoft 98] and [Sun 98].[Heidemann 97] discusses this problem in persistent HTTP. He states that the problem does not occur inHTTP, versions 1.0 and earlier, but we find it does.Microsoft acknowledges the problem [Microsoft 97] and [Microsoft 97], but indicates that it only occurswhen making small sends.[Nielsen 98] and [Nielsen 97] measure the cumulative performance of a set of accesses to arepresentative Web site. In [Nielsen 98] enabling Nagle in an HTTP/1.1 server slows performance:Situation
Nagle0.480.27NoNagle0.450.21However, the client ran on a Digital Alpha station 400 4/233, UNIX 4.0a, rather than Win32 whichconcerns us.
4The lab experiments
4.1A simple client-server test application
Before discussing production cases, we analyze the performance behavior of a simple client-server laboratory application that triggers the bug. The test application runs 100 identical transactions. Eachtransaction consists of the exchange of a 10-byte client request and a 20-byte server response. The 20-byte response is written to the socket by two 10-byte
calls. No computational overhead isinvolved. All communication is
strictly sequential 
: the client only initiates a subsequent transaction after the entire 20-byte server response has been received.We ran the same test application on Win32, Solaris and Linux, in different client/server combinations. Thesource code remained unchanged for all systems. The application uses the standard Berkeley socketinterface. There is no delay between the writes. The resulting fragmentation in the application layer at theserver is maintained in lower layers.In half of our tests, the Nagle algorithm was activated. In the other half, Nagle was deactivated. To turnNagle on or off, the
system call was used with the
4.2Experimental setup
Table 1lists the clients and servers used in our experiments. All computers except Win98 are connectedto the same 100Mbit Ethernet. Win98 is connected to a 100Mbit Ethernet separated from the others byone router. Network congestion was insignificant during our experiments. IP segment traces werecollected with Network General's NetXRay network monitoring tool. The traces appear complete andaccurate.3

You're Reading a Free Preview

/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->