e-PG Pathshala

Subject : Computer Science

Paper: Computer Networks

Module: Application layer protocols - HTTP
Module No: CS/CN/3

Quadrant 1 – e-text

We have had an overview of why and how the networking function is organized in terms of
layers and protocols across the end systems and the different networking devices. We
have also briefly looked at the performance requirements and parameters associated with
reporting performance. Now, it is time to start examining the different protocols at each of
the layers in a fair amount of detail. Since, we use a number of network applications, you
are likely to be familiar with their working. Hence, we will start with understanding of the
protocols involved in such applications. We start with application layer protocols, and then
go down the network protocol stack, taking a top-down approach. The objectives of this
module are as follows.

Learning Objectives

 To understand application layer protocols, application architectures and their

requirements

 To study the details of Hyper Text Transfer Protocol (HTTP) used for web-browsing

3.1 Application architectures

An interesting aspect of developing networking applications is that we only need to write
the software at the end systems, and not at the core. It is after all the applications running
on the end systems that are talking to each other. Since they are running on different
machines, they typically employ a message-passing approach to communicate with each
other. What kind of messages are passed, and who initiates the process vary depending
on the application, and the architecture it uses.
As we have seen in the introductory module, we know that there are predominantly, two
architectures as far as networking applications are concerned, viz., client-server
architecture, and Peer-to-peer (P2P) architecture. We also have a third model, which is
actually a combination of these two called the hybrid architecture, i.e., hybrid of client-
server and P2P.
In the Client-server architecture (Fig. 3.1), the server is an always-on host which
responds to requests from many clients. The server should have the bandwidth and the
processing power to handle many requests simultaneously. For this purpose of scale,
often a number of servers grouped together as a server farm are used. In this architecture,
clients may be intermittently connected with the server, and get their requests serviced.
The clients do not communicate directly with each other. Web browsers sending request to
web servers are a typical example of this architecture.

Figure 3.1 Client-Server architecture

In a Peer-to-Peer architecture (Fig. 3.2), there is no concept of an always-on server.
Instead, arbitrary end systems which are peers directly communicate with each other.
Peers are connected intermittently as and when required. Any peer may request
information, and get its response from a connected peer. This is highly scalable, but being
a purely distributed architecture, is difficult to manage. An example of a P2P application is
the file-sharing application, Gnutella, where, a node contacts a set of peer nodes, to find
the file that a user needs. There is no central server, hence the peers talk to each other
and locate the file they are looking for and share it.

Figure 3.2 P2P model

In the hybrid model, you will see features of both the client-server and the P2P models.
The old Napster music (file) sharing application is a good example of this approach.
Though it was predominantly P2P in the sense that peers served each other, a central
server was used to store the location of the files. Thus a peer would first contact the
server, identify the peer where the file it is seeking is located, and then contact that peer
directly for file transfer. So you can see that it uses a hybrid architecture. Some chat
applications also work in a similar fashion. You log into a server to find the friends who are
logged in, get their location or IP addresses, and then connect to the friend in a p2p
fashion for the chat session. Of course, if a log of the chat session is to be maintained at
the server, then all the chat content will have to go through the server (as in the case of
google chat for instance), and it becomes more of a client-server model if the messages
are relayed through the server.
3.2 Application interface to the network
We know that the application layer sits right above the transport layer, and that it is
through the transport layer that it accesses the network. At the sending end, the transport
layer takes data from different applications and sends it to the network layer. At the
receiving end, the transport layer, gets the data from the network layer, and delivers it to
the right application.
The IP address is used to arrive at the correct destination node. But how do we identify the
right application at that node? The transport layer uses what are called as port numbers
to identify applications. Port numbers (16-bit numbers) are the ids for the application
processes for networking purposes. Many common server applications have standard,
well-known port numbers. For example, HTTP server runs at port number 80. This is done
to enable any client process to directly talk to these servers.
The application programmers interface to the transport layer happens through a structure
called ‘socket’. It is through these sockets that the application processes communicate
with the network stack below (Fig. 3.3).

Figure 3.3 Socket Interface

Now, we know that there are two different protocols available at the transport layer in the
TCP/IP network stack. So, which transport service does an application use? The answer to
this is in identifying the needs of the application. In the previous module, we have identified
certain parameters that describe the type or quality of service that an application may
need. Recall them to be – bandwidth or throughput, delay or latency, and integrity or data
loss. Different applications have different requirements with respect to these parameters.
Some applications (eg., multimedia) require a certain minimum guaranteed throughput,
while some (non-realtime) can manage with whatever is available (elastic). Some (eg., IP
telephony, interactive games) need the timing or delay to be within certain limits, while
others (eg. File transfer) can tolerate an arbitrary amount of delay. Similarly, some (web
access or file transfer) require 100% reliable data transfer, while some can tolerate a
certain amount of loss (eg., video). Thus, applications can be time-sensitive, loss-
sensitive, or throughput sensitive. We also use the terms delay-tolerant, and loss-tolerant
to describe the applications.
The table (Table 3.1) below gives the requirements for some of the common applications.
Table 3.1 Application QoS Requirements

Application Data loss Throughput Time sensitive

File transfer no loss elastic No

E-mail no loss elastic No

Web documents no loss elastic No

Real-time loss-tolerant audio: 5kbps-1Mbps yes, 100’s msec

audio/video
video:10kbps-
5Mbps

Stored audio/video loss-tolerant same as above yes, few secs

Interactive games loss-tolerant few kbps up yes, 100’s msec

Text messaging no loss elastic yes and no

Given these requirements, the applications can choose between TCP and the UDP
protocols available at the transport layer. The TCP protocol provides reliable service, while
the UDP protocol does not provide any sort of guarantees. But the UDP protocol is faster.
Hence for applications that are delay-sensitive, but loss-tolerant, UDP is a good choice.
Applications which require guaranteed delivery, i.e., loss-sensitive, but delay-tolerant
would go for TCP.
Shown in Table 3.2 is the choice of the transport protocol used by common applications.
Table 3.2 Applications and Transport protocols

Application Application Underlying

layer protocol Transport protocol

e-mail SMTP [RFC 2821] TCP

 remote terminal access Telnet [RFC 854] TCP

Web HTTP [RFC 2616] TCP

file transfer FTP [RFC 959] TCP

streaming multimedia HTTP (e.g., YouTube), TCP or UDP

RTP [RFC 1889]

Internet telephony SIP, RTP, proprietary TCP or UDP

(e.g., Skype)

Over and above these requirements, security may be another aspect that the applications
need. They may not want the data they send to be sent as plain-text throughout the
network, they may require encryption services from the network. Both UDP and TCP do
not provide this service, but there is an SSL (Secure Socket Layer) protocol which
provides encrypted TCP connections. SSL is actually an application layer protocol, which
can be used by other applications to get secure data transfer. We will look at this in later
modules.

3.3 Application Layer Protocols

Application layer protocols specify all details of the exchange between the two ends. They
specify :

 Types of messages exchanged, e.g., request & response messages,

 Syntax of message types: what are the fields in the messages & how these fields
are delineated,

 Semantics of the fields, i.e., meaning of the information in the fields, and

 Rules for when and how processes send & respond to messages
We will examine some application layer protocols, and it will become clear as to how this is
achieved.
Table 3.2 gives the application layer protocols used by some of the common applications.
Note that, in Table 3.2, we have mentioned a term called RFC. RFCs (Request For
Comments) are open standards for protocols available at (https://www.ietf.org/rfc.html), as
defined by the IETF (Internet Engineering Task Force). There are over 7000 RFCs
describing various network related protocols!
The protocols in general may be public-domain protocols as defined in the RFCs or
proprietary. The advantage of public-domain or open protocols is that they allow for
interoperability, and further enhancements.
We will now look at the details of one of the most common protocols – the Hyper Text
Transfer Protocol (HTTP) used for information retrieval on the web.

3.4 Web and HTTP

HTTP is a protocol that is used to access web pages. So let us start with what a web page
is! A web page consists of objects, which could be a HTML file, JPEG image, Java applet,
audio file, video file, etc. A web page is organized as a base HTML-file which includes
several referenced objects. Each object is addressable by a URL (Uniform Resource
Locator). A URL consists of a host name that refers to the machine or system in the web,
and a path name that gives the path on that machine where the resource is located. An
example URL of the form, www.someschool.edu/someDept/pic.gif, can be interpreted as
having a host name – www.someschool.edu, and a path name – someDept/pic.gif (object
is a gif image).
Thus retrieving a web page involves retrieving the base file, and each of the objects
referenced in that file. Let us look at how HTTP allows you to do that.
3.4.1 HTTP overview
HTTP is the web’s application layer protocol. It uses the client/server model. The client is
the browser (IE, Firefox, Chrome, Safari etc.) that requests web objects, receives them
and “displays” them. The server is the web server (implemented using Apache, IIS etc.)
that sends objects in response to requests. Multiple clients can simultaneously talk to the
server (Fig. 3.4). There are two versions of HTTP that have been developed, version 1.0
given by RFC 1945, and HTTP 1.1 given by RFC 2068. Version 1.1 is the one that is
currently prominent.

Figure 3.4 HTTP clients and server

Since web access required 100% data integrity, it uses TCP. There may be some delays,
but we are often willing to tolerate the delays as long as the data retrieved is not lost or
corrupted. Also, HTTP can tolerate variations in bandwidth. If you access from a low
bandwidth link, you anyway know that it is going to take longer. So it uses TCP.
TCP is a connection-oriented mechanism, i.e, it opens a logical connection with the other
end before data transfer takes place. The HTTP server runs at the well-known port number
80. Hence, a HTTP client first initiates a TCP connection (by creating a TCP socket) to the
server at port 80. This obviously means that the server would already have opened a
socket at port 80, and will be waiting for requests to come from the clients. Once the
server accepts the TCP connection request from the client, HTTP messages (application-
layer protocol messages) are exchanged between the browser (HTTP client) and the Web
server (HTTP server).
It is interesting to note that HTTP is a “stateless” protocol. What this implies is that HTTP
does not keep track of what messages or objects have been previously requested. Every
HTTP request is treated as a fresh request, and the response is given by the server. This
may appear very inefficient, (which it is to an extent), but it has a major advantage in terms
of scalability. Also, there are additional features that have been added to HTTP to
compensate for this inefficiency. If state were to be maintained, handling large number of
clients as in the case of most web-servers would become too heavy and complex. So, to
keep things simple HTTP has adopted a state-less approach, and this decision has proved
itself, with the growth of the world-wide-web.
3.4.2 HTTP connections
We said that a web page could consist of many objects referenced in a base file.
Therefore, as many HTTP requests have to be sent to the server, to retrieve all the
objects. Do we request all these objects with a single TCP connection or do we need many
TCP connections? Here, HTTP provides certain options. Basically, it gives two options –
one called ‘non-persistent HTTP’, and the other called ‘persistent HTTP’. In non-
persistent HTTP, at most one object is sent over a TCP connection. Once an object is
retrieved, the TCP connection is closed. To request every other object in that page, a
separate TCP connection is opened, object retrieved and the connection is closed. This
apparently is not very efficient, but this was the default mode in HTTP version 1.0. To
improve on this a persistent mode has been introduced. In this mode, once you have
opened a TCP connection with a server, the same connection can be used to request all
the objects referenced in the base page in that server, one after another. We reduce the
multiple TCP connection establishment overhead in this mode. HTTP version 1.1 uses this
as the default mode of operation.
Even, in the persistent mode, response can be speeded up if all the requests are sent
back to back – in a pipelined manner, instead of waiting for a response for a previous
object before sending the request for the next object. This is referred to as ‘persistent
HTTP with pipelining’.
Let us consider an example of retrieving a file which has three jpg objects, and which is
located at URL www.someSchool.edu/someDepartment/index.html. Let us enumerate the
steps involved in each of the cases – non-persistent, persistent (non-pipelined) and
persistent with pipelining.
(i) Non-persistent HTTP
 The steps involved at the client and the server are given below in Table 3.3.
Table 3.3 Steps involved in Non-persistent HTTP

Client Server

1 (a). HTTP client initiates TCP connection

to HTTP server (process) at
www.someSchool.edu on port 80

1 (b). HTTP server at host

www.someSchool.edu waiting for TCP
connection at port 80. “accepts”
connection, notifying client.

2. HTTP client sends HTTP request

message (containing URL) into TCP
connection socket. Message indicates that
client wants object
someDepartment/index.html

3. HTTP server receives request message,

forms response message containing
requested object, and sends message into
its socket.

4. HTTP server closes TCP connection.

5. HTTP client receives response message

containing html file, displays html. Parsing
html file, finds 3 referenced (jpeg) objects.
Client closes the connection.

Steps 1-5 repeated for each of the 3 jpeg objects

(ii) Persistent HTTP (Non-pipelined)

The steps involved at the client and the server are given below in Table 3.4.
Table 3.4 Steps involved in Persistent HTTP non-pipelined

Client Server

1 (a). HTTP client initiates TCP connection

to HTTP server (process) at
www.someSchool.edu on port 80

1 (b). HTTP server at host

www.someSchool.edu waiting for TCP
 connection at port 80. “accepts”
connection, notifying client.

2. HTTP client sends HTTP request

message (containing URL) into TCP
connection socket. Message indicates that
client wants object
someDepartment/index.html

3. HTTP server receives request message,

forms response message containing
requested object, and sends message into
its socket.

4. HTTP client receives response message

containing html file, displays html. Parsing
html file, finds 3 referenced (jpeg) objects.

5. HTTP client sends HTTP request

message for object1 into TCP connection
socket.

6. HTTP server receives request message,

forms response message containing
object1, and sends message into its
socket.

Steps 5-6 repeated for the next 2 jpeg objects.

Client displays the full page.
The TCP connection is closed.

(iii) Persistent HTTP (Pipelined)

The steps involved at the client and the server are given below in Table 3.6.
Table 3.6 Steps involved in Persistent HTTP (pipelined)

Client Server

1 (a). HTTP client initiates TCP connection

to HTTP server (process) at
www.someSchool.edu on port 80.

1 (b). HTTP server at host

www.someSchool.edu waiting for TCP
 connection at port 80. “accepts”
connection, notifying client.

2. HTTP client sends HTTP request

message (containing URL) into TCP
connection socket. Message indicates that
client wants object
someDepartment/index.html

3. HTTP server receives request message,

forms response message containing
requested object, and sends message into
its socket.

4. HTTP client receives response message

containing html file, displays html. Parsing
html file, finds 3 referenced (jpeg) objects.

5. HTTP client sends 3 HTTP request

message for objects 1 to 3 respectively
into TCP connection socket, (back-to-
back).

6. HTTP server receives the 3 request

messages, forms 3 corresponding
response messages containing objects 1
to 3 respectively, and sends messages into
its socket.

7. Client receives the three objects and

displays the full page.

The TCP connection is closed.

On comparing the three, it can easily be seen that the persistent connection with pipelining
is the most efficient option, and current browsers implement this option. To speed up
performance even in the case of non-persistent connection, sometimes parallel
connections are used. That is, once the base file or main page is retrieved, the three
objects in it could be requested in three parallel connections, thus reducing the overall
time.
3.4.2.1 Response time modeling
It would be more insightful to quantify the response time in each of these cases. Since it is
difficult to estimate the actual time taken given the vagaries of the network, we use Round-
Trip Time (RTT) as a unit of measure to relatively grade the different options. We define
RTT as the time taken for a small packet to travel from client to server and back. The
response time (Fig. 3.5) can then be calculated as follows.
Response time for non-persistent HTTP :
Response time has the following components :
(i) One RTT to initiate TCP connection.
(ii) One RTT for HTTP request and first few
bytes of HTTP response to return
(iii) File transmission time
Time for base file = 2RTT+transmit time
(iv) Repeat components (i) to (iii) for each
referenced object.
Total time = 4 x (2RTT+transmit time).
Figure 3.5 HTTP response time

Response time for Persistent HTTP with pipelining

Response time (Fig. 3.6) has the following components :
(i) One RTT to initiate TCP connection.
(ii) One RTT for HTTP request and first
few bytes of HTTP response to return
(iii) File transmission time
Time for base file =
2RTT+transmit time
(iv) One RTT for fetching the three objects
+ transmit time
Total time = 3RTT+transmit
times.

Figure 3.6 HTTP Persistent response time

It can be clearly seen that persistent HTTP with pipelining gives the fastest response time.
3.4.3 HTTP message formats
We will now look at the formats of the messages used in HTTP, and understand how the
different fields are interpreted. HTTP messages are in ASCII format and hence in human-
readable format. Thus interpreting them becomes very easy. There are two types of HTTP
messages: request, and response.
HTTP Request : The general format of a HTTP request message is given in Figure 3.7.
 Figure 3.7 HTTP request message: general format
The message consists of a mandatory request line followed by an arbitrary number of
optional header lines. This forms the header of the message, which ends with a cr+lf
characters appearing on a single line. The body of the message (which has the actual
content) follows this header. We are not going into an exhaustive list of all fields and their
options. We will discuss a few so that you get the general idea of the protocol design.
The request line consists of the method, the URL, and the version number. The method
field determines the action to be taken at the server. GET, POST and HEAD are three
common methods found in both versions of HTTP. GET is used to request an object from
the server, POST to send content to the server (the content is given in the body of the
message), and HEAD is used to get only the header lines leaving out the actual requested
object from the response. Other methods in version 1.1 include PUT, DELETE,
CONNECT, OPTIONS, and TRACE. The purpose of these methods is given in Fig. 3.8.

Figure 3.8 HTTP Request Methods

Web pages often includes form input. To upload form input, two methods can be used.
The obvious one is to use the post method, where the input in entity body is uploaded to
the server. Another way of doing this especially when the content to be uploaded is small
is to use the GET method itself, and specify the content to be uploaded as part of the URL
field of the request line. You would have seen this technique being used in many search
engines. For example, a search for monkeys and banana from somesite.com would have
the following URL with a GET method - www.somesite.com/animalsearch?
monkeys&banana.
An example of a typical HTTP request message is given in Fig. 3.9. Since the message is
in plain text, it is easy to guess and understand the purpose of the various fields.

Figure 3.9 HTTP Request Message Example

HTTP Response : The general format of a HTTP response message is given in Fig.3.10.

status-line HTTP-Version SP Status-Code SP Reason-Phrase CRLF

general-headers
response-headers
entity-headers
empty-line
message-body
message-trailers
Figure 3.10 HTTP Response format
The first line in the response gives the status, in terms of a status code and a message.
The status code is a three-digit integer, and the first digit identifies the general category of
response:
1xx indicates an informational message only
2xx indicates success of some kind
3xx redirects the client to another URL
4xx indicates an error on the client's part
5xx indicates an error on the server's part
Some sample status codes that you would have encountered are given below.
200 OK request succeeded, requested object later in this message
301 Moved Permanently requested object moved, new location specified later in this
 message (Location:)
400 Bad Request request message not understood by server
404 Not Found requested document not found on this server
505 HTTP Version Not Supported
An example HTTP response message is given below in Fig. 3.11.

Figure 3.11 HTTP response example

The header lines are easy to interpret. We will leave it as an exercise for you to figure
them out.
3.4.3 Playing HTTP client
Since HTTP is a plain ASCII based protocol, it is easy to send the HTTP request
messages to a web server, by telneting into the server.
1. Telnet to your favorite Web server. For example, telnet cis.poly.edu 80. You are
connected to the server running on that machine.
2. Type in a GET HTTP request.
GET /~ross/ HTTP/1.1

Host: cis.poly.edu

And press enter twice to complete the header and the request message. This is a minimal
but complete request message.
3. Look at response message sent by HTTP server and displayed on your terminal!
3.4.4 Other features of HTTP
In addition to the basic operation of HTTP that we have discussed, HTTP has several
headers that help improve the versatility and performance of the web-browsing
experience. Since HTTP is stateless, a concept called cookies is used to provide some
notion of state. Many major Web sites use cookies to track their users, their choices,
behavior etc. Cookie is actually a small file that is used to identify the user, and is
maintained by the browser. This identity is used by a back-end database at the server to
track the user. HTTP provides two header lines to exchange this cookie information.
1) Set-Cookie header line of HTTP response message
2) Cookie header line in HTTP request message
If a user accesses Internet always from the same machine, cookies can be used. When a
user visits a specific e-commerce site for the first time, and when the initial HTTP request
arrives at site, the site creates a unique ID and creates an entry in backend database for
that ID. It then sends this ID in the form of a cookie using the set-cookie header line in the
response message. This information is stored by the browser on the users machine.
Subsequently, whenever a request is sent from the same machine to the said server, the
browser checks if a cookie has been set, and if it has been set it sends that information in
the request using the cookie header line. This id is then used to check the back-end
database and user-specific service is provided.
Thus cookies help to provide a notion of state, even though HTTP is stateless.
Similarly, HTTP provides support for caching of content, so that we do not have to
redundantly fetch static content. We will look at these in more detail in the next module.
3.5 Summary
To summarize, the following are the topics we have discussed in this module.
• Application Types & requirements
• Choosing the Transport protocol
• WWW & HTTP
• Persistent & Non-persistent connections
• Request & response messages
As an exercise, we would urge you to read the RFCs of the HTTP protocol, both for
curiosity, and deeper knowledge.