Professional Documents
Culture Documents
Application Layer
Professor Strauss
Office: LC123
Voice: 718-260-3308
Email: fstrauss@poly.edu
2: Application Layer 1
2: Application Layer 2
Chapter 2: Application layer
2.1 Principles of 2.6 P2P applications
network applications 2.7 Socket programming
2.2 Web and HTTP with UDP
2.3 FTP 2.8 Socket programming
2.4 Electronic Mail with TCP
SMTP, POP3, IMAP
2.5 DNS
2: Application Layer 3
Chapter 2: Application Layer
Chapter goals: learn about protocols
conceptual, by examining popular
implementation application-level
aspects of network protocols
application protocols HTTP
transport-layer FTP
service models SMTP / POP3 / IMAP
DNS
client-server
2: Application Layer 4
2.0 Some network apps
e-mail social networks
web voice over IP
instant messaging real-time video
remote login conferencing
P2P file sharing grid computing
multi-user network
games
streaming stored video
clips
2: Application Layer 5
2.0 Creating a network app application
transport
network
data link
Peer-to-peer (P2P)
Hybrid of client-server and P2P
2: Application Layer 7
2.1 Client-server architecture
server:
always-on host
permanent IP address
server farms for
scaling
clients:
client/server communicate with server
may be intermittently
connected
may have dynamic IP
addresses
do not communicate
directly with each other
2: Application Layer 8
2.1 Google Data Centers
Estimated cost of data center: $600M
Google spent $2.4B in 2007 on new data
centers
Each data center uses 50-100 megawatts
of power
2.1 Pure P2P architecture
no always-on server
arbitrary end systems
directly communicate peer-peer
peers are intermittently
connected and change IP
addresses
2: Application Layer 10
2.1 Hybrid of client-server and P2P
Skype
voice-over-IP P2P application
centralized server: finding address of remote
party:
client-client connection: direct (not through
server)
Instant messaging
chatting between two users is P2P
centralized service: client presence
detection/location
• user registers its IP address with central
server when it comes online
• user contacts central server to find IP
addresses of buddies
2: Application Layer 11
2.1 Processes communicating
Process: program running Client process: process
within a host. that initiates
within same host, two
communication
processes communicate Server process: process
using inter-process that waits to be
communication (defined contacted
by OS).
processes in different Note: applications with
hosts communicate by P2P architectures have
exchanging messages client processes &
server processes
2: Application Layer 12
2.1 Sockets
process sends/receives
host or host or
server server
messages to/from its
socket controlled by
app developer
socket analogous to door process process
2: Application Layer 16
2.1 Transport service requirements of common apps
2: Application Layer 17
2.1 Internet transport protocols services
Application Underlying
Application layer protocol transport protocol
2: Application Layer 19
2.2 Web and HTTP
First some jargon
Web page consists of objects
Object can be HTML file, JPEG image, Java
applet, audio file,…
Web page consists of base HTML-file which
includes several referenced objects
Each object is addressable by a URL
Example URL:
www.someschool.edu/someDept/pic.gif
2: Application Layer 20
2.2 HTTP overview
HTTP: hypertext
transfer protocol
Web’s application layer PC running
protocol Explorer
client/server model
client: browser that
requests, receives, Server
“displays” Web objects running
Apache Web
server: Web server
server
sends objects in
response to requests
Mac running
Navigator
2: Application Layer 21
2.2 HTTP overview (continued)
Uses TCP: HTTP is “stateless”
client initiates TCP server maintains no
connection (creates socket) information about
to server, port 80 past client requests
server accepts TCP
connection from client aside
Protocols that maintain
HTTP messages (application- “state” are complex!
layer protocol messages) past history (state) must
exchanged between browser be maintained
(HTTP client) and Web
if server/client crashes,
server (HTTP server)
their views of “state” may
TCP connection closed
be inconsistent, must be
reconciled
2: Application Layer 22
2.2 HTTP connections
Nonpersistent HTTP Persistent HTTP
At most one object is Multiple objects can
sent over a TCP be sent over single
connection. TCP connection
between client and
server.
2: Application Layer 23
2.2 Nonpersistent HTTP
(contains text,
Suppose user enters URL references to 10
www.someSchool.edu/someDepartment/home.index jpeg images)
time
2: Application Layer 24
2.2 Nonpersistent HTTP (cont.)
4. HTTP server closes TCP
connection.
5. HTTP client receives response
message containing html file,
displays html. Parsing html
file, finds 10 referenced jpeg
objects
time 6. Steps 1-5 repeated for each
of 10 jpeg objects
2: Application Layer 25
2.2 Non-Persistent HTTP: Response time
Definition of RTT: time for
a small packet to travel
from client to server
and back. initiate TCP
connection
Response time: RTT
one RTT to initiate TCP request
file
connection time to
RTT
transmit
one RTT for HTTP file
request and first few file
received
bytes of HTTP response
to return time time
file transmission time
total = 2RTT+transmit time
2: Application Layer 26
2.2 Persistent HTTP
Nonpersistent HTTP issues: Persistent HTTP
requires 2 RTTs per object server leaves connection
OS overhead for each TCP open after sending
connection response
browsers often open parallel subsequent HTTP messages
TCP connections to fetch between same
referenced objects client/server sent over
open connection
client sends requests as
soon as it encounters a
referenced object
as little as one RTT for all
the referenced objects
2: Application Layer 27
2.2 HTTP request message
two types of HTTP messages: request, response
HTTP request message:
ASCII (human-readable format)
request line
(GET, POST, GET /somedir/page.html HTTP/1.1
HEAD commands) Host: www.someschool.edu
User-agent: Mozilla/4.0
header Connection: close
lines Accept-language:fr
Carriage return,
line feed (extra carriage return, line feed)
indicates end
of message
2: Application Layer 28
2.2 HTTP request message: general format
2: Application Layer 29
2.2 Uploading form input
Post method:
Web page often
includes form input URL method:
Input is uploaded to Uses GET method
server in entity body Input is uploaded in
URL field of request
line:
www.somesite.com/animalsearch?monkeys&banana
2: Application Layer 30
2.2 Method types
HTTP/1.0 HTTP/1.1
GET GET, POST, HEAD
POST PUT
HEAD uploads file in entity
body to path specified
asks server to leave
in URL field
requested object out of
response DELETE
deletes file specified in
the URL field
2: Application Layer 31
2.2 HTTP response message
status line
(protocol
status code HTTP/1.1 200 OK
status phrase) Connection close
Date: Thu, 06 Aug 1998 12:00:15 GMT
Server: Apache/1.3.0 (Unix)
header
Last-Modified: Mon, 22 Jun 1998 …...
lines
Content-Length: 6821
Content-Type: text/html
2: Application Layer 32
2.2 HTTP response status codes
In first line in server->client response message.
A few sample codes:
200 OK
request succeeded, requested object later in this message
301 Moved Permanently
requested object moved, new location specified later in
this message (Location:)
400 Bad Request
request message not understood by server
404 Not Found
requested document not found on this server
505 HTTP Version Not Supported
2: Application Layer 33
2.2 Trying out HTTP (client side) for yourself
2: Application Layer 34
2.2 User-server state: cookies
Example:
Many major Web sites
use cookies Susan always access
Four components: Internet always from PC
1) cookie header line of visits specific e-
HTTP response message commerce site for first
2) cookie header line in time
HTTP request message
3) cookie file kept on when initial HTTP
user’s host, managed by requests arrives at site,
user’s browser
site creates:
4) back-end database at
Web site unique ID
entry in backend
database for ID
2: Application Layer 35
2.2 Cookies: keeping “state” (cont.)
client server
ebay 8734
usual http request msg
Amazon server
cookie file usual http response creates ID
Set-cookie: 1678 1678 for user create
ebay 8734 entry
amazon 1678
usual http request msg
cookie: 1678 cookie- access
specific
one week later: usual http response msg action backend
database
access
ebay 8734 usual http request msg
amazon 1678 cookie: 1678 cookie-
spectific
usual http response msg action
2: Application Layer 36
2.2 Cookies (continued)
aside
What cookies can bring: Cookies and privacy:
authorization cookies permit sites to
shopping carts
learn a lot about you
you may supply name
recommendations
and e-mail to sites
user session state
(Web e-mail)
How to keep “state”:
protocol endpoints: maintain state
at sender/receiver over multiple
transactions
cookies: http messages carry state
2: Application Layer 37
2.2 Web caches (proxy server)
Goal: satisfy client request without involving origin server
institutional
cache
2: Application Layer 41
2.2 Caching example (cont)
origin
possible solution: install servers
cache public
suppose hit rate is 0.4 Internet
consequence
40% requests will be
satisfied almost immediately
15 Mbps
60% requests satisfied by
access link
origin server
utilization of access link institutional
reduced to 60%, resulting in network
100 Mbps LAN
negligible delays (say 10
msec)
total avg delay = Internet
delay + access delay + LAN institutional
delay = .6*(2.01) secs +
.4*milliseconds < 1.4 secs cache
2: Application Layer 42
2.2 Conditional GET
2: Application Layer 44
2.3 FTP: separate control, data connections
TCP control connection
FTP client contacts FTP server port 21
at port 21, TCP is transport
protocol TCP data connection
client authorized over control FTP port 20 FTP
connection client server
client browses remote
server opens another TCP
directory by sending commands
data connection to transfer
over control connection.
another file.
when server receives file
control connection: “out of
transfer command, server
band”
opens 2nd TCP connection (for
file) to client FTP server maintains “state”:
current directory, earlier
after transferring one file,
authentication
server closes data connection.
2: Application Layer 45
2.3 FTP commands, responses
Sample commands: Sample return codes
sent as ASCII text over status code and phrase (as
control channel in HTTP)
USER username 331 Username OK,
PASS password password required
125 data connection
LIST return list of file in
already open;
current directory
transfer starting
RETR filename retrieves 425 Can’t open data
(gets) file connection
STOR filename stores 452 Error writing
(puts) file onto remote file
host
2: Application Layer 46
2.4 Electronic Mail outgoing
message queue
user mailbox
user
Three major components: agent
user agents mail
user
server
mail servers agent
simple mail transfer SMTP mail
protocol: SMTP server user
SMTP agent
User Agent
a.k.a. “mail reader” SMTP
mail user
composing, editing, reading agent
server
mail messages
e.g., Eudora, Outlook, elm, user
Mozilla Thunderbird agent
user
outgoing, incoming messages agent
stored on server
2: Application Layer 47
2.4 Electronic Mail: mail servers
user
Mail Servers agent
mailbox contains incoming mail
user
messages for user server
agent
message queue of outgoing
SMTP
(to be sent) mail messages mail
server user
SMTP protocol between mail
servers to send email SMTP agent
messages SMTP
client: sending mail mail user
agent
server server
“server”: receiving mail
user
server agent
user
agent
2: Application Layer 48
2.4 Electronic Mail: SMTP [RFC 2821]
uses TCP to reliably transfer email message from client
to server, port 25
direct transfer: sending server to receiving server
three phases of transfer
handshaking (greeting)
transfer of messages
closure
command/response interaction
commands: ASCII text
response: status code and phrase
2: Application Layer 49
2.4 Scenario: Alice sends message to Bob
1) Alice uses UA to compose 4) SMTP client sends Alice’s
message and “to” message over the TCP
bob@someschool.edu connection
2) Alice’s UA sends message 5) Bob’s mail server places the
to her mail server; message message in Bob’s mailbox
placed in message queue 6) Bob invokes his user agent
3) Client side of SMTP opens to read message
TCP connection with Bob’s
mail server
1 mail
mail
server user
user server
2 agent
agent 3 6
4 5
2: Application Layer 50
2.4 Sample SMTP interaction
S: 220 hamburger.edu
C: HELO crepes.fr
S: 250 Hello crepes.fr, pleased to meet you
C: MAIL FROM: <alice@crepes.fr>
S: 250 alice@crepes.fr... Sender ok
C: RCPT TO: <bob@hamburger.edu>
S: 250 bob@hamburger.edu ... Recipient ok
C: DATA
S: 354 Enter mail, end with "." on a line by itself
C: Do you like ketchup?
C: How about pickles?
C: .
S: 250 Message accepted for delivery
C: QUIT
S: 221 hamburger.edu closing connection
2: Application Layer 51
2.4 Try SMTP interaction for yourself:
telnet servername 25
see 220 reply from server
enter HELO, MAIL FROM, RCPT TO, DATA, QUIT
commands
above lets you send email without using email client
(reader)
2: Application Layer 52
2.4 SMTP: final words
SMTP uses persistent Comparison with HTTP:
connections
HTTP: pull
SMTP requires message
(header & body) to be in 7- SMTP: push
bit ASCII both have ASCII
SMTP server uses command/response
CRLF.CRLF to determine interaction, status codes
end of message
HTTP: each object
encapsulated in its own
response msg
SMTP: multiple objects
sent in multipart msg
2: Application Layer 53
2.4 Mail message format
SMTP: protocol for
exchanging email msgs header
blank
RFC 822: standard for text
line
message format:
header lines, e.g.,
To:
body
From:
Subject:
different from SMTP
commands!
body
the “message”, ASCII
characters only
2: Application Layer 54
2.4 Mail access protocols
SMTP SMTP access user
user
agent protocol agent
2: Application Layer 55
2.4 POP3 protocol S: +OK POP3 server ready
C: user bob
authorization phase S: +OK
C: pass hungry
client commands: S: +OK user successfully logged on
user: declare username
C: list
pass: password S: 1 498
server responses S: 2 912
S: .
+OK
C: retr 1
-ERR S: <message 1 contents>
transaction phase, client: S: .
C: dele 1
list: list message numbers C: retr 2
retr: retrieve message by S: <message 1 contents>
number S: .
C: dele 2
dele: delete
C: quit
quit S: +OK POP3 server signing off
2: Application Layer 56
2.4 POP3 (more) and IMAP
More about POP3 IMAP
Previous example uses Keep all messages in
“download and delete” one place: the server
mode. Allows user to
Bob cannot re-read e- organize messages in
mail if he changes folders
client IMAP keeps user state
“Download-and-keep”: across sessions:
copies of messages on names of folders and
different clients mappings between
message IDs and folder
POP3 is stateless
name
across sessions
2: Application Layer 57
Chapter 2: Application layer
2.1 Principles of 2.6 P2P applications
network applications 2.7 Socket programming
2.2 Web and HTTP with UDP
2.3 FTP 2.8 Socket programming
2.4 Electronic Mail with TCP
SMTP, POP3, IMAP
2.5 DNS
2: Application Layer 58
2.5 DNS: Domain Name System
People: many identifiers: Domain Name System:
SSN, name, passport # distributed database
Internet hosts, routers: implemented in hierarchy of
many name servers
IP address (32 bit) -
application-layer protocol
used for addressing
host, routers, name servers to
datagrams
communicate to resolve names
“name”, e.g., (address/name translation)
ww.yahoo.com - used by
note: core Internet
humans
function, implemented as
Q: map between IP application-layer protocol
addresses and name ? complexity at network’s
“edge”
2: Application Layer 59
2.5 DNS
DNS services Why not centralize DNS?
hostname to IP single point of failure
address translation traffic volume
host aliasing distant centralized
Canonical, alias names database
mail server aliasing maintenance
load distribution
replicated Web doesn’t scale!
servers: set of IP
addresses for one
canonical name
2: Application Layer 60
2.5 Distributed, Hierarchical Database
Root DNS Servers
TLD namesrvrs
com DNS servers org DNS servers edu DNS servers
a Verisign, Dulles, VA
c Cogent, Herndon, VA (also LA)
d U Maryland College Park, MD k RIPE London (also 16 other locations)
g US DoD Vienna, VA
h ARL Aberdeen, MD i Autonomica, Stockholm (plus
j Verisign, ( 21 locations) 28 other locations)
e NASA Mt View, CA m WIDE Tokyo (also Seoul,
f Internet Software C. Palo Alto, Paris, SF)
CA (and 36 other locations)
13 root name
servers worldwide
b USC-ISI Marina del Rey, CA
l ICANN Los Angeles, CA
2: Application Layer 62
2.5 TLD and Authoritative Servers
Top-level domain (TLD) servers:
responsible for com, org, net, edu, etc, and all
top-level country domains uk, fr, ca, jp.
Network Solutions maintains servers for com TLD
Educause for edu TLD
Authoritative DNS servers:
organization’s DNS servers, providing
authoritative hostname to IP mappings for
organization’s servers (e.g., Web, mail).
can be maintained by organization or service
provider
2: Application Layer 63
2.5 Local Name Server
does not strictly belong to hierarchy
each ISP (residential ISP, company,
university) has one.
also called “default name server”
when host makes DNS query, query is sent
to its local DNS server
acts as proxy, forwards query into hierarchy
Organizations local name server and
authoritative name typically combined on
one machine (BIND software).
2: Application Layer 64
2.5 DNS name root DNS server
resolution example
2
Host at cis.poly.edu 3
TLD DNS server
wants IP address for 4
gaia.cs.umass.edu 5
gaia.cs.umass.edu
2: Application Layer 65
2.5 DNS name
root DNS server
resolution example
recursive query: 2 3
puts burden of name 6
7
resolution on
TLD DNS server
contacted name
server
heavy load? local DNS server
dns.poly.edu 5 4
1 8
gaia.cs.umass.edu
2: Application Layer 66
2.5 DNS: caching and updating records
once (any) name server learns mapping, it caches
mapping
cache entries timeout (disappear) after some
time
TLD servers typically cached in local name
servers
• Thus root name servers not often visited
update/notify mechanisms under design by IETF
RFC 2136
http://www.ietf.org/html.charters/dnsind-charter.html
2: Application Layer 67
2.5 DNS records
DNS: distributed db storing resource records (RR)
RR format: (name, value, type, ttl)
Type=A Type=CNAME
name is hostname name is alias name for some
value is IP address “canonical” (the real) name
www.ibm.com is really
Type=NS
servereast.backup2.ibm.com
name is domain (e.g.
value is canonical name
foo.com)
value is hostname of
Type=MX
authoritative name
value is name of mailserver
server for this domain
associated with name
2: Application Layer 68
2.5 DNS protocol, messages
DNS protocol : query and reply messages, both with
same message format
msg header
identification: 16 bit #
for query, reply to query
uses same #
flags:
query or reply
recursion desired
recursion available
reply is authoritative
2: Application Layer 69
2.5 DNS protocol, messages
RRs in response
to query
records for
authoritative servers
additional “helpful”
info that may be used
2: Application Layer 70
2.5 Inserting records into DNS
example: new startup “Network Utopia”
register name networkuptopia.com at DNS registrar
(e.g., Network Solutions)
provide names, IP addresses of authoritative name server
(primary and secondary)
registrar inserts two RRs into com TLD server:
2: Application Layer 71
2.6 Pure P2P architecture
no always-on server
arbitrary end systems
directly communicate peer-peer
peers are intermittently
connected and change IP
addresses
Three topics:
File distribution
Searching for information
Case Study: Skype
2: Application Layer 72
2.6 File Distribution: Server-Client vs P2P
Question : How much time to distribute file
from one server to N peers?
us: server upload
bandwidth
Server
ui: peer i upload
u1 d1 u2 bandwidth
us d2
di: peer i download
File, size F bandwidth
dN
Network (with
uN abundant bandwidth)
2: Application Layer 73
2.6 File distribution time: server-client
Server
server sequentially F u1 d1 u2
sends N copies: us d2
Time to distribute F
to N clients using = dcs = max { NF/us, F/min(di) }
i
client/server approach
increases linearly in N
(for large N) 2: Application Layer 74
2.6 File distribution time: P2P
Server
server must send one
F u1 d1 u2
copy: F/us time us d2
downloaded (aggregate)
fastest possible upload rate: us + Σui
3.5
P2P
Minimum Distribution Time
3
Client-Server
2.5
1.5
0.5
0
0 5 10 15 20 25 30 35
N
2: Application Layer 76
2.6 File distribution: BitTorrent
P2P file distribution
obtain list
of peers
trading
chunks
peer
2: Application Layer 77
2.6 BitTorrent (1)
2: Application Layer 79
2.6 BitTorrent: Tit-for-tat
(1) Alice “optimistically unchokes” Bob
(2) Alice becomes one of Bob’s top-four providers; Bob reciprocates
(3) Bob becomes one of Alice’s top-four providers
15 3
4
12
5
10
8
Each peer only aware of immediate successor
and predecessor.
“Overlay network”
2.6 Circle DHT (2)
1110 0100
1110
1100
1110
1110 0101
Define closest 1110
as closest 1010
successor 1000
2.6 Circular DHT with Shortcuts
1 Who’s resp
for key 1110?
3
15
4
12
5
10
8
Each peer keeps track of IP addresses of predecessor,
successor, short cuts.
Reduced from 6 to 2 messages.
Possible to design shortcuts so O(log N) neighbors, O(log
N) messages in query
2.6 Peer1 Churn
•To handle peer churn, require
2: Application Layer 88
2.6 Peers as relays
Problem when both
Alice and Bob are
behind “NATs”.
NAT prevents an outside
peer from initiating a call
to insider peer
Solution:
Using Alice’s and Bob’s
SNs, Relay is chosen
Each peer initiates
session with relay.
Peers can now
communicate through
NATs via relay
2: Application Layer 89
2.7 Socket programming
Goal: learn how to build client/server application that
communicate using sockets
2: Application Layer 90
2.7 Socket programming basics
Server must be Socket is locally
running before identified with a port
client can send number
anything to it. Analogous to the apt #
Server must have a in a building
socket (door) Client needs to know
through which it server IP address and
receives and sends socket port number.
segments
Similarly client
needs a socket
2: Application Layer 91
2.7 Socket programming with UDP
UDP: no “connection” between
client and server
no handshaking
sender explicitly attaches application viewpoint
IP address and port of
destination to each segment UDP provides unreliable transfer
of groups of bytes (“datagrams”)
OS attaches IP address and
between client and server
port of sending socket to
each segment
Server can extract IP
address, port of sender Note: the official terminology
from received segment for a UDP packet is “datagram”.
In this class, we instead use “UDP
segment”.
2: Application Layer 92
2.7 Running example
Client:
User types line of text
Client program sends line to server
Server:
Server receives line of text
Capitalizes all the letters
Sends modified line to client
Client:
Receives line of text
Displays
2: Application Layer 93
2.7 Client/server socket interaction: UDP
Server (running on hostid) Client
write reply to
serverSocket
specifying read datagram from
client address, clientSocket
port number close
clientSocket
2: Application Layer 94
2.7 UDP Client
from socket import * address format
2: Application Layer 96
2.7 UDP observations & questions
Both client server use socket type: SOCK_DGRAM
Dest IP and port are explicitly attached to
segment.
What would happen if change both clientSocket
and serverSocket to “mySocket”?
Can the client send a segment to server without
knowing the server’s IP address and/or port
number?
Can multiple clients use the server?
2: Application Layer 97
2.8 Socket-programming using TCP
controlled by
controlled by process application
application process
developer
developer socket socket
controlled by TCP with TCP with controlled by
buffers, operating
operating buffers, internet system
system variables variables
host or host or
server server
2: Application Layer 98
2.8 Socket programming with TCP
Client must contact server When contacted by client,
server process must first server TCP creates new
be running socket for server process to
server must have created communicate with client
socket (door) that allows server to talk with
welcomes client’s contact multiple clients
source port numbers
Client contacts server by:
used to distinguish
creating client-local TCP
clients (more in Chap 3)
socket
specifying IP address, port application viewpoint
number of server process
TCP provides reliable, in-order
When client creates
transfer of bytes (“pipe”)
socket: client TCP
between client and server
establishes connection to
server TCP
2: Application Layer 99
2.8 Client/server socket interaction: TCP
Server (running on serverIP) Client
create socket,
port=x, for
incoming request:
serverSocket =
socket()
TCP create socket,
wait for incoming
connection request connection setup connect to serverIP, port=x
connectionSocket = clientSocket =socket()
serverSocket.accept()
write reply to
connectionSocket read reply from
clientSocket
close
connectionSocket close
clientSocket
2: Application Layer 100
2.8 TCP client
from socket import *
P2P FTP