UNIX

Network Programming
with TCP/IP
UNIX
Network Programming
with TCP/IP
UNIX
Network Programming
with TCP/IP
UNIX
Network Programming
with TCP/IP
UNIX
Network Programming
with TCP/IP
UNIX Network
Programming with TCP/IP
Short Course Notes
Alan Dix © 1996
http://www.hiraeth.com/alan/tutorials
UNIX
Network Programming
with TCP/IP
Course
Outline
Alan Dix
http://www.hcibook.com/alan
Session 1 Internet Basics
Session 2 First Code
Session 3 Standard Applications
Session 4 Building Clients
Session 5 Servers I
Session 6 Servers II
Session 7 Security
Three interrelated aspects:
¯ TCP/ IP protocol suite
¯ standard Internet applications
¯ coding using UNIX sockets API
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 1
UNIX
Network Programming
with TCP/IP
Reading
Books:
1. W. Richard Stevens, "TCP/ IP Illustrated. Vol. 1: The protocols", Addison
Wesley, 1994, (ISBN 0-201-63346-9).
Explains the protocols using network monitoring tools without programming.
2. Douglas E. Comer and David L. Stevens, "Internetworking with TCP/ IP.
Vol.3: Client-server programming and applications BSD socket version",
Prentice Hall, 1993, (ISBN 0-13-020272-X).
Good book about principles of client/ server design. Assumes you have some
knowledge or at least some other reference for actual programming.
3. Michael Santifaller , translated by Stephen S. Wilson, "TCP/ IP and ONC/ NFS
internetworking in a UNIX environment", 2nd Edition, Addison Wesley, 1994,
(ISBN 0-201-42275-1).
Covers more ground less deeply. Translation from German seems good.
4. W. Richard Stevens, "UNIX Network Programming", Prentice Hall, 1990,
(ISBN 0-13-949876-1).
A programming book. I'm waiting for a copy, but Stevens is a good writer and
this book is recommended by other authors.
See also:
• your local manual pages (ma n 2)
• RFCs
Requests for comments (RFCs)
• these are the definition of the Internet protocols
• obtain via anonymous ftp from s un. doc . i c . a c . uk (193. 63. 255. 1)
login as a nonymous
give your email address as password
cd to r f c
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 2
UNIX
Network Programming
with TCP/IP
UNIX
Network Programming
with TCP/IP
UNIX
Network Programming
with TCP/IP
UNIX
Network Programming
with TCP/IP
UNIX
Network Programming
with TCP/IP
Session 1

Internet Basics
UNIX
Network Programming
with TCP/IP
Session 1
Alan Dix
http://www.hcibook.com/alan
• origins
• internets and the Internet
• protocol layers
• addressing
• common applications
ª using them
• TCP and UDP
• port numbers
• APIs
ª information calls
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 1
Origins
Development of Internet & TCP/IP
1968 First proposal for ARPANET – military & gov’t research
Contracted to Bolt, Beranek & Newman
1971 ARPANET enters regular use
1973/ 4 redesign of lower level protocols
leads to TCP/ IP
1983 Berkeley TCP/ IP implementation for 4.2BSD
public domain code
1980s rapid growth of NSFNET – broad academic use
1990s WWW and public access to the Internet
The Internet Now
• growing commercialisation of the Internet
• 50,000 networks
• 6 million hosts
• 30 million users
• WWW dominating Internet growth
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 2
internets and the Internet
an internet is
a collection of
• interconnected networks
• (possibly different)
e.g. X25, AppleTalk
the Internet is
a particular internet which
• uses the TCP/ IP protocols
• is global
• is hardware and network independent
• is non-proprietary
in addition
• supports commonly used applications
• publicly available standards (RFCs)
the Internet is not (just) the web !
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 3
Characteristics of the Internet
To communicate you need:
• continuous connection
• common language
• means of addressing
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 4
Global Connectivity
ethernet
token ring
PPP
routers
router
sub-network
star
network
lots of networks:
• ethernet, FDDI, token ring
• AppleTalk (itself an internet!)
• etc. etc. etc.
connected (possibly indirectly)
• to each other
• to the central ‘ARPAnet’ backbone in the US
protocols can be used in isolation
? but is it the Internet
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 5
Protocols – the Language of the
Internet
electrical signals
low-level networks
(e.g. ethernet)
IP layer (end-to-end)
TCP/ UDP layer
application protocols
(e.g. FTP, telnet, http)
application user interfaces
(e.g. Fetch, mosaic)
OSI
ICMP (control and routing)
Physical
Link
Transport
Network
Session,
Presentation,
Application
routers
end-points
TCP/IP
Standardisation:
• RFC (request for comments) and DoD MIL
RFCs also include (defined but not required):
• PPP, ethernet packaging, etc.
• FTP and other protocols
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 6
Addressing
J. C. T. Jennings,
Linbury Court School,
Dunhambury,
Sussex,
England,
Europe,
Eastern Hemisphere,
Earth,
near Moon,
Solar System,
Space,
near More Space

Without addresses can only broadcast
Four types of address:
' location independent e.g. personal names
Z physical location e.g. letter addresses
· logical location e.g. organisational hierarchy
J route based e.g. old email addresses
Two kinds of Internet address:
IP address – type Z (sort of)
e.g. 161.12.188.167
domain name – type ·
e.g. zeus.hud.ac.uk

extract from Jennings Goes to School, Anthony Buckeridge, Collins, 1950.
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 7
IP addresses
• 32 bit integer – 2701966503
• Often represented as 4 octets – 161.12.188.167
• General structure:
net id { sub-net id } host id
• N.B. octets do not map simply onto components
Five classes of IP address:
Class A 0 netid subnet/ hostid
7 bits 24 bits
Class B 1 0 netid subnet/ hostid
14 bits 16 bits
Class C 1 1 0 netid hostid
21 bits 8 bits
Class D & Class E – experimental
• hostids may divided using subnet mask
¯ different for each major network (netid)
⇒ needs to be set for each machine on network
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 8
Domain names
• human readable names
..... or at least ASCII !
• Heirarchical (roughly organisational)
z e us . hud. a c . uk
uk – United Kingdom
a c – academic
hud – huddersfield
z e us – local machine
N.B. USA is implicit – c s . wa s hi ngt on. e du
• Decentralised administration
• Mapping
from name to IP address
– domain name servers
also reverse mapping
• C API :
gethostbyname – name → IP address
gethostbyaddr – IP address → name
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 9
Common applications
• FTP (file transfer protocol)
• SMTP (simple mail transfer protocol)
• telnet (remote logins)
• rlogin (simple remote login between UNIX machines)
• World Wide Web (built on http)
• NFS (network filing system – originally for SUNs)
• TFTP (trivial file transfer protocol – used for booting)
• SNMP (simple network management protocol)
+ In each case protocols are defined
+ User interfaces depend on platform
(where relevant)
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 10
Hands on
ª connect to zeus using telenet:
% t e l ne t z e us . hud. a c . uk
l ogi n: c 5
. . . etc.
ª what happens if you just say “t e l ne t z e us ”?
ª what is zeus’ IP address?
ª try “t e l ne t a a . bb. c c . dd”
(where ‘aa.bb.cc.dd’ is zeus’ IP address)
ª connect to zeus using ftp:
% f t p z e us . hud. a c . uk
connect as yourself and then as anonymous
Read between the lines
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 11
Network communications
Communication can be:
• Connectionless
¯ address every message
+ like letters
• Connection based
¯ use address to establish a fixed link
¯ send each message using the link
+ like telephone
N.B. both need an address
⇒ some sort of system address book
or, publicly known addresses
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 12
Network communications – 2
Other issues:
• Reliability
Do all messages arrive?
Do they arrive in the right order?
• Buffering
effects responsiveness
hides potential deadlock
• Messages or byte-stream
sent:
write 1 (len=26): “abcde....vwxyz”
write 2 (len=10): “0123456789”
received:
read 1 (len=20): “abcde....qrst”
read 2 (len=16): “uvwxyz012...89”
⇒ fixed length messages or prefix with length
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 13
IP – the fundamental Internet protocol
point to point
¯ between machines
¯ addressed using IP address
message (packet) based
unreliable
¯ network failures
¯ router buffers fill up
dynamic routing
⇒ order may be lost
heterogeneous intermediate networks
⇒ fragmentation
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 14
TCP & UDP
Both
• built on top of IP
• addressed using port numbers
process to process
(on UNIX platforms)
TCP
• connection based
• reliable
• byte stream
used in: FTP, telnet, http, SMTP
UDP
• connectionless
• unreliable
• datagram (packet based)
used in: NFS, TFTP
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 15
Port numbers
• 16 bit integers
• unique within a machine
• to connect need IP address + port no
TCP
• connection defined by
IP address & port of server
+ IP address & port of client
UNIX
• port < 1023 – root only
• used for authentication
(e.g. rlogin)
How do you find them?
v well known port numbers
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 16
Well known port numbers
Service Port no Protocol
echo 7 UDP/ TCP sends back what it receives
discard 9 UDP/ TCP throws away input
daytime 13 UDP/ TCP returns ASCII time
chargen 19 UDP/ TCP returns characters
ftp 21 TCP file transfer
telnet 23 TCP remote login
smtp 25 TCP email
daytime 37 UDP/ TCP returns binary time
tftp 69 UDP trivial file transfer
finger 79 TCP info on users
http 80 TCP World Wide Web
login 513 TCP remote login
who 513 UDP different info on users
Xserver 6000 TCP X windows (N.B. >1023)
N.B. different ‘name’ spaces for TCP & UDP
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 17
API – the language of the programmer
Application Programmer Interfaces
Not part of the Internet standard – but very important!
A story about DOS
TCP/ IP stacks supplied by different vendors
⇒ different device drivers
different APIs
chaos
APIs depend on platform:
UNIX – sockets (original Berkley system calls)
– TLI (transport layer interface)
Apple Mac – MacTCP
MS Windows – WinSock (similar to sockets)
• UNIX TCP/ IP API are kernel system calls
• Mac & Windows are extensions/ drivers (+DLL)
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 18
Hands on
ª copy s ke l e t on. c from t c p directory
ª edit to make two programs:
ge t i d. c – returns IP address of machine
ge t na me . c – returns name of machine
ª use the following C calls:
ge t hos t i d( )
returns (lon unsigned) integer result
ge t hos t na me ( buf f , l e n)
returns error code
puts name into buff (maximum len bytes)
ª if you have time, play with telnet on different ports
% t e l ne t z e us . hud. a c . uk port_no
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 19
UNIX
Network Programming
with TCP/IP
UNIX
Network Programming
with TCP/IP
UNIX
Network Programming
with TCP/IP
UNIX
Network Programming
with TCP/IP
UNIX
Network Programming
with TCP/IP
Session 2

First Code
UNIX
Network Programming
with TCP/IP
Session 2
Alan Dix
http://www.hcibook.com/alan
• features of sockets API
• establishing TCP connections
• simple client/ server program
ª use it
• read & write with sockets
• wrapper functions
• what they do
ª an echo server
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 2/ 1
Sockets
• developed for Berkeley UNIX
¯ recall early Berkeley TCP/ IP implementation
¯ first delivered with BSD 2.1
• central features
¯ central abstraction - the socket - an end-point
like an electrical connector
¯ not TCP/ IP specific (e.g. UNIX named pipes)
¯ uses normal read/ write system calls
¯ sockets associated with UNIX file descriptors
but some not for normal I/ O
¯ some extra system calls
• sits more comfortably with TCP than with UDP
because of byte-stream nature of UNIX I/ O
• special UDP functions
e.g., recv( ...) – accepts a UDP datagram
• additional non-socket functions
e.g., get host byname( ...) – domain name server
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 2/ 2
Establishing a TCP Connection
Initial State
different
processes
151.100.17.25
different
processes
161.112.192.5
different
processes
161.112.4.3
Internet
• TCP is connection based
... establishing it is a complex multistage process
• initially all machines are the same
• no special ‘server’ machines
• the difference is all in the software
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 2/ 3
Establishing a TCP Connection
Passive Open
161.112.192.5
server
21
151.100.17.25
• server process does a ‘passive’ open on a port
• it waits for a client to connect
• at this stage there is no Internet network traffic
• tells the TCP layer which process to connect to
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 2/ 4
Establishing a TCP Connection
Active Open
151.100.17.25
client
2397
161.112.192.5
server
21
161.112.192.5
server
21
161.112.192.5 : 21
• client process usually on a different machine
• performs an ‘active’ open on the port
• port number at the client end is needed
usually automatic (e.g., 2397)
but can be chosen
• network message → server machine
requests connection
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 2/ 5
Establishing a TCP Connection
Rendezvous
151.100.17.25
client
161.112.192.5
server
21
161.112.192.5
server
21
• server side accepts and TCP connection established
• a bi-directional reliable byte-stream
• connection identified by both host/ port numbers
e.g. <151.10017.25:2397/ 161.112.192.5:21>
• server port is not consumed
can stay ‘passive’ open for more connections
• like telephone call desk: one number many lines
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 2/ 6
Establishing a TCP Connection
and more ...
151.100.17.25
client
161.112.192.5
server
21
161.112.192.5
server
21
161.112.4.3
client
• other clients can connect to the same port
• state for connections in the client/ server only
• no information needed in the network
not like old style relay-based exchanges
• server can restrict access to specified host or port
• server can find out connected host/ port
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 2/ 7
Passive & Active Open
passive – patient but lazy
active – industrious but impatient
passive active
waits for request for
connection
sends out request for
connection
waits for ever times out
• normally server does passive open
– waiting for client
• but not always (e.g. ftp)
• active opens can rendezvous ...
... but may miss due to time-outs
• either can specify local port
but if not specified, allocated automatically
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 2/ 8
Simple client/server ‘talk’
• uses simplified calls
• server handles only one client
• strict turntaking
user 1 user 2
zeus: s i mp l e - s e r v e r
st art up complet e
io: s i mp l e - c l i e n t - h o s t z e u s
You can send now
speak: hi t her e
client says: hi t here
speak: ni c e d ay i sn' t i t
server says: nice day isn' t it
speak: b i t c o l d her e
client says: bit cold here
speak: ^D (EOF) bye bye
zeus:
server f inished t he conversat ion
io:
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 2/ 9
Server Code
establish port
port_sk = tcp_passive_open(port)
/* only done once */
wait for client to connect
client_sk = tcp_accept(port_sk)
/* repeated for multiple clients */
then talk to client
for(;;) {
/* wait for client’s message */
len = read(client_sk,buff,buf_len);
buff[len] = '\0';
printf("client says: %s\n",buff);
/* now it’s our turn */
printf("speak: ");
gets(buff);
write(client_sk,buff,strlen(buff));
}
N.B. strict turn taking: client–server–client–server ...
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 2/ 10
Client Code
request connection to server
serv_sk = tcp_active_open(host,port)
/* waits for server to accept */
/* returns negative on failure */
/* host is server’s machine */
then talk to server
for(;;) {
/* our turn first */
printf("speak: ");
gets(buff);
write(serv_sk,buff,strlen(buff));
/* wait for server’s message */
len = read(serv_sk,buff,buf_len);
buff[len] = '\0';
printf("server says: %s\n",buff);
}
N.B. ' opposite turn order
^ no error checking!
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 2/ 11
ª ª ª ª Hands on · · · ·
ª copy s i mpl e - c l i e nt . c from t c p/ s e s s i on2 directory
• s i mpl e - c l i e nt . c
• s i mpl e - s e r ve r . c
• ma ke f i l e
ª compile and run the programs:
• ma ke s i mpl e – compiles them both
• on one machine type:
s i mpl e - s e r ve r
• on another type:
s i mpl e - c l i e nt machine-name
where machine-name is the name of the first
ª what happens if you re-run the server straight after
it finishes?
ª use the - por t option
zeus: s i mpl e - s e r ve r - por t 3865
i o: s i mpl e - c l i e nt - hos t z e us - por t 3865
ª try a port less than 1024!
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 2/ 12
read & write
Reminder:
r e t = r e a d( f d, buf f , l e n)
i nt f d – a file descriptor (int), open for reading
c ha r *buf f – buffer in which to put chars
i nt l e n – maximum number of bytes to read
i nt r e t – returns actual number read
• r e t is 0 at end of file, negative for error
• buf f is not NULL terminated
leave room if you need to add ‘\ 0’!
r e t = wr i t e ( f d, buf f , l e n)
i nt f d – a file descriptor (int), open for writing
c ha r *buf f – buffer from which to get chars
i nt l e n – number of bytes to write
i nt r e t – returns actual number written
• r e t is negative for error, 0 means “end of file”
r e t may be less than l e n e.g. if OS buffers full
* should really check and repeat until all gone *
• buf f need not be NULL terminated
if buf f is a C string, use s t r l e n to get its length
N.B. Both may return negative after interrupt (signal)
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 2/ 13
read & write with sockets
• similar to normal UNIX pipes
• bi-directional byte stream
¯ read and write to same file descriptor
× difficult to close one direction
v special socket call s hut down( s oc k, di r )
• reading may block
¯ reading from a file either:
(i) succeeds
(ii) gets end of file (r e t = 0)
¯ reading from a socket waits until
(i) network data received (r e t > 0)
(ii) connection closed (r e t = 0)
(iii) network error (r e t < 0)
• writing may block
¯ writing to a socket may
(i) send to the network (r e t > 0)
(ii) find connection is closed (r e t = 0)
(iii) network error (r e t < 0)
¯ it may return instantly
¯ but may block if buffers are full
× BEWARE – may work during testing
then fail in use
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 2/ 14
Wrapper Functions (1)
• not real socket functions
• simplified versions for examples
r e t = pa r s e _ne t wor k_a r gs ( &a r gc , a r gv,
&hos t , &por t , &e r r me s s )
scan command arguments for network options
por t _s k = t c p_pa s s i ve _ope n( por t )
server performs passive open
s e r v_s k = t c p_a c t i ve _ope n( hos t , por t )
client performs active open
c l i e nt _s k = t c p_a c c e pt ( por t _s k)
server accepts client connection
¯ pa r s e _ne t wor k_a r gs does not use socket calls
¯ the rest package one or more socket calls
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 2/ 15
Wrapper Functions (2)
r e t = pa r s e _ne t wor k_a r gs ( &a r gc , a r gv,
&hos t , &por t , &e r r me s s )
• scans and edits argument list
• looks for options: - hos t na me - por t nos
• removes them from argument list
• sets the arguments hos t and por t if options found
• set either hos t or por t to NULL to disable options
• returns 0 for success
non-zero failed – e r r me s s set to appropriate message
por t _s k = t c p_pa s s i ve _ope n( por t )
i nt por t – port number to use
i nt por t _s k – file descriptor of socket
' creates Internet TCP socket
por t _s k = s oc ke t ( AF_I NET, SOCK_STREAM, 0 ) ;
Z ‘binds’ socket with right port and address 0.0.0.0
(special address means “this machine”)
bi nd( por t _s k, &bi nd_a ddr , a ddr _l e n ) ;
N.B. por t _s k is not used for normal reading and writing
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 2/ 16
Wrapper Functions (3)
s e r v_s k = t c p_a c t i ve _ope n( hos t na me , por t )
c ha r *hos t na me – name of server’s machine
i nt por t – port number to use
i nt s e r v_s k – file descriptor of socket
' finds IP address of host
hos t I P = ge t hos t byna me ( hos t na me ) ;
Z creates Internet TCP socket
s e r v_s k = s oc ke t ( AF_I NET, SOCK_STREAM, 0 ) ;
· ‘connects’ socket to appropriate port and host
c onne c t ( s e r v_s k, &bi nd_a ddr , a ddr _l e n ) ;
• rendezvous with the server happens at ·
socket s e r v_s k can then be used to talk to the server
c l i e nt _s k = t c p_a c c e pt ( por t _s k)
i nt por t _s k – file descriptor of socket
' performs raw accept call
c l i e nt _s k = a c c e pt ( por t _s k, &bi nd_a ddr , &l e n) ;
• waits for rendezvous at '
when it returns c l i e nt _s k can be used to talk to client
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 2/ 17
Special IP addresses
• bi nd call in t c p_pa s s i ve _ope n uses IP address 0.0.0.0
One of several special IP addresses
0.0.0.0
• source only
• default IP address – ‘local machine’
• filled in by socket API call
127.0.0.0
• loopback address,
• also means ‘the local machine’
• usually used as recipient for local server
• doesn’t normally hit network
• N.B. can also connect to own IP address
255.255.255.255
• limited broadcast (doesn’t pass routers)
a ny ne t i d – s ubne t i d/ hos t i d = –1
a ny ne t i d & a ny s ubne t i d –hos t i d = –1
• broadcast to specified net or subnet
• N.B. need to know subnet mask
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 2/ 18
ª ª ª ª Hands on · · · ·
build an echo server
ª copy s i mpl e - s e r ve r . c and call it e c ho- s e r ve r . c
ª alter code so that instead of asking the user for
input (ge t s ) it simply uses the last message from the
client (in buf f )
ª you will need to add to the makefile:
e c ho- s e r ve r : e c ho- s e r ve r . o $( MYLI BS)
c c $( CFLAGS) - o e c ho- s e r ve r e c ho- s e r ve r . o $( MYLI BS)

N.B. this must be a tab
ª compile and run your code
ª does your server echo everything once or twice to
its terminal?
ª the server exits after it has finished echoing
make it continue to wait for additional clients
(don’t try for two at once!)
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 2/ 19
Session 3
UNIX
Network Programming
with TCP/IP
UNIX
Network Programming
with TCP/IP
UNIX
Network Programming
with TCP/IP
UNIX
Network Programming
with TCP/IP
UNIX
Network Programming
with TCP/IP
Application
Protocols
UNIX
Network Programming
with TCP/IP
Session 3
Alan Dix
http://www.hcibook.com/alan
Standard Applications
• trusted login – rlogin
• negotiating options – telnet
• world wide web– http
ª peeking
• file transfer – ftp
• standard response codes
• electronic mail – SMTP
ª drive it by hand
• argc , argv & makefiles
ª build your own mail client
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 3/ 1
Types of Protocol
user character stream
¯ used by remote terminal/ login applications
( rlogin & telnet )
¯ most of the traffic is uninterpretted data
¯ some embedded control sequences
ascii turn-taking protocols
¯ includes ftp, SMTP, http
¯ human readable client & server messages
¯ usually line oriented
¯ turn-taking typically: client command
server response
· · ·
but roles may reverse
¯ bulk data may be embedded (SMTP, http)
or use separate connection (ftp)
binary protocols
¯ used for low level protocols:
TCP/ IP itself!
SNMP – simple network management protocol
NFS (built on top of RPC – remote procedure call)
¯ issues such as byte order important
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 3/ 2
Remote Terminal Access:
rlogin and telnet
• one of the earliest Internet application areas
• the client end – interacts with the user
• the server end – shell or command interpreter
I
´
·
I
^
'
Internet
client
server
shell
basic pattern:
' user types characters
Z the client sends them to the server
· the server passes them on to the shell
J shell generates output
· server passes output to client
± client puts output on user’s screen
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 3/ 3
Remote Terminals – Issues
• initialisation and authentication
' how does the server know who you are?
Z how do you know the server is official?
answer to Z:
¯ the server is on a reserved port (<1024)
N.B. only works for UNIX servers!
• how to deal with special characters
... including end-of-line !
• which end performs different things:
¯ user flow control (crtl-S, ctrl-Q)
¯ line editing
¯ echoing
• how do the client and server communicate:
¯ user interrupts
¯ window size changes
¯ who does what
• if embedded control characters are used
what happens if the user types them?
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 3/ 4
rlogin
• simple stable protocol
• designed for UNIX–UNIX logins
⇒ can make more assumptions
( terminal handling, interrupts, etc. )
• authentication by ‘trusted’ hosts
¯ no password required if:
client uses port <1024
and
client host is in ‘. r hos t s ’ file
¯ means that client must be s e t ui d to r oot
• responsibility
¯ echoing – server
¯ flow-control – client on server request
• client–server communication
¯ client→server initialisation string
¯ client→server window size change:
ctrl chars – 2 bytes of 255
followed by window size in 2 bytes
no protection against user typing it!
¯ server→client requests:
special characters (bytes x02, x10, x20, x80)
marked by URG (urgent) pointer
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 3/ 5
Urgent Data
• sometimes called out-of-band data
. . . but it’s not!
• data sent in normal TCP stream
• special URG pointer set
¯ officially to the last byte of urgent data
¯ BSD set it one beyond!
o m e t e x t 0x80 m o r
URG pointer
Berkeley URG pointer!
• client should:
' read until urgent data reached
Z if necessary discard intervening data
(e.g. if insufficient buffer space to store it)
problem with '
¯ URG pointer says where it ends . . .
. . . but how do you know where it starts?
¯ have to have special codes again
• with UNIX sockets
¯ send urgent data with ‘s e nd’ system call
¯ recipient gets a SI GURG signal
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 3/ 6
telnet
• cross platform ⇒ more complex
• many downward-compatible options
• can be used to connect to non-login services
• client authentication
¯ not in protocol – application specific
e.g. ge t t y
• responsibility
¯ client may handle echoing, line editing etc.
subject to option negotiation
• NVT character set
¯ needed because cross-platform
¯ 7 bit US ASCII
¯ end-of-line sent as “\ r \ n” (carriage return, line feed)
¯ carriage return sent as “\ r \ 0”
¯ also used by SMTP, ftp, finger etc.
v high bit free for control characters!
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 3/ 7
telnet – 2
control codes
• introduced by byte 255
¯ called: IAC – interpret as command
• following byte is actual control code
examples:
255 – the actual byte 255 (needed for binary mode)
236 – end of file
241 – no op
243 – break
option negotiation control codes:
251 – WILL
252 – WONT
253 – DO
254 – DONT
250 – sub-option begin
240 – sub-option end
option negotiation
• many different options:
¯ echoing ¯ line editing,
¯ flow control ¯ window size information
• client and server play “will you/ wont you”
to determine common protocol
• just like fax machines and modems
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 3/ 8
http
• the World Wide Web protocol
• protocol:
¯ ASCII control messages
¯ standard data formats for pages/ images
• uses single step transactions
' establish TCP connection
Z client sends request
· server sends reply + page
J connection closed
• why transaction based?
¯ client end – many different servers
(hypertext links to different sites)
¯ server end – many clients
¯ load time < interaction time (ideally!)
• why use TCP?
× high cost of establishing connection
¯ wide area, large messages & simple clients ⇒
v reliable communication needed
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 3/ 9
ª ª ª ª Hands on · · · ·
peeking
ª use the program pr oxy in t c p/ s e s s i on3
ª it sits between client and server
ª use it to see how http works:
' run: pr oxy www. hud. a c . uk 80 - por t 8800
Z start up Netscape using background menu
· go to the url:
ht t p:/ / www.hud.ac.uk/ schools/ comp+mat hs/ privat e/ alan/ alandix.ht ml
J now edit the host name in the url field
if your machine is i o
change / / www.hud.ac.uk to / / io.hud.ac.uk:8800
the 8800 is to set the port number used by pr oxy
· hit return and watch the pr oxy window
ª you can do the same with telnet:
' run: pr oxy z e us . hud. a c . uk 23 - por t 2300
' then: t e l ne t i o 2300
N.B. cannot be used for protected ports (ftp, mail etc.)
ª try using the - v option of f t p
type:
f t p - v pr ome t he us . hud. a c . uk
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 3/ 10
File Transfer Protocol
FTP
• used to transfer files and list directory contents
• uses two types of connection:
control – for commands and responses
data – for files and listings
• protocol for control is ascii turn-taking
client command, server response, ...
• client commands nearly user level, including:
USER
user name for connection
often ‘a nonymous ’ is accepted
PASS
password, email address for a nonymous
GET
receive a file from remote machine
PUT
send file to remote machine
CWD
change remote directory
LI ST
change remote directory
PORT
tell server what data port to use
HELP
info about commands supported
QUI T
finish session
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 3/ 11
FTP - 2
control and data
control connection
¯ server waits (passive open) on port 21
¯ client establishes connection (active open)
¯ client sends ascii commands – one per line
¯ server responds: single or multi-line response
¯ when required a data connection is established
data connection
¯ client performs a passive open on some port
(may leave OS to determine port number)
¯ client tells server using control connection
PORT 161. 112. 192. 5. 9. 93
port 2397 (=9*256+93) on host 161. 112. 192. 5
when data transfer is required
¯ client sends appropriate command
e.g. GET s i mpl e - c l i e nt . c
then waits listening for connection
¯ server performs an active open on port
then sends data
¯ server tells client when transfer is complete
e.g. 226 Tr a ns f e r c ompl e t e .
then both sides (usually) close the data port
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 3/ 12
standard response codes
• ftp server replies with lines such as:
200 PORT c omma nd s uc c e s s f ul
• SMTP and some other protocols use similar codes
• three digit codes – type given by first digit:
1yz – expect further reply from server
2yz – all OK
3yz – more required from client
4yz – temporary failure (try again)
5yz – error or permanent failure
• single-line response general format
999 a t e xt me s s a ge
space here
• multi-line response
either:
hyphen means ‘more to come’
999- f i r s t l i ne
999- one or mor e f ur t he r l i ne s
999 t he l a s t l i ne
space here on last line
or
999- f i r s t l i ne
l ot s of l i ne s a l l s t a r t i ng wi t h
a t l e a s t one s pa c e
999 t he l a s t l i ne
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 3/ 13
Simple Mail Transfer Protocol
SMTP
• allows:
¯ mail client (user interface) to send via server
¯ servers to talk to one another
(one server takes ‘client’ role)
• note:
• not used by user interface for receipt
• s e ndma i l is common SMTP server under UNIX
• client commands:
HELO
client tells server who it is
MAI L
initiates message and sets sender
RCPT
sets one of the recipients
DATA
says actual message content follows
VRFY
check that recipient exists (no mail sent)
EXPN
expand mail alias (no mail sent)
RSET
start from scratch
EHLO
see if server handles advanced features
QUI T
finish session
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 3/ 14
SMTP – 2
• authentication, servers typically:
¯ do not trust HELO
use reverse name mapping instead
¯ do trust sender name (Fr om: )
how could they verify it?
• SMTP specifies delivery not content
• other standards used for content:
¯ non-ASCII characters in headers
=? I SO- 8859- 1? Q? Al a n=20Di x? =
¯ MIME for multi-part mixed content messages
• simple mail message is just:
¯ header
Fr om: a l a n@z e us . hud. uk
To: R. Be a l e @c s . bha m. uk. a c
Subj e c t : HCI book 2E
¯ blank line
¯ body
Rus s e l l ,
ha ve you he a r d f r om Pr e nt i c e Ha l l
ye t c onc e r ni ng t he we b pa ge s ?
Al a n
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 3/ 15
ª ª ª ª Hands on · · · ·
see what it does
ª we want to send a mail message using raw SMTP!
ª first of all see how ‘ma i l ’ does it
cannot use pr oxy as SMTP is at port 25 (protected)
ª instead try the - v option of ma i l
type:
ma i l - v c 3 – or whoever you want to send mail to!
see the messages from the server and the client
N.B. not all messages are shown
ª when does mail establish the connection?
why?
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 3/ 16
ª ª ª ª Hands on · · · ·
drive it by hand
ª use telnet to send a message
type:
t e l ne t z e us . hud. a c . uk 25
ª you are connected to the SMTP server on z e us
ª say hello! which machine you are on
HELO wa l t . di s ne y. c om
did it believe you?
how does it know?
ª now say who the message is from and who it is to
MAI L Fr om: <Dona l d_Duc k>
RCPT To: <c 3@z e us . hud. a c . uk>
ª next send the message
DATA
f i r s t l i ne of me s s a ge
. . dot t y
s he a r qua c ke r y
.
ª finally say goodbye
QUI T
ª run ma i l to see if any celebrity has sent you any
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 3/ 17
argc & argv
• recall: i nt ma i n( i nt a r gc , c ha r **a r gv ) ...
or: i nt ma i n( i nt a r gc , c ha r *a r gv[ ] ) ...
• one of the ways to get information into a C program
• in UNIX you type:
mypr og a " b c " d
the program gets:
a r gc = 4 – length of a r gv
a r gv[ 0] = " mypr og" – program name
a r gv[ 1] = " a "
a r gv[ 2] = " b c " – single second argument
a r gv[ 3] = " d"
a r gv[ 4] = NULL – terminator
N.B. ¯ DOS is identical (except a r gv[ 0] is NULL early versions)
¯ a r gc is one less than the number of arguments!
• other ways to get information in (UNIX & DOS):
¯ configuration file (known name)
¯ standard input
¯ environment variables using ge t e nv( )
or (UNIX only) third arg to main:
ma i n( i nt a r gc , c ha r **a r gv, c ha r **e nvp)
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 3/ 18
Make
‘ma ke ’ is a UNIX

command which:
• automates program construction and linking
• tracks dependencies
• keeps things up-to-date after changes
to use it:
¯ construct a file with rules in it
you can call it anything, but ‘ma ke f i l e ’ is the default
¯ run ‘make’ itself
ma ke t a r ge t
– (uses the default ma ke f i l e )
ma ke - f myf i l e t a r ge t
– (uses the rule file myf i l e )
either rebuilds the program ‘t a r ge t ’ if necessary
• each makefile consists of:
¯ definitions
¯ rules
• rules say how one thing depends on another
they are either:
¯ specific – e.g. to make ma i l - c l i e nt do this ...
¯ generic – e.g. to make any ‘. o’ from its ‘.c’ ...
† ma ke is also available in many other programming environments
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 3/ 19
Makefile format
Definitions
• general form:
variable = value
• example:
SDI R = t c p
MYLI BS = $( SDI R) / l i b
N.B. one variable used in another's definition
• make variables are referred to later using $
e.g. $( SDI R) , $( MYLI BS)
• expanded like #de f i ne s or shell variables
(some versions of make will expand shell variables also)
Rules (just specific rules)
• general form:
target : dependent1 dependent2 ...
command-line

N.B. this must be a tab
• example:
mypr og: mypr og. o a not he r . o
c c - o mypr og mypr og. o a not he r . o $( MYLI BS)
this says:
to make mypr og you need mypr og. o and a not he r . o
if either of them is newer than mypr og rebuild it using the
then rebuild it using the command: “c c - o mypr og ...”
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 3/ 20
Helper Functions
standard response lines
• to make life easier!
• my own helper functions
¯ to read standard response lines
#i nc l ude " pr ot oc ol . h"
¯ to interact with SMTP server
#i nc l ude " ma i l - he l pe r . h"
i nt ge t _r e s pons e _f d( i nt s e r ve r _f d, i nt e c ho_f d,
c ha r *buf f , i nt l e n ) ;
• reads from s e r ve r _f d
• parses a single or multi-line response
• returns the response code (of last line)
• echoes full response to e c ho_f d
• also copies it into buf f if non-NULL
i nt ge t _r e s pons e _f p( FI LE *s e r ve r _f p, FI LE *e c ho_f p,
c ha r *buf f , i nt l e n ) ;
• similar with s t di o files
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 3/ 21
Helper Functions – 2
for sending mail
i nt do_ma i l _i ni t ( i nt s e r v_f d) ;
• awaits first response and does ‘HELO’
• checks response and returns 0 if OK
i nt do_ma i l _f r om( i nt s e r v_f d, c ha r *f r om) ;
i nt do_ma i l _t o( i nt s e r v_f d, c ha r *t o) ;
• sends ‘MAI L’ and ‘RCPT’ commands respectively
• sender (f r om) and recipient (t o) are C strings
i nt do_ma i l _da t a _f p( i nt s e r v_f d, FI LE *us e r _f p) ;
i nt do_ma i l _da t a _buf f ( i nt s e r v_f d, c ha r *buf f , i nt l e n) ;
• send ‘DATA’ command and send message
copied from us e r _f p or buf f respectively
i nt do_ma i l _qui t ( i nt s e r v_f d) ;
• does ‘QUI T’ command
All optionally echo all exchanges to a file (or terminal) set by:
FI LE *do_ma i l _s e t _e c ho_f p( FI LE *ne w_e c ho_f p)
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 3/ 22
ª ª ª ª Hands on · · · ·
build your own mail client
ª copy s i mpl e - c l i e nt . c and call it ma i l - c l i e nt . c
ª copy the following from t c p/ s e s s i on3:
ma i l - he l pe r . c
ma ke 3
the makefile is ready to compile your mail client
you can type (when ready!):
ma ke - f ma ke 3 ma i l - c l i e nt
N.B. ' SMTP obeys strict turn-taking:
server–client–server–client–server
Z server starts with a return code
· but client ‘in control’
ª modify the client code
' set default host (z e us ) and port (25)
Z to and from addresses:
either read in or use argv
· message: initially read a single line
J ‘unwrap’ loop to give fixed turns
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 3/ 23
ª ª ª ª Hands on · · · ·
mail client – 2
ª resulting program structure:
(a) r e a d ( pa r s e ) t o/ f r om a ddr e s s e s f r om us e r
(b) r e a d me s s a ge f r om us e r ( ge t s or s c a nf )
(c) ope n t c p c onne c t i on t o ma i l s e r ve r on c or r e c t por t
(d) wa i t f or s e r ve r r e s pons e l i ne ( s )
(e) s a y he l l o t o s e r ve r
(f) wa i t f or s e r ve r r e s pons e l i ne ( s )
(g) s a y who t he ma i l i s f r om
(h) wa i t f or s e r ve r r e s pons e l i ne ( s )
(i) s a y who t he ma i l i s t o
(j) wa i t f or s e r ve r r e s pons e l i ne ( s )
(k) s a y t ha t da t a i s c omi ng
(l) wa i t f or s e r ve r r e s pons e l i ne ( s )
(m) s e nd one l i ne me s s a ge
(n) s e nd l i ne wi t h j us t f ul l s t op
(o) wa i t f or s e r ve r r e s pons e l i ne ( s )
(p) s a y goodbye
(q) wa i t f or s e r ve r r e s pons e l i ne ( s )
(r) c l os e c onne c t i on
ª compile and run your code!
ª if you have time modify it to send longer messages
either: change step (b) and (m) to accept long messages
or: remove step (b) and
make (m) read from user before sending each line
or: whatever you like ...
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 3/ 24
Session 4
UNIX
Network Programming
with TCP/IP
UNIX
Network Programming
with TCP/IP
UNIX
Network Programming
with TCP/IP
UNIX
Network Programming
with TCP/IP
UNIX
Network Programming
with TCP/IP
Concurrent
Clients
UNIX
Network Programming
with TCP/IP
Session 4
Alan Dix
http://www.hcibook.com/alan
• sequential and concurrent clients
• techniques for concurrency
• call-backs
• knowing what you’re doing
• callback–based client
ª using it
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 4/ 1
Sequential Clients
e.g. FTP
1. client waits for user input
2. user types “DI R”
3. client performs passive open on data port (2397)
4. client sends “PORT 161. 112. 192. 5. 9. 93” to server
5. client waits for standard ‘200’ reply line
6. if not OK then fail
7. client sends “LI ST” to server
8. client waits for standard ‘150’ reply line
9. if not OK then fail
10. client reads from data port
11. client waits for standard ‘226’ reply line
12. if not OK then fail
13. report success to user
• client is in control
• next client action depends on:
¯ what happened last
e.g. what commend the user types
¯ NOT on when it happens
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 4/ 2
Naturally Concurrent Clients
e.g. telnet
• at any moment either
user may type something
or
output may come from server end
• client must respond whichever happens
• program a bit like:
¯ when user types
then send to the server
¯ when server sends message
then print on terminal
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 4/ 3
Concurrency for Usability
e.g. Netscape – WWW client
• basic protocol transaction based
×. but response can be slow
/ interaction allowed during transaction
~ scrolling
~ ‘STOP’ button
⇒ client has to listen to
server – more data
user – mouse and keyboard
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 4/ 4
Programming Concurrency
Problem
• doing more than one thing at once
listening user terminal & TCP server port
Solutions
• polling
¯ use non-blocking I/ O
× keeps processor busy
• threads
¯ needs built-in support (language or OS)
¯ program written as several sequential parts
¯ all executed at the same time
¯ communicate using shared data
(also semaphores etc.)
• event driven programming
¯ low-level – e.g. UNIX select
¯ event-loop – e.g., raw X and Mac
¯ program paradigm – e.g. Visual Basic, HyperCard
¯ call-backs – e.g., Windows, X Motif
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 4/ 5
Event Loop
Typical program structure
f or ( ; ; ) { / * l oop f or e ve r */
s t r uc t e ve nt _s t e ve nt ;
r e a d_e ve nt ( &e ve nt ) ;
i f ( e ve nt . t ype == BUTTON
&& e ve nt . t a r ge t = qui t _but t on)
r e t ur n OK;
e l s e i f ( e ve nt . t ype == KEYPRESS )
i ns e r t _c ha r ( e ve nt . c ha r ) ;
e l s e i f ( e ve nt . t ype == I NPUT_READY )
do_ne t wor k_t a s k( e ve nt . buf f ) ;
. . .
}
/ programmer in control
× related code gets spread out in if/ case statements
• often written with sub-loops e.g. for dialogue boxes
⇒ unforeseen events (e.g. network I/ O)
may be delayed or even ignored!
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 4/ 6
Event–Based Languages
program = collection of event handlers
e.g. HyperCard
on mous e Up
s e t c ur s or t o wa t c h
put ge t Se r ve r Addr e s s ( ) i nt o s e r ve r Addr
put ge t Us e r Na me ( ) i nt o us e r Na me
put c d f l d " ToOr Fr om" i nt o t oNa me
put c d f l d " Me s s a ge " i nt o t he Me s s
s e nd " t oSe r ve r Se ndMa i l " ¬
&& quot e & t oNa me & quot e & c omma ¬
&& quot e & us e r Na me & quot e & c omma ¬
&& quot e & t he Me s s & quot e ¬
t o pr ogr a m s e r ve r Addr
e nd mous e Up
on Appl e Eve nt c l a s s , i d, s e nde r
a ns we r " Appl e Eve nt " && c l a s s && “ f r om” && s e nde r
- - di a l ogue box f or us e r
e nd Appl e Eve nt
/ concurrency naturally part of language
× network I/ O not always treated uniformly
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 4/ 7
Call-backs
used in many toolkits and window mgrs:
e.g.:
¯ WinSock (TCP/ IP under Windows)
¯ X Motif
General pattern
Program
' define a function
Z tell toolkit to attach it to event
· give control to the toolkit
Toolkit
· when event happens
call user defined function
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 4/ 8
Example – X Motif Call-backs
Xt AddCa l l ba c k( widget, callback-type, func, my-data )
widget – a widget such as a button
type – a callback resource name:
which type of event to respond to
e.g., XmNactivateCallback
func – pointer to C function defined by you
e.g., quit_func
my-data – an integer or pointer to your data
passed on to your callback
The callback function definition:
voi d qui t _f unc ( widget, my-data, event-data )
widget – where the event occurred
my-data – the integer or pointer passed in
the call to XtAddCallback
event-data – the X event structure which caused
the callback
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 4/ 9
What’s going on?
Sequential Programs
f or ( ; ; ) { / * N. B. ps e udo- C ! ! ! */
ge t s ( c omma nd) ;
i f ( . . . )
i f ( c omma nd i s " qui t " ) {
c ha r r e s pons e [ MAX_LI NE_SI ZE+1] ; ← Z
wr i t e ( s e r v_s d, " QUI T\ n" , 5) ;
' ª r e a d( s e r v_f d, r e s pons e , MAX_LI NE_SI ZE) ;
i f ( r e s pons e [ 0] ! = ' 2' ) . . .
pr i nt f ( " s e s s i on c ompl e t e \ n" ) ;
e xi t ( 0) ;
}
i f ( . . . )
}
features for free
' program counter (ª)
– what you are doing
Z local variables
– what you are doing it to
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 4/ 10
What’s going on? - 2
sequential concurrent
implicit explicit
• local variables
global variables
or dynamic data structures
e.g. partial line of user input
• program counter
mode variable
or finite state machines!
e.g. TELNET command sequences
server output modes:
' normal echoing
Z waiting for command
· waiting for option
not byte 255
byte 255
3 2 1
253 – DO
254 – DONT
any
option
other bytes
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 4/ 11
Callback based client – 1
' Initialisation
ma i n( . . . ) {
/ * request connection to server */
s d = t c p_a c t i ve _ope n( hos t , por t )
/ * set-up callback for server */
i nf or m_i nput ( s d, r e a d_s oc ke t , NULL) ;
/ * set-up call-backs for interface */
. . .
/ * give control to toolkit */
i nf or m_l oop( ) ;
}
^ When server sends a message ...
... read_socket is called
r e a d_s oc ke t ( i nt s d, . . . ) {
/ * read server’s message */
l e n = r e a d( s d, buf f , buf _l e n) ;
/ * process message */
/ * probably update interface */
}
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 4/ 12
Callback based client – 2
· When user does something ...
... appropriate function is called
t e r m_l i ne ( i nt f d, voi d *i d, c ha r *buf f ) {
/ * process interface event */
me s s ( " s e ndi ng {%s }\ n" , buf f ) ;
/ * possibly send message to server */
wr i t e ( s d, buf f , s t r l e n( buf f ) ) ;
}
step ' once at initialisation
steps ^ & · any number of times in any order
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 4/ 13
ª ª ª ª Hands on · · · ·
an electronic conference
ª copy the following from t c p/ s e s s i on4:
c l i e nt . c
s e r ve r . c
ma ke 4
the makefile is ready to compile, type :
ma ke - f ma ke 4 c onf
ª one person run the server:
i o: s e r ve r
ª two or more others run the client:
ot he r : c l i e nt - hos t i o
N.B. you cannot participate from the server
to join in launch a client in another
window of the server’s machine
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 4/ 14
UNIX
Network Programming
with TCP/IP
UNIX
Network Programming
with TCP/IP
UNIX
Network Programming
with TCP/IP
UNIX
Network Programming
with TCP/IP
UNIX
Network Programming
with TCP/IP
Session 5

Server Design
UNIX
Network Programming
with TCP/IP
Session 5
Alan Dix
http://www.hcibook.com/alan
• types of server
• handling server concurrency
• server state
• stateless servers
• when things go wrong!
• survival – the 3 Rs
• callback–based server
ª modify server
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 5/ 1
Servers
Kinds of server
' transaction based
e.g. database: 1 query → 1 result
Z strict turn-taking
e.g. ftp
· inherent concurrency
e.g. electronic conferences, MUDs
for lots of clients either:
• serve one at a time in turn
⇒ ' may be slow
Z may take forever!
• serve several at the same time
⇒ both require concurrency
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 5/ 2
Server Concurrency
• similar solutions to client
¯ polling
/ acceptable if machine dedicated to server
¯ threads
¯ UNIX select
¯ event driven
× less likely to run in event-based system
/ some web based servers do
• in addition:
¯ when no intrinsic concurrency
¯ can use UNIX fork
/ launch separate process to serve each client
so each is simpler
/ uses standard UNIX process concurrency
× can be expensive (process creation)
especially with lots of small transactions
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 5/ 3
Server State
• concurrent server needs to remember
¯ how many clients
¯ state of their connection
¯ state of each transaction/ protocol
etc. etc. etc.
×. many clients ⇒ large state
×. disaster scenarios
¯ client establishes connection
¯ client crashes
¯ client restarts
¯ client establishes a new connection
¯ it crashes again ...
/ solution – no state
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 5/ 4
Stateless Servers
stateless = no per client state
• for transaction based services
¯ client makes request
¯ server performs action
¯ server returns result
• really only possible with UDP
e.g. http – transaction based, but uses TCP
⇒ may need several reads for request
need to store partially filled buffer ...
N.B. in general, buffers part of the per client state
× not all plain sailing ...
¯ clients have to maintain more state
¯ requests more complex (no context)
¯ unreliable protocol ⇒
transactions must be idempotent
time-outs for lots transactions ...
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 5/ 5
When things go wrong
PC crash ⇒ one sad user
server crash ⇒ lots of angry users
• take special care with servers!
• probability of failure:
clients – prob. of failure = p
server – prob. of failure = q
n clients and only 1 server, so:
probability of some failure ≈ np+q
• good news!
¯ server failure less likely (or is it?)
• bad news!
¯ servers are more complex (q > p)
¯ what if client brings server down?
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 5/ 6
Causes of failure
' hardware failures
^ programming errors
· unforeseen sequences of events
´ system does not scale
Large number of components
⇒ ' more frequent
Complexity of algorithms
⇒ ^ more likely
Interleaving and delays
⇒ · difficult to debug
Limited testing conditions
⇒ ´ unexercised
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 5/ 7
Survival
Network or server failure
standard solutions
Client fails — three Rs for server
• robust
server should survive
• never wait for response from client
• non-blocking network I/ O
• reconfigure
detect and respond to failure
• time-out or failure of I/ O operations
• reset internal data structures
• inform other clients
• resynchronise
catch up when client restarts
• similar to new client
• N.B. client may not know (network)
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 5/ 8
Software faults
Defensive programming
• inconsistent client/ server data structures
Use simple algorithms
• fixed sized structures – but check bounds!
• may conflict with scaleability – document
Verify
• close hand checks
• for production code – formal methods
Unforeseen sequences of events
• deadlock – never use blocking I/ O
• never assume particular orders of events
• back-to-back messages
network packet ≠ logical message
Debugging and testing
• logging – to reproduce failure
• random data – at interface or network
• ask your friends
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 5/ 9
Callback based server – 1
' Initialisation
ma i n( . . . ) {
/ * establish port */
pd = t c p_pa s s i ve _ope n( por t )
/ * set-up callback for port */
i nf or m_i nput ( pd, a c c e pt _c l i e nt , NULL) ;
/ * give control to notifier */
i nf or m_l oop( ) ;
}
^ When client requests connection ...
... notifier calls accept_client
a c c e pt _c l i e nt ( . . . ) {
/ * accept client’s connection */
f d = t c p_a c c e pt ( por t _f d) ;
/ * record connection details */
c l i e nt _f d[ c ount ] = f d;
/ * set-up callback for client */
i nf or m_i nput ( f d, r e a d_c l i e nt , c ount ) ;
/ * keep track of number of clients */
c ount = c ount +1;
/ * probably tell other clients also */
}
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 5/ 10
Callback based server – 2
· When client sends message ...
... notifier calls read_client
r e a d_c l i e nt ( c _f d, i d ) {
/ * read client’s message */
l e n = r e a d( c _f d, buf f , buf _l e n) ;
/ * broadcast to other clients */
f or ( c =0; c <c l i e nt _c ount ; c ++) {
i f ( c l i e nt _f d == c _f d ) {
/ * special reply for sender */
}
e l s e {
/ * relay message to other clients */
}
}
N.B. step ' performed once at initialisation
steps ^ & · happen any number of times ...
... in any order
• similar to client code, but with extra ‘accept’ stage.
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 5/ 11
My window-less callbacks – 1
• so you can experience the pain of
callbacks without the added pain of
windows ...
# i n c l u d e " i n f o r m. h "
i nt i nf or m_i nput ( i nt f d, i nf or m_f un f ,
i nf or m_i d i d ) ;
• function f is your callback
• f is called when a buffer can be read from f d
... without blocking
• the identifier i d is also passed to f
i nt i nf or m_out put ( i nt f d, i nf or m_f un f ,
i nf or m_i d i d ) ;
• similar to i nf or m_i nput but for output
• f is called when a buffer can be written to f d
i nt i nf or m_l oop( ) ;
• gives control to the 'notifier' which performs
callbacks for you
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 5/ 12
My window-less callbacks – 2
# i n c l u d e " l i n e _ b y _ l i n e . h "
i nt i nf or m_l i ne _by_l i ne ( i nt f d, l i ne _f un l i ne _f ,
e of _f un e of _f , i d_t ype i d ) ;
• the file f d is monitored by notifier
• two callbacks: l i ne _f and e of _f
• l i ne _f is called when a complete line is read
• e of _f is called when the end of file is reached
# i n c l u d e " mo n i t o r . h "
s t r uc t mon_t a b_s t r uc t moni t or _t a b[ ] = {
{ 0, " command" , callback, " description" },
{ 0, 0, 0, 0 }
};
i nt pe r f or m_l i ne ( c ha r *buf f ) ;
• helper for simple command interface
• you make moni t or _t a b with suitable functions
• the first word in buf f is regarded as a command
• it is looked up in moni t or _t a b
. . . and the relevant callback is run
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 5/ 13
ª ª ª ª Hands on · · · ·
< the conference server is not very friendly
it refers to everyone by number
you are going to make this better!
ª copy s e r ve r . c call it ne w- s e r ve r . c
ª edit the makefile ma ke 4 so that you can
compile ne w- s e r ve r . c by using:
ma ke - f ma ke 4 ne w- c onf
ª locate the place where the server first establishes
contact with the client.
ª make the server wait for a line (or buffer) of input
from the client (the clients name)
ª modify the notification message it sends to all the
clients to make it name the user
ª compile and run (use the same client)
run several clients, do you notice delays?
< Harder bits
ª add the user name to the per-client data structure
ª alter the server so that all messages use the name
rather than client number
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 5/ 14
UNIX
Network Programming
with TCP/IP
UNIX
Network Programming
with TCP/IP
UNIX
Network Programming
with TCP/IP
UNIX
Network Programming
with TCP/IP
UNIX
Network Programming
with TCP/IP
Session 6
Forking Servers
& more TCP/IP
UNIX
Network Programming
with TCP/IP
Session 6
Alan Dix
http://www.hcibook.com/alan
Forking Servers
& TCP/IP behaviour
• UNIX processes and fork
• forking servers
• fork system call
• example code
• dup, exec and wait
ª remote shell
• inet demon and remote login
ª another echo server
• IP fragmentation
• TCP flow control
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 6/ 1
Loosely coupled services
• closely coupled:
strong client interaction
e.g. electronic conference
• loosely coupled:
little or no client interaction
e.g. WWW
• no interaction at all ⇒
separate process to serve each client
• weak interaction ⇒
need locking, database server etc.
i.e. some central point of control
server
process
client
server
process
client
server
process
client
server
process
client
database server file locking demon
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 6/ 2
A UNIX process
UNIX process:
• identified by process id (pid)
• process includes:
¯ program code
¯ application data
¯ system data
× including file descriptors
code
data
system data
e.g. file descriptors
pid = 597
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 6/ 3
Forking
UNIX 'fork' duplicates process:
• copies complete process state:
¯ program data + system data
¯ including file descriptors
• code immutable – shared
$ e c ho $$
597
$ ( e c ho $$)
632
$
code
data
system data
597
code
data
system data
632
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 6/ 4
Forking – 2
• old process called the parent
• new process called the child
• process ids
¯ allocated sequentially
¯ so effectively unique
(but do wrap after a very long time)
• finding process ids
¯ at the shell prompt:
use 'p s '
¯ in a C program:
use 'i n t p = g e t p i d ( ) ; '
¯ in a shell script:
use '$ $'
N.B. useful for naming temporary files:
t mp f i l e = " / t mp / my f i l e $ $ "
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 6/ 5
Use in servers
' the server passive opens a port
and waits for a client
client
server
Z the client performs an active open
a connection is established
client
server
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 6/ 6
Use in servers – 2
· the server forks a child
client
child server
fork
• child is a copy of the server
• both socket connections are duplicated
⇒ server waiting on port . . .
. . . and child waiting on port
⇒ child connected to client . . .
. . . and server connected to client
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 6/ 7
Use in servers – 3
J server closes the connection
child closes the passive port
client
child server
· server waits for further connections
child talks to client
client
child server
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 6/ 8
Fork system call
pi d_t p = f or k( ) ;
( pi d_t ≈ i nt )
• if successful
¯ process
¯ successful fork returns:
0 – to child process
child pid – to parent process
⇒ parent and child are different!
• negative result on failure
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 6/ 9
Execution – 1
• parent forks
597
~
i nt i = 3, c _pi d = - 1;
c _pi d = f or k( ) ;
i f ( c _pi d == 0 )
pr i nt f ( " c hi l d\ n" ) ;
e l s e i f ( c _pi d > 0 )
pr i nt f ( " pa r e nt \ n" ) ;
e l s e
pr i nt f ( " f a i l e d\ n" ) ;
DATA
i = 3
c _pi d = - 1
• after fork parent and child identical
597 632
~
i nt i = 3, c _pi d = - 1;
c _pi d = f or k( ) ;
i f ( c _pi d == 0 )
pr i nt f ( " c hi l d\ n" ) ;
e l s e i f ( c _pi d > 0 )
pr i nt f ( " pa r e nt \ n" ) ;
e l s e
pr i nt f ( " f a i l e d\ n" ) ;
~
i nt i = 3, c _pi d = - 1;
c _pi d = f or k( ) ;
i f ( c _pi d == 0 )
pr i nt f ( " c hi l d\ n" ) ;
e l s e i f ( c _pi d > 0 )
pr i nt f ( " pa r e nt \ n" ) ;
e l s e
pr i nt f ( " f a i l e d\ n" ) ;
DATA
i = 3
DATA
i = 3
c _ p i d = 6 3 2 c _ p i d = 0
• except for the return value of fork
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 6/ 10
Execution – 2
• because data are different
597 632
~
i nt i = 3, c _pi d = - 1;
c _pi d = f or k( ) ;
i f ( c _pi d == 0 )
pr i nt f ( " c hi l d\ n" ) ;
e l s e i f ( c _pi d > 0 )
pr i nt f ( " pa r e nt \ n" ) ;
e l s e
pr i nt f ( " f a i l e d\ n" ) ;
~
i nt i = 3, c _pi d = - 1;
c _pi d = f or k( ) ;
i f ( c _pi d == 0 )
pr i nt f ( " c hi l d\ n" ) ;
e l s e i f ( c _pi d > 0 )
pr i nt f ( " pa r e nt \ n" ) ;
e l s e
pr i nt f ( " f a i l e d\ n" ) ;
DATA
i = 3
DATA
i = 3
c _ p i d = 6 3 2 c _ p i d = 0
• program execution differs
597 632
~
i nt i = 3, c _pi d = - 1;
c _pi d = f or k( ) ;
i f ( c _pi d == 0 )
pr i nt f ( " c hi l d\ n" ) ;
e l s e i f ( c _pi d > 0 )
pr i nt f ( " pa r e nt \ n" ) ;
e l s e
pr i nt f ( " f a i l e d\ n" ) ;
~
i nt i = 3, c _pi d = - 1;
c _pi d = f or k( ) ;
i f ( c _pi d == 0 )
pr i nt f ( " c hi l d\ n" ) ;
e l s e i f ( c _pi d > 0 )
pr i nt f ( " pa r e nt \ n" ) ;
e l s e
pr i nt f ( " f a i l e d\ n" ) ;
DATA
i = 3
DATA
i = 3
c _ p i d = 6 3 2 c _ p i d = 0
• so parent and child behaviour diverge
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 6/ 11
fork based shell server – 1
Basic structure:
• establish port
• loop forever
• on each loop:
¯ accept a single client connection
¯ fork a child to manage client
• child execs a copy of the shell
N.B. no login – very insecure !
' Main loop
ma i n( . . . ) {
/ * ope n por t */
por t _s k = t c p_pa s s i ve _ope n( por t )
/ * l oop f or e ve r a c c e pt i ng c l i e nt s */
whi l e ( a c c e pt _one ( por t _s k) > 0 ) ;
/ * on e r r or c l os e a nd e xi t */
c l os e ( por t _s k) ;
e xi t ( 0) ;
}
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 6/ 12
fork based shell server – 2
^ Process each client in turn
a c c e pt _one ( i nt por t _s k ) {
/ * accept a single connection */
c l i e nt _s k = t c p_a c c e pt ( por t _s k) ;
/ * perform fork */
c hi l d_pi d = f or k( ) ;
• child gets zero return from fork
i f ( c hi l d_pi d == 0 ) {
/ * child closes passive port */
c l os e ( por t _s k) ;
/ * then starts its own behaviour */
e xe c _a _s he l l ( c l i e nt _s k) ;
}
• parent gets child process id returned from fork
e l s e i f ( c hi l d_pi d > 0 ) {
/ * parent closes client socket */
c l os e ( c l i e nt _s k) ;
/ * N.B. child has open descriptor */
/ * so client is not cut off */
/ * returns child pid to main loop */
r e t ur n c hi l d_pi d;
}
• negative result on failure
e l s e r e t ur n 0;
}
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 6/ 13
fork based shell server – 3
· Child e xe c s a copy of the shell
N.B. only the child process calls this function
i nt e xe c _a _s he l l ( i nt f d) / * doe s n' t r e t ur n */
{
i nt t t y_f d; ;
• shell will expect I/ O from standard file descriptors
use 'dup2' system call to link them to f d
dup2( f d, 0) ; / * standard input from fd */
dup2( f d, 1) ; / * standard output to fd */
dup2( f d, 2) ; / * standard error to fd */
c l os e ( f d) ;
e xe c v( " / bi n/ s h" , a r gv) ;
• exec only returns if it fails
• standard error has been closed
so need to open / de v/ t t y explicitly
t t y_f d = ope n( " / de v/ t t y" , 1) ;
wr i t e ( t t y_f d, e xe c _f a i l _me s s ) ;
_e xi t ( 1) ;
}
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 6/ 14
dup2 system call
i nt r e s = dup2( ol d_f d, ne w_f d) ;
• makes ne w_f d point to same file/ stream as ol d_f d
• ne w_f d is closed if already open
• most often used with standard I/ O descriptors:
dup2( f d, 0) ;
– standard input reads from f d
• can close the old descriptor
. . . but new descriptor still works
dup2( f d, 0) ;
c l os e ( f d) :
n = r e a d( 0, buf f , buf f _l e n) ;
• negative return on failure
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 6/ 15
exec system call
e xe c v( c ha r *pr og, c ha r **a r gv) ;
• replaces the current process with pr og
• never returns except on failure
• a r gv is passed to the 'ma i n' of pr og
N.B. needs at least a r gv[ 0] set to program name
• new process:
¯ code – replaced by pr og
¯ data – reinitialised
¯ system data – partly retained
+ file descriptors still open
• several variants (e xe c l , e xe c vp, . . . )
• often used after f or k to spawn a fresh program
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 6/ 16
exec vs. fork
• fork duplicates process
• exec replaces process
code
data
system
597
code
data
system
632
code
data
system
597
code
data
system
493
code
data
system
493
fork exec
• fork child shares open file descriptors
• exec-ed process retains open fds
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 6/ 17
death of a forked process
• when parent dies
¯ children become orphans !
¯ system init process 'adopts' them
• when child dies
¯ parent (or init) informed by signal
( SI GCHLD )
¯ child process partly destroyed
¯ rump retained until parent 'reaps'
– using wa i t or wa i t 3 system call
¯ until then child is 'zombie'
– ps says <e xi t i ng> or <de f unc t >
N.B. zombie state necessary so parent
can discover which child died
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 6/ 18
SIGCHLD & wait3
• if parent does not reap children
. . . they stay zombies forever
. . . system resources may run out
' first catch your signal
s i gna l ( my_r e a pe r , SI GCHLD) ;
• function 'my_r e a pe r ' called when signal arrives
Z then reap a child
i nt my_r e a pe r ( )
{
uni on wa i t s t a t us ;
whi l e ( wa i t 3( &s t a t us , WNOHANG, NULL) >= 0 ) ;
}
• use WNOHANG so that wa i t 3 doesn't block
• loop to reap multiple children
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 6/ 19
fork and I/O
low-level I/ O

open file descriptors shared so:
¯ output is merged
¯ input goes to first read
– accept similar
¯ close down may be delayed
until all processes close fd
⇒ c l os e all unwanted fds
or use i oc t l to set c l os e - on- e xe c
high-level I/ O

C stdio is buffered:
¯ duplicated at fork
¯ may get flushed after fork
⇒ duplicate writes
/ s t de r r OK – unbuffered
⇒ careful with stdio
use s t de r r or s e t buf f ( f d, NULL)
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 6/ 20
ª ª ª Hands on · · ·
ª copy the following from t c p/ s e s s i on6:
kni f e . c
ma ke 6
ª compile kni f e . c :
ma ke - f ma ke 6 kni f e
ª launch the knife server: kni f e . c :
i o 3% kni f e - por t 2345
ª connect to it from a different machine or window
kl a h 7% t e l ne t i o 2345
ª do you get a shell prompt?
ª try something simple like
e c ho he l l o
ª then try ps
ª what happens?
ª try typing a # at the end of each line
e c ho he l l o#
ps #
ª what is happening?
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 6/ 21
inet demon
• there are many Internet services:
ftp, telnet, rlogin, echo, etc.
• a server for each is expensive
• i ne t d is a multi-service server
• it does a passive open on lots of ports:
21 – ftp, 25 – SMTP, etc.
• when a client connects
it forks the appropriate service
• remote logins somewhat complicated
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 6/ 22
remote login
First solution . . .
. . . simply fork a shell or getty
× no translation of codes
e.g. end of line sequence
× no terminal driver at server end
⇒ no tty control by application
e.g. editors need tty raw mode
Actual solution . . .
. . . intermediate process
• server-end process
between client and shell/ getty
/ can perform translation
/ pseudo-tty between it and shell
⇒ server-end tty control
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 6/ 23
remote login – 2
' remote login client connects to server
client
server
Z server forks child to handle login
client
child server
fork
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 6/ 24
remote login – 3
· child then forks another process
/ de v
client
child
server
fork and
exec
shell
J the new process connects to the child
using a pseudo-terminal
· and finally execs a shell (or getty etc.)
+ user is now connected to shell
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 6/ 25
remote login – 4
• client and server-side child similar
¯ both connected to network
¯ both connected to (pseudo)terminal
• general algorithm:
¯ echo terminal input to network
¯ echo network input to terminal
N.B. both concurrent
• difference in use of terminal:
¯ where
client – application end of tty
child – 'user' end of pseudo-tty
¯ how
client – tty always in raw mode
child – pseudo-tty mode set by shell
⇒ only one layer of tty processing
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 6/ 26
ª ª ª Hands on · · ·
echo server
ª modify kni f e . c to make a forking echo server
your previous echo server (session 2) only dealt
with one client – this one will deal with any number
ª copy kni f e . c into e c ho- a l l
ª locate the sub-routine where the shell is exec-ed
ª replace the code duplicating file descriptors and
exec-ing the shell – simply have a loop which reads
from the socket and writes back to it
ª compile and run e c ho- a l l
i o 15% ma ke - f ma ke 6 e c ho- a l l
i o 16% e c ho- a l l - por t 2345
ª an connect to it:
kl a h 23% t e l ne t i o 2345
ª there is an alternative solution which only involves
replacing 2 characters of kni f e . c
ª hint: the answer doesn't involve any dogs
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 6/ 27
MTUs
• the Internet is heterogeneous
¯ heterogeneous transport layers
⇒ different packet sizes
¯ dynamic routing
⇒ hops on different layers
⇒ unpredictable packet size
• transport layer limit called MTU:
– maximum transmission unit
transport layer MTU in bytes
Hyperchannel 65535
16Mbps IBM token ring 17914
4Mbps IEEE 802.5 token ring 4464
FDDI 4352
Ethernet 1500
IEEE 802.3/ 802.2 1492
X.25 576
PPP (performance limit) 296
(from RFC 1191)
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 6/ 28
IP fragmentation
• what happens when size is too small?
• fragmentation
¯ any intermediate router detects problem
¯ IP datagram broken into pieces
¯ each sent separately (possibly different routes)
¯ reconstructed at further router or destination
• real limit is recipient's buffer size
¯ 576 bytes IP datagram guaranteed
... but this includes headers
¯ UDP limit = 512 bytes user data
¯ TCP divides data up for you
limit is UNIX read/ write buffers
• only end points matter
⇒ in a controlled environment . . .
. . . larger datagrams possible
e.g. NFS = 8192 bytes
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 6/ 29
fragmentation considered harmful
• fragmentation
⇒ IP transparent to underlying link layer MTU
. . . well almost . . .
• IP is not reliable
⇒ some packets (fragments) may be lost
• no re-transmission
¯ IP handles reconstruction . . .
. . . but not fragment retransmission
¯ fragment lost
⇒ whole IP datagram lost
¯ probability one fragment lost = p
n fragments
⇒ probability IP datagram lost ≈ n p
• avoiding fragmentation
¯ UDP – most protocols ≤ 512 bytes
¯ TCP – uses local (end-point) MTU
+ path MTU discovery algorithm
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 6/ 30
TCP reliability
• underlying IP unreliable
⇒ TCP must handshake
• stream protocol
¯ sender: this is bytes n–m of the data
¯ recipient: ack m – last byte received
• retransmission
¯ recipient: out of order receipt → repeat ack
¯ timeout or several repeat acks → retransmit
• too many acks
¯ avoid lots of little acknowledgement packets
¯ ack of last packet ⇒ previous packets arrived
¯ piggyback A→B ack on B→A message
¯ delay acks to allow piggyback
¯ turn off delay for some protocols (e.g. X)
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 6/ 31
TCP flow control
Cannot send without limits:
B network capacity → packet loss
¯ exponential backoff
rapid resend → nightmare scenario
⇒ long delay before failure (2-9 mins)
¯ slow-start algorithm
B link-layer buffer
¯ MSS announcement
B TCP buffer
¯ window size announcement
only send to last ack + window size
known to
be received
sent but not
acknowleged
may be sent
before next ack
must be held
at sender end
window size
last ack last byte sent
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 6/ 32
UNIX
Network Programming
with TCP/IP
UNIX
Network Programming
with TCP/IP
UNIX
Network Programming
with TCP/IP
UNIX
Network Programming
with TCP/IP
UNIX
Network Programming
with TCP/IP
Session 7
Select and
Security
UNIX
Network Programming
with TCP/IP
Session 7
Alan Dix
http://www.hcibook.com/alan
Select and
Security
• UNIX events
• select system call
• proxy server
ª raw client
• security, secrecy and privacy
• under attack: viruses & worm
• the Internet worm
• levels of security
• encryption and authentication
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 7/ 1
UNIX Events
Computational programs:
• busy most of the time
• read/ write when they are ready
Interactive programs:
• servers & clients
• idle most of the time
• respond to events
UNIX processes – 4 types of event
' signal (interrupt)
Z time (alarm)
· input ready
read will not block
J output can accept (more) data
write will not block
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 7/ 2
Responding to events
Events:
' signal (interrupt)
Z time (alarm)
· input (read) ready
J output (write) ready
Responding
• interrupt handler – '
&
Z
use s i gna l system call
use s e t i t i me r to send SI GALRM
• turntaking – Z
,
·
&
J
call r e a d/ wr i t e when ready
use s l e e p for delays
• polling – Z
,
·
&
J
use non-blocking r e a d/ wr i t e
use t i me to do things at specific times
• wait for several events
use s e l e c t system call
timeout or SI GALRM
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 7/ 3
polling in UNIX
#i nc l ude <s ys / f i l i o. h>
i oc t l ( f d, FI ONBI O, 1) ;
• call to i oc t l tells system:
don’t block on r e a d/ wr i t e
• polling therefore possible
• structure of polling telnet-like client:
i oc t l ( t t y_f d, FNBI O, 1) ;
i oc t l ( ne t _f d, FNBI O, 1) ;
f or ( ; ; ) {
/ * a ny t e r mi na l i nput ? */
n = r e a d( t t y_f d, buf f , buf f _l e n) ;
i f ( n > 0 ) { / * ye s ! do s ome t hi ng */ }
/ * a ny ne t wor k i nput ? */
n = r e a d( ne t _f d, buf f , buf f _l e n) ;
i f ( n > 0 ) { / * ye s ! do s ome t hi ng */ }
}
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 7/ 4
read & write
r e a d:
• waits on one file descriptor
• returns when input data is ready
• and reads the data into a buffer
r e a d( 0, buf f , l e n)
wr i t e :
• waits on one file descriptor
• returns when output is possible
• and writes the data from the buffer
wr i t e ( 1, buf f , l e n)
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 7/ 5
select
s e l e c t :
• waits on many file descriptor
• returns when input or output ready
• but does no actual I/ O
+ also allows timeout
s e l e c t ( wi dt h, &i n_f ds , &out _f ds , &e r r _f ds , &t i me out )
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 7/ 6
select system call – 2
i nt r e t =
s e l e c t ( s i z e , &i n_f ds , &out _f ds , &e r r _f ds , &t i me out ) ;
• i n_f ds , out _f ds :
– bitmaps of file descriptors
¯
i n_f ds – wait for input
i.e. read will not block
¯
out _f ds – wait for output
i.e. write will not block
• s i z e : – size of i n_f ds , out _f ds , e r r _f ds
• t i me out : – when to timeout
in seconds and milliseconds
Returns when:
• input ready on one of i n_f ds (r e t > 0)
• output ready on one of out _f ds (r e t > 0)
• error occurs on one of e r r _f ds (r e t > 0)
• timeout expires (r e t == 0)
• signal has been caught (r e t < 0)
• some other error occurs (r e t < 0)
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 7/ 7
select and I/O
#i nc l ude <s ys / t ype s . h>
f d_s e t i n_f ds , out _f ds , e r r _f ds
• modified by call:
call – bit set = wait for file desc
return – bit set = file desc ready
return value from select
= number ready
• long integer in early UNIX systems
i n_f ds = i n_f ds | | ( 1<<f d ) ;
⇒ limit of 32 file descriptors
. . . but some systems allow more
• now a special f d_s e t structure
actually an array of integers!
¯
setting:
FD_ZERO( &i n_f ds ) ;
FD_SET( f d, &i n_f ds ) ;
FD_CLR( f d, &i n_f ds ) ;
¯
testing:
i f ( FD_I SSET( f d, &i n_f ds ) ) . . .
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 7/ 8
select and I/O – 2
• input
¯ terminal/ socket
– r e a d will not block
¯ passive socket
– a c c e pt will not block
• output
¯ terminal/ socket
– wr i t e ‘ready’
¯ wr i t e relies on system resources
¯ change between s e l e c t and wr i t e ?
⇒ wr i t e may block
+ use non-blocking wr i t e
• can ‘get away’ without s e l e c t on wr i t e
. . . but dangerous!
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 7/ 9
select and timeouts
#i nc l ude <s ys / t i me . h>
s t r uc t t i me va l t i me out ;

t i me out . t v_s e c s
t i me out . t v_ms
– maximum time to wait in seconds and ms

if no I/ O ready and no signals in time limit
then select returns with zero result
N.B. i n_f ds , out _f ds , e r r _f ds all zero also

modified by call?
¯
ideally should return time remaining
¯
doesn’t now . . .
. . . but may do one day

don’t rely on t i me out not being changed
reset for each call to s e l e c t
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 7/ 10
select and signals
• signal occurs during system call:
r e a d, wr i t e , or s e l e c t
• signal not caught . . .
. . . process aborts!
• signal caught . . .
' relevant handler called
Z systems call returns with ‘error’
• how do you know?
¯ negative return value
¯ e r r no set to EI NTR
• negative return & e r r no ≠ EI NTR
⇒ really an error!
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 7/ 11
care with signals
• signal handlers can run at any time
i nt i = 0
i nt my_ha ndl e r ( )
{
i = i + 1
}
ma i n( )
{
s i gna l ( my_ha ndl e r , SI GI NTR) ;
f or ( ; ; )
i f ( i > 0 ) {
do_s ome t hi ng( ) ;
i = i - 1;
}
}
• intention:
execute do_s ome t hi ng once per interrupt
• what actually happens:
' interupt processed (i =1)
Z do_s ome t hi ng executes
· main calculates i - 1 gets result 0
J before it stores the result . . .
. . . another interupt (i =2)
· ma i n stores result (i =0)
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 7/ 12
when to use select
• servers:
¯ where concurrency essential
¯ possibly ftp server
– listen to control & data
¯ telnet server
– listen to user over network
+ listen to shell/ application
• clients
¯ not with most window managers
– instead use callback
¯ some event stream WMs
– single fd for WM events
– listen to WM and network
¯ terminal based clients
– not needed for turn-taking
– e.g. telnet/ rlogin clients
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 7/ 13
proxy server
• proxy server used in session 3
• structure of code
' passive open on own port
Z wait for client connection
· active open on remote server
J loop forever
waiting for client or server input:
¯ when client data ready
read it
send to server
echo it to terminal
¯ when server data ready
read it
send to client
echo it to terminal
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 7/ 14
proxy code – 1
' Main loop
ma i n( . . . ) {
/ * e s t a bl i s h por t */
por t _s k = t c p_pa s s i ve _ope n( por t ) ;
/ * wa i t f or c l i e nt t o c onne c t */
c l i e nt _s k = t c p_a c c e pt ( por t _s k) ;
/ * onl y wa nt one c l i e nt , */
/ * s o c l os e por t _s k */
c l os e ( por t _s k) ;
/ * now c onne c t t o r e mot e s e r ve r */
s e r v_s k = t c p_a c t i ve _ope n( r e m_hos t , r e m_por t ) ;
r e t = do_pr oxy( c l i e nt _s k, s e r v_s k ) ;
e xi t ( 0) ;
}
• when do_pr oxy is called both network sockets open
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 7/ 15
proxy code – 2
Z perform proxy loop
i nt do_pr oxy( i nt c l i e nt _s k, i nt s e r v_s k )
{
• first declare and initialise fd bitmaps
f d_s e t r e a d_f ds , wr i t e _f ds , e x_f ds ;
FD_ZERO( &r e a d_f ds ) ; FD_ZERO( &wr i t e _f ds ) ;
FD_ZERO( &e x_f ds ) ;
FD_SET( c l i e nt _s k, &r e a d_f ds ) ;
FD_SET( s e r v_s k , &r e a d_f ds ) ;
• then loop forever
f or ( ; ; ) {
i nt num, l e n;
• copy bitmaps because s e l e c t modifies them
f d_s e t r e a d_c opy = r e a d_f ds ;
f d_s e t wr i t e _c opy = wr i t e _f ds ;
f d_s e t e x_c opy = e x_f ds ;
s t a t i c s t r uc t t i me va l t i me out = {0, 0};
• then call s e l e c t
num = s e l e c t ( MAX_FD, &r e a d_c opy, &wr i t e _c opy,
&e x_c opy, &t i me out ) ;
~ check return – ·, J & · at this point
}
r e t ur n 0;
}
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 7/ 16
proxy code – 3
· check for signals, errors and timeout
• first check for signals:
in this case, we are not expecting any so return
in general, we may need to do some processing
following the interrupt
it is usually better for the interrupt to set some
flag and let the main loop do most of the work
this reduces the risk of stacked interrupts and
mistakes in concurrent access to data structures
i f ( num < 0 && e r r no == EI NTR ) {
/ * stopped by signal */
pe r r or ( " EI NTR" ) ; r e t ur n 1;
}
• if there has been no signal num < 0 is an error
i f ( num < 0 ) { / * not stopped by signal */
pe r r or ( " s e l e c t " ) ; r e t ur n 1;
}
• if num is zero then a timeout has occurred
again, in this case no processing
but in general this is the opportunity for animation
or other periodic activity
i f ( num == 0 ) c ont i nue ; / * timeout */
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 7/ 17
proxy code – 4
J check for client input
client ready if bit is set in r e a d_c opy
i f ( FD_I SSET( c l i e nt _s k, &r e a d_c opy) ) {
i nt l e n = r e a d( c l i e nt _s k, buf f , buf _l e n ) ;
• on end of file or error exit the loop
i f ( l e n <= 0 ) { / * error or close */
c l os e ( s e r v_s k) ; r e t ur n l e n;
}
• if there is some input data, write it to the server and log it
e l s e {
wr i t e ( s e r v_s k, buf f , l e n) ;
l og_f r om_c l i e nt ( buf f , l e n ) ;
}
}
· server input similar
i f ( FD_I SSET( s e r v_s k , &r e a d_c opy) ) {
i nt l e n = r e a d( s e r v_s k , buf f , buf _l e n ) ;
i f ( l e n <= 0 ) { / * error or close */
c l os e ( c l i e nt _s k) ;
r e t ur n l e n;
}
e l s e {
wr i t e ( c l i e nt _s k, buf f , l e n) ;
l og_f r om_s e r ve r ( buf f , l e n ) ;
}
}
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 7/ 18
ª ª ª ª Hands on · · · ·
< the proxy server is a bit similar to a telnet client
both open a connection to a remote server
both echo from the user to the server . . .
. . . and from the server to the user
the major difference is that the proxy server
operates on the ‘other end’ of a network connection
ª you are going make a simple telnet-like client
ª copy pr oxy. c and ma ke 7 from t c p/ s e s s i on7
copy pr oxy. c and call it r a w- c l i e nt . c
< pr oxy. c reads and writes the client socket
you want to read from standard input (0)
and write to standard output (1)
ª proceed as follows:
' remove the code to open the client connection
(passive open and accept)
Z remove the parameter to do_pr oxy which
corresponds to the client socket
· modify the FD_SET calls so that s e l e c t waits
for standard input (0) rather than the client
J change all r e a d calls from the client so that
they read from standard input (0)
· change all wr i t e calls to the client so that
they write to standard output (1)
ª now compile and run your raw client, e.g.:
r a w- c l i e nt ha de s 25
(send mail as in session 3 page 3/ 17)
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 7/ 19
Security
• types of security:
¯ information:
– secrecy
– privacy
¯ resources:
– destructive access
– virus infection
• linked
¯ information → resources
e.g. password → login
¯ resources → information
e.g. modify / e t c / pa s s wd
• chain reaction
¯ small breach → complete loss
e.g. root password!
¯ N.B. special problem for computers
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 7/ 20
who are you afraid of?
• internal
¯ selling your secrets
¯ personal data
– payroll, debtor files etc
¯ using resources
– surfing, doom!
¯ downloading material
– indecent, possibly illegal
¯ backdoors
c l i e nt _s k = t c p_a c c e pt ( por t _s k) ;
n= r e a d( c l i e nt _s k, buf f , buf f _l e n) ;
buf f ( l e n) = ' \ 0' ;
i f ( s t r c mp( buf f , " Al a n' s s e c r e t wa y i n" ) == 0 ) {
/ * c onne c t c l i e nt _s k t o a r oot s he l l */
}
/ * nor ma l ope r a t i on * /
• external
¯ hackers
¯ accidental release
– e.g. forgotten portable on the train
¯ industrial espionage
¯ viruses
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 7/ 21
under attack
• viruses a real risk?
v heterogeneous
⇒ cross-infection more difficult
× lots of machines just like yours
? interpreted languages?
– can be made secure (e.g. JAVA)
• types of attack
¯ virus
– embeds itself in another program
¯ Trojan horse
– masquerades as another program
¯ worm
– independent self-replicating program
N.B. names and definitions differ
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 7/ 22
viruses on the web?
• explicit download of code
¯ helpers – machine specific code
¯ general software
× both risk infection
• implicit download
¯ semi-compiled – JAVA
¯ interpreted – JAVA script
¯ embedded in HTML
× you may never know!
v the good news
¯ JAVA & JAVA script ‘safe’
¯ cannot read or write to local disk
× the bad news
¯ JAVA script can connect remotely
¯ send details of browsing patterns
¯ minor breach of privacy
? the only breach possible?
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 7/ 23
The Internet Worm
• for 2 days in 1988, the Internet was
under siege
November 2nd, 1988
17:00 worm launched from Cornell University
21:00 worm detected at Stanford
22:04 worm detected at Berkeley
23:40 Berkeley discover one means of attack (sendmail)
23:45 infects Dartmouth and Army Ballistics Res. Lab.
November 3rd, 1988
00:21 Princeton University main machine crashes due to load
02:38 email from Berkeley: “We are under attack”
03:15 anonymous warning from foo@bar.arpa
05:54 patches to sendmail distributed
06:45 National Computer Security Centre (NCSC) informed
11:30 Milnet severs itself from Arpanet to prevent infection
16:00 inoculation method found (directory s h in / us r / t mp)
21:30 Berkeley start to decompile ‘captured’ worm
November 4th, 1988
05:00 MIT finish decompiling worm
11:00 Milnet rejoins Arpanet
17:20 final set of preventative patches mailed
21:30 worm’s author identified – named in the next day’s
newspaper as Robert T. Morris son of the NCSC’s chief
scientist Robert Morris!
• infections still noted as late as December 1988
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 7/ 24
What went wrong?
• several means of attack
• between machines:
¯ debug mode in s e ndma i l
¯ buffer overflow in f i nge r d
¯ once broken into a user on a machine
– r l ogi n/ r s h to other hosts
• within a machine:
¯ simple password attacks
– permutations of user’s own name
– internal list of 432 common passwords
– system dictionary
• attempted to prevent repeat infection
¯ didn’t always work
¯ main damage was excessive load due
to repeat infections (often 100s)
¯ also how it was detected
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 7/ 25
sendmail attack
• s e ndma i l had a debug mode
¯ worm connects to s e ndma i l
¯ worm sends ‘de bug’ command
¯ s e ndma i l will then execute
any command!
¯ should have been disabled
but s e ndma i l is complex!
• similar attacks still possible
¯ system engineer accounts
¯ remote vendor maintenance
• any debug modes on your system?
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 7/ 26
fingerd attack
• f i nge r d uses ge t s – buffer overflow
¯ worm connects to f i nge r d
¯ worm sends 536 byte line
¯ overflows f i nge r d’s buffer (512 bytes)
. . . and corrupts stack
¯ extra 24 bytes executed as code!
• lessons:
¯ never use ge t s !
– at best may crash
- at worst is a loophole
¯ always be careful of buffer lengths
• never again?
¯ a popular WWW browser . . .
– corrected in later versions
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 7/ 27
physical security
• physical security:
¯ are the machines secure
can someone reboot, substitute disks etc.?
¯ is the network secure
can someone link-in their own computer?
• local or global?
' local network and machines
Z backbone and routers
· remote network and machines
• secure?
' possible
Z reasonable for non-critical data
· no way!
N.B. ‘listening in’ easy on many networks
e.g. ethernet
• never trust transport layer
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 7/ 28
logical security
• secrecy:
¯ TCP/ IP packets not secure
e.g. credit card by email
¯ use encryption
e.g. Netscape secure sockets layer for WWW
• authentication:
¯ who am I talking to?
¯ is it the real server?
v rely on correct routing and protected ports
× impostor machine, non-UNIX server host
¯ is it an acceptable client?
v user passwords
× often sent as plain text! – e.g. telnet
• audit:
¯ risk of detection deters
¯ keeping logs
¯ relies on authentication
v SMTP reverse name lookup
× can’t check FROMfield – e.g. worm warning
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 7/ 29
low-level protection – firewalls
• simple measures
¯ isolation
– don’t connect to the global Internet
... but lose the benefits too
¯ anonymity
– don’t publish domain machine names
... but IP addresses still valid
• firewalls
¯ application independent
¯ act at router/ gateway
¯ can only look at IP or TCP headers
• what is possible
¯ only allow friendly IP addresses
– N.B. impostors
¯ limited internal routing
– protect sensitive machines/ data
¯ restrict incoming TCP packets
– only allow connection to protected ports
... but difficult for ftp
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 7/ 30
high-level protection – ring fences
• rlogin
¯ beware external root logins!
¯ passwords:
– if reasonable no ‘equiv’ hosts
– certainly no root ‘equiv’ hosts
? means lots of duplicate password files?
• servers
¯ never run as root?
– impossible! e.g. inetd, rshd
¯ never unnecessarily run as root?
– special login e.g. user ‘f t p’
– run as user ‘nobody’
• the rest of the system – normal measures
¯ backups – damage limitation
¯ permissions – restrict ‘other’ access
¯ setuid – dangerous, no write perm!
¯ / e t c / pa s s wd – encrypt or restricted read
– may cause problems
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 7/ 31
encryption
• one way function:
c yphe r = f ( i nput ) – easy
i nput = ? ( c yphe r ) – hard
¯ used in / etc/ passwd
¯ brute force attack:
for each possible input i np
if f ( i np) is c yphe r – got it!
• single key
c yphe r = c ode ( ke y, i nput )
i nput = de c ode ( ke y, c yphe r )
¯ in DES – c ode = de c ode
• public key encryption
c yphe r = c ode ( ke y1, i nput )
i nput = de c ode ( ke y2, c yphe r )
¯ ke y1 – given to everyone – public
ke y2 – kept by you – private
¯ anyone can send a message
only you can decrypt it
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 7/ 32
session keys and authentication
• public keys good, but:
¯ expensive
¯ the more you use a key
the easier it is to break
• use public keys to exchange single key
' machine A generates session key K
S
Z A encrypts it using B’s public key
K
SB
= code(K
B1
, K
S
)
· A sends K
SB
to B
J B decrypts K
SB
to obtain K
S
K
S
= decode(K
B2
, K
SB
)
· B generates value X
· B encrypts X and Ks using A’s public key
KX
A
= code(K
A1
, X.K
S
)
± B sends KX
A
to A
1 A decrypts KX
A
X.K
S
= decode(K
A2
, K
SA
)
* A encrypts X using B’s public key
X
B
= code(K
B1
, X )
" and sends it to B
• result:
¯ A and B share a secret key
¯ A and B sure of each other’s identity
• discard key after session or fixed time
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 7/ 33
authentication servers
• how do you find out B’s public key?
• answers:
' B tells you
Z someone else, C, tells you
· use physical means (post, hand)
• if ' or Z: how do you know it is B/ C?
• if Z: why should you believe C?
⇒ · ?
× no good for broad distribution
v use an authentication server
¯ trusted machine
¯ everyone tells it their public key
(using its public key or physical)
¯ ask it for other’s public keys
¯ or ask it for session keys
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 7/ 34
don’t panic!
• how secure is a fax?
• credit card number by phone
• hacker ≈ burglar
– if they want in, you won’t stop them
• main differences
– rate of loss (Mbytes/ sec)
– hidden loss (electronic copies)
– automatic attack
• ease of use ≈ ease of access
– where do you draw the line
UNIX
TCP/IP
Short Course Notes Alan Dix © 1996 7/ 35

UNIX

Network Programming with TCP/IP

Course Outline

Alan Dix
http://www.hcibook.com/alan

Session 1 Session 2 Session 3

Internet Basics First Code Standard Applications

Session 4 Session 5 Session 6 Session 7

Building Clients Servers I Servers II Security

Three interrelated aspects: r TCP/IP protocol suite r standard Internet applications r coding using UNIX sockets API

TCP/IP UNIX

Short Course Notes

Alan Dix © 1996

1

UNIX
Books:
1. 2.

Network Programming with TCP/IP

Reading

W. Richard Stevens, "TCP/IP Illustrated. Vol. 1: The protocols", Addison Wesley, 1994, (ISBN 0-201-63346-9). Explains the protocols using network monitoring tools without programming. Douglas E. Comer and David L. Stevens, "Internetworking with TCP/IP. Vol.3: Client-server programming and applications BSD socket version", Prentice Hall, 1993, (ISBN 0-13-020272-X). Good book about principles of client/server design. Assumes you have some knowledge or at least some other reference for actual programming.

3.

Michael Santifaller , translated by Stephen S. Wilson, "TCP/IP and ONC/NFS internetworking in a UNIX environment", 2nd Edition, Addison Wesley, 1994, (ISBN 0-201-42275-1). Covers more ground less deeply. Translation from German seems good.

4.

W. Richard Stevens, "UNIX Network Programming", Prentice Hall, 1990, (ISBN 0-13-949876-1). A programming book. I'm waiting for a copy, but Stevens is a good writer and this book is recommended by other authors.

See also:
• • your local manual pages (man 2) RFCs

Requests for comments (RFCs)
• • these are the definition of the Internet protocols obtain via anonymous ftp from sun.doc.ic.ac.uk (193.63.255.1) login as anonymous give your email address as password cd to rfc

TCP/IP UNIX

Short Course Notes

Alan Dix © 1996

2

UNIX Session 1 Internet Basics Network Programming with TCP/IP UNIX Network Programming with TCP/IP UNIX Network Programming with TCP/IP UNIX UNIX Network Programming with TCP/IP Network Programming with TCP/IP .

UNIX • origins Network Programming with TCP/IP Session 1 Alan Dix http://www.hcibook.com/alan • internets and the Internet • protocol layers • addressing • common applications  using them • TCP and UDP • port numbers • APIs  information calls TCP/IP UNIX Short Course Notes Alan Dix © 1996 1 .

Beranek & Newman ARPANET enters regular use 1973/4 redesign of lower level protocols leads to TCP/IP 1983 1980s 1990s Berkeley TCP/IP implementation for 4.000 networks 6 million hosts 30 million users WWW dominating Internet growth TCP/IP UNIX Short Course Notes Alan Dix © 1996 2 .2BSD public domain code rapid growth of NSFNET – broad academic use WWW and public access to the Internet The Internet Now • • • • • growing commercialisation of the Internet 50.Origins Development of Internet & TCP/IP 1968 1971 First proposal for ARPANET – military & gov’t research Contracted to Bolt.

AppleTalk the Internet is a particular internet which • uses the TCP/IP protocols • is global • is hardware and network independent • is non-proprietary in addition • supports commonly used applications • publicly available standards (RFCs) the Internet is not (just) the web ! TCP/IP UNIX Short Course Notes Alan Dix © 1996 3 .g. X25.internets and the Internet an internet is a collection of • interconnected networks • (possibly different) e.

Characteristics of the Internet To communicate you need: • continuous connection • common language • means of addressing TCP/IP UNIX Short Course Notes Alan Dix © 1996 4 .

etc. token ring • AppleTalk (itself an internet!) • etc. etc.Global Connectivity ethernet routers token ring sub-network PPP star network router lots of networks: • ethernet. FDDI. connected (possibly indirectly) • to each other • to the central ‘ARPAnet’ backbone in the US protocols can be used in isolation ? but is it the Internet TCP/IP UNIX Short Course Notes Alan Dix © 1996 5 .

Presentation. Application Standardisation: • RFC (request for comments) and DoD MIL RFCs also include (defined but not required): • PPP. ethernet packaging. http) Transport application user interfaces (e. etc. FTP. Fetch.g.Protocols – the Language of the Internet TCP/IP electrical signals low-level networks (e. ethernet) OSI Physical Link routers IP layer (end-to-end) ICMP (control and routing) Network TCP/UDP layer end-points application protocols (e. • FTP and other protocols TCP/IP UNIX Short Course Notes Alan Dix © 1996 6 .g.g. telnet. mosaic) Session.

Sussex.g.g. TCP/IP UNIX Short Course Notes Alan Dix © 1996 7 . Earth. near More Space† Without addresses can only broadcast Four types of address: x location independent y physical location z logical location { route based e. near Moon.ac. Eastern Hemisphere.g.12. Dunhambury.188. organisational hierarchy e. personal names e.g. England. C. old email addresses Two kinds of Internet address: IP address – type y (sort of) e. T. Anthony Buckeridge.167 domain name – type z e. Solar System. Europe.g. letter addresses e.Addressing J. 1950. Linbury Court School.g. Jennings.hud.uk † extract from Jennings Goes to School. Collins. zeus. 161. Space.

IP addresses • 32 bit integer • Often represented as 4 octets – – 2701966503 161.B.167 • General structure: net id { sub-net id } host id • N. octets do not map simply onto components Five classes of IP address: Class A 0 netid 7 bits subnet/hostid 24 bits Class B 1 0 netid 14 bits subnet/hostid 16 bits Class C 1 1 0 netid 21 bits hostid 8 bits Class D & Class E – experimental • hostids may divided using subnet mask r different for each major network (netid) ⇒ needs to be set for each machine on network TCP/IP UNIX Short Course Notes Alan Dix © 1996 8 .12.188.

edu uk • Decentralised administration • Mapping from name to IP address – domain name servers also reverse mapping • C API : gethostbyname gethostbyaddr – – name → IP address IP address → name TCP/IP UNIX Short Course Notes Alan Dix © 1996 9 . or at least ASCII ! • Heirarchical (roughly organisational) zeus..hud...ac.Domain names • human readable names . USA is implicit – cs.uk – United Kingdom ac – academic hud – huddersfield zeus – local machine N.B.washington..

Common applications
• FTP
(file transfer protocol) (simple mail transfer protocol) (remote logins) (simple remote login between UNIX machines) (built on http)

• SMTP • telnet • rlogin

• World Wide Web • NFS • TFTP

(network filing system – originally for SUNs) (trivial file transfer protocol – used for booting) (simple network management protocol)

• SNMP

g In each case protocols are defined g User interfaces depend on platform
(where relevant)

TCP/IP UNIX

Short Course Notes

Alan Dix © 1996

10

Hands on 

connect to zeus using telenet:
% telnet zeus.hud.ac.uk login: c5

...

etc.
zeus”? 

  

what happens if you just say “telnet what is zeus’ IP address? try “telnet

aa.bb.cc.dd” (where ‘aa.bb.cc.dd’ is zeus’ IP address)

connect to zeus using ftp:
% ftp zeus.hud.ac.uk

connect as yourself and then as anonymous

Read between the lines

TCP/IP UNIX

Short Course Notes

Alan Dix © 1996

11

Network communications
Communication can be: • Connectionless r address every message g like letters • Connection based r use address to establish a fixed link r send each message using the link g like telephone

N.B. both need an address

some sort of system address book or, publicly known addresses

TCP/IP UNIX

Short Course Notes

Alan Dix © 1996

12

...Network communications – 2 Other issues: • Reliability Do all messages arrive? Do they arrive in the right order? • Buffering effects responsiveness hides potential deadlock • Messages or byte-stream sent: write 1 (len=26): “abcde..qrst” read 2 (len=16): “uvwxyz012....89” ⇒ fixed length messages or prefix with length TCP/IP UNIX Short Course Notes Alan Dix © 1996 13 .vwxyz” write 2 (len=10): “0123456789” received: read 1 (len=20): “abcde..

IP – the fundamental Internet protocol point to point r between machines r addressed using IP address message (packet) based unreliable r network failures r router buffers fill up dynamic routing ⇒ order may be lost heterogeneous intermediate networks ⇒ fragmentation TCP/IP UNIX Short Course Notes Alan Dix © 1996 14 .

TCP & UDP Both • built on top of IP • addressed using port numbers process to process (on UNIX platforms) TCP • connection based • reliable • byte stream used in: FTP. http. SMTP UDP • connectionless • unreliable • datagram (packet based) used in: NFS. telnet. TFTP TCP/IP UNIX Short Course Notes Alan Dix © 1996 15 .

rlogin) How do you find them? well known port numbers TCP/IP UNIX Short Course Notes Alan Dix © 1996 16 .g.Port numbers • 16 bit integers • unique within a machine • to connect need IP address + port no TCP • connection defined by IP address & port of server + IP address & port of client UNIX • port < 1023 – root only • used for authentication (e.

different ‘name’ spaces for TCP & UDP TCP/IP UNIX Short Course Notes Alan Dix © 1996 17 . >1023) N.B.B.Well known port numbers Service echo discard daytime chargen ftp telnet smtp daytime tftp finger http login who Xserver Port no 7 9 13 19 21 23 25 37 69 79 80 513 513 6000 Protocol UDP/TCP UDP/TCP UDP/TCP UDP/TCP TCP TCP TCP UDP/TCP UDP TCP TCP TCP UDP TCP sends back what it receives throws away input returns ASCII time returns characters file transfer remote login email returns binary time trivial file transfer info on users World Wide Web remote login different info on users X windows (N.

API – the language of the programmer Application Programmer Interfaces Not part of the Internet standard – but very important! A story about DOS TCP/IP stacks supplied by different vendors ⇒ different device drivers different APIs chaos APIs depend on platform: UNIX – sockets (original Berkley system calls) – TLI (transport layer interface) Apple Mac – MacTCP MS Windows – WinSock (similar to sockets) • UNIX TCP/IP API are kernel system calls • Mac & Windows are extensions/drivers (+DLL) TCP/IP UNIX Short Course Notes Alan Dix © 1996 18 .

c – returns name of machine use the following C calls: gethostid()  returns (lon unsigned) integer result gethostname(buff.c – returns IP address of machine getname.hud.ac.c from tcp directory edit to make two programs: getid. play with telnet on different ports % telnet zeus.len) returns error code puts name into buff (maximum len bytes)  if you have time.uk port_no TCP/IP UNIX Short Course Notes Alan Dix © 1996 19 .Hands on   copy skeleton.

UNIX Session 2 First Code Network Programming with TCP/IP UNIX Network Programming with TCP/IP UNIX Network Programming with TCP/IP UNIX UNIX Network Programming with TCP/IP Network Programming with TCP/IP .

hcibook.UNIX Network Programming with TCP/IP Session 2 Alan Dix http://www.com/alan • features of sockets API • establishing TCP connections • simple client/server program  use it • read & write with sockets • wrapper functions • what they do  an echo server TCP/IP UNIX Short Course Notes Alan Dix © 1996 2/1 .

gethostbyname(.an end-point like an electrical connector r not TCP/IP specific (e. UNIX named pipes) r uses normal read/write system calls r sockets associated with UNIX file descriptors but some not for normal I/O r some extra system calls • sits more comfortably with TCP than with UDP because of byte-stream nature of UNIX I/O • special UDP functions e..g.g..g..1 • central features r central abstraction .the socket .Sockets • developed for Berkeley UNIX r recall early Berkeley TCP/IP implementation r first delivered with BSD 2..) – accepts a UDP datagram • additional non-socket functions e.. recv(..) – domain name server TCP/IP UNIX Short Course Notes Alan Dix © 1996 2/2 .

.17..5 different processes 161.Establishing a TCP Connection Initial State Internet different processes 151.100.112.192.112.3 • TCP is connection based .4.25 different processes 161. establishing it is a complex multistage process initially all machines are the same no special ‘server’ machines the difference is all in the software • • • TCP/IP UNIX Short Course Notes Alan Dix © 1996 2/3 .

17.Establishing a TCP Connection Passive Open 21 151.112.192.100.5 • server process does a ‘passive’ open on a port • it waits for a client to connect • at this stage there is no Internet network traffic • tells the TCP layer which process to connect to TCP/IP UNIX Short Course Notes Alan Dix © 1996 2/4 .25 server 161.

g.Establishing a TCP Connection Active Open 161.112.100..5 • client process usually on a different machine • performs an ‘active’ open on the port • port number at the client end is needed usually automatic (e.192. 2397) but can be chosen • network message → server machine requests connection TCP/IP UNIX Short Course Notes Alan Dix © 1996 2/5 .192.112.5 : 21 2397 21 server client 151.17.25 161.

112.10017.25:2397/161.25 server 161.192.100.112.192.5 • server side accepts and TCP connection established • a bi-directional reliable byte-stream • connection identified by both host/port numbers e.Establishing a TCP Connection Rendezvous 21 client 151.g.5:21> • server port is not consumed can stay ‘passive’ open for more connections • like telephone call desk: one number many lines TCP/IP UNIX Short Course Notes Alan Dix © 1996 2/6 . <151.17.

Establishing a TCP Connection and more .100.112.192.3 • other clients can connect to the same port • state for connections in the client/server only • no information needed in the network not like old style relay-based exchanges • server can restrict access to specified host or port • server can find out connected host/port TCP/IP UNIX Short Course Notes Alan Dix © 1996 2/7 . 21 client 151..4.112.25 server 161.17.5 client 161..

.. . allocated automatically TCP/IP UNIX Short Course Notes Alan Dix © 1996 2/8 . ftp) • active opens can rendezvous ..Passive & Active Open passive – patient but lazy active – industrious but impatient passive waits for request for connection waits for ever active sends out request for connection times out • normally server does passive open – waiting for client • but not always (e.. but may miss due to time-outs • either can specify local port but if not specified.g.

Simple client/server ‘talk’ • uses simplified calls • server handles only one client • strict turntaking user 1 zeus: simple-server start up complete io: simple-client -host zeus You can send now speak: hi there client says: hi there speak: nice day isn't it client says: bit cold here speak: ^D (EOF) bye bye zeus: server finished the conversation io: server says: nice day isn't it speak: bit cold here user 2 TCP/IP UNIX Short Course Notes Alan Dix © 1996 2/9 .

printf("client says: %s\n".buff. } N. write(client_sk.strlen(buff))..buff. gets(buff).. buff[len] = '\0'.) { /* wait for client’s message */ len = read(client_sk.Server Code establish port port_sk = tcp_passive_open(port) /* only done once */ wait for client to connect client_sk = tcp_accept(port_sk) /* repeated for multiple clients */ then talk to client for(.B.buff).buf_len). /* now it’s our turn */ printf("speak: "). strict turn taking: client–server–client–server .. TCP/IP UNIX Short Course Notes Alan Dix © 1996 2/10 .

B. Œ opposite turn order  no error checking! TCP/IP UNIX Short Course Notes Alan Dix © 1996 2/11 ..Client Code request connection to server serv_sk = tcp_active_open(host. printf("server says: %s\n".) { /* our turn first */ printf("speak: ").port) /* waits for server to accept */ /* returns negative on failure */ /* host is server’s machine */ then talk to server for(.buff. } N.buff. /* wait for server’s message */ len = read(serv_sk.buff).buf_len). buff[len] = '\0'. write(serv_sk. gets(buff).strlen(buff)).

     Hands on     copy simple-client.c • makefile compile and run the programs: • make simple – compiles them both • on one machine type: • on another type: where simple-server simple-client machine-name machine-name is the name of the  first   what happens if you re-run the server straight after it finishes? use the -port option zeus: simple-server -port 3865 io: simple-client -host zeus -port 3865  try a port less than 1024! TCP/IP UNIX Short Course Notes Alan Dix © 1996 2/12 .c • simple-server.c from tcp/session2 directory • simple-client.

use strlen to get its length ret ret N.read & write Reminder: ret = read(fd. negative for error buff is not NULL terminated leave room if you need to add ‘\0’! ret ret = write(fd.buff.B. 0 means “end of file” may be less than len e. open for reading buffer in which to put chars maximum number of bytes to read returns actual number read int fd char *buff int len int ret • • is 0 at end of file.buff. Both may return negative after interrupt (signal) TCP/IP UNIX Short Course Notes Alan Dix © 1996 2/13 .g. if OS buffers full * should really check and repeat until all gone * • buff need not be NULL terminated if buff is a C string.len) – – – – a file descriptor (int).len) – – – – a file descriptor (int). open for writing buffer from which to get chars number of bytes to write returns actual number written int fd char *buff int len int ret • is negative for error.

dir) reading from a file either: (i) succeeds (ii) gets end of file (ret = 0) reading from a socket waits until (i) network data received (ret > 0) (ii) connection closed (ret = 0) (iii) network error (ret < 0) writing to a socket may (i) send to the network (ret > 0) (ii) find connection is closed (ret (iii) network error (ret < 0) it may return instantly but may block if buffers are full may work during testing then fail in use • reading may block r r • writing may block r = 0) r r $ BEWARE – TCP/IP UNIX Short Course Notes Alan Dix © 1996 2/14 .read & write with sockets • similar to normal UNIX pipes • bi-directional byte stream r $ read and write to same file descriptor difficult to close one direction special socket call shutdown(sock.

&host. argv. &errmess ) scan command arguments for network options port_sk = tcp_passive_open(port) server performs passive open serv_sk = tcp_active_open(host.port) client performs active open client_sk = tcp_accept(port_sk) server accepts client connection r r parse_network_args does not use socket calls the rest package one or more socket calls TCP/IP UNIX Short Course Notes Alan Dix © 1996 2/15 . &port.Wrapper Functions (1) • not real socket functions • simplified versions for examples ret = parse_network_args( &argc.

addr_len ).Wrapper Functions (2) ret = parse_network_args( &argc. N.0.0 (special address means “this machine”) bind( port_sk. &host. port_sk is not used for normal reading and writing TCP/IP UNIX Short Course Notes Alan Dix © 1996 2/16 .0. &errmess ) • • • • • • scans and edits argument list looks for options: -host name -port nos removes them from argument list sets the arguments host and port if options found set either host or port to NULL to disable options returns 0 for success non-zero failed – errmess set to appropriate message port_sk = tcp_passive_open(port) – – port number to use file descriptor of socket int port int port_sk x creates Internet TCP socket port_sk = socket( AF_INET. &port. 0 ). SOCK_STREAM.B. &bind_addr. argv. y ‘binds’ socket with right port and address 0.

&bind_addr. SOCK_STREAM. &len).Wrapper Functions (3) serv_sk = tcp_active_open(hostname. &bind_addr. y creates Internet TCP socket serv_sk = socket( AF_INET. • waits for rendezvous at x when it returns client_sk can be used to talk to client TCP/IP UNIX Short Course Notes Alan Dix © 1996 2/17 . addr_len ). 0 ). z ‘connects’ socket to appropriate port and host connect( serv_sk.port) – – – name of server’s machine port number to use file descriptor of socket char *hostname int port int serv_sk x finds IP address of host hostIP = gethostbyname(hostname). • rendezvous with the server happens at z socket serv_sk can then be used to talk to the server client_sk = int tcp_accept(port_sk) – file descriptor of socket port_sk x performs raw accept call client_sk = accept(port_sk.

255.255 • limited broadcast (doesn’t pass routers) any netid – subnetid/hostid = –1 any netid & any subnetid –hostid = • • broadcast to specified net or subnet N. need to know subnet mask –1 TCP/IP UNIX Short Course Notes Alan Dix © 1996 2/18 . can also connect to own IP address 255.0.B.0 One of several special IP addresses 0.0.0 • • • 127.255.0.Special IP addresses • bind call in tcp_passive_open uses IP address 0. also means ‘the local machine’ usually used as recipient for local server doesn’t normally hit network N.B.0.0.0.0 source only default IP address – ‘local machine’ filled in by socket API call • • • • • loopback address.

o $(MYLIBS) cc $(CFLAGS) -o echo-server echo-server.B. this must be a tab    compile and run your code does your server echo everything once or twice to its terminal? the server exits after it has finished echoing make it continue to wait for additional clients (don’t try for two at once!) TCP/IP UNIX Short Course Notes Alan Dix © 1996 2/19 .c and call it echo-server.o $(MYLIBS)  N.    Hands on     build an echo server   copy simple-server.c alter code so that instead of asking the user for input (gets) it simply uses the last message from the client (in buff) you will need to add to the makefile: echo-server: echo-server.

UNIX Session 3 Application Protocols Network Programming with TCP/IP UNIX Network Programming with TCP/IP UNIX Network Programming with TCP/IP UNIX UNIX Network Programming with TCP/IP Network Programming with TCP/IP .

argv & makefiles  build your own mail client TCP/IP UNIX Short Course Notes Alan Dix © 1996 3/1 .UNIX Network Programming with TCP/IP Session 3 Alan Dix http://www.hcibook.com/alan Standard Applications • trusted login – rlogin • negotiating options – telnet • world wide web– http  peeking • file transfer – ftp • standard response codes • electronic mail – SMTP  drive it by hand • argc .

Types of Protocol user character stream r r r used by remote terminal/login applications ( rlogin & telnet ) most of the traffic is uninterpretted data some embedded control sequences ascii turn-taking protocols r r r r includes ftp. http) or use separate connection (ftp) binary protocols r used for low level protocols: TCP/IP itself! SNMP – simple network management protocol NFS (built on top of RPC – remote procedure call) r issues such as byte order important TCP/IP UNIX Short Course Notes Alan Dix © 1996 3/2 . SMTP. http human readable client & server messages usually line oriented turn-taking typically: client command server response ··· r but roles may reverse bulk data may be embedded (SMTP.

Remote Terminal Access: rlogin and telnet • one of the earliest Internet application areas • the client end – interacts with the user • the server end – shell or command interpreter Internet ‘ client Œ  Ž shell  basic pattern: x user types characters y the client sends them to the server z the server passes them on to the shell { shell generates output | server passes output to client } client puts output on user’s screen server  TCP/IP UNIX Short Course Notes Alan Dix © 1996 3/3 .

Remote Terminals – Issues • initialisation and authentication x how does the server know who you are? y how do you know the server is official? answer to y: r the server is on a reserved port (<1024) N.. including end-of-line ! which end performs different things: r user flow control (crtl-S.. only works for UNIX servers! how to deal with special characters . ctrl-Q) r line editing r echoing how do the client and server communicate: r user interrupts r window size changes r who does what if embedded control characters are used what happens if the user types them? • • • • TCP/IP UNIX Short Course Notes Alan Dix © 1996 3/4 .B.

x10. etc.x80) marked by URG (urgent) pointer TCP/IP UNIX Short Course Notes Alan Dix © 1996 3/5 . ) authentication by ‘trusted’ hosts r no password required if: client uses port <1024 and client host is in ‘.rhosts’ file r means that client must be setuid to root responsibility r echoing – server r flow-control – client on server request client–server communication r r • • • client→server initialisation string client→server window size change: ctrl chars – 2 bytes of 255 followed by window size in 2 bytes no protection against user typing it! r server→client requests: special characters (bytes x02.rlogin • • simple stable protocol designed for UNIX–UNIX logins ⇒ can make more assumptions ( terminal handling. interrupts.x20.

but it’s not! data sent in normal TCP stream special URG pointer set r officially to the last byte of urgent data r BSD set it one beyond! o m e t e x t 0x80 m o r • • URG pointer Berkeley URG pointer! • client should: x read until urgent data reached y if necessary discard intervening data (e. . . . . . .g. .Urgent Data • sometimes called out-of-band data . if insufficient buffer space to store it) problem with x r URG pointer says where it ends . but how do you know where it starts? r have to have special codes again • with UNIX sockets r send urgent data with ‘send’ system call r recipient gets a SIGURG signal TCP/IP UNIX Short Course Notes Alan Dix © 1996 3/6 .

telnet • • • • cross platform ⇒ more complex many downward-compatible options can be used to connect to non-login services client authentication r not in protocol – application specific e. subject to option negotiation NVT character set r needed because cross-platform r 7 bit US ASCII r end-of-line sent as “\r\n” (carriage return. ftp. line feed) r carriage return sent as “\r\0” r also used by SMTP. high bit free for control characters! • TCP/IP UNIX Short Course Notes Alan Dix © 1996 3/7 . getty • responsibility r client may handle echoing. finger etc. line editing etc.g.

r flow control r window size information client and server play “will you/wont you” to determine common protocol just like fax machines and modems • • TCP/IP UNIX Short Course Notes Alan Dix © 1996 3/8 .telnet – 2 control codes • • introduced by byte 255 r called: IAC – interpret as command following byte is actual control code examples: 255 236 241 243 251 252 253 254 250 240 – – – – – – – – – – the actual byte 255 (needed for binary mode) end of file no op break WILL WONT DO DONT sub-option begin sub-option end option negotiation control codes: option negotiation • many different options: r echoing r line editing.

large messages & simple clients ⇒ reliable communication needed TCP/IP UNIX Short Course Notes Alan Dix © 1996 3/9 .http • the World Wide Web protocol • protocol: r r ASCII control messages standard data formats for pages/images • uses single step transactions x y z { establish TCP connection client sends request server sends reply + page connection closed • why transaction based? r r r client end – many different servers (hypertext links to different sites) server end – many clients load time < interaction time (ideally!) • why use TCP? $ r high cost of establishing connection wide area.

uk to //io.hud.ac.uk 80 -port 8800 y start up Netscape using background menu z go to the url: { http://www.hud.ac.hud.html |  now edit the host name in the url field if your machine is io change //www.B.uk/schools/comp+maths/private/alan/alandix.ac.ac.hud.uk x then: telnet io 2300 23 -port 2300 N.uk:8800 the 8800 is to set the port number used by proxy hit return and watch the proxy window you can do the same with telnet: x run: proxy zeus.ac. mail etc.ac.hud.uk TCP/IP UNIX Short Course Notes Alan Dix © 1996 3/10 .)  try using the -v option of ftp type: ftp -v prometheus. cannot be used for protected ports (ftp.hud.    Hands on     peeking    use the program proxy in tcp/session3 it sits between client and server use it to see how http works: x run: proxy www.

. email address for anonymous receive a file from remote machine send file to remote machine change remote directory change remote directory tell server what data port to use info about commands supported finish session • • TCP/IP UNIX Short Course Notes Alan Dix © 1996 3/11 .File Transfer Protocol FTP • • used to transfer files and list directory contents uses two types of connection: control – for commands and responses data – for files and listings protocol for control is ascii turn-taking client command. client commands nearly user level. server response.. including: USER PASS GET PUT CWD LIST PORT HELP QUIT user name for connection often ‘anonymous’ is accepted password. .

93 port 2397 (=9*256+93) on host 161.5 when data transfer is required r client sends appropriate command e.FTP .g. GET simple-client. 226 Transfer complete.5.192.g.112.192.c then waits listening for connection r server performs an active open on port then sends data r server tells client when transfer is complete e. then both sides (usually) close the data port TCP/IP UNIX Short Course Notes Alan Dix © 1996 3/12 .9.112.2 control and data control connection r r r r r server waits (passive open) on port 21 client establishes connection (active open) client sends ascii commands – one per line server responds: single or multi-line response when required a data connection is established data connection r r client performs a passive open on some port (may leave OS to determine port number) client tells server using control connection PORT 161.

standard response codes • ftp server replies with lines such as: 200 PORT command successful • • SMTP and some other protocols use similar codes three digit codes – type given by first digit: 1yz – expect further reply from server 2yz – all OK 3yz – more required from client 4yz – temporary failure (try again) 5yz – error or permanent failure single-line response general format 999 a text message • space here • multi-line response either: hyphen means ‘more to come’ 999-first line 999-one or more further lines 999 the last line space here on last line or 999-first line lots of lines all starting with at least one space 999 the last line TCP/IP UNIX Short Course Notes Alan Dix © 1996 3/13 .

Simple Mail Transfer Protocol SMTP • allows: r mail client (user interface) to send via server r servers to talk to one another (one server takes ‘client’ role) note: • not used by user interface for receipt • sendmail is common SMTP server under UNIX client commands: HELO MAIL RCPT DATA VRFY EXPN RSET EHLO QUIT client tells server who it is initiates message and sets sender sets one of the recipients says actual message content follows check that recipient exists (no mail sent) expand mail alias (no mail sent) start from scratch see if server handles advanced features finish session • • TCP/IP UNIX Short Course Notes Alan Dix © 1996 3/14 .

ac Subject: HCI book 2E r r blank line body Russell.Beale@cs. have you heard from Prentice Hall yet concerning the web pages? Alan TCP/IP UNIX Short Course Notes Alan Dix © 1996 3/15 . servers typically: r do not trust HELO use reverse name mapping instead r do trust sender name (From:) how could they verify it? SMTP specifies delivery not content other standards used for content: r non-ASCII characters in headers r • • =?ISO-8859-1?Q?Alan=20Dix?= MIME for multi-part mixed content messages • simple mail message is just: r header From: alan@zeus.hud.uk.SMTP – 2 • authentication.uk To: R.bham.

    Hands on     see what it does   we want to send a mail message using raw SMTP! first of all see how ‘mail’ does it cannot use proxy as SMTP is at port 25 (protected) instead try the -v option of mail type: mail -v c3 – or whoever you want to send mail to!  see the messages from the server and the client N.B. not all messages are shown  when does mail establish the connection? why? TCP/IP UNIX Short Course Notes Alan Dix © 1996 3/16 .

uk> DATA first line of message .ac.ac..    Hands on     drive it by hand  use telnet to send a message type: telnet zeus.uk 25   you are connected to the SMTP server on zeus say hello! which machine you are on HELO walt.hud. QUIT  next send the message  finally say goodbye  run mail to see if any celebrity has sent you any TCP/IP UNIX Short Course Notes Alan Dix © 1996 3/17 .dotty shear quackery .com did it believe you? how does it know?  now say who the message is from and who it is to MAIL From:<Donald_Duck> RCPT To:<c3@zeus.hud.disney.

. • one of the ways to get information into a C program • in UNIX you type: the program gets: argc = = = = = = argv[0] argv[1] argv[2] argv[3] argv[4] myprog a "b c" d 4 "myprog" "a" "b c" "d" NULL – – – – length of argv program name single second argument terminator N..argc & argv • recall: or: int main( int argc. char **argv.. char *argv[ ] ) .. char **envp) TCP/IP UNIX Short Course Notes Alan Dix © 1996 3/18 . char **argv ) . int main( int argc. r r DOS is identical (except argv[0] is NULL early versions) argc is one less than the number of arguments! • other ways to get information in (UNIX & DOS): r configuration file (known name) r standard input r environment variables using getenv() or (UNIX only) third arg to main: main(int argc.B.

r generic – e.Make ‘make’ is a UNIX† command which: • automates program construction and linking • tracks dependencies • keeps things up-to-date after changes to use it: r construct a file with rules in it you can call it anything..g.c’ . † make is also available in many other programming environments TCP/IP UNIX Short Course Notes Alan Dix © 1996 3/19 .o’ from its ‘... to make mail-client do this ..g. to make any ‘. but ‘makefile’ is the default r target – (uses the default makefile) make -f myfile target – (uses the rule file myfile) either rebuilds the program ‘target’ if run ‘make’ itself make necessary • each makefile consists of: r definitions r rules • rules say how one thing depends on another they are either: r specific – e.

this must be a tab • general form: • example: myprog: myprog... command-line N.o another. $(SDIR).B.. one variable used in another's definition • make variables are referred to later using $ e.o cc -o myprog myprog.o if either of them is newer than myprog rebuild it using the then rebuild it using the command: “cc -o myprog .” TCP/IP UNIX Short Course Notes Alan Dix © 1996 3/20 .Makefile format Definitions • general form: variable = value • example: SDIR = tcp MYLIBS = $(SDIR)/lib N.o $(MYLIBS) this says: to make myprog you need myprog.o and another..o another. $(MYLIBS) • expanded like #defines or shell variables (some versions of make will expand shell variables also) Rules (just specific rules) target : dependent1 dependent2 .B.g.

• • • • • reads from server_fd parses a single or multi-line response returns the response code (of last line) echoes full response to echo_fd also copies it into buff if non-NULL int get_response_fp( FILE *server_fp. int len ). int echo_fd. FILE *echo_fp. int len ). char *buff. char *buff. • similar with stdio files TCP/IP UNIX Short Course Notes Alan Dix © 1996 3/21 .Helper Functions standard response lines • to make life easier! • my own helper functions r r to read standard response lines to interact with SMTP server #include "protocol.h" #include "mail-helper.h" int get_response_fd( int server_fd.

• awaits first response and does ‘HELO’ • checks response and returns 0 if OK int do_mail_from(int serv_fd. • does ‘QUIT’ command All optionally echo all exchanges to a file (or terminal) set by: FILE *do_mail_set_echo_fp(FILE *new_echo_fp) TCP/IP UNIX Short Course Notes Alan Dix © 1996 3/22 . int len). int do_mail_data_buff(int serv_fd. int do_mail_to(int serv_fd. char *buff. char *from). • sends ‘MAIL’ and ‘RCPT’ commands respectively • sender (from) and recipient (to) are C strings int do_mail_data_fp(int serv_fd. FILE *user_fp). char *to). • send ‘DATA’ command and send message copied from user_fp or buff respectively int do_mail_quit(int serv_fd).Helper Functions – 2 for sending mail int do_mail_init(int serv_fd).

c copy the following from mail-helper. x y z SMTP obeys strict turn-taking: server–client–server–client–server server starts with a return code but client ‘in control’  modify the client code x set default host (zeus) and port (25) y to and from addresses: either read in or use argv z message: initially read a single line { ‘unwrap’ loop to give fixed turns TCP/IP UNIX Short Course Notes Alan Dix © 1996 3/23 .B.    Hands on     build your own mail client   copy simple-client.c and call it mail-client.c make3 tcp/session3: the makefile is ready to compile your mail client you can type (when ready!): make -f make3 mail-client N.

TCP/IP UNIX Short Course Notes Alan Dix © 1996 3/24 .    Hands on     mail client – 2  resulting program structure: (a) (b) (c) (d) (e) (f) (g) (h) (i) (j) (k) (l) (m) (n) (o) (p) (q) (r) read (parse) to/from addresses from user read message from user (gets or scanf) open tcp connection to mail server on correct port wait for server response line(s) say hello to server wait for server response line(s) say who the mail is from wait for server response line(s) say who the mail is to wait for server response line(s) say that data is coming wait for server response line(s) send one line message send line with just full stop wait for server response line(s) say goodbye wait for server response line(s) close connection   compile and run your code! if you have time modify it to send longer messages either: change step (b) and (m) to accept long messages or: remove step (b) and or: make (m) read from user before sending each line whatever you like ...

UNIX
Session 4
Concurrent Clients

Network Programming with TCP/IP

UNIX

Network Programming with TCP/IP

UNIX

Network Programming with TCP/IP

UNIX
UNIX

Network Programming with TCP/IP

Network Programming with TCP/IP

UNIX

Network Programming with TCP/IP

Session 4
Alan Dix
http://www.hcibook.com/alan

• sequential and concurrent clients • techniques for concurrency • call-backs • knowing what you’re doing • callback–based client  using it

TCP/IP UNIX

Short Course Notes

Alan Dix © 1996

4/1

Sequential Clients
e.g. FTP
1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. client waits for user input user types “DIR” client performs passive open on data port (2397) client sends “PORT 161.112.192.5.9.93” to server client waits for standard ‘200’ reply line if not OK then fail client sends “LIST” to server client waits for standard ‘150’ reply line if not OK then fail client reads from data port client waits for standard ‘226’ reply line if not OK then fail report success to user

• •

client is in control next client action depends on: r r what happened last
e.g. what commend the user types

NOT on when it happens

TCP/IP UNIX

Short Course Notes

Alan Dix © 1996

4/2

Naturally Concurrent Clients e.g. telnet • at any moment either user may type something or output may come from server end • client must respond whichever happens • program a bit like: r when user types then send to the server r when server sends message then print on terminal TCP/IP UNIX Short Course Notes Alan Dix © 1996 4/3 .

g. Netscape – WWW client • basic protocol transaction based $. but response can be slow  interaction allowed during transaction ¯ scrolling ¯ ‘STOP’ button ⇒ client has to listen to server – more data user – mouse and keyboard TCP/IP UNIX Short Course Notes Alan Dix © 1996 4/4 .Concurrency for Usability e.

X Motif TCP/IP UNIX Short Course Notes Alan Dix © 1996 4/5 .Programming Concurrency Problem • doing more than one thing at once listening user terminal & TCP server port Solutions • polling r $ use non-blocking I/O keeps processor busy • threads r r r r needs built-in support (language or OS) program written as several sequential parts all executed at the same time communicate using shared data (also semaphores etc...) • event driven programming r r r r low-level – e. Visual Basic.g. raw X and Mac program paradigm – e. HyperCard call-backs – e. UNIX select event-loop – e. Windows.g.g.g.

target = quit_button) return OK. for dialogue boxes unforeseen events (e. read_event(&event). else if ( event. if ( event.char). }  programmer in control $ related code gets spread out in if/case statements • often written with sub-loops ⇒ e. .type == KEYPRESS ) insert_char(event. . .Event Loop Typical program structure for (. . else if ( event.type == BUTTON && event.buff).type == INPUT_READY ) do_network_task(event. ) { /* loop forever */ struct event_st event.g.g. network I/O) may be delayed or even ignored! TCP/IP UNIX Short Course Notes Alan Dix © 1996 4/6 .

Event–Based Languages program = collection of event handlers e. sender end AppleEvent answer "AppleEvent" && class && “from” && sender -.g.dialogue box for user  $ concurrency naturally part of language network I/O not always treated uniformly TCP/IP UNIX Short Course Notes Alan Dix © 1996 4/7 . id. HyperCard on mouseUp set cursor to watch put getServerAddress() into serverAddr put getUserName() into userName put cd fld "ToOrFrom" into toName put cd fld "Message" into theMess send "toServerSendMail" ¬ && quote & toName & quote & comma ¬ && quote & userName & quote & comma ¬ && quote & theMess & quote ¬ to program serverAddr end mouseUp on AppleEvent class.

: r r WinSock X Motif (TCP/IP under Windows) General pattern Program x y z define a function tell toolkit to attach it to event give control to the toolkit Toolkit i when event happens call user defined function TCP/IP UNIX Short Course Notes Alan Dix © 1996 4/8 .g.Call-backs used in many toolkits and window mgrs: e.

..Example – X Motif Call-backs XtAddCallback( widget.g. my-data ) widget – type – a widget such as a button a callback resource name: which type of event to respond to e. XmNactivateCallback func – pointer to C function defined by you e. event-data ) widget my-data – – where the event occurred the integer or pointer passed in the call to XtAddCallback the X event structure which caused the callback event-data – TCP/IP UNIX Short Course Notes Alan Dix © 1996 4/9 . quit_func my-data – an integer or pointer to your data passed on to your callback The callback function definition: void quit_func( widget. my-data. callback-type. func.g.

response. ) { /* N. .B. ) } y features for free x program counter () – what you are doing y local variables – what you are doing it to TCP/IP UNIX Short Course Notes Alan Dix © 1996 4/10 . printf("session complete\n"). . ← write(serv_sd. exit(0).MAX_LINE_SIZE). if ( response[0] != '2' ) .5). . . if ( . } if ( . ) if ( command is "quit" ) { char response[MAX_LINE_SIZE+1]. pseudo-C !!! */ gets(command). ."QUIT\n".What’s going on? Sequential Programs for ( . . x read(serv_fd. .

command sequences server output modes: x normal echoing y waiting for command z waiting for option TELNET 253 – DO 254 – DONT other bytes not byte 255 1 byte 255 2 3 any option TCP/IP UNIX Short Course Notes Alan Dix © 1996 4/11 . partial line of user input • program counter mode variable or finite state machines! e.g.g.2 sequential implicit concurrent explicit • local variables global variables or dynamic data structures e.What’s going on? .

. read_socket is called read_socket( int sd.. }  When server sends a message .buf_len)...) { /* request connection to server */ sd = tcp_active_open(host.NULL).. . .Callback based client – 1 Œ Initialisation main(.read_socket.buff. ) { /* read server’s message */ len = read(sd.. /* process message */ /* probably update interface */ } TCP/IP UNIX Short Course Notes Alan Dix © 1996 4/12 ....port) /* set-up callback for server */ inform_input(sd.. /* give control to toolkit */ inform_loop(). /* set-up call-backs for interface */ .

appropriate function is called term_line( int fd. /* possibly send message to server */ write(sd.buff)... . void *id.buff.. char *buff ) { /* process interface event */ mess("sending {%s}\n". } step Œ once at initialisation steps  & Ž any number of times in any order TCP/IP UNIX Short Course Notes Alan Dix © 1996 4/13 ..Callback based client – 2 Ž When user does something .strlen(buff)).

type : make -f make4 conf   one person run the server: io: server two or more others run the client: other: client -host io N.B.c server. you cannot participate from the server to join in launch a client in another window of the server’s machine TCP/IP UNIX Short Course Notes Alan Dix © 1996 4/14 .    Hands on     an electronic conference  copy the following from client.c make4 tcp/session4: the makefile is ready to compile.

UNIX Session 5 Server Design Network Programming with TCP/IP UNIX Network Programming with TCP/IP UNIX Network Programming with TCP/IP UNIX UNIX Network Programming with TCP/IP Network Programming with TCP/IP .

hcibook.com/alan • types of server • handling server concurrency • server state • stateless servers • when things go wrong! • survival – the 3 Rs • callback–based server  modify server TCP/IP UNIX Short Course Notes Alan Dix © 1996 5/1 .UNIX Network Programming with TCP/IP Session 5 Alan Dix http://www.

ftp z inherent concurrency e. database: 1 query → 1 result y strict turn-taking e. MUDs for lots of clients either: • serve one at a time in turn ⇒ x may be slow y may take forever! both require concurrency • serve several at the same time ⇒ TCP/IP UNIX Short Course Notes Alan Dix © 1996 5/2 .g.g.Servers Kinds of server x transaction based e.g. electronic conferences.

Server Concurrency • similar solutions to client r polling  acceptable if machine dedicated to server r r r threads UNIX select event driven $ less likely to run in event-based system  some web based servers do • in addition: r when no intrinsic concurrency r can use UNIX fork  launch separate process to serve each client so each is simpler  uses standard UNIX process concurrency $ can be expensive (process creation) especially with lots of small transactions TCP/IP UNIX Short Course Notes Alan Dix © 1996 5/3 .

Server State • concurrent server needs to remember r r r how many clients state of their connection state of each transaction/protocol etc. etc.. etc. many clients ⇒ large state $..  solution – no state TCP/IP UNIX Short Course Notes Alan Dix © 1996 5/4 . $. disaster scenarios r r r r r client establishes connection client crashes client restarts client establishes a new connection it crashes again .

B. in general.. http – transaction based.. r r r clients have to maintain more state requests more complex (no context) unreliable protocol ⇒ transactions must be idempotent time-outs for lots transactions . TCP/IP UNIX Short Course Notes Alan Dix © 1996 5/5 ... N.Stateless Servers stateless = no per client state • for transaction based services r r r client makes request server performs action server returns result • really only possible with UDP e.. but uses TCP ⇒ may need several reads for request need to store partially filled buffer .. buffers part of the per client state $ not all plain sailing .g.

of failure = q n clients and only 1 server. so: probability of some failure ≈ np+q • good news! r server failure less likely (or is it?) • bad news! r r servers are more complex (q > p) what if client brings server down? TCP/IP UNIX Short Course Notes Alan Dix © 1996 5/6 .When things go wrong PC crash server crash ⇒ ⇒ one sad user lots of angry users • take special care with servers! • probability of failure: clients – prob. of failure = p server – prob.

Causes of failure Œ  Ž  hardware failures programming errors unforeseen sequences of events system does not scale Large number of components ⇒ Œ more frequent Complexity of algorithms ⇒  more likely Interleaving and delays ⇒ Ž difficult to debug Limited testing conditions ⇒  unexercised TCP/IP UNIX Short Course Notes Alan Dix © 1996 5/7 .

B.Survival Network or server failure standard solutions Client fails — three Rs for server • robust server should survive • never wait for response from client • non-blocking network I/O • reconfigure detect and respond to failure • time-out or failure of I/O operations • reset internal data structures • inform other clients • resynchronise catch up when client restarts • similar to new client • N. client may not know (network) TCP/IP UNIX Short Course Notes Alan Dix © 1996 5/8 .

Software faults Defensive programming • inconsistent client/server data structures Use simple algorithms • fixed sized structures – but check bounds! • may conflict with scaleability – document Verify • • close hand checks for production code – formal methods Unforeseen sequences of events • deadlock – never use blocking I/O • never assume particular orders of events • back-to-back messages network packet ≠ logical message Debugging and testing • logging – to reproduce failure • random data – at interface or network • ask your friends TCP/IP UNIX Short Course Notes Alan Dix © 1996 5/9 .

. /* probably tell other clients also */ } TCP/IP UNIX Short Course Notes Alan Dix © 1996 5/10 .. /* keep track of number of clients */ count = count+1..) { /* establish port */ pd = tcp_passive_open(port) /* set-up callback for port */ inform_input(pd.NULL)...accept_client.count). /* record connection details */ client_fd[count] = fd.. /* set-up callback for client */ inform_input(fd.Callback based server – 1 Œ Initialisation main(.read_client.) { /* accept client’s connection */ fd = tcp_accept(port_fd). notifier calls accept_client accept_client(.. }  When client requests connection .. . /* give control to notifier */ inform_loop().

. but with extra ‘accept’ stage...B. c<client_count.Callback based server – 2 Ž When client sends message . in any order similar to client code. • TCP/IP UNIX Short Course Notes Alan Dix © 1996 5/11 . .. /* broadcast to other clients */ for( c=0... notifier calls read_client read_client( c_fd.buff.buf_len)... . id ) { /* read client’s message */ len = read(c_fd. step Œ performed once at initialisation steps  & Ž happen any number of times . c++) { if ( client_fd == c_fd ) { /* special reply for sender */ } else { /* relay message to other clients */ } } N.

• function f is your callback • f is called when a buffer can be read from fd .h" int inform_input( int fd. #include "inform. without blocking • the identifier id is also passed to f int inform_output( int fd.. inform_fun f.. • similar to inform_input but for output • f is called when a buffer can be written to fd int inform_loop( ).My window-less callbacks – 1 • so you can experience the pain of callbacks without the added pain of windows . inform_fun f. inform_id id ).. • gives control to the 'notifier' which performs callbacks for you TCP/IP UNIX Short Course Notes Alan Dix © 1996 5/12 . inform_id id )..

"description" }. line_fun line_f. { 0. "command". • • • • helper for simple command interface you make monitor_tab with suitable functions the first word in buff is regarded as a command it is looked up in monitor_tab . • the file fd is monitored by notifier • two callbacks: line_f and eof_f • line_f is called when a complete line is read • eof_f is called when the end of file is reached #include "monitor.h" struct mon_tab_struct monitor_tab[] = { { 0. id_type id ). 0 } }. eof_fun eof_f.My window-less callbacks – 2 #include "line_by_line. 0. int perform_line( char *buff ).h" int inform_line_by_line( int fd. 0. . . callback. and the relevant callback is run TCP/IP UNIX Short Course Notes Alan Dix © 1996 5/13 .

do you notice delays? Harder bits add the user name to the per-client data structure alter the server so that all messages use the name rather than client number TCP/IP UNIX Short Course Notes Alan Dix © 1996 5/14 . make the server wait for a line (or buffer) of input from the client (the clients name) modify the notification message it sends to all the clients to make it name the user compile and run (use the same client) run several clients.c by using: make -f make4 new-conf locate the place where the server first establishes contact with the client.c call it new-server.    h       h   Hands on     the conference server is not very friendly it refers to everyone by number you are going to make this better! copy server.c edit the makefile make4 so that you can compile new-server.

UNIX Session 6 Network Programming with TCP/IP UNIX Forking Servers & more TCP/IP Network Programming with TCP/IP UNIX Network Programming with TCP/IP UNIX UNIX Network Programming with TCP/IP Network Programming with TCP/IP .

hcibook. exec and wait  remote shell • inet demon and remote login  another echo server • IP fragmentation • TCP flow control TCP/IP UNIX Short Course Notes Alan Dix © 1996 6/1 .UNIX Network Programming with TCP/IP Session 6 Alan Dix http://www.com/alan Forking Servers & TCP/IP behaviour • UNIX processes and fork • forking servers • fork system call • example code • dup.

g. some central point of control client client client client server process server process server process server process database server file locking demon TCP/IP UNIX Short Course Notes Alan Dix © 1996 6/2 . WWW • no interaction at all ⇒ separate process to serve each client • weak interaction ⇒ need locking. i.e.Loosely coupled services • closely coupled: strong client interaction e.g. electronic conference • loosely coupled: little or no client interaction e. database server etc.

A UNIX process UNIX process: • identified by process id (pid) • process includes: r program code r application data r system data p including file descriptors pid = 597 code data system data e.g. file descriptors TCP/IP UNIX Short Course Notes Alan Dix © 1996 6/3 .

Forking UNIX 'fork' duplicates process: • copies complete process state: r r program data + system data including file descriptors • code immutable – shared 597 code 632 code data data system data system data $ echo $$ 597 $ (echo $$) 632 $ TCP/IP UNIX Short Course Notes Alan Dix © 1996 6/4 .

' r in a shell script: use '$$' N.Forking – 2 • old process called the parent • new process called the child • process ids r allocated sequentially r so effectively unique (but do wrap after a very long time) • finding process ids r at the shell prompt: use 'ps' r in a C program: use 'int p = getpid().B. useful for naming temporary files: tmpfile = "/tmp/myfile$$" TCP/IP UNIX Short Course Notes Alan Dix © 1996 6/5 .

Use in servers x the server passive opens a port and waits for a client client server y the client performs an active open a connection is established client server TCP/IP UNIX Short Course Notes Alan Dix © 1996 6/6 .

. .Use in servers – 2 z the server forks a child client child fork server • child is a copy of the server • both socket connections are duplicated ⇒ ⇒ server waiting on port . . . . . . . . and server connected to client TCP/IP UNIX Short Course Notes Alan Dix © 1996 6/7 . . and child waiting on port child connected to client .

Use in servers – 3 { server closes the connection child closes the passive port client child server | server waits for further connections child talks to client client child server TCP/IP UNIX Short Course Notes Alan Dix © 1996 6/8 .

Fork system call pid_t ( pid_t ≈ int ) p = fork(). • if successful r process r successful fork returns: 0 – to child process child pid – to parent process ⇒ parent and child are different! • negative result on failure TCP/IP UNIX Short Course Notes Alan Dix © 1996 6/9 .

else printf("failed\n"). c_pid = 0 ¬ ¬ DATA i = 3 DATA i = 3 • except for the return value of fork TCP/IP UNIX Short Course Notes Alan Dix © 1996 6/10 . c_pid = -1. else if ( c_pid > 0 ) printf("parent\n"). else printf("failed\n"). c_pid = fork(). else printf("failed\n"). c_pid = fork(). if ( c_pid == 0 ) printf("child\n"). else if ( c_pid > 0 ) printf("parent\n"). c_pid = fork(). else if ( c_pid > 0 ) printf("parent\n"). if ( c_pid == 0 ) printf("child\n"). c_pid = -1. c_pid = 632 632 int i = 3. c_pid = -1 DATA i = 3 • after fork parent and child identical 597 int i = 3. c_pid = -1.Execution – 1 • parent forks 597 ¬ int i = 3. if ( c_pid == 0 ) printf("child\n").

else if ( c_pid > 0 ) printf("parent\n"). c_pid = -1. c_pid = fork(). c_pid = -1. else printf("failed\n"). if ( c_pid == 0 ) printf("child\n"). c_pid = fork(). c_pid = fork(). if ( c_pid == 0 ) printf("child\n"). c_pid = 632 632 int i = 3. else printf("failed\n").Execution – 2 • because data are different 597 int i = 3. c_pid = 632 632 int i = 3. else printf("failed\n"). else printf("failed\n"). c_pid = -1. if ( c_pid == 0 ) printf("child\n"). if ( c_pid == 0 ) printf("child\n"). else if ( c_pid > 0 ) printf("parent\n"). else if ( c_pid > 0 ) printf("parent\n"). c_pid = -1. c_pid = fork(). c_pid = 0 ¬ ¬ DATA i = 3 DATA i = 3 • program execution differs 597 int i = 3. c_pid = 0 ¬ ¬ DATA i = 3 DATA i = 3 • so parent and child behaviour diverge TCP/IP UNIX Short Course Notes Alan Dix © 1996 6/11 . else if ( c_pid > 0 ) printf("parent\n").

.B.fork based shell server – 1 Basic structure: • establish port • loop forever • on each loop: r accept a single client connection r fork a child to manage client • child execs a copy of the shell N.. exit(0).) { /* open port port_sk = tcp_passive_open(port) /* loop forever accepting clients while ( accept_one(port_sk) > 0 ). /* on error close and exit close(port_sk). } */ */ */ TCP/IP UNIX Short Course Notes Alan Dix © 1996 6/12 . no login – very insecure ! Œ Main loop main(.

/* N. /* perform fork */ child_pid = fork().B. } */ */ */ */ • negative result on failure } else return 0.fork based shell server – 2  Process each client in turn accept_one( int port_sk ) { /* accept a single connection */ client_sk = tcp_accept(port_sk). child has open descriptor /* so client is not cut off /* returns child pid to main loop return child_pid. } */ */ • parent gets child process id returned from fork else if ( child_pid > 0 ) { /* parent closes client socket close(client_sk). • child gets zero return from fork if ( child_pid == 0 ) { /* child closes passive port close(port_sk). TCP/IP UNIX Short Course Notes Alan Dix © 1996 6/13 . /* then starts its own behaviour exec_a_shell(client_sk).

} TCP/IP UNIX Short Course Notes Alan Dix © 1996 6/14 . /* standard output to fd dup2(fd.1).2). /* doesn't return */ • shell will expect I/O from standard file descriptors use 'dup2' system call to link them to fd dup2(fd. /* standard input from fd dup2(fd.1).exec_fail_mess).argv). execv("/bin/sh". _exit(1). only the child process calls this function int exec_a_shell(int fd) { int tty_fd.0). */ */ */ • exec only returns if it fails • standard error has been closed so need to open /dev/tty explicitly tty_fd = open("/dev/tty". /* standard error to fd close(fd). write(tty_fd.B..fork based shell server – 3 Ž Child execs a copy of the shell N.

. • negative return on failure TCP/IP UNIX Short Course Notes Alan Dix © 1996 6/15 .0).buff. – standard input reads from fd • can close the old descriptor .dup2 system call int res = dup2(old_fd. • makes new_fd point to same file/stream as old_fd • new_fd is closed if already open • most often used with standard I/O descriptors: dup2(fd. but new descriptor still works dup2(fd. new_fd).buff_len). . close(fd): n = read(0.0).

char **argv).exec system call execv(char *prog.B. execvp. . . . • replaces the current process with prog • never returns except on failure • argv is passed to the 'main' of prog N. needs at least argv[0] set to program name • new process: r code – r data – r system data – f replaced by prog reinitialised partly retained file descriptors still open • several variants (execl. ) • often used after fork to spawn a fresh program TCP/IP UNIX Short Course Notes Alan Dix © 1996 6/16 .

exec vs. fork • fork duplicates process • exec replaces process 597 493 code data system code data system fork exec 597 632 493 code data system code data system code data system • fork child shares open file descriptors • exec-ed process retains open fds TCP/IP UNIX Short Course Notes Alan Dix © 1996 6/17 .

death of a forked process • when parent dies r children become orphans ! r system init process 'adopts' them • when child dies r parent (or init) informed by signal ( SIGCHLD ) r r r child process partly destroyed rump retained until parent 'reaps' – using wait or wait3 system call until then child is 'zombie' – ps says <exiting> or <defunct> N. zombie state necessary so parent can discover which child died TCP/IP UNIX Short Course Notes Alan Dix © 1996 6/18 .B.

• function 'my_reaper' called when signal arrives y then reap a child int my_reaper() { union wait status. . they stay zombies forever .NULL) >= 0 ). system resources may run out x first catch your signal signal(my_reaper.WNOHANG. while( wait3(&status. . . .SIGCHLD). } • use WNOHANG so that wait3 doesn't block • loop to reap multiple children TCP/IP UNIX Short Course Notes Alan Dix © 1996 6/19 .SIGCHLD & wait3 • if parent does not reap children .

NULL) TCP/IP UNIX Short Course Notes Alan Dix © 1996 6/20 .fork and I/O low-level I/O • open file descriptors shared so: r output is merged r input goes to first read – accept similar r close down may be delayed until all processes close fd ⇒ close all unwanted fds or use ioctl to set close-on-exec high-level I/O • C stdio is buffered: r duplicated at fork r may get flushed after fork ⇒ duplicate writes  stderr OK – unbuffered ⇒ careful with stdio use stderr or setbuff(fd.

c : io 3% knife -port 2345 connect to it from a different machine or window klah 7% telnet io 2345 do you get a shell prompt? try something simple like echo hello then try ps what happens? try typing a # at the end of each line echo hello# ps # what is happening? TCP/IP UNIX Short Course Notes Alan Dix © 1996 6/21 .c : make -f make6 knife launch the knife server: knife.c make6 compile knife.   Hands on tcp/session6:              copy the following from knife.

inet demon
• there are many Internet services: ftp, telnet, rlogin, echo, etc. • a server for each is expensive • inetd is a multi-service server • it does a passive open on lots of ports: 21 – ftp, 25 – SMTP, etc. • when a client connects it forks the appropriate service • remote logins somewhat complicated

TCP/IP UNIX

Short Course Notes

Alan Dix © 1996

6/22

remote login
First solution . . . . . . simply fork a shell or getty $ no translation of codes
e.g. end of line sequence

$ no terminal driver at server end ⇒ no tty control by application
e.g. editors need tty raw mode

Actual solution . . . . . . intermediate process • server-end process between client and shell/getty  can perform translation  pseudo-tty between it and shell ⇒ server-end tty control

TCP/IP UNIX

Short Course Notes

Alan Dix © 1996

6/23

remote login – 2
x remote login client connects to server

client

server

y server forks child to handle login

client

child

fork

server

TCP/IP UNIX

Short Course Notes

Alan Dix © 1996

6/24

remote login – 3 z child then forks another process server client child /dev fork and exec shell { the new process connects to the child using a pseudo-terminal | and finally execs a shell (or getty etc.) f user is now connected to shell TCP/IP UNIX Short Course Notes Alan Dix © 1996 6/25 .

both concurrent • difference in use of terminal: r where client child r – – – – application end of tty 'user' end of pseudo-tty tty always in raw mode pseudo-tty mode set by shell how client child ⇒ only one layer of tty processing TCP/IP UNIX Short Course Notes Alan Dix © 1996 6/26 .remote login – 4 • client and server-side child similar r both connected to network r both connected to (pseudo)terminal • general algorithm: r echo terminal input to network r echo network input to terminal N.B.

c to make a forking echo server your previous echo server (session 2) only dealt with one client – this one will deal with any number copy knife.c into echo-all locate the sub-routine where the shell is exec-ed replace the code duplicating file descriptors and exec-ing the shell – simply have a loop which reads from the socket and writes back to it compile and run echo-all io 15% io 16% make -f make6 echo-all echo-all -port 2345 an connect to it: klah 23% telnet io 2345   there is an alternative solution which only involves replacing 2 characters of knife.   Hands on    echo server       modify knife.c hint: the answer doesn't involve any dogs TCP/IP UNIX Short Course Notes Alan Dix © 1996 6/27 .

3/802.5 token ring FDDI Ethernet IEEE 802.MTUs • the Internet is heterogeneous r r heterogeneous transport layers ⇒ different packet sizes dynamic routing ⇒ hops on different layers ⇒ unpredictable packet size • transport layer limit called MTU: – maximum transmission unit transport layer Hyperchannel 16Mbps IBM token ring 4Mbps IEEE 802.25 PPP (performance limit) MTU in bytes 65535 17914 4464 4352 1500 1492 576 296 (from RFC 1191) TCP/IP UNIX Short Course Notes Alan Dix © 1996 6/28 .2 X.

. . .. NFS = 8192 bytes TCP/IP UNIX Short Course Notes Alan Dix © 1996 6/29 . larger datagrams possible e. .IP fragmentation • what happens when size is too small? • fragmentation r r r r any intermediate router detects problem IP datagram broken into pieces each sent separately (possibly different routes) reconstructed at further router or destination • real limit is recipient's buffer size r r r 576 bytes IP datagram guaranteed .g. .. but this includes headers UDP limit = 512 bytes user data TCP divides data up for you limit is UNIX read/write buffers • only end points matter ⇒ in a controlled environment .

• IP is not reliable ⇒ some packets (fragments) may be lost • no re-transmission r r r IP handles reconstruction . . . well almost .fragmentation considered harmful • fragmentation ⇒ IP transparent to underlying link layer MTU . . . . . . but not fragment retransmission fragment lost ⇒ whole IP datagram lost probability one fragment lost = p n fragments ⇒ probability IP datagram lost ≈ n p • avoiding fragmentation r r UDP – most protocols ≤ 512 bytes TCP – uses local (end-point) MTU + path MTU discovery algorithm TCP/IP UNIX Short Course Notes Alan Dix © 1996 6/30 . . .

X) TCP/IP UNIX Short Course Notes Alan Dix © 1996 6/31 .TCP reliability • underlying IP unreliable ⇒ TCP must handshake • stream protocol r r sender: this is bytes n–m of the data recipient: ack m – last byte received • retransmission r r recipient: out of order receipt → repeat ack timeout or several repeat acks → retransmit • too many acks r r r r r avoid lots of little acknowledgement packets ack of last packet ⇒ previous packets arrived piggyback A→B ack on B→A message delay acks to allow piggyback turn off delay for some protocols (e.g.

TCP flow control Cannot send without limits: q network capacity → packet loss r exponential backoff rapid resend → nightmare scenario ⇒ long delay before failure (2-9 mins) r slow-start algorithm q link-layer buffer r MSS announcement TCP buffer r window size announcement only send to last ack + window size window size q known to be received last ack last byte sent may be sent before next ack sent but not acknowleged must be held at sender end TCP/IP UNIX Short Course Notes Alan Dix © 1996 6/32 .

UNIX Session 7 Select and Security Network Programming with TCP/IP UNIX Network Programming with TCP/IP UNIX Network Programming with TCP/IP UNIX UNIX Network Programming with TCP/IP Network Programming with TCP/IP .

secrecy and privacy • under attack: viruses & worm • the Internet worm • levels of security • encryption and authentication TCP/IP UNIX Short Course Notes Alan Dix © 1996 7/1 .hcibook.UNIX Network Programming with TCP/IP Session 7 Alan Dix http://www.com/alan Select and Security • UNIX events • select system call • proxy server  raw client • security.

UNIX Events Computational programs: • busy most of the time • read/write when they are ready Interactive programs: • servers & clients • idle most of the time • respond to events UNIX processes – 4 types of event x signal (interrupt) y time (alarm) z input ready read will not block { output can accept (more) data write will not block TCP/IP UNIX Short Course Notes Alan Dix © 1996 7/2 .

Responding to events
Events: x signal (interrupt) y time (alarm) z input (read) ready { output (write) ready Responding • interrupt handler – x&y
use signal system call use setitimer to send SIGALRM

• turntaking – y,z&{
call read/write when ready use sleep for delays

• polling – y,z&{
use non-blocking read/write use time to do things at specific times

• wait for several events
use select system call timeout or SIGALRM

TCP/IP UNIX

Short Course Notes

Alan Dix © 1996

7/3

polling in UNIX
#include <sys/filio.h>

ioctl(fd,FIONBIO,1);

• call to ioctl tells system: don’t block on read/write • polling therefore possible • structure of polling telnet-like client:
ioctl(tty_fd,FNBIO,1); ioctl(net_fd,FNBIO,1); for(;;) { /* any terminal input? */ n = read(tty_fd,buff,buff_len); if ( n > 0 ) { /* yes! do something */ /* any network input? */ n = read(net_fd,buff,buff_len); if ( n > 0 ) { /* yes! do something */ }

} }

TCP/IP UNIX

Short Course Notes

Alan Dix © 1996

7/4

read & write
read: • waits on one file descriptor • returns when input data is ready • and reads the data into a buffer

read(0,buff,len)

write: • waits on one file descriptor • returns when output is possible • and writes the data from the buffer
write(1,buff,len)

TCP/IP UNIX

Short Course Notes

Alan Dix © 1996

7/5

select select: • waits on many file descriptor • returns when input or output ready • but does no actual I/O + also allows timeout select(width.&in_fds.&err_fds.&out_fds.&timeout) TCP/IP UNIX Short Course Notes Alan Dix © 1996 7/6 .

e.&err_fds.&in_fds. err_fds • • size: timeout: when to timeout in seconds and milliseconds Returns when: • input ready on one of in_fds • output ready on one of out_fds • error occurs on one of err_fds • timeout expires • signal has been caught • some other error occurs (ret (ret (ret (ret (ret (ret > 0) > 0) > 0) == 0) < 0) < 0) TCP/IP UNIX Short Course Notes Alan Dix © 1996 7/7 .&out_fds. read will not block wait for output i.&timeout).select system call – 2 int ret = select(size. out_fds. • in_fds. write will not block size of in_fds. out_fds: r r in_fds out_fds – – – – – bitmaps of file descriptors wait for input i.e.

h> fd_set in_fds.select and I/O #include <sys/types. limit of 32 file descriptors . err_fds • modified by call: call – bit set = wait for file desc return – bit set = file desc ready return value from select = number ready • long integer in early UNIX systems in_fds = in_fds || ( 1<<fd ). out_fds. r ⇒ testing: if ( FD_ISSET(fd. but some systems allow more • now a special fd_set structure actually an array of integers! r setting: FD_ZERO( &in_fds ). FD_SET( fd. TCP/IP UNIX Short Course Notes Alan Dix © 1996 7/8 . . &in_fds )...&in_fds) ) . FD_CLR( fd. &in_fds ). .

. . but dangerous! TCP/IP UNIX Short Course Notes Alan Dix © 1996 7/9 .select and I/O – 2 • input r terminal/socket – read will not block r passive socket – accept will not block • output r terminal/socket – write ‘ready’ r write relies on system resources r change between select and write? ⇒ write may block c use non-blocking write • can ‘get away’ without select on write .

but may do one day don’t rely on timeout not being changed reset for each call to select ⇒ TCP/IP UNIX Short Course Notes Alan Dix © 1996 7/10 .tv_ms – maximum time to wait in seconds and ms • if no I/O ready and no signals in time limit then select returns with zero result N.select and timeouts #include <sys/time.B. .h> struct timeval timeout. . out_fds. in_fds. err_fds all zero also • modified by call? r r ideally should return time remaining doesn’t now .tv_secs timeout. . . • timeout. .

. . .select and signals • signal occurs during system call: read. . or select • signal not caught . . . process aborts! • signal caught . . x relevant handler called y systems call returns with ‘error’ • how do you know? r negative return value r errno set to EINTR • negative return & errno ≠ EINTR ⇒ really an error! TCP/IP UNIX Short Course Notes Alan Dix © 1996 7/11 . write.

.SIGINTR).1. } } • intention: execute do_something once per interrupt • what actually happens: x y z { | interupt processed (i=1) do_something executes main calculates i-1 gets result 0 before it stores the result . . ..care with signals • signal handlers can run at any time int i = 0 int my_handler() { i = i + 1 } main() { signal(my_handler. . for(. another interupt (i=2) main stores result (i=0) TCP/IP UNIX Short Course Notes Alan Dix © 1996 7/12 . i = i .) if ( i > 0 ) { do_something(). .

telnet/rlogin clients TCP/IP UNIX Short Course Notes Alan Dix © 1996 7/13 .g.when to use select • servers: r where concurrency essential r possibly ftp server – listen to control & data r telnet server – listen to user over network + listen to shell/application • clients r not with most window managers – instead use callback r some event stream WMs – single fd for WM events – listen to WM and network r terminal based clients – not needed for turn-taking – e.

proxy server • proxy server used in session 3 • structure of code x passive open on own port y wait for client connection z active open on remote server { loop forever waiting for client or server input: r when client data ready read it send to server echo it to terminal r when server data ready read it send to client echo it to terminal TCP/IP UNIX Short Course Notes Alan Dix © 1996 7/14 .

rem_port).proxy code – 1 Œ Main loop main(. /* wait for client to connect client_sk = tcp_accept(port_sk). • when do_proxy is called both network sockets open TCP/IP UNIX Short Course Notes Alan Dix © 1996 7/15 .. ret = do_proxy( client_sk. */ */ */ */ /* now connect to remote server */ serv_sk = tcp_active_open(rem_host.) { /* establish port port_sk = tcp_passive_open(port). } exit(0). serv_sk ).. /* so close port_sk close(port_sk). /* only want one client.

• then call select num = select(MAX_FD. • then loop forever for(. FD_ZERO(&write_fds).&read_fds). { & | at this point } } return 0.. ¯ check return – z. FD_ZERO(&ex_fds). write_fds. FD_SET(client_sk. FD_SET(serv_sk . FD_ZERO(&read_fds). fd_set write_copy = write_fds.0}. &ex_copy. fd_set ex_copy = ex_fds. &timeout). int serv_sk ) { • first declare and initialise fd bitmaps fd_set read_fds.&read_fds). static struct timeval timeout = {0. &write_copy.proxy code – 2 y perform proxy loop int do_proxy( int client_sk. len. &read_copy.) { int num. • copy bitmaps because select modifies them fd_set read_copy = read_fds. ex_fds. TCP/IP UNIX Short Course Notes Alan Dix © 1996 7/16 .

return 1. we may need to do some processing following the interrupt it is usually better for the interrupt to set some flag and let the main loop do most of the work this reduces the risk of stacked interrupts and mistakes in concurrent access to data structures if (num < 0 && errno == EINTR ) { /* stopped by signal */ perror("EINTR"). /* timeout */ TCP/IP UNIX Short Course Notes Alan Dix © 1996 7/17 . • if num is zero then a timeout has occurred again.proxy code – 3 z check for signals. we are not expecting any so return in general. in this case no processing but in general this is the opportunity for animation or other periodic activity if ( num == 0 ) continue. } < 0 is an error /* not stopped by signal */ return 1. } • first check for signals: • if there has been no signal num if (num < 0 ) { perror("select"). errors and timeout in this case.

&read_copy) ) { int len = read( serv_sk . } else { write(client_sk. • on end of file or error exit the loop if ( len <= 0 ) { /* error or close close(serv_sk). log_from_server( buff. if ( len <= 0 ) { /* error or close */ close(client_sk). buf_len ). log_from_client( buff. buff. write it to the server and log it else { write(serv_sk.len).&read_copy) ) { int len = read( client_sk.proxy code – 4 { check for client input client ready if bit is set in read_copy if ( FD_ISSET(client_sk. } */ • if there is some input data. return len.len). return len.buff. buf_len ). buff. len ). } } | server input similar if ( FD_ISSET(serv_sk . } } TCP/IP UNIX Short Course Notes Alan Dix © 1996 7/18 . len ).buff.

g.c and make7 from tcp/session7 copy proxy. and from the server to the user the major difference is that the proxy server operates on the ‘other end’ of a network connection you are going make a simple telnet-like client copy proxy. e. . . .c   h  proceed as follows: x y z { | remove the code to open the client connection (passive open and accept) remove the parameter to do_proxy which corresponds to the client socket modify the FD_SET calls so that select waits for standard input (0) rather than the client change all read calls from the client so that they read from standard input (0) change all write calls to the client so that they write to standard output (1)  now compile and run your raw client.c reads and writes the client socket you want to read from standard input (0) and write to standard output (1) proxy.c and call it raw-client. .    h Hands on     the proxy server is a bit similar to a telnet client both open a connection to a remote server both echo from the user to the server . .: (send mail as in session 3 page 3/17) raw-client hades 25 TCP/IP UNIX Short Course Notes Alan Dix © 1996 7/19 .

special problem for computers TCP/IP UNIX Short Course Notes Alan Dix © 1996 7/20 . modify /etc/passwd • chain reaction r small breach → complete loss e. password → login r resources → information e.g.B.Security • types of security: r information: – secrecy – privacy r resources: – destructive access – virus infection • linked r information → resources e. root password! r N.g.g.

if ( strcmp(buff."Alan's secret way in") == 0 ) { /* connect client_sk to a root shell */ } /* normal operation * / • external r hackers r accidental release – e. debtor files etc r using resources – surfing. forgotten portable on the train r industrial espionage r viruses TCP/IP UNIX Short Course Notes Alan Dix © 1996 7/21 .g. doom! r downloading material – indecent. possibly illegal r backdoors client_sk = tcp_accept(port_sk).who are you afraid of? • internal r selling your secrets r personal data – payroll.buff.buff_len). buff(len) = '\0'. n= read(client_sk.

names and definitions differ TCP/IP UNIX Short Course Notes Alan Dix © 1996 7/22 .B.g.under attack • viruses a real risk? heterogeneous ⇒ cross-infection more difficult $ lots of machines just like yours ? interpreted languages? – can be made secure (e. JAVA) • types of attack r virus – embeds itself in another program r Trojan horse – masquerades as another program r worm – independent self-replicating program N.

viruses on the web? • explicit download of code r helpers – machine specific code r general software $ both risk infection • implicit download r semi-compiled – JAVA r interpreted – JAVA script r embedded in HTML $ you may never know! the good news r JAVA & JAVA script ‘safe’ r cannot read or write to local disk $ the bad news r JAVA script can connect remotely r send details of browsing patterns r minor breach of privacy ? the only breach possible? TCP/IP UNIX Short Course Notes Alan Dix © 1996 7/23 .

The Internet Worm • for 2 days in 1988. the Internet was under siege November 2nd. November 3rd. 1988 00:21 02:38 03:15 05:54 06:45 11:30 16:00 21:30 Princeton University main machine crashes due to load email from Berkeley: “We are under attack” anonymous warning from foo@bar. Lab. 1988 05:00 11:00 17:20 21:30 MIT finish decompiling worm Milnet rejoins Arpanet final set of preventative patches mailed worm’s author identified – named in the next day’s newspaper as Robert T. Morris son of the NCSC’s chief scientist Robert Morris! • infections still noted as late as December 1988 TCP/IP UNIX Short Course Notes Alan Dix © 1996 7/24 .arpa patches to sendmail distributed National Computer Security Centre (NCSC) informed Milnet severs itself from Arpanet to prevent infection inoculation method found (directory sh in /usr/tmp) Berkeley start to decompile ‘captured’ worm November 4th. 1988 17:00 21:00 22:04 23:40 23:45 worm launched from Cornell University worm detected at Stanford worm detected at Berkeley Berkeley discover one means of attack (sendmail) infects Dartmouth and Army Ballistics Res.

What went wrong? • several means of attack • between machines: r r r debug mode in sendmail buffer overflow in fingerd once broken into a user on a machine – rlogin/rsh to other hosts • within a machine: r simple password attacks – permutations of user’s own name – internal list of 432 common passwords – system dictionary • attempted to prevent repeat infection r r r didn’t always work main damage was excessive load due to repeat infections (often 100s) also how it was detected TCP/IP UNIX Short Course Notes Alan Dix © 1996 7/25 .

sendmail attack • sendmail r r r r had a debug mode worm connects to sendmail worm sends ‘debug’ command sendmail will then execute any command! should have been disabled but sendmail is complex! • similar attacks still possible r system engineer accounts r remote vendor maintenance • any debug modes on your system? TCP/IP UNIX Short Course Notes Alan Dix © 1996 7/26 .

at worst is a loophole r always be careful of buffer lengths • never again? r a popular WWW browser .fingerd attack • fingerd r r r r uses gets – buffer overflow worm connects to fingerd worm sends 536 byte line overflows fingerd’s buffer (512 bytes) . . . . and corrupts stack extra 24 bytes executed as code! • lessons: r never use gets! – at best may crash . – corrected in later versions TCP/IP UNIX Short Course Notes Alan Dix © 1996 7/27 . .

substitute disks etc. ethernet • never trust transport layer TCP/IP UNIX Short Course Notes Alan Dix © 1996 7/28 .g. ‘listening in’ easy on many networks e.physical security • physical security: r are the machines secure r is the network secure can someone reboot.B.? can someone link-in their own computer? • local or global? x local network and machines y backbone and routers z remote network and machines • secure? x possible y reasonable for non-critical data z no way! N.

telnet • audit: r risk of detection deters r keeping logs r relies on authentication $ SMTP reverse name lookup can’t check FROM field – e. Netscape secure sockets layer for WWW • authentication: r who am I talking to? r is it the real server? r $ is it an acceptable client? $ rely on correct routing and protected ports impostor machine. credit card by email r use encryption e.g. non-UNIX server host user passwords often sent as plain text! – e.logical security • secrecy: r TCP/IP packets not secure e.g.g. worm warning TCP/IP UNIX Short Course Notes Alan Dix © 1996 7/29 .g.

. but difficult for ftp r restrict incoming TCP packets – TCP/IP UNIX Short Course Notes Alan Dix © 1996 7/30 .. but lose the benefits too don’t publish domain machine names ...low-level protection – firewalls • simple measures r isolation – r don’t connect to the global Internet . impostors limited internal routing – protect sensitive machines/data only allow connection to protected ports .B. but IP addresses still valid anonymity – • firewalls r application independent r act at router/gateway r can only look at IP or TCP headers • what is possible r only allow friendly IP addresses – r N...

high-level protection – ring fences • rlogin r beware external root logins! r passwords: – – ? if reasonable no ‘equiv’ hosts certainly no root ‘equiv’ hosts means lots of duplicate password files? • servers r never run as root? – impossible! e. no write perm! r /etc/passwd – encrypt or restricted read – may cause problems TCP/IP UNIX Short Course Notes Alan Dix © 1996 7/31 . rshd r never unnecessarily run as root? – special login e. user ‘ftp’ – run as user ‘nobody’ • the rest of the system – normal measures r backups – damage limitation r permissions – restrict ‘other’ access r setuid – dangerous.g. inetd.g.

encryption • one way function: cypher = f(input) input r r = ?(cypher) – – easy hard used in /etc/passwd brute force attack: for each possible input inp if f(inp) is cypher – got it! • single key cypher = code(key.cypher) r r – given to everyone – – kept by you – anyone can send a message only you can decrypt it key1 key2 public private TCP/IP UNIX Short Course Notes Alan Dix © 1996 7/32 .cypher) r in DES – code = decode • public key encryption cypher = code(key1.input) input = decode(key2.input) input = decode(key.

KS ) B sends KXA to A A decrypts KXA X. KS ) A sends KSB to B B decrypts KSB to obtain KS KS = decode(KB2 .KS = decode(KA2 . KSA ) A encrypts X using B’s public key XB = code(KB1 .session keys and authentication • public keys good. X ) and sends it to B A and B share a secret key A and B sure of each other’s identity • result: • discard key after session or fixed time TCP/IP UNIX Short Course Notes Alan Dix © 1996 7/33 . KSB ) B generates value X B encrypts X and Ks using A’s public key KXA = code(KA1 . X. but: r expensive r the more you use a key the easier it is to break • use public keys to exchange single key x y z { | | } ~  € r r machine A generates session key KS A encrypts it using B’s public key KSB = code(KB1 .

hand) • if x or y: how do you know it is B/C? • if y: why should you believe C? ⇒ z? $ no good for broad distribution use an authentication server r trusted machine r everyone tells it their public key (using its public key or physical) r ask it for other’s public keys r or ask it for session keys TCP/IP UNIX Short Course Notes Alan Dix © 1996 7/34 . tells you z use physical means (post. C.authentication servers • how do you find out B’s public key? • answers: x B tells you y someone else.

you won’t stop them • main differences – rate of loss (Mbytes/sec) – hidden loss (electronic copies) – automatic attack • ease of use ≈ ease of access – where do you draw the line TCP/IP UNIX Short Course Notes Alan Dix © 1996 7/35 .don’t panic! • how secure is a fax? • credit card number by phone • hacker ≈ burglar – if they want in.

Sign up to vote on this title
UsefulNot useful