High

High Performance
Performance
Computing
Computing

Lecture 3
Network Interconnection
and Protocols
Topic
Topic Overview
Overview
“ Network topologies
“ Logical topologies
“ Network implementations
“ Network protocols
Network
Network Topologies
Topologies
“ Bus
“ Ring
“ Star
“ Extended Star
“ Hybrid
Bus
Bus Topology
Topology
All devices are connected to a central cable, called the bus or
backbone. Bus networks are relatively inexpensive and easy to install
for small networks. Ethernet systems use a bus topology.
Bus
Bus Topology
Topology
“ Standard is IEEE 802.3
“ Thin Ethernet (10Base2) has a maximum
segment length of 200m
“ Max no. of connections is 30 devices
“ Four repeaters may be used to a total cable
length of 1000m
“ Max no. of nodes is 150
Bus
Bus Topology
Topology
“ Thick Ethernet (10Base5) used for
backbones
“ Limited to 500m
“ Max of 100 nodes per segment
“ Total of four repeaters , 2500m, with a
total under 500 nodes
Bus
Bus Topology
Topology
Advantages Disadvantages
“ Backbone breaks, whole
“ Inexpensive to install
network down
“ Easy to add stations “ Limited amount of devices
“ Use less cable than can be attached
other topologies “ Difficult to isolate problems
“ Sharing same cable slows
“ Works well for small
response rates
networks
Ring
Ring Topology
Topology
All devices are connected to one another in the shape of a closed loop, so that each
device is connected directly to two other devices, one on either side of it. Ring
topologies are relatively expensive and difficult to install, but they offer high
bandwidth and can span large distances.
Ring
Ring Topology
Topology
“ All devices of equality of access to media
“ Single ring – data travels in one direction only,
guess what a double ring allows !?
“ Each device has to wait its turn to transmit
“ Most common type is Token Ring (IEEE 802.5)
“ A token contains the data, reaches the destination,
data extracted, acknowledgement of receipt sent
back to transmitting device, removed, empty token
passed on for another device to use
Ring
Ring Topology
Topology
Advantages
“ Data packets travel at
great speed
“ No collisions
Disadvantages
“ Easier to fault find
“ Requires more cable
“ No terminators required
than a bus
“ A break in the ring will
bring it down
“ Not as common as the
bus – less devices
available
Star
Star Topology
Topology
All devices are connected to a central hub. Star networks are relatively easy to install
and manage, but bottlenecks can occur because all data must pass through the hub.
This is not much of a problem anymore with the widespread deployment of switches.
Star
Star Topology
Topology

“ Centre point is a Hub

“ Segments meet at the Hub
“ Each device needs its own cable to the
Hub
“ Predominant type of topology
“ Easy to maintain and expand
Star
Star Topology
Topology
Advantages
“ Easy to add devices as the
network expands
“ One cable failure does not bring
down the entire network
(resilience) Disadvantages
“ Hub provides centralised “ A star network requires more
management cable than a ring or bus
network
“ Easy to find device and cable
problems “ Failure of the central hub can
bring down the entire network
“ Can be upgraded to faster speeds
“ Costs are higher (installation
“ Lots of support as it is the most and equipment) than for most
used bus networks
Extended
Extended Star
Star Topology
Topology

A Star
Network
which has
been
expanded to
include an
additional
hub or hubs.
Hybrid
Hybrid Topology
Topology
Hybrid
Hybrid Topology
Topology

“ Old networks are updated and replaced,

leaving older segments (legacy)
“ Hybrid Topology – combines two or more
different physical topologies
“ Commonly Star-Bus or Star-Ring
“ Star-Ring uses a MAU (Multistation Access
Unit)
“ A MAU is a device to attach multiple
network stations in a star topology in a
Token Ring network, internally wired to
connect the stations into a logical ring.
Types
Types of
of Logical
Logical Topology
Topology
“ Previous slides showed Physical Topologies
“ Only two Logical Topologies (Bus or Ring)
“ Physical Bus or Ring easy to conceptualise
“ Physical Star could be either a Bus or Ring in
logical terms
“ Confused ? See next slides
Logical
Logical Bus
Bus

•Modern Ethernet networks are Star Topologies (physically)

•The Hub is at the centre, and defines a Star Topology
•The Hub itself uses a Logical Bus Topology internally, to
transmit data to all segments
Logical
Logical Bus
Bus
Advantages
“ A single node failure does
not bring the network down
“ Most widely implemented
topology
Disadvantages
“ Network can be added to
“ Collisions can occur easily
or changed without
affecting other stations “ Only one device can
access the network
media at a time
Logical
Logical Ring
Ring
“ Data in a Star Topology can transmit data in a
Ring
“ The MAU (Multistation Access Unit) looks like an
ordinary Hub, but data is passed internally using
a logical ring
“ It is superior to a Logical Bus Hub
Logical
Logical Ring
Ring
Logical
Logical Ring
Ring
Advantages
“ The amount of data
that can be carried in
a single message is
greater than on a Disadvantages
logical bus “ A broken ring will
“ There are no collisions stop all transmissions
“ A device must wait
for an empty token
to be able to
transmit
Ethernet
Ethernet
“ 10BASE5 -- This is the original 10 Mbit/s implementation
of Ethernet. The early IEEE standard uses a single 50-
ohm coaxial cable of a type designated RG-8, of
maximum length 500 meters.

“ 10BASE2 -- 50 ohm RG-58 coaxial cable, of maximum

length 200 meters, connects machines together, each
machine using a T-adaptor to connect to its NIC, which
has a BNC connector. Requires termination at each end.
For many years this was the dominant 10 Mbit/s
Ethernet standard.

“ StarLAN 10 -- First implementation of Ethernet on

twisted pair wiring at 10 Mbit/s. Later evolved into
10BASE-T.
Ethernet
Ethernet
“ 10BASE-T -- Runs over 4 wires (two twisted pairs) on a cat-3 or cat-
5 cable up to 100 meters in length. A hub or switch sits in the
middle and has a port for each node.

“ FOIRL -- Fiber-optic inter-repeater link. The original standard for

Ethernet over fiber.

“ 10BASE-F (also called 10BASE-FX) -- A generic term for the family

of 10 Mbit/s Ethernet standards using fiber optic cable: 10BASE-FL,
10BASE-FB and 10BASE-FP. Of these only 10BASE-FL is in
widespread use.

Š10BASE-FL -- An updated version of the FOIRL standard.

Š10BASE-FB -- Intended for backbones connecting a number of hubs or
switches, it is now obsolete.
Š10BASE-FP -- A passive star network that required no repeater, it was never
implemented
Fast
Fast Ethernet
Ethernet (100
(100 Mbit/s)
Mbit/s)
“ 100BASE-T -- A term for any of the three standards for
100 Mbit/s Ethernet over twisted pair cable up to 100
meters long. Includes 100BASE-TX, 100BASE-T4 and
100BASE-T2.
Š100BASE-TX -- Similar star-shaped configuration to 10BASE-T. It also
uses two pairs, but requires cat-5 cable to achieve 100Mbit/s.
Š100BASE-T4 -- 100 Mbit/s Ethernet over cat-3 cabling (as used for
10BASE-T installations). Uses all four pairs in the cable. Now obsolete,
as cat-5 cabling is the norm. Limited to half-duplex.
Š100BASE-T2 -- No products exist. 100 Mbit/s Ethernet over cat-3
cabling. Supports full-duplex, and uses only two pairs. It is functionally
equivalent to 100BASE-TX, but supports old telephone cable (cat-3).
Fast
Fast Ethernet
Ethernet (100
(100 Mbit/s)
Mbit/s)
“ 100BASE-FX -- 100 Mbit/s Ethernet over multimode
fiber. Maximum length is 400 meters for half-duplex
connections (to ensure collisions are detected) or 2
kilometers for full-duplex.

“ 100Base-VG -- Championed by only HP, VG was the

earliest in the market. It needed four pair of cat-3 cables.
Its however questionable whether VG was really
Ethernet.
Gigabit
Gigabit Ethernet
Ethernet
“ 1000BASE-T -- 1 Gbit/s over cat-5e or cat-6 copper cabling.

“ 1000BASE-SX -- 1 Gbit/s over multi-mode fiber (up to 550 m).

“ 1000BASE-LX -- 1 Gbit/s over multi-mode fiber (up to 550 m).

Optimized for longer distances (up to 10 km) over single-mode
fiber.

“ 1000BASE-LH -- 1 Gbit/s over single-mode fiber (up to 100 km). A

long-haul solution.

“ 1000BASE-CX -- A short-haul solution (up to 25 m) for running 1

Gbit/s Ethernet over special copper cable. Predates 1000BASE-T,
and now obsolete.
10
10 Gigabit
Gigabit Ethernet
Ethernet
“ The new 10 gigabit Ethernet standard encompasses seven different
media types for LAN and WAN. It is currently specified by a
supplementary standard, IEEE 802.3ae, and will be incorporated
into a future revision of the IEEE 802.3 standard.

“ 10GBASE-CX4 -- designed to support short distances over copper

cabling, it uses InfiniBand 4x connectors and CX4 cabling and
allows a cable length of up to 15 m.

“ 10GBASE-SR -- designed to support short distances over deployed

multi-mode fiber cabling, it has a range of between 26 m and 82 m
depending on cable type. It also supports 300 m operation over a
new 2km multi-mode fiber.

“ 10GBASE-LX4 -- uses wavelength division multiplexing to support

ranges of between 240 m and 300 m over deployed multi-mode
cabling. Also supports 10 km over single-mode fiber.
10
10 Gigabit
Gigabit Ethernet
Ethernet
“ 10GBASE-LR and 10GBASE-ER -- these standards
support 10 km and 40 km respectively over single-mode
fiber.

“ 10GBASE-SW, 10GBASE-LW and 10GBASE-EW --

designed to interoperate with OC-192 / STM-64
SONET/SDH equipment. They correspond at the
physical layer to 10GBASE-SR, 10GBASE-LR and
10GBASE-ER respectively, and hence use the same
types of fiber and support the same distances.

“ 10GBASE-T -- Uses unshielded twisted-pair wiring.

10GBASE-T should be ready by August 2006.
Myrinet
Myrinet
“ Myrinet has much less protocol overhead than standards such as
Ethernet, and therefore provides better throughput, less
interference, and less latency while using the host CPU.

“ Myrinet physically consists of two fibre optic cables, upstream

and downstream, connected to the host computers with a single
connector. Machines are connected via low-overhead routers and
switches, as opposed to connecting one machine directly to
another.

“ Myrinet includes a number of fault-tolerance features, mostly

backed by the switches. These include flow control, error
control, and "heartbeat" monitoring on every link.

“ The first generation provided 512 Mbit/s data rates in both

directions, and later versions supported 1.28 Gbit/s and 2
Gbit/s.
Myrinet
Myrinet
“ Newest "Fourth-generation Myrinet" suppors 10
Gbit/s data rate, and is interoperable with 10 Gigabit
Ethernet

“ Myrinet's throughput is close to the theoretical

maximum of the physical layer. On the latest 2.0
Gbit/s links Myrinet often runs at 1.98 Gbit/s of
sustained throughput, considerably better than what
Ethernet offers, which varies from 0.6 and 1.9
Gbit/s depending on load.

“ However, for high performance, the low latency of

Myrinet is even more important than its throughput
performance.
InfiniBand
InfiniBand
“ Uses a bidirectional serial bus for low cost and
low latency.
“ 2.5 gigabits per second (Gbit/s) links in each
direction per connection.
“ InfiniBand also supports double and quad data
rates for 5 Gbit/s or 10 Gbit/s respectively.
“ Links can be aggregated in units of 4 or 12,
called 4X or 12X. A quad-rate 12X link therefore
carries 120 Gbit/s raw, or 100 Gbit/s of user
data.
InfiniBand
InfiniBand
“ Most systems today use a 4X single data rate
connection, though first double data rate
products are already entering the market.
“ InfiniBand uses a switched fabric topology so
several devices can share the network at the
same time (as opposed to a bus topology).
OSI
OSI vs.
vs. TCP/IP
TCP/IP
OSI
Communication TCP/IP
Model Protocol Suite

Application

SMTP
Presentation Process/Application
TELNET

Session
FTP

Transport TCP Host-Host

Network IP Internet

Data Link

Network Access

Physical
TCP/IP
TCP/IP Protocol
Protocol Suite
Suite
OSI TCP/IP Protocol Suite

Session Presentation Application

SNMP
TFTP SMTP
FTP LPD NFS (Simple
(Trivial (Simple
(File (Line (Network Network X
Telnet File Mail Others
Transfer Printer File Managem Window
Transfer Transfer
Protocol) Daemon) System) ent
Protocol) Protocol)
Protocol)
Transport

TCP UDP
(Transmission Control Protocol) (User Datagram Protocol)
Network

ICMP ARP RARP

BootP IP
(Internet Control (Address Resolution (Reverse Address
(Bootstrap Protocol) (Internet Protocol)
Message Protocol) Protocol) Resolution Protocol)
Physical Data Link

FDDI
Ethernet Token Ring (Fibre Distributed Frame Relay Others
Data Interface)
Data
Data Transport
Transport Protocols
Protocols
“ Typically provide reliable, source-ordered
delivery of messages with excellent flow control
and buffer management.
“ Transmission Control Protocol (TCP)
“ Xpress Transfer Protocol (XTP)
“ Scalable Reliable Multicast Protocol (SRMP)
Streaming
Streaming Protocols
Protocols
“ Typically used for audio, video, and multimedia
and also for certain instrumentation
applications.
“ For applications that do not require reliable
delivery and can accept message lost.
“ The datagram orientation of the Internet does
not support streaming protocols; the
connection-oriented, negotiate quality of
service approach of ATM generally provides a
better foundation for streaming protocols
“ Real-Time Transport Protocol (RTP)
“ Resource reSerVation Protocol (RSVP)
Group
Group Communication
Communication Protocols
Protocols
“ Concerned with more than just movement of
data.
“ In a distributed system, to permit effective
cooperation and to provide fault tolerance, a
group of processes must all keep copies of the
same data, and the copies of the data must be
kept consistent as the application executes.
“ This protocols assist application programmers
in maintaining the consistency of replicated
data by maintaining the membership of process
groups and by multicasting messages to those
process groups.
“ InterGroup Protocols (IGPs)
Distributed
Distributed Object
Object Protocols
Protocols
“ Protocols for heterogeneous distributed
systems
“ Suited for distributed collaborative virtual
environments and real-time instrumentation
applications.
“ Internet Inter-Orb Protocol (IIOP) for CORBA
“ Java’s Remote Method Invocation (RMI)
Network
Network Tools
Tools
ŠRocketfuel”: ISP maps
Šhttp://www.cs.washington.edu/research/networking/rocketfuel/
ŠSkitter: Internet Maps
Šhttp://www.caida.org/tools/measurement/skitter/
ŠPathchar: bandwidth measurement
Šhttp://www.caida.org/tools/utilities/others/pathchar/
ŠPathrate/Pathload: load/available bandwidth measurement
Šhttp://www.pathrate.org/
ŠVisualization tools & utilities:
Šhttp://www.caida.org/tools/visualization/
Šhttp://www.caida.org/tools/utilities/
ŠKing: end-to-end latency estimator
Šhttp://www.cs.washington.edu/homes/gummadi/king/
ŠInternet Traffic Archive:
Šhttp://ita.ee.lbl.gov/index.html
ŠNetwork Emulator
Šhttp://www.spelio.net.ru/ne/ne3/
ŠCisco Router Configuration/Emulation
Šhttp://www.cisco.com/warp/public/cc/pd/nemnsw/cm/index.shtml