RAIDs and availability

n Store more data than a single disk’s worth:

n Databases/file systems
n Improves bandwidth/throughput
Networks n How should arrange data across disks?

n Option 1: treat disks as huge pool of disk blocks

n Disk1 has blocks 1, 2, …, N
Disk2 has blocks N+1, N+2, …, 2N
Arvind Krishnamurthy
n

Spring 2001 n Option 2: RAID (Redundant Arrays of Inexpensive Disks)

Stripe data across disks, with k disks:
n Disk1 has blocks 1, k+1, 2k+1, …

n Disk2 has blocks 2, k+2, 2k+2, …

More on RAIDs Adding parity bits to RAID

n Evaluation: n With k+1 disks
n Disk1 has blocks 1, k+1, 2k+1, …
n Load distribution
n Disk2 has blocks 2, k+2, 2k+2, …
n Small files vs. large files n …
n Reads vs. writes n Parity disk has blocks parity(1..k), parity (k+1..2k), …

n If lose any disk, can recover data from other disks plus parity
n Problem --- what if one disk fails ? n Disk1 holds 1001
n Goal --- availability --- never lose access to data n Disk2 holds 0101
Disk3 holds 1000
System should continue to work even if some components are not
n
n
Parity disk: 0100
working.
n

What if we lose disk2? Its contents are parity of remainder!

Thus can lose any disk and data would still be available.
n Solution: dedicate one disk to hold bitwise parity for other disks in
stripe. n Updating a disk block needs to update both data and parity --- need to
use write ahead logging to support crash recovery

Centralized vs. Distributed

Networking Software Goals Systems
n Simple n Distributed systems: physically separate computers working
n Scalability together
n Predict what will happen in the future n Why do we need distributed systems?
n Everything will have a network address n Cheaper & easier to build lots of simple computers
n Heterogeneity (not a goal – but have to support it) n Easier to add power incrementally
n Robustness: failure, structural changes n Promise of distributed systems:
n Something is changing n Higher availability of services
n Not a clean reboot n Better reliability of data
More security
n Performance: n

n Latency: minimum cost (or amount of work to get nothing done!) n Reality:
n Measured in time
n Worse availability
n Bandwidth: incremental cost: bytes/second n Worse reliability
n Latency more important than bandwidth n Worse security
n Most common mistake in systems is to ignore latency

1
 Definitions Broadcast Networks
n Network: physical connection to allow two computers to communicate n Shared communication medium
n Packet: unit of transfer, sequence of bits carried over network n Example: wire, all hosts listen to wire
n Network carries packet from one CPU to another
n Inside a computer: bus
n Destination gets interrupts when packet arrives
n Ethernet (10 Mbits/sec)
n Protocol: agreement between two parties as to how information is to
be transmitted n Cellular phones (100Kb – 1Mbit/sec)

PC n Issues:
n Delivery
n Arbitration
PC
n Composing broadcast networks

PC
PC

Delivery
n When you broadcast a packet, how does receiver know
who it is for?
n Put header in front of packet: Dest || Packet
n Header would contain unique machine # of target
n Everyone gets packet, discards if not the target
n Ethernet: check is done in h/w, no interrupt if not for you
n Huge security problem:
Anybody can listen in!
n
Arbitration
n How do machines arbitrate use of shared medium?
n Aloha network (’70s): packet radio between Hawaiian
islands
n Arbitration: blind broadcast, with checksum at end of
packet; if received ok, send back an acknowledgment
n Need checksum anyway in case packet gets garbled
n Sender waits for a while, if no ack, retransmits
n Problem: if load increases è more collisions è less gets
thru’ è more re-send è more load

n Break into a machine, become root, reprogram it to pick up every

packet, can see passwords go over the network!
n TCPDUMP program does that for you
n Ethernet: first practical local area network (Xerox Parc)
n Wire, still broadcast, everyone taps into single wire
n New arbitration: carrier sense (don’t send unless idle), collision
detect (sender checks if packet is trampled)
n How long before retransmission?

Internet Routing
n What happens if you need more b/w than a single n How do packets get to their destination?
ethernet? n Simple if there is a single machine that straddles all networks
n Does not scale!
n SUN has > 10000 workstations
n If packet has to go several hops before it gets to destination, and
n Buy two ethernets; how do two machines on different router straddles several networks:
networks talk to each other? n How do routers know how to forward packets?
n Put machine that straddles both networks (router, n Some definitions:
n Name
gateway, bridge, repeater)
n Address: phone number, postal
n Basically a switch
n Route: how do we get there
n Machine watches packets on both sides
n Internet solution: routing tables
n If a packet is for machine on other side, then copy n Look at packet header
n Internet: generalization of this n Do table lookup (destination LAN à output link to follow)
n Local networks can be anything: ethernet, FDDI etc. n How do you setup the table?

2
 Routing table Point-to-point networks
n Internet has no centralized state! n Different way of thinking about all this:
Why have a shared bus for ethernet at all?
No single machine knows entire topology, and topology is
n
n
n Why not simplify and have only point-to-point links, plus switches?
constantly changing
n Central idea behind ATM (asynchronous transfer mode)
n Instead: n First commercial point-to-point LAN
n Routing table has “cost” – number of hops to destination (in n Inspiration from telephone network
practice, consider how heavily used it is, bandwidth) n Advantages:
n Neighbors periodically exchange routing tables n Higher link performance
n If neighbor has cheaper route, use that one n Greater aggregate bandwidth than single link
n Initially, routers don’t know about any destination node n Can add capacity incrementally
n Get routing table from neighbors n Better fault tolerance
n Update routing table n Lower latency (no need for arbitration to send, but do need a buffer in the
switch)
n Same idea: workstations, multiprocessors, “switched ethernets”

Issues Issues (Problems to solve)

n Switches look just like computers: n Link transmission: how do you get a packet of data from
n Inputs, memory, outputs one machine to another machine “connected” to it
n What if everyone sends to the same output? n Routing: from one arbitrary place to another arbitrary place
n What happens when buffers fill up? (fastest path, how do you deal with machines going down)
Option 1:
n
n Naming: mapping names to network addresses
n Packets get dropped if they arrive and there’s no space
n If one machine sends a lot, it’ll grab the buffers
n Multiplexing (how do you share)
n Option 2: n Reliable delivery (cannot guarantee that every packet will
n Feedback between switches: “don’t send me more” signals be delivered) [ack, timeout, retransmit]
n Problem: Cross-traffic somewhere down-stream n Duplicate packets
n Option 3: n Sequencing (process packets in the same order as it was
n Per flow control; allocate a separate set of buffers and send “don’t send me sent; one approach is to have only packets outstanding)
more” signals to the source n Fragmentation & reassembly
n Problem: topology dependent

Issues (contd.)
n Flow control
n Sender generating data faster than the receiver can handle
n Feedback required from receiver to sender
n Congestion control
n Related to flow control; similar in many ways
n There is more than the sender & receiver
n Problem gets rediscovered every once in a while!
n Presentation
n Endian-ness, floating point format
n Security (authentication)