You are on page 1of 47


Server Fundamentals
The performance of a server depends on the speed of data flow
between its processor and the peripheral device that are connected.
These peripheral device use a set of pathways to communicate with
the processor. This set of pathways are called as BUS. In server we
mainly use PCI bus technology.
Types of Buses
System Bus
Data Bus
System Bus
Use to connect CPU to its MEMORY.
Also known as Local Bus
Faster than other Buses.

Data Bus
Connects the other computer peripherals with the memory.
PCI is an example of data bus.
Usage of PCI
In the early 1990, INTEL introduce PCI with the aim to improve I/O
operation of computers.
PCI is also used with other processors, eg. AMD
PCI Design
Initially PCI was a 32 bit bus that operated a speed of 33 MHz
Now 64 bit PCI with a speed of 33 MHz whose DTR=264 Mbps
64 bit PCI with a speed of 66 MHz, with DTR=512 mbps

Bus Speed Setup
Synchronized--- bus speed is not depends with the memory bus
Mainly used in today's bus architecture

Asynchronized--- bus speed depends with the memory bus speed.
Normally controlled by jumpers or CMOS setup.

PCI Implementation
Data buses connect to system buses by single chip bridge device.
PCI can be implemented on servers through peer PCI bus or
hierarchical PCI bus technology.
Peer PCI Bus
In Peer PCI, several PCI buses are connected directly to a host bus. These
host buses provide connectivity between server and storage devices.
This enables the CPU to access each bus directly.
Peer PCI improves the performance of a server because the processor or
the bus bandwidth is evenly shared between the buses.
Also helps to separate high bandwidth and low bandwidth traffic on different
PCI buses.
Hierarchical PCI
When multiple expansion slots and connectors are needed
hierarchical PCI can be implemented.
A bridge controller is used to connect independent PCI buses so
that they can communicate with each other.
Hot plug
The hot plug facility of PCI enables to install and remove PCI device
without turning off the server.
Today most servers are equipped with the hot plug facility.
Microsoft Windows Server 2003 have native support for hot plug
Hot Plug Devices
NIC Card
Hot Swap
Cards inserted or remove without failure when servers are running.
To support hot swap PCI hot swap adapters are required. These
device provide circuit breaker functions that protect electric supplies
against fault conditions resulting from over current. A PCI hot swap
adapter provides protection against 12V, 5V, 3.3V and -12V
PCI- Hot Swap Levels
Basic hot swap
Full hot swap
High availability
Basic Hot Swap VS Full Hot Swap
Requires human intervention for interaction with the host operating
system when a card is being inserted or removed.
It automates the procedure of replacing the card.
High Availability
The high availability level focuses on having cards that can be
logically and electrically added to or removed from a server.
This ensure hot spare or fail over capability for the server. The high
availability hot swap level also enables an automatic reconfiguration
of software and hardware components in a running server.
PCI is also intelligent input/output (I2O) compliant.
I2O is a standard designed by a consortium of computer companies called as
I2O Special Interest Group (SIG). This standard simplifies and speed up the
I/O operations on servers.
According to I2O the drivers used for the operating system of a server must be same
as those used for the SCSI and network cards on the server.
It is recommended to use INTEL 960 chip on the motherboard of a server, to increase
the speed of its I/O subsystem.
PCI- Bus Mastering
Bus mastering is the capability of devices other than the system chipset, on
the PCI bus to take control of the bus and perform transfer directly.
Bus mastering allows the transfer of data between devices on a PCI bus
and the main system memory even when the other devices are using CPU.
However these devices should have their own built-in processors to operate
independently of the CPU.
Bus Mastering facilitates multitasking by allowing data transfer
between the devices on a PCI bus and other devices in the system
without involving the CPU.
The devices that take control of the bus keep track of the
transactions. There fore the CPU utilization is low. This enables the
CPU to handle other task in the meantime.
Bus Arbitration
Bus Mastering is achieved through the bus arbitration process. This process
determines priority levels for the request to a PCI bus so that no devices
lock each other out.
For example, when more than one processor is connected to the same
system bus, there are simultaneous requests for the bus. This request are
resolved using bus arbitration that allows devices to use the bus based on
their priority level.

The Peripheral Component Interconnect Express architecture is a
serial interconnect technology designed to meet the bandwidth
requirement of advanced processors, memory subsystem and
It impose some advance features than a normal PCI.
PCI-e is a point-to-point serial switch bus technology. It aims at reducing
implementation costs and providing higher bandwidth per pin and a scalable
The advantage of PCI-e over PCI is its point to point bus topology. It
enables each PCI-e device to communicate the PCI controller directly and
autonomously. This communication occurs through a PCI link.
PCI-e link
A PCI-e link comprise of one or more lanes. Depending on the
number of lanes used in a PCI-e link the link is called as x1 link, x2
link and so on. A link which made up for 4 lanes is known as x4 link.
PCI-e supports x1,x2,x4,x8,x12,x16 and x34 lane widths.
PCI-e Bandwidth
Each PCI-e lane can transfer one byte at a time in both the
directions by using full duplex communication. An individual lane
can transfer data at the rate of 2 Gbps in each direction
The next table shows its detail.
PCI-e bandwidth
PCI-e implementation Encoded data transfer
Unencoded data transfer
x1 5 gbps 4 gbps
x2 10 gbps 8 gbps
x4 20 gbps 16 gbps
x8 40 gbps 32 gbps
x16 80 gbps 64 gbps
x32 160 gbps

128 gbps
8b/10b Clock-encoding mechanism
A PCI-e mechanism comprises dual simplex channels, where each
channel is composed of a transmit pair and a receive pair for
concurrent transmission in both directions. Every pair is composed
of dual pairs of signal that are driven by Low Voltage Differential
(LVD). To attain a high speed data transfer rate, a data clock is built
into each pair by using an 8b/10b data encoding scheme.
Point to Point Bus Topology of PCI-e
As PCI-e supports the point to point bus topology the total
bandwidth of each PCI bus is dedicated to the device at the end of
the link. This in turn enables multiple PCI-e devices to be active at
the same time without affecting each other.
Advanced Features of PCI-e
Advanced power management
Support for real-time data traffic
Hot plug and hot swap
Data integrity and error handling
Advanced Power Management
PCI-e maintains an active interface constantly to synchronize the
transmitter and receiver. The interface is kept active continually
transmitting idle characters when data is not sent. As additional
power is consumed by the receiver to decode and discard these idle
characters, PCI-e uses two low-power link states and the active-
state power management (ASPM) protocol.
Support for Real-Time Data Traffic
PCI-e provides native support for isochronous data transfers and different
Quality of Service (QoS) levels through virtual channels. These channels
ensure timely and reliable delivery of data packets. PCI-e supports
numerous isochronous virtual channels per lane, which may have varying
QoS levels. This feature is developed for applications that require real-time
delivery such as real-time audio and video.
Hot Plug and Hot Swap
The hot plug and hot swap features of PCI-e are extremely
beneficial to servers and portable computers. The facility to hot plug
I/O devices reduces server downtime when peripheral cards need to
be installed or replaced.
Data Integrity and Error Handling
PCI-e provides support for link-level data integrity for all kinds of
transactions and data-link packets. As a result, it provides end-to-end data
integrity required by high availability application running on server systems.
Additionally, fault isolation and recovery solutions can be enhanced by
using the PCI error handling and advanced error reporting and handling
capabilities of a PCI-e.
PCI-X was developed jointly by IBM, HP and Compaq and was
approved by PCI Special Interest Group(PCI-SIG). This group
provides technical support, training and compliance testing. Now
PCI-X has become an open standard that is used by all computer
PCI-X Data Speed
The PCI-X design, uses one 64 bit slot that runs at 133 MHz while
the other slots run at 66 MHz. This enables data to be exchanged at
the speed of 1.6 GBps.
So using PCI-X technology existing PCI bus can be increased from
133 MBps to 1 GBps.
PCI-X backward compatibility
PCI-X is backward compatible. It means, a PCI-X card can be
installed in a standard 32 PCI slot. However, this reduces the speed
of the bus to 33 MHz
PCI-X Fault Tolerance
A PCI-X system is more fault tolerant than a Normal PCI system.
When a faulty card is installed on a computer, the PCI-X bus can re-
initialize the faulty card or take it offline before the computer fails.
PCI-X power requirements
Normal PCI----- 5 VOLT
PCI-X ----- 3.3 VOLT
Comparison of PCI and PCI-X Systems
Bus Width Bus Frequency PCI/PCI-X
No. of slots(PCI) No. of slots(PCI-
32 bit 66 MHz 264 Mbps 4 N/A
64 bit 66 MHz 528 MBps 2 4
64 bit 100 MHz 800 MBps N/A 2
64 bit 133 MHz 1064 Mbps N/A 1
Adapter Teaming
Adapter Teaming is a process of combining two or more physical
adapters into a single logical adapter. This enables the network
administrator to assign a single IP address to the logical adapter.
A software driver is usually used to link the physical adapter to the
logical adapter.
Adapter Teaming- number of members
An adapter team can have a minimum of two and a maximum of
eight adapters. The main advantage of a adapter teaming is that it
increases server uptime. This is because if a physical adapter in a
team fails, it does not affect other adapters in the team as the IP
address is assigned to the logical adapter and not to a physical
Adapter teaming also enables Adapter Fault Tolerance (AFT)
against broken or loose or loose cables, hubs or switch ports.
It can also provide fault tolerance to problems caused by a
Peripheral Component Interconnect (PCI) slot malfunction or
breakdown in adapter hardware.
Adapter Teaming load distribution
Adapter teaming ensures that the failure of an adapter in a team
does not affect the load distribution through failover capabilities. If
any member in an adapter team fails because of problems in a
network interface card (NIC), cable, switch port, or switch, the load
distribution is evaluated and then reassigned to the other members
of the team.
Primary Adapter
For an effective failover, a primary adapters must be specified in an
adapter team. The primary adapter is specified based on the
priorities of the adapters.
Usually, the primary adapter is the adapter that has highest prority
Failover Restoration
In adapter teaming, only a single active adapter transmits and
receives traffic.
The single adapter is usually the primary adapter. However, if the
primary adapter fails, a secondary adapter takes control.
After the connection of the primary adapter is restored, the control is
automatically passed back to the primary adapter.
Adapter teaming also enables Adapter Load Balancing (ALB). ALB provides
fault tolerance by ensuring load balancing data traffic.
When load balancing takes place, all adapters in an adapter team share the
transmission load.
To enable load balancing, all the adapters in the team should be connected
to a switch and an IP Address must be assigned to the adapter team.

Thank you