Pipelining


What is it?

How does it work?

What are the benefits?

What could go wrong?
What is pipelining?

Pipelining is a design feature that allows

individual common processor tasks to run
simultaneously, such as:

Fetch
Decode
Execute
What is pipelining?

Or, more specifically in modern computers,

Instruction read
Decode
Operand read
Execute
Operand write
What is pipelining?

No more “one instruction at a time”
processing

Processor works simultaneously on
multiple instructions

The cycle time of the processor is reduced,
thus increasing instruction issue-rate in
most cases.
How does pipelining work?

First instruction is fetched from memory
How does pipelining work?

First instruction is fetched from memory

First instruction is decoded;
second instruction is fetched
How does pipelining work?

First instruction is fetched from memory

First instruction is decoded;
second instruction is fetched

First instruction’s operands are fetched;
second instruction is decoded;
third instruction is fetched
How does pipelining work?

• First instruction is fetched from memory

• First instruction is decoded;
second instruction is fetched
• First instruction’s operands are fetched;
second instruction is decoded;
third instruction is fetched
• And on, and on, and on…
How does pipelining work?

Because each instruction demands one
stage of the processor, the maximum
number of simultaneous instructions is the
number of stages in the processor
How does pipelining work?

A short animation
of simple processor routines
How does pipelining work?

A short(er) animation
of pipelined processor routines
What are the benefits
of pipelining?

Uses most (if not all) of a processor’s
ability at all times
Efficiency goes up

Relatively minor hardware changes give
more completed cycles per second
Speed goes up
What can go wrong in
pipelines?

Data hazards

Control hazards

Structural hazards
What can go wrong…?
Data hazards

Two instructions at different stages
One instruction needs the results of
the other instruction’s operation

…
add %r1, 5, %r2
add %r2, 7, %r3
…
What can go wrong…?
Data hazards

Well-designed programs can anticipate this, and
spread the connected commands out with other
operations
…
add %r1, 5, %r2 (i1 fetch)
sub %r4, 7, %r8 (i1 decode)
mov %r5, %r6 (i1 read)
add %r2, 7, %r3 (i1 execute, i2 fetch)
What can go wrong…?
Data hazards

Hardware can detect some data hazards and
“forward” the information to the instructions that
need it, even as the data goes to the registers

Otherwise, programs may simply have bad data -
testing and good programming are the best cures!
What can go wrong…?
Control hazards

Instructions are read sequentially from
memory - even though loops may return to
previous statements

Instructions following the loop are already
in the pipeline!

Or, simple if/else blocks of code. Which
block do you get instructions from?
What can go wrong…?
Control hazards
• At worst, the entire pipeline must be
flushed: hardware can stall the influx of
new instructions until the conditional is
evaluated
What can go wrong…?
Control hazards
• Branch prediction inserts a couple
instructions of its own to guess the outcome
of the conditional
• If prediction is usually correct,
fewer cycles lost on average
• If prediction is wrong,
flush the pipeline and undo any improper
commands
What can go wrong…?
Structure hazards
• Two different instructions try to use the
same hardware resource
• One instruction will obviously get access
first, second instruction must be stalled
(maybe along with all other instructions
behind it)
• Again, careful programming will prevent
this