Chapter 4B: The Processor, Part B: Mary Jane Irwin

Chapter 4B: The Processor, Part B
Mary Jane Irwin ( www.cse.psu.edu/~mji )
[Adapted from Computer Organization and Design, 4th Edition, Patterson & Hennessy, Hennessy 2008, 2008 MK]
CSE431 Chapter 4B.1
Irwin, PSU, 2008
Review: Why Pipeline? For Performance!

Ti Time (clock ( l k cycles) l )
I n s t r. O r d e r
Inst 0 Inst 1 Inst 2 Inst 3 Inst 4
IM
R Reg
DM
R Reg
IM
Reg g
DM
Reg g
Once the pipeline i li i is f full, ll one instruction is completed every cycle, cycle so CPI = 1
Reg
IM
Time to fill the pipeline
ALU A
Reg
IM
ALU
Reg
IM
ALU
DM
Reg
ALU
DM
Reg
ALU
DM
Reg
CSE431 Chapter 4B.2
Irwin, PSU, 2008
Review: MIPS Pipeline Data and Control Paths

PCSrc ID/EX EX/MEM Control IF/ID Add 4 RegWrite Shift left 2 Add Branch MEM/WB
Instruction Memory
Read Address PC
Read Addr 1
Register Read
Data 1 Read Addr 2
Data Memory
ALUSrc ALU Address Write Data ALU cntrl Read Data
MemtoReg
File
Write Addr Write Data Read R d Data 2
16
Sign Extend
MemWrite MemRead
32
ALUOp
R D t RegDst
CSE431 Chapter 4B.3
Irwin, PSU, 2008
Review: Can Pipelining Get Us Into Trouble?
Yes: Pipeline Hazards

z
structural hazards: attempt to use the same resource by two different instructions at the same time data hazards: attempt to use data before it is ready
- An instructions source operand(s) are produced by a prior instruction still in the pipeline
control hazards: attempt to make a decision about program control flow before the condition has been evaluated and the new PC target address calculated
- branch and jump instructions, exceptions
Pipeline control must detect the hazard and then take action to resolve hazards
CSE431 Chapter 4B.4
Irwin, PSU, 2008
Review: Register Usage Can Cause Data Hazards
Read before write data hazard

10 10 10
ALU
Value of $1 add $1, sub $4,$1,$5 and $6,$1,$7 or $8,$1,$9
10 10/-20 -20 -20
-20
-20
IM
Reg
DM
Reg
ALU
IM
Reg
DM
Reg
ALU
IM
Reg
DM
Reg
ALU A
IM
Reg
DM
Reg
AL LU
xor $4 $4,$1,$5 $1 $5
CSE431 Chapter 4B.5
IM
Reg
DM
Reg
Irwin, PSU, 2008
One Way to Fix a Data Hazard

Can fix data hazard by waiting stall but impacts CPI
ALU U
I n s t r r. O r d e r
add $1, stall stall
IM
Reg
DM
Reg
ALU
sub $4,$1,$5 and $6,$1,$7
IM
Reg
DM
Reg
ALU
IM
Reg
DM
Reg
CSE431 Chapter 4B.6
Irwin, PSU, 2008
Another Way to Fix a Data Hazard

add $1,
IM Reg DM Reg
sub $4,$1,$5
IM
Reg
DM
Fix data hazards by forwarding results as soon as they are available to where they are Reg needed
DM Reg
and $6,$1,$7 or $8,$1,$9
IM
ALU
Reg
IM
ALU
Reg
xor $4,$1,$5
IM
ALU
Reg
ALU
DM
Reg
ALU
DM
Reg
CSE431 Chapter 4B.7
Irwin, PSU, 2008
Another Way to Fix a Data Hazard

add $1,
IM Reg DM Reg
sub $4,$1,$5
IM
Reg
DM
Fix data hazards by forwarding results as soon as they are available to where they are Reg needed
DM Reg
and $6,$1,$7 or $8,$1,$9
IM
ALU
Reg
IM
ALU
Reg
xor $4,$1,$5
IM
ALU
Reg
ALU
DM
Reg
ALU
DM
Reg
CSE431 Chapter 4B.8
Irwin, PSU, 2008
Data Forwarding (aka Bypassing)
Take T k the th result lt from f the th earliest li t point i t that th t it exists i t in i any of the pipeline state registers and forward it to the functional units (e.g., the ALU) that need it that cycle For ALU functional unit: the inputs can come from any pipeline register rather than just from ID/EX by
z z
adding multiplexors to the inputs of the ALU connecting the Rd write data in EX/MEM or MEM/WB to either (or both) of the EXs stage Rs and Rt ALU mux inputs adding the proper control hardware to control the new muxes
Other functional units may need similar forwarding logic ( (e.g., th DM) the With forwarding can achieve a CPI of 1 even in the presence of data dependencies
Irwin, PSU, 2008
CSE431 Chapter 4B.9
Data Forwarding Control Conditions

1.
EX Forward Unit:
!= 0) = ID/EX ID/EX.RegisterRs)) RegisterRs)) != 0) = ID/EX.RegisterRt)) Forwards the result from the previous instr. to either input of the ALU
if (EX/MEM.RegWrite and (EX/MEM.RegisterRd and (EX/MEM (EX/MEM.RegisterRd RegisterRd ForwardA = 10 if (EX/MEM.RegWrite and (EX/MEM.RegisterRd and (EX/MEM.RegisterRd ForwardB = 10
2 2.
MEM Forward Unit:

!= 0) = ID/EX.RegisterRs)) != 0) = ID/EX ID/EX.RegisterRt)) RegisterRt)) Forwards the result from the second previous instr. to either input of the ALU
Irwin, PSU, 2008
if (MEM/WB.RegWrite and (MEM/WB.RegisterRd and (MEM/WB.RegisterRd ForwardA = 01 if (MEM/WB.RegWrite and (MEM/WB.RegisterRd and (MEM/WB (MEM/WB.RegisterRd RegisterRd ForwardB = 01
CSE431 Chapter 4B.10
Forwarding Illustration
ALU U
add $1,
IM
Reg
DM
Reg
ALU
sub $4,$1,$5
IM
Reg
DM
Reg
ALU
and $ $6,$7,$1 ,$ ,$
IM
Reg
DM
Reg
EX forwarding
MEM forwarding
Irwin, PSU, 2008
Yet Another Complication!
Another A th potential t ti l data d t hazard h d can occur when h th there i is a conflict between the result of the WB stage instruction and the MEM stage instruction which should be forwarded?
ALU
add $1,$1,$2 add $1,$1,$3
IM
Reg
DM
Reg
ALU
IM
Reg
DM
Reg
ALU
add $1,$1,$4
IM
Reg
DM
Reg
Irwin, PSU, 2008
Yet Another Complication!
Another A th potential t ti l data d t hazard h d can occur when h th there i is a conflict between the result of the WB stage instruction and the MEM stage instruction which should be forwarded?
ALU
add $1,$1,$2 add $1,$1,$3
IM
Reg
DM
Reg
ALU
IM
Reg
DM
Reg
ALU
add $1,$1,$4
IM
Reg
DM
Reg
Irwin, PSU, 2008
Corrected Data Forwarding Control Conditions

1.
EX Forward Unit:
!= 0) = ID/EX.RegisterRs)) != 0) = ID/EX.RegisterRt)) Forwards the result from the previous instr. to either input of the ALU
if (EX/MEM.RegWrite and (EX/MEM.RegisterRd and (EX/MEM.RegisterRd ForwardA = 10 if (EX/MEM.RegWrite and (EX/MEM.RegisterRd and (EX/MEM.RegisterRd ForwardB = 10
2.
MEM Forward Unit:

Forwards the result from the previous or second previous instr. to either input of the ALU
Irwin, PSU, 2008
if (MEM/WB.RegWrite and (MEM/WB.RegisterRd != 0) and (EX/MEM.RegisterRd != ID/EX.RegisterRs) and (MEM/WB.RegisterRd = ID/EX.RegisterRs)) ForwardA = 01 if (MEM/WB.RegWrite and (MEM/WB.RegisterRd != 0) and d ( (EX/MEM.RegisterRd / i d != ! ID/EX.RegisterRt) / i ) and (MEM/WB.RegisterRd = ID/EX.RegisterRt)) ForwardB = 01 CSE431 Chapter 4B.14
Datapath with Forwarding Hardware

PCSrc ID/EX EX/MEM Control IF/ID Add 4 Shift left 2 Add Branch MEM/WB
Instruction Memory
Read Address PC
Read Addr 1
Register Read
Data 1 Read Addr 2
Data Memory
ALU Address Read Data Write Data ALU cntrl
File
Write Addr Write Data 16 Sign Extend Read R d Data 2 32
Forward Unit
Irwin, PSU, 2008
Datapath with Forwarding Hardware

PCSrc ID/EX EX/MEM Control IF/ID Add 4 Shift left 2 Add Branch MEM/WB
Instruction Memory
Read Address PC
Read Addr 1
Register Read
Data 1 Read Addr 2
Data Memory
File
EX/MEM.RegisterRd ID/EX.RegisterRt ID/EX.RegisterRs Forward Unit MEM/WB.RegisterRd
Irwin, PSU, 2008
Memory-to-Memory Copies
For loads F l d i immediately di t l f followed ll db by stores t ( (memory-tot memory copies) can avoid a stall by adding forwarding hardware from the MEM/WB register to the data memory input.
z
Would need to add a Forward Unit and a mux to the MEM stage
ALU A
lw $1,4($2)
IM
Reg
DM
Reg
ALU
sw $1,4($3)
IM
Reg
DM
Reg
Irwin, PSU, 2008
Forwarding with Load-use Data Hazards

ALU
lw
IM $1,4($2)
Reg
DM
Reg
ALU
sub $4,$1,$5 and $6,$1,$7 or $8 $8,$1,$9 $1 $9
IM
Reg
DM A ALU
Reg
IM
R Reg
DM
R Reg
AL LU
IM
Reg g
DM
Reg g
ALU U
xor $4,$1,$5
IM
Reg
DM
Reg
ALU U
IM
Reg
DM
Irwin, PSU, 2008
Forwarding with Load-use Data Hazards

ALU
lw
IM $1,4($2)
Reg
DM
Reg
ALU
stall $4,$1,$5 sub and d $4 $6 $6,$1,$7 $7 sub b $4,$1,$5 $1 $5 or $6 $8,$1,$9 and $6,$1,$7 $1 $7 or xor $8,$1,$9 $4,$1,$5 xor $4,$1,$5
IM
Reg
DM A ALU
Reg
IM
R Reg
DM
R Reg
AL LU
IM
Reg g
DM
Reg g
ALU U
IM
Reg
DM
Reg
ALU U
IM
Reg
DM
Will still need one stall cycle even with forwarding

Irwin, PSU, 2008
Load-use Hazard Detection Unit
Need N daH Hazard d detection d t ti Unit U it i in th the ID stage t th that t inserts i t a stall between the load and its use
ID Hazard detection Unit: if (ID/EX.MemRead and ((ID/EX.RegisterRt = IF/ID.RegisterRs) or ( (ID/EX.RegisterRt g = IF/ID.RegisterRt))) g ))) stall the pipeline
1 1.
The first line tests to see if the instruction now in the EX stage is a lw; the next two lines check to see if the destination register of the lw matches either source register of the instruction in the ID stage (the load load-use use instruction) After this one cycle y stall, , the forwarding g logic g can handle the remaining data hazards
Irwin, PSU, 2008
Hazard/Stall Hardware

Along with the Hazard Unit Unit, we have to implement the stall Prevent the instructions in the IF and ID stages from progressing p g g down the p pipeline p done by yp preventing g the PC register and the IF/ID pipeline register from changing
z
Hazard detection Unit controls the writing of the PC (PC.write) and IF/ID (IF/ID.write IF/ID write) registers
Insert a bubble between the lw instruction (in the EX stage) and the load-use instruction (in the ID stage) (i.e., insert a noop in the execution stream)
z
Set the control bits in the EX, MEM, and WB control fields of the ID/EX / pipeline p pe e register eg s e to o 0 (noop oop) ). The e Hazard a adU Unit co controls o s the e mux that chooses between the real control values and the 0s.
Let the lw instruction and the instructions after it in the pipeline i li (b (before f it i in th the code) d ) proceed d normally ll d down th the pipeline
Irwin, PSU, 2008
Adding the Hazard/Stall Hardware

PCSrc Hazard Unit IF/ID Control Add 4 Shift left 2 Add Branch MEM/WB ID/EX EX/MEM
0 1
Instruction Memory
Read Address PC
Read Addr 1
Register Read
Data 1 Read Addr 2
Data Memory
File
Forward Unit
Irwin, PSU, 2008
Adding the Hazard/Stall Hardware

PCSrc Hazard Unit IF/ID Control 0 Add 4 Shift left 2 Add Branch MEM/WB ID/EX ID/EX.MemRead EX/MEM
0 1
Instruction Memory
Read Address PC
Read Addr 1
Register Read
Data 1 Read Addr 2
Data Memory
File
ID/EX.RegisterRt
Forward Unit
Irwin, PSU, 2008
Control Hazards
When the flow of instruction addresses is not sequential (i.e., PC = PC + 4); incurred by change of flow instructions
z z z
Unconditional branches (j, jal, jr) C Conditional branches (beq, bne) Exceptions
Possible approaches
z z z z
Stall (impacts CPI) Move decision point as early in the pipeline as possible, thereby reducing the number of stall cycles Delay decision (requires compiler support) Predict and hope p for the best !
Control hazards occur less frequently than data hazards, but there is nothing as effective against control hazards as f forwarding is for f data hazards
Irwin, PSU, 2008
Datapath Branch and Jump Hardware

Jump PCSrc CS Shift left 2 IF/ID Add PC+4[31-28] 4 Shift left 2 Add Control Branch MEM/WB ID/EX EX/MEM
Instruction Memory
Read Address PC
Read Addr 1
Register Read
Data 1 Read Addr 2
Data Memory
File
Forward Unit
Irwin, PSU, 2008
Jumps Incur One Stall

Jumps J
z
not t decoded d d d until til ID ID, so one flush fl h is i needed d d
To flush, set IF.Flush to zero the instruction field of the IF/ID pipeline register (turning it into a noop)
IM Reg DM Reg
j flush j target
IM
Reg
DM ALU
Reg
Fix jump hazard by waiting flush
IM
Fortunately, F t t l jumps j are very infrequent i f t only l 3% of f th the SPECint instruction mix
Irwin, PSU, 2008
ALU
Reg
ALU
DM
Reg
Two Types of Stalls
Noop instruction i t ti ( (or b bubble) bbl ) inserted i t d between b t two t instructions in the pipeline (as done for load-use situations)
z
z z
Keep the instructions earlier in the pipeline (later in the code) from progressing down the pipeline for a cycle (bounce them in place with write control signals) Insert noop by zeroing control bits in the pipeline register at the appropriate stage Let the instructions later in the pipeline (earlier in the code) progress normall normally do down n the pipeline
Flushes (or instruction squashing) were an instruction in the pipeline is replaced with a noop instruction (as done for instructions located sequentially after j instructions)
z
Zero the control bits for the instruction to be flushed
Irwin, PSU, 2008
Supporting ID Stage Jumps

Jump PCSrc CS Shift left 2 IF/ID Add PC+4[31-28] 4 Shift left 2 Add Control Branch MEM/WB ID/EX EX/MEM
Instruction Memory
Read Address PC
Read Addr 1
Register Read 0
Data 1 Read Addr 2
Data Memory
File
Forward Unit
Irwin, PSU, 2008
Review: Branch Instrs Cause Control Hazards
Dependencies backward in time cause hazards

ALU A
beq lw Inst 3 Inst 4
IM
R Reg
DM
R Reg
ALU
IM
Reg g
DM
Reg g
ALU
IM
Reg
DM
Reg
ALU
IM
Reg
DM
Reg
Irwin, PSU, 2008
One Way to Fix a Branch Control Hazard

beq flush flush flush beq ta target get Inst 3
IM
Reg
DM
Reg
IM
Reg IM
DM ALU A
Reg DM AL LU
Fix branch hazard by waiting flush but affects CPI

Reg
ALU
Reg
IM
ALU
Reg g
DM
Reg g
ALU
IM
Reg
DM
Reg
ALU
IM
Reg
DM
Irwin, PSU, 2008
Another Way to Fix a Branch Control Hazard
Move branch decision hardware back to as early in the pipeline as possible i.e., during the decode cycle
ALU
beq flush beq target Inst 3
IM
Reg
DM
Reg
IM
Reg
DM
Reg
Fix branch hazard by waiting flush
ALU
ALU
IM
Reg
DM
Reg
ALU
IM
Reg
DM
Irwin, PSU, 2008
Reducing the Delay of Branches
Move the branch decision hardware back to the EX stage

z z
Reduces the number of stall (flush) cycles to two Adds an and gate and a 2x1 mux to the EX timing path
Add hardware to compute the branch target address and evaluate the branch decision to the ID stage
z
Reduces the number of stall (flush) cycles to one (like with jumps)
- But now need to add forwarding g hardware in ID stage g
Computing branch target address can be done in parallel with RegFile read (done for all instructions only used when needed) Comparing the registers cant can t be done until after RegFile read, read so comparing and updating the PC adds a mux, a comparator, and an and gate to the ID timing path
For deeper pipelines, branch decision points can be even later in the pipeline, incurring more stalls
Irwin, PSU, 2008
ID Branch Forwarding Issues
MEM/WB forwarding forwarding is taken care of by the normal RegFile write before read operation Need to forward from the EX/MEM pipeline stage to the ID comparison h d hardware f for cases lik like
if (IDcontrol.Branch and (EX/MEM.RegisterRd and (EX/MEM.RegisterRd / ForwardC = 1 if (IDcontrol.Branch (EX/MEM.RegisterRd / g and ( and (EX/MEM.RegisterRd ForwardD = 1
WB MEM EX ID IF WB MEM EX ID IF
add3 $1 $1, add2 $3, add1 $4, beq $1,$2,Loop $ ,$ , oop next_seq_instr add3 $3 $3, add2 $1, add1 $4, beq $1,$2,Loop next_seq_instr
!= 0) = IF/ID.RegisterRs)) /
Forwards the result from the second previous instr. != 0) ) to either input = IF/ID.RegisterRt)) of the compare
Irwin, PSU, 2008
ID Branch Forwarding Issues, cont
If the th instruction i t ti immediately i di t l WB add3 $3, MEM add2 $4, before the branch produces EX add1 $1, one of the branch source ID b beq $1 $2 L $1,$2,Loop operands, then a stall needs IF next_seq_instr to be inserted (between the beq and add1) since the EX stage ALU operation is occurring at the same time as the ID stage branch compare operation
z
Bounce the beq (in ID) and next_seq_instr (in IF) in place (ID Hazard Unit deasserts PC.Write and IF/ID.Write) Insert a stall between the add in the EX stage and the beq in the ID stage by zeroing the control bits going into the ID/EX pipeline register (done by the ID Hazard Unit)
If the branch is found to be taken taken, then flush the instruction currently in IF (IF.Flush)
Irwin, PSU, 2008
Supporting ID Stage Branches

PCSrc Branch Hazard Unit IF/ID Add IF.F Flush 4 Control ID/EX
1 0
Com mpare Add
EX/MEM
Shift left 2
MEM/WB
Instruction Memory
Read Address PC
Read Addr 1
RegFile 0
Read Addr 2 Read Data 1 Write Addr ReadData 2 Write Data 16 Sign Extend 32 Forward Unit Forward Unit ALU
Data Memory
Read Data Address Write Data ALU cntrl
Irwin, PSU, 2008
Delayed Branches
If the branch hardware has been moved to the ID stage stage, then we can eliminate all branch stalls with delayed branches which are defined as always executing the next sequential instruction after the branch instruction the branch takes effect after that next instruction
z
MIPS compiler moves an instruction to immediately after the branch that is not affected by the branch (a safe instruction) thereby hiding the branch delay
With deeper pipelines, the branch delay grows requiring more than one delay slot
z
Delayed branches have lost popularity compared to more expensive but more flexible (dynamic) hardware branch prediction Growth in available transistors has made hardware branch prediction relatively cheaper
Irwin, PSU, 2008
Scheduling Branch Delay Slots

A From before branch A. add $1,$2,$3 if $2=0 then delay slot B From branch target B. sub $4,$5,$6 add $1,$2,$3 if $1=0 then delay slot becomes C From fall through C. add $1,$2,$3 if $1=0 then delay slot sub $4,$5,$6 becomes add $1,$2,$3 if $1=0 then sub $4,$5,$6
becomes if $2=0 then add $1,$2,$3
add $1,$2,$3 if $1=0 then sub $4,$5,$6 $4 $5 $6
A is the best choice, fills delay slot and reduces IC I B and In d C, C th the sub instruction i t ti may need dt to b be copied, i d i increasing i IC In B and C, must be okay to execute sub when branch fails
Irwin, PSU, 2008
Static Branch Prediction
Resolve branch hazards by assuming a given outcome and proceeding without waiting to see the actual branch outcome Predict not taken always predict branches will not be taken, continue to fetch from the sequential instruction stream only when branch is taken does the pipeline stall stream,
z
1.
If taken, flush instructions after the branch (earlier in the pipeline)

in IF, ID, and EX stages if branch logic in MEM three stalls In IF and ID stages if branch logic in EX two stalls in IF stage if branch logic in ID one stall
ensure that those flushed instructions havent changed the machine hi state t t automatic t ti in i th the MIPS pipeline i li since i machine hi state changing operations are at the tail end of the pipeline (MemWrite (in MEM) or RegWrite (in WB)) restart the pipeline at the branch destination
Irwin, PSU, 2008
Flushing with Misprediction (Not Taken)

ALU
4 beq $1,$2,2
IM
Reg
DM
Reg
ALU
8 sub $4,$1,$5
IM
Reg
DM
Reg
To flush the IF stage instruction, assert IF.Flush to zero the instruction field of the IF/ID pipeline register (transforming it into a noop)
Irwin, PSU, 2008
Flushing with Misprediction (Not Taken)

ALU
4 beq $1,$2,2 8 flush sub $4,$1,$5 16 and $6,$1,$7 20 or r8 r8,$1,$9 $1 $9
IM
Reg
DM
Reg
ALU
IM
Reg
DM
Reg
A ALU
IM
Reg
DM
Reg
AL LU
IM
Reg
DM
Reg
To flush the IF stage instruction, assert IF.Flush to zero the instruction field of the IF/ID pipeline register (transforming it into a noop)
Irwin, PSU, 2008
Branching Structures
Predict not taken works well for top top of the loop loop branching structures
z
But such loops have jumps at the bottom of the loop to return to the top of the loop and incur the jump stall overhead
Loop: beq $1,$2,Out 1nd loop instr . . . last loop p instr j Loop Out: fall out instr
Predict not taken doesnt doesn t work well for bottom bottom of the loop branching structures Loop: 1st loop instr
2nd loop instr . . . last loop instr b bne $1 $1,$2,Loop $2 L fall out instr
Irwin, PSU, 2008
Static Branch Prediction, cont
Resolve branch hazards by assuming a given outcome and proceeding Predict taken predict branches will always be taken
z
2 2.
Predict taken always incurs one stall cycle (if branch destination hardware has been moved to the ID stage) Is there a way to cache the address of the branch target instruction ??
As the branch penalty increases (for deeper pipelines), a simple static prediction scheme will hurt performance. With more h hardware, d it i is possible ibl t to t try t to predict di t branch behavior dynamically during program execution Dynamic branch prediction predict branches at runrun time using run-time information
Irwin, PSU, 2008
3 3.
Dynamic Branch Prediction
A branch prediction buffer (aka branch history table (BHT)) in the IF stage addressed by the lower bits of the PC, contains bit(s) passed to the ID stage through the IF/ID pipeline register that tells whether the branch was taken the last time it was execute
z
Prediction bit may predict incorrectly (may be a wrong prediction for this branch this iteration or may be from a different branch with the same low order PC bits) but the doesnt affect correctness, just performance
- Branch decision occurs in the ID stage g after determining g that the fetched instruction is a branch and checking the prediction bit(s)
If the prediction is wrong, flush the incorrect instruction(s) in pipeline, restart the pipeline with the right instruction, and invert th prediction the di ti bit(s) bit( )
- A 4096 bit BHT varies from 1% misprediction (nasa7, tomcatv) to 18% (eqntott)
Irwin, PSU, 2008
Branch Target Buffer
The BHT predicts when a branch is taken taken, but does not tell where its taken to!
z
A branch target buffer (BTB) in the IF stage caches the branch target address address, but we also need to fetch the next sequential instruction. The prediction bit in IF/ID selects which next instruction will be loaded into IF/ID at the next clock edge
- Would need a two read port p instruction memory
BTB
Or the BTB can cache the branch taken instruction while the instruction memory is fetching the next sequential instruction
P PC
Instruction Memory 0 Read

Address
If the p prediction is correct, , stalls can be avoided no matter which direction they go
Irwin, PSU, 2008
1-bit Prediction Accuracy
A 1-bit predictor will be incorrect twice when not taken

z
Assume predict_bit = 0 to start (indicating branch not taken) and loop control is at L Loop: 1st loop l i instr t th b the bottom tt of f the th loop l code d First time through the loop, the predictor mispredicts the branch since the branch is t k back taken b k to t th the t top of f th the loop; l invert i t prediction bit (predict_bit = 1) As long as branch is taken (looping), prediction is correct Exiting the loop, the predictor again mispredicts the branch since this time the branch is not taken falling out of the loop; invert prediction bit (predict_bit = 0)
2nd loop instr . . . last loop instr bne $1,$2,Loop fall out instr
1.
2.
3.
For 10 times through the loop we have a 80% prediction accuracy for a branch that is taken 90% of the time
Irwin, PSU, 2008
2-bit Predictors
A 2-bit 2 bit scheme can give 90% accuracy since a prediction must be wrong twice before the prediction bit is changed
L Loop: 1st l loop i instr t 2nd loop instr . . . last loop instr bne $1,$2,Loop fall out instr
Taken Predict Taken Taken Not taken Predict Not Taken Taken Not taken Taken Predict Taken
Not taken Predict Not Taken Not taken
Irwin, PSU, 2008
2-bit Predictors
A 2-bit 2 bit scheme can give 90% accuracy since a prediction must be wrong twice before the prediction bit is changed
L Loop: 1st l loop i instr t 2nd loop instr . . . last loop instr bne $1,$2,Loop fall out instr
right 9 times wrong on loop Taken fall out 1 Predict 11

Taken Taken Not taken Taken Predict 1 10 Taken
t right on 1st iteration
Not taken
0 Predict 01
Not Taken
Not taken Taken
00Predict 0
Not Taken Not taken
BHT also stores the initial FSM state

Irwin, PSU, 2008
Dealing with Exceptions
Exceptions (aka interrupts) are just another form of control hazard. Exceptions arise from
z z z z z
R-type arithmetic overflow T i t Trying to execute t an undefined d fi d i instruction t ti An I/O device request An OS service request (e.g., a page fault, TLB exception) A hardware malfunction
The pipeline has to stop executing the offending instruction in midstream, let all prior instructions complete, l t flush fl h all ll f following ll i i instructions, t ti set t a register i t t to show the cause of the exception, save the address of the offending instruction, and then jump to a prearranged address dd (the (th address dd of f th the exception ti h handler dl code) d ) The software (OS) looks at the cause of the exception and deals with it
CSE431 Chapter 4B.48 Irwin, PSU, 2008
Two Types of Exceptions
Interrupts asynchronous to program execution

z z
caused by external events may be handled between instructions, so can let the instructions currently active in the pipeline complete before passing control to the OS interrupt handler simply suspend and resume user program
Traps (Exception) synchronous to program execution

z z
caused by internal events condition must be remedied by the trap handler for that instruction, so much stop the offending instruction midstream in the pipeline and pass control to the OS trap handler the offending instruction may be retried (or simulated by the OS) and the program may continue or it may be aborted
Irwin, PSU, 2008
Where in the Pipeline Exceptions Occur

ALU IM Reg DM Reg
Stage(s)?

Synchronous?
Arithmetic overflow Undefined instruction TLB or page fault I/O service request Hardware malfunction
Irwin, PSU, 2008
Where in the Pipeline Exceptions Occur

ALU IM Reg DM Reg
Stage(s)?

Arithmetic overflow Undefined instruction TLB or page fault I/O service request Hardware malfunction
EX ID IF, , MEM any any
Synchronous? yes yes yes no no
Beware Be are that multiple m ltiple e exceptions ceptions can occ occur r simultaneously in a single clock cycle
Irwin, PSU, 2008
Multiple Simultaneous Exceptions

ALU
IM
Reg
DM
Reg
ALU
IM
Reg
DM
Reg
ALU
IM
Reg
DM
Reg
ALU
IM
Reg
DM
Reg
ALU A
IM
Reg
DM
Reg
Hardware sorts the exceptions so that the earliest instruction is the one interrupted first
Irwin, PSU, 2008
Multiple Simultaneous Exceptions

ALU
IM
Reg
DM
Reg
D$ page fault
ALU IM Reg DM Reg
arithmetic ith ti overflow fl

ALU IM Reg DM Reg
undefined instruction
DM Reg
ALU
IM
Reg
ALU A
IM
Reg
DM
Reg
I$ page fault
Hardware sorts the exceptions so that the earliest instruction is the one interrupted first
Irwin, PSU, 2008
Additions to MIPS to Handle Exceptions (Fig 6.42)
Cause register (records exceptions) hardware to record in Cause the exceptions and a signal to control writes to it (CauseWrite) EPC register (records the addresses of the offending instructions) hardware to record in EPC the address of the offending g instruction and a signal g to control writes to it (EPCWrite)
z
Exception software must match exception to instruction
A way to load the PC with the address of the exception handler

z
Expand the PC input mux where the new input is hardwired to the exception handler address - (e.g., 8000 0180hex for arithmetic overflow)
A way to flush offending instruction and the ones that follow it

Irwin, PSU, 2008
Datapath with Controls for Exceptions

PCSrc Branch EX.Flush Hazard ID.Flush Unit
8000 0180hex
ID/EX
1 0
EX/MEM
IF/ID Add IF.F Flush 4
Control
Cause
Com mpare Shift left 2 Add
MEM/WB
EPC Data Memory

ALU Read Data Address Write Data ALU cntrl
Instruction Memory
Read Address PC
Read Addr 1
RegFile 0
Read Addr 2 Read Data 1 Write Addr ReadData 2 Write Data 16 Sign Extend 32 Forward Unit Forward Unit
Irwin, PSU, 2008
Summary
All modern day processors use pipelining for f performance (a CPI of 1 and fast a CC) Pipeline p clock rate limited by y slowest p pipeline p stage g so designing a balanced pipeline is important Must detect and resolve hazards
z z
Structural hazards resolved by designing the pipeline correctly Data hazards

- Stall (impacts CPI) - Forward (requires hardware support)
Control hazards put the branch decision hardware in as early a stage in the pipeline as possible
- St Stall ll (i (impacts t CPI) - Delay decision (requires compiler support) - Static and dynamic prediction (requires hardware support)
Pipelining complicates exception handling

Irwin, PSU, 2008

Chapter 4B: The Processor, Part B: Mary Jane Irwin

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Chapter 4B: The Processor, Part B: Mary Jane Irwin

Uploaded by

Copyright:

Available Formats

Chapter 4B: The Processor, Part B

Mary Jane Irwin ( www.cse.psu.edu/~mji )

CSE431 Chapter 4B.1

Irwin, PSU, 2008

Review: Why Pipeline? For Performance!

Inst 0 Inst 1 Inst 2 Inst 3 Inst 4

Time to fill the pipeline

CSE431 Chapter 4B.2

Irwin, PSU, 2008

Review: MIPS Pipeline Data and Control Paths

CSE431 Chapter 4B.3

Irwin, PSU, 2008

Review: Can Pipelining Get Us Into Trouble?

Yes: Pipeline Hazards

CSE431 Chapter 4B.4

Irwin, PSU, 2008

Review: Register Usage Can Cause Data Hazards

Read before write data hazard

Value of $1 add $1, sub $4,$1,$5 and $6,$1,$7 or $8,$1,$9

10 10/-20 -20 -20

Irwin, PSU, 2008

One Way to Fix a Data Hazard

add $1, stall stall

sub $4,$1,$5 and $6,$1,$7

CSE431 Chapter 4B.6

Irwin, PSU, 2008

Another Way to Fix a Data Hazard

and $6,$1,$7 or $8,$1,$9

CSE431 Chapter 4B.7

Irwin, PSU, 2008

Another Way to Fix a Data Hazard

and $6,$1,$7 or $8,$1,$9

CSE431 Chapter 4B.8

Irwin, PSU, 2008

Data Forwarding (aka Bypassing)

CSE431 Chapter 4B.9

Data Forwarding Control Conditions

MEM Forward Unit:

CSE431 Chapter 4B.11

Irwin, PSU, 2008

Yet Another Complication!

add $1,$1,$2 add $1,$1,$3

CSE431 Chapter 4B.12

Irwin, PSU, 2008

Yet Another Complication!

add $1,$1,$2 add $1,$1,$3

CSE431 Chapter 4B.13

Irwin, PSU, 2008

Corrected Data Forwarding Control Conditions

MEM Forward Unit:

Datapath with Forwarding Hardware

CSE431 Chapter 4B.15

Irwin, PSU, 2008

Datapath with Forwarding Hardware

EX/MEM.RegisterRd ID/EX.RegisterRt ID/EX.RegisterRs Forward Unit MEM/WB.RegisterRd

CSE431 Chapter 4B.16

Irwin, PSU, 2008

CSE431 Chapter 4B.17

Irwin, PSU, 2008

Forwarding with Load-use Data Hazards

sub $4,$1,$5 and $6,$1,$7 or $8 $8,$1,$9 $1 $9

CSE431 Chapter 4B.18

Irwin, PSU, 2008