8 - RISCV - Pipelined - Arch2

Structural Hazards Data Hazards Resolving Data Hazards Performance Evaluation
RISC-V Pipelined Architecture II
Muhammad Tahir
Lecture 10-11
Electrical Engineering Department

University of Engineering and Technology Lahore
Contents
1 Structural Hazards
2 Data Hazards
3 Resolving Data Hazards
Pipeline Forwarding
4 Performance Evaluation
2/49
Pipeline Limitations
• Major drawback of pipelining: Hazards

• Hazards are a consequence of instruction dependence
3/49
Pipeline Limitations
• Major drawback of pipelining: Hazards

• Hazards are a consequence of instruction dependence
• Broadly there are three types of hazards
• Structural hazards
• Data hazards
• Control hazards
3/49
Structural Hazards
• A Structural Hazard arises when two instructions (in the
pipeline) vie for the same (hardware) resource
4/49
Structural Hazards
• Consider the pipelined execution of the following program
(assuming single memory for instructions and data)
4/49
Structural Hazards
• Consider the pipelined execution of the following program
(assuming single memory for instructions and data)
Example program.
lw x1, 8(x2)
sw x3, 24(x4)
or x5, x6, x7
add x8, x6, x7
lw x1, 4(x2)
4/49
Instruction Execution
Fetch Decode ALU Memory Writeback
lw x1, 8(x2) sw x3, 24(x2) or x5, x6, x7 add x8, x6, x7 lw x1, 0(x2)
clock
5/49
Instruction Execution: Cont’d
clock
6/49
clock
7/49
clock
8/49
clock
9/49
Structural Hazard in Our RV32I Design?

Fetch Decode Execute Memory Writeback
+ 4 PC PC PC PC + 4
Register File
IR [19:15]
raddr1
IR [24:20] ALU
raddr2 rs1
Inst. Mem rdata1 A Data Mem

IR [11:7]
waddr ALU
ALU addr RD
rs2
PC wdata rdata2 rdata
Addr B
IR wdata
Inst.
Imm WD
Immediate
Gen.
Branch
Cond. br_taken
IR [31:0]
IR IR IR
clock
10/49
No Structural Hazard in Our RV32I Design

+ 4 PC PC PC PC + 4
Register File
IR [19:15]
raddr1
IR [24:20] ALU
raddr2 rs1

IR [11:7]
waddr ALU
ALU addr RD
rs2
Addr B
IR wdata
Inst.
Imm WD
Immediate
Gen.
Branch
Cond. br_taken
IR [31:0]
IR IR IR
clock
11/49
Data Hazards
• Data Hazard is a consequence of operand(s) dependence

• Applicable to those instructions which are simultaneously
present in the pipeline
• Data hazards can be resolved by waiting
12/49
Data Hazards Cont’d
• Data hazards can be of three different types

• Read After Write (RAW) hazard, true data dependence
x3 ← x1 op x2
x5 ← x3 op x4
• Write After Read (WAR) hazard, anti-dependence (a type of
name dependence)
x1 ← x3 op x2
x3 ← x4 op x5
• Write After Write (WAW) hazard, output-dependence (a type
of name dependence)
x3 ← x1 op x2
x3 ← x6 op x7
13/49
RAW Data Hazards

• True Data Dependence (RAW Hazard): An instruction j is
said to be data dependent on instruction i if
• Instruction j uses a result that is produced by instruction i, or
• Instruction j is data dependent on instruction k, and
instruction k is data dependent on instruction i
RAW Hazard
add x3, x2, x1

addi x4 x3, 5
RAW Data Hazards

• True Data Dependence (RAW Hazard): An instruction j is
said to be data dependent on instruction i if
• Instruction j uses a result that is produced by instruction i, or
• Instruction j is data dependent on instruction k, and
instruction k is data dependent on instruction i
RAW Hazard
add x3, x2, x1

add x3, x2, x1
or x5, x6, x7
addi x4 x3, 5
addi x4 x3, 5
14/49
Resolving RAW Data Hazards
• Solution 1: A simple approach is to freeze the earlier pipeline

stages while waiting for the result to be available (Interlock or
Stall).
• Solution 2: Route data, as soon as it is available, to the
earlier pipeline stage (Bypassing or Forwarding).
15/49
RAW Data Hazard Illustration
• Example program for data hazards
Example program.
add x3, x2, x1

addi x4, x3, 5
sub x7, x6, x8
and x9, x6, x8
lw x10, 4(x12)
add x11, x8, x2
16/49
Instructions Execution
Fetch
Decode
ALU

Memory
Writeback

Fetch
Decode
Bubble

Bubble
ALU

Memory
Bubble

Fetch
Decode
Bubble
Bubble
Fetch
Decode
Bubble
Fetch Decode
ALU

Memory
Writeback

Fetch
Decode
ALU

Memory

Fetch
Decode
ALU
ADD x3, x2, x1 ADDI x4, x3, 5 ADDI x4, x3, 5 ADDI x4,x6,
add x8, x3,x7
5 ADDI x4,0(x2)
lw x1, x3, 5 SUB x7, x6, x8 AND x9, x6, x8
clock
Stall Stall Stall
17/49
Instructions Execution Cont’d
Fetch
Decode
ALU

Memory
Writeback

Fetch
Decode
Bubble

Bubble
ALU

Memory
Bubble

Fetch
Decode
Bubble
Bubble
Fetch
Decode
Bubble
Fetch Decode
ALU

Memory
Writeback

Fetch
Decode
ALU

Memory

Fetch
Decode
ALU
add x8, x3,x7
5 ADDI x4,0(x2)
clock
Stall Stall Stall
18/49
Fetch
Decode
ALU

Memory
Writeback

Fetch
Decode
Bubble

Bubble
ALU

Memory
Bubble

Fetch
Decode
Bubble
Bubble
Fetch
Decode
Bubble
Fetch Decode
ALU

Memory
Writeback

Fetch
Decode
ALU

Memory

Hazard
Fetch
Decode
ALU
add x8, x3,x7
5 ADDI x4,0(x2)
clock
Stall Stall Stall
19/49
Fetch
Decode
ALU

Memory
Writeback

Fetch
Decode
Bubble

Bubble
ALU

Memory
Bubble

Fetch
Decode
Bubble
Bubble
Fetch
Decode
Bubble
Fetch Decode
ALU

Memory
Writeback

Fetch
Decode
ALU

Memory
Hazard
detection unit
Fetch
Decode
ALU
add x8, x3,x7
5 ADDI x4,0(x2)
clock
Stall Stall Stall
20/49
Instructions Execution: Pipeline Stalling
Fetch
Decode
ALU

Memory
Writeback

Fetch
Decode
Bubble

Bubble
ALU

Memory
Bubble

Fetch
Decode
Bubble
Bubble
Fetch
Decode
Bubble
Fetch Decode
ALU

Memory
Writeback

Fetch
Decode
ALU

Memory
Hazard
detection unit
Fetch
Decode
ALU
add x8, x3,x7
5 ADDI x4,0(x2)
clock
Stall Stall Stall
21/49
Fetch
Decode
ALU

Memory
Writeback

Fetch
Decode
Bubble

Bubble
ALU

Memory
Bubble

Fetch
Decode
Bubble
Bubble
Fetch
Decode
Bubble
Fetch Decode
ALU

Memory
Writeback

Fetch
Decode
ALU

Memory
Hazard
detection unit
Fetch
Decode
ALU
add x8, x3,x7
5 ADDI x4,0(x2)
clock
Stall Stall Stall
22/49
Fetch
Decode
ALU

Memory
Writeback

Fetch
Decode
Bubble

Bubble
ALU

Memory
Bubble

Fetch
Decode
Bubble
Bubble
Fetch
Decode
Bubble
Fetch Decode

ALU

Memory
Writeback

Fetch
Decode
ALU

Memory

Fetch
Decode
ALU
add x8, x3,x7
5 ADDI
lw x1,
x4,0(x2)
x3, 5 SUB x7, x6, x8 AND x9, x6, x8
clock
Stall Stall Stall
23/49
Fetch
Decode
ALU

Memory
Writeback

Fetch
Decode
Bubble

Bubble
ALU

Memory
Bubble

Fetch
Decode
Bubble
Bubble
Fetch
Decode
Bubble
Fetch Decode

ALU

Memory
Writeback

Fetch
Decode
ALU

Memory

Fetch
Decode
ALU
add x8, x3,x7
5 ADDI
lw x1,
x4,0(x2)
x3, 5 SUB x7, x6, x8 AND x9, x6, x8
clock
Stall Stall Stall
24/49
Fetch
Decode
ALU

Memory
Writeback

Fetch
Decode
Bubble

Bubble
ALU

Memory
Bubble

Fetch
Decode
Bubble
Bubble
Fetch
Decode
Bubble
Fetch Decode

ALU

Memory
Writeback

Fetch
Decode
ALU

Memory

Fetch
Decode
ALU
add x8, x3,x7
5 ADDI
lw x1,
x4,0(x2)
x3, 5 SUB x7, x6, x8 AND x9, x6, x8
clock
Stall Stall Stall
25/49
Fetch
Decode
ALU

Memory
Writeback

Fetch
Decode
Bubble

Bubble
ALU

Memory
Bubble

Fetch
Decode
Bubble
Bubble
Fetch
Decode
Bubble
Fetch Decode

ALU

Memory
Writeback

Fetch
Decode
ALU

Memory

Fetch
Decode
ALU
add x8, x3,x7
5 ADDI
lw x1,
x4,0(x2)
x3, 5 SUB x7, x6, x8 AND x9, x6, x8
clock
Stall Stall Stall
26/49
t1 t2 t3 t4 t5 t6 t7 t8 t9
I1 IF1 ID1 EX1 MA1 WB1
I2 IF2 ID2 ID2 ID2 ID2 EX2 MA2 WB2
I3 IF3 IF3 IF3 IF3 ID3 EX3 MA3
27/49
Stalling Drawback
• Using Stalling to resolve RAW data hazard

• The strategy of inserting NOP’s is not a good idea
• Introduces delays in executing a particular set of instructions
28/49
Stalling Drawback
• Using Stalling to resolve RAW data hazard

• The strategy of inserting NOP’s is not a good idea
• Introduces delays in executing a particular set of instructions
• Pipelining was introduced to improve performance
Time Instructions Cycles Time

= x x
Program Program Instruction Cycle
• But with stalling, CPI (cycles per instruction) will go up

• Need an alternative method to overcome stalling limitation
28/49
Resolving RAW Data Hazard: Forwarding
• Forwarding to resolve RAW data hazard

• An alternative method with additional hardware for data
forwarding
• Route data to the input of pipeline execution stage (for
subsequent instruction(s)) as soon as it becomes available
• Requires additional signal path and multiplexer
• Does not work for all scenarios
29/49
Fetch
Decode
ALU

Memory
Writeback

Fetch Decode

ALU

Memory
Writeback

Fetch
Decode
ALU

Memory
Writeback

Fetch
Decode
ALU

Memory
Writeback

Hazard
detection unit
ADD x3, x2, x1 ADDI x4, x3, 5 SUB x7, x6, x8 AND x9, x6, x8
clock
30/49
Fetch
Decode
ALU

Memory
Writeback

Fetch Decode

ALU

Memory
Writeback

Fetch
Decode
ALU

Memory
Writeback

Fetch
Decode
ALU

Memory
Writeback

Forwarding
Hazard
detection unit
clock
31/49
Fetch
Decode
ALU

Memory
Writeback

Fetch Decode

ALU

Memory
Writeback

Fetch
Decode
ALU

Memory
Writeback

Fetch
Decode
ALU

Memory
Writeback

clock
32/49
Load-Use RAW Data Hazard
• Forwarding performance for Load-Use hazard
Load-Use Hazard with Forwarding
lw x3, 4(x12) lw x3, 4(x12)

addi x4, x3, 5 sub x7, x6, x8
sub x7, x6, x8 addi x4, x3, 5
Stall cycles = 1 Stall cycles = 0
33/49
Resolving Load-Use Hazard: Forwarding

Fetch
Decode
ALU

Memory
Writeback

Fetch Decode
Bubble

ALU
Bubble

Memory
Bubble
Writeback
Bubble

Fetch
Decode
ALU

Memory
Writeback

LW x3, 4(x12) ADDI x4, x3, 5 ADDI x4, x3, 5
34/49
Resolving Load-Use Hazard: Forwarding

Fetch
Decode
ALU

Memory
Writeback

Fetch Decode
Bubble

ALU
Bubble

Memory
Bubble
Writeback
Bubble

Fetch
Decode
ALU

Memory
Writeback

LW x3, 4(x12) ADDI x4, x3, 5 ADDI x4, x3, 5
Fetch
Decode
ALU

Memory
Writeback

Fetch Decode

ALU

Memory
Writeback

Fetch
Decode
ALU

Memory
Writeback

LW x3, 4(x12) SUB x7, x6, x8 ADDI x4, x3, 5
34/49
RAW Data Hazard: Forwarding Performance
• Pipeline with single execution stage
RAW Hazards with Forwarding
add x3, x2, x1 lw x3, 4(x12)

addi x4 x3, 5 addi x4, x3, 5
Stall cycles = 0 Stall cycles = 1
• Pipeline with n execution stages

• No. of stall cycles ≥ n − 1
35/49
Datapath without Forwarding

+ 4 PC PC PC PC + 4
Register File
IR [19:15]
raddr1
IR [24:20] ALU
raddr2 rs1

IR [11:7]
waddr ALU
ALU addr RD
rs2
Addr B
IR wdata
Inst.
Imm WD
Immediate
Gen.
Branch
Cond. br_taken
IR [31:0]
IR IR IR
clock
36/49
Datapath with Forwarding

+ 4 PC PC PC PC + 4
Register File
IR [19:15]
raddr1
IR [24:20] ALU
raddr2 rs1

IR [11:7]
waddr ALU
ALU addr RD
rs2
Addr B
IR wdata
Inst.
Imm WD
Immediate
Gen.
Branch
Cond. br_taken
IR [31:0]
IR IR IR
clock
37/49
Datapath with Forwarding Cont’d

+ 4 PC PC PC PC + 4
Register File
IR [19:15]
raddr1
IR [24:20] ALU
raddr2 rs1

IR [11:7]
waddr ALU
ALU addr RD
rs2
Addr B
IR wdata
Inst.
Imm WD
Immediate
Gen.
Branch
Cond. br_taken
IR [31:0]
IR IR IR
clock
38/49
Updated Datapath with Forwarding

+ 4 PC PC PC PC + 4
Register File
IR [19:15]
raddr1
IR [24:20] ALU
raddr2 rs1

IR [11:7]
waddr ALU
ALU addr RD
rs2
Addr B
IR wdata
Inst.
Imm WD
Immediate
Gen.
Branch
Cond. br_taken
IR [31:0]
IR IR IR
clock
39/49
Resolving RAW DATA Hazards: Hardware Solution
• Interlocking or Stalling: Freeze earlier pipeline stages till

the data becomes available
• Bypassing or Forwarding: Route the data to the earlier
pipeline stage(s) as soon as it has become available
• Out of Order Execution: Dynamic instruction scheduling
to avoid data hazards (to be discussed later)
40/49
Resolving RAW DATA Hazards: Software Solution

Example program
x = a + b;
y = b + c;
Code Scheduling to resolve RAW Hazard
lw x3, 0(x10)
lw x4, 4(x10)
add x5 x3, x4
sw x5, 12(x10)
lw x6, 8(x10)
add x7 x3, x6
sw x7, 16(x10)
Resolving RAW DATA Hazards: Software Solution

Example program
x = a + b;
y = b + c;
Code Scheduling to resolve RAW Hazard
lw x3, 0(x10) lw x3, 0(x10)

lw x4, 4(x10) lw x4, 4(x10)
add x5 x3, x4 lw x6, 8(x10)
sw x5, 12(x10) add x5 x3, x4
lw x6, 8(x10) sw x5, 12(x10)
add x7 x3, x6 add x7 x3, x6
sw x7, 16(x10) sw x7, 16(x10)
41/49
RAW Hazard Detection Unit Cont’d
• Assume Current instruction is in the D/E pipeline register

• Then Previous instruction is in the E/M pipeline register
and Second-previous instruction is in the M/W pipeline
register
42/49
RAW Hazard Detection Unit Cont’d
• Assume Current instruction is in the D/E pipeline register

• Then Previous instruction is in the E/M pipeline register
and Second-previous instruction is in the M/W pipeline
register
• Detecting RAW hazards between Current and Previous
instructions
(E/M.IR.Rd == D/E.IR.Rs1) OR (E/M.IR.Rd == D/E.IR.Rs2)
• Detecting RAW hazards between Current and
Second-previous instructions
(M/W.IR.Rd == D/E.IR.Rs1) OR (M/W.IR.Rd == D/E.IR.Rs2)
42/49
Forwarding for RAW Hazard
• Forwarding for RAW hazards between Current and Previous

instructions
if ( (E/M.CR.WB REG == WB ALU) && (E/M.IR.Rd 6= 0) &&
(E/M.IR.Rd == D/E.IR.Rs1) )
then Forward to A input of ALU
43/49
Pipeline Performance
• Let n instructions (excluding branch or jump) are executed

using k stage pipeline
clock cycles required = k + (n − 1)
• CPI for pipelined implementation
k + (n − 1)
CPIpiplined =
n
k + (n − 1)
CPIpiplined ≈ lim
n→∞ n
≈ 1.
44/49
Pipeline Performance Cont’d
• Recall
time
Execution Time = N × CPI ×
cycle
• For datapath implementation having 5 phases (fetch, decode,

execute, memory, write-back)
time
Implementation No. of Inst. CPI cycle
Single Cycle N 1 Large (≈ 5x)
Multicycle N > 1 (≈ 4) Small (≈ 1x)
Pipelined N ≈1 Small (≈ 1x)
45/49
Pipeline Performance Cont’d
• Speedup due to pipelining
CPIunpipelined Cycle timeunpipelined

Speedup = ×
CPIpipelined Cycle timepipelined
1
Speedup ≈ × Pipeline depth
1 + stall cycles per instruction
where
Cycle timeunpipelined
Pipeline depth ≈
Cycle timepipelined
46/49
Suggested Reading
• Read relevant sections of Chapter 4 of

[Patterson and Hennessy, 2021].
47/49
Acknowledgment
• Preparation of this material was partly supported by Lampro

Mellon Pakistan.
48/49
References
Patterson, D. and Hennessy, J. (2021).

Computer Organization and Design RISC-V Edition: The Hardware
Software Interface, 2nd Edition.
Morgan Kaufmann.
49/49

8 - RISCV - Pipelined - Arch2

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

8 - RISCV - Pipelined - Arch2

Uploaded by

Copyright:

Available Formats

Structural Hazards Data Hazards Resolving Data Hazards Performance Evaluation

RISC-V Pipelined Architecture II

Electrical Engineering Department

3 Resolving Data Hazards

• Major drawback of pipelining: Hazards

• Major drawback of pipelining: Hazards

Fetch Decode ALU Memory Writeback

Fetch Decode ALU Memory Writeback

Fetch Decode ALU Memory Writeback

Fetch Decode ALU Memory Writeback

Fetch Decode ALU Memory Writeback

Instruction Execution: Cont’d

Fetch Decode ALU Memory Writeback

Fetch Decode ALU Memory Writeback

Fetch Decode ALU Memory Writeback

Fetch Decode ALU Memory Writeback

Fetch Decode ALU Memory Writeback

Instruction Execution: Cont’d

Fetch Decode ALU Memory Writeback

Fetch Decode ALU Memory Writeback

Fetch Decode ALU Memory Writeback

Fetch Decode ALU Memory Writeback

Fetch Decode ALU Memory Writeback

Instruction Execution: Cont’d

Fetch Decode ALU Memory Writeback

Fetch Decode ALU Memory Writeback

Fetch Decode ALU Memory Writeback

Fetch Decode ALU Memory Writeback

Fetch Decode ALU Memory Writeback

Instruction Execution: Cont’d

Fetch Decode ALU Memory Writeback

Fetch Decode ALU Memory Writeback

Fetch Decode ALU Memory Writeback

Fetch Decode ALU Memory Writeback

Fetch Decode ALU Memory Writeback

Structural Hazard in Our RV32I Design?

Inst. Mem rdata1 A Data Mem

No Structural Hazard in Our RV32I Design

Inst. Mem rdata1 A Data Mem

• Data Hazard is a consequence of operand(s) dependence

Data Hazards Cont’d

• Data hazards can be of three different types

RAW Data Hazards

add x3, x2, x1

RAW Data Hazards

add x3, x2, x1

Resolving RAW Data Hazards

• Solution 1: A simple approach is to freeze the earlier pipeline

RAW Data Hazard Illustration

• Example program for data hazards

add x3, x2, x1

Instructions Execution Cont’d

Instructions Execution Cont’d

Instructions Execution Cont’d

Instructions Execution: Pipeline Stalling

Instructions Execution: Pipeline Stalling

Instructions Execution: Pipeline Stalling

Instructions Execution: Pipeline Stalling

Instructions Execution: Pipeline Stalling

Instructions Execution: Pipeline Stalling

Instructions Execution: Pipeline Stalling

• Using Stalling to resolve RAW data hazard