The Berkeley Out - of - Order Machine (Boom!) : An Open - Source Industry - Compeeeve, Synthesizable, Parameterized Risc - V Processor

The
Berkeley Out-‐of-‐Order Machine (BOOM!):

An Open-‐source Industry-‐CompeEEve, Synthesizable,
Parameterized RISC-‐V Processor
Christopher Celio, Krste Asanovic, David Pa6erson
2016 Jan
celio@eecs.berkeley.edu
UC Berkeley
UC Berkeley What is BOOM?
◾ superscalar, out-‐of-‐order processor wri6en in
Berkeley’s Chisel RTL
◾ It is synthesizable
◾ It is parameterizable
◾ We hope to use it as a plaOorm for architecture
research
◾ It is open-‐source!
BOOM is a work-in-progress.
Results shown in the talk are
preliminary and subject to
change!
2
UC Berkeley Open-‐source Berkeley RISC-‐V Processors
◾ Sodor CollecEon
- RV32I -‐ Eny, educaEonal, not synthesizable
◾ Rocket-‐chip SoC generator
◾ Z-‐scale
- RV32IM -‐ micro-‐controller
◾ Rocket
- RV64G -‐ in-‐order, single-‐issue applicaEon core
◾ BOOM
- RV64G -‐ out-‐of-‐order, superscalar applicaEon core
3
UC Berkeley Why Out-‐of-‐Order?
add r1, r0, r0
ld r2, M[r10]
sub r3, r2, r13
mul r4, r14, r15
◾ Great for ...

- toleraEng variable latencies
- finding ILP in code (instrucEon-‐level parallelism)
- complex method for fine-‐grain data prefetching
- plays nicely with poor compilers and lazily wri6en code
◾ Downsides
- complicated
- expensive (area & power)
Performance! (and easy to program!)

4
UC Berkeley OoO is widely used in industry
◾ Intel Xeon/i-‐series (10-‐100W)
◾ ARM Cortex mobile chips (1W)
◾ Sun/Oracle Niagra UltraSPARC
◾ Intel Atom
◾ Play StaEon
5
UC Berkeley Goals
◾ build a prototypical OoO core
- provide an open-‐source implementaEon for educaEon,
research, and industry
- serve as a baseline for micro-‐architecture research
◾ enable studies that need an OoO core
- research memory systems
- research accelerators
- research VLSI methodologies
◾ use BOOM to get detailed performance, area, and
power running real programs
- most research uses sw simulators for performance
(100 KIPS), RTL for area
6
Berkeley Architecture Research
Infrastructure
UC Berkeley
◾ RISC-‐V ISA
◾ Rocket-‐chip SoC generator
◾ Chisel HCL (hardware construcEon language)
◾ bar.eecs.berkeley.edu
7
UC Berkeley BOOM Implements IMAFD
◾ “M” (mul/div/rem)
- imul can be either pipelined or unpipelined
◾ "A”
- AMOs+LR/SC
◾ “FD”
- single, double-‐precision floaEng point
- IEEE 754-‐2008 compliant FPU
- SP, DP FMA with hw support for subnormals
- parameterized latency
◾ RV64G + privileged spec (page-‐based virtual memory)
8
UC Berkeley The RISC-‐V ISA is easy to implement!
◾ relaxed memory model
◾ accrued FP excepEon flags
◾ no integer side-‐effects (e.g., condiEon codes)
◾ no cmov or predicaEon
◾ no implicit register specifiers
- JAL requires explicit rd
◾ rs1, rs2, rs3, rd always in same space
- allows decode, rename to proceed in parallel
9
UC Berkeley Rocket-‐Chip SoC Generator
◾ open-‐source
◾ taped out 11 Emes by
Berkeley
◾ runs at 1.6 GHz in IBM
45nm
◾ makes for a great library
of processor
components!
◾ swap out Rocket Ele and
replace with a BOOM Ele
◾ as Rocket-‐chip gets
be6er, so does BOOM
*slightly out-of-date uncore
(now uses AXI instead of MemIO)
10
UC Berkeley Rocket-‐Chip Updates
◾ uncached loads/stores
◾ memory-‐mapped IO
◾ replaced memIO with AXI
◾ Work in Progress
◾ remove HTIF (untether)
*slightly out-of-date uncore

(now uses AXI instead of MemIO)
11
UC Berkeley Chisel
◾ Hardware ConstrucEon Language embedded in
Scala
◾ not a high-‐level synthesis language
◾ hardware module is a data structure in Scala
◾ Full power of Scala for wriEng generators
- object-‐oriented programming
- factory objects, traits, overloading
- funcEonal programming
- high-‐order funs, anonymous funcs, currying
◾ generated C++ simulator is 1:1 copy of Verilog
designs
◾ version 3.0 coming soon!
- uses an IR called FIRRTL
12
UC Berkeley BOOM Pipeline
Issue
Window
Unified
Decode & Physical Functional Unit
Fetch Rename Register
File
in-‐order out-‐of-‐order
front-‐half back-‐half
13
UC Berkeley BOOM Pipeline
Rename Map Tables & Freelist
Issue
Window
ALU
Unified
Physical
Decode &
Fetch Rename
Register
File FPU
(PRF)
ROB
◾ PRF Commit
- explicit renaming
- holds speculaEve and commi6ed data
- holds both x-‐regs, f-‐regs
◾ split ROB/issue window design
◾ Unified Issue Window
- holds all instrucEons
14
UC Berkeley Parameterized Superscalar
bypassing
dual-issue (5r,3w)
val exe_units = ArrayBuffer[ExecutionUnit]()
ALU
exe_units += Module(new ALUExeUnit(is_branch_unit = true
, has_fpu = true
FPU , has_mul = true
))
exe_units += Module(new ALUMemExeUnit(fp_mem_support = true
imul , has_div = true
Issue Regfile ))
Regfile
bypass
Select Read network Writeback
bypassing
ALU
div Quad-issue (9r,4w) bypassing

Agen ALU
LSU D$
ALU
OR Issue
Select
Regfile bypass
Read network
FPU
imul
Regfile
Writeback
exe_units += Module(new ALUExeUnit(is_branch_unit = true)) ALU

exe_units += Module(new ALUExeUnit(has_fpu = true
, has_mul = true div
))
exe_units += Module(new ALUExeUnit(has_div = true))
exe_units += Module(new MemExeUnit()) Agen
LSU D$
15
UC Berkeley A FuncZonal Unit Hierarchy
Functional Unit
• Abstract FunctionalUnit Pipelined UnPipelined

(req.ready ==true)
– describes common IO

• Pipelined/Unpipelined iMul/iDiv/
iMul fDiv/fSqrt
iRem
– handles storing uop ALU
FPU
(w/ optional MemAddrCalc
metadata, branch Br Unit)
resolution, branch kills FPTo

FP
• Concrete Subclasses ALU DFMA IntTo
– instantiates the actual SFMA
FPTo
FP
expert-‐written FU
Int
– no modifications
required to get FU
working with speculative
OoO
– allows easy “stealing” of
external code 16
UC Berkeley Some of the Parameters
◾ fetch/decode/commit width (1,2,4)
◾ issue width
- funcEonal unit mix
◾ un-‐ordered vs age-‐ordered issue scheduler (R10k vs R12k)
◾ enable commit map table (R10k roll-‐back vs. Alpha 21264 single-‐cycle flush)
◾ rob size
◾ issue window size
◾ lsu size
◾ number of physical registers
◾ max inflight branches
◾ FPU latencies, iMul latencies, unpipelined mul/div bits per stage
◾ fetch buffer size
◾ flow-‐through fetch buffer
◾ enable backing branch predictor (and it's set of parameters...)
◾ enable uarch counters
◾ RAS size, BTB size
◾ L1 cache sets, ways, mshrs
◾ enable L2 (and L2 parameters...)
17
UC Berkeley Synthesizable
TSMC 45nm floorplan
◾ TargeEng ASIC fpu
◾ Runs on FPGA dcache

- Zynq zc706
issue
Fetch unit
2
lsu
rob
2
ROB
inst Decode & alu imul uncore
Rename rename
Issue alu
Window muldiv
uop tags
2
rob
Commit
wakeup 2
Physical regfile
Register
File (5x3)
regread icache
(PRF)
ALU LSU ALU

FPU
Data 2-wide BOOM, 16kB L1 caches
1.2 mm2
cache
2
preliminary results 18
UC Berkeley How fast can BOOM be clocked?
◾ depends on parameters
◾ +1.5 GHz for two-‐wide
◾ (currently) designed for single-‐cycle SRAM access as
criEcal path
◾ upcoming tape-‐out will keep us honest!
◾ ... and 50 MHz on FPGA
- bo6leneck is FPGA tools which can't register-‐reEme the FPU
19
UC Berkeley Full Branch SpeculaZon Support
◾ next-‐line
predictor (NLP) NPC Fetch1 Fetch2
- BTB, BHT, RAS
- combinaEonal μDec
backing predictor
TakePC PC1 NLP PC2
◾
(BPD) BHT
Target I$ >>
- global history Fetch

predictor ExeBrTarget Front-end Buffer
- SRAM (1 r/w port) BPD

Branch
Prediction
Front-end
20
UC Berkeley Load/Store Unit
◾ talks to Rocket's non-‐blocking data cache
◾ load/store queue with store ordering
- loads execute fully out-‐of-‐order wrt stores, other loads
- (based on current understanding of the RISC-‐V memory model)
- store-‐data forwarded to loads as required
21
UC Berkeley Load/Store Unit
rs1 imm
Address ROB
Calculation
Adder com_st_mask
com_ld_mask
LOAD/STORE retry load address (needs translation)

retry store address (needs translation)
UNIT vaddr
woken-up load address
TLB miss or !ready TLB store address
br page fault paddr TLB load address

kill Exe Stage
Store Address Load Address
miss xcpt Addr Data Addr
SAQ assoc
LAQ Data
A val addr V search assoc A val addr V E R F st_mask forward_
kill* search std_idx cache
Data
Array
addrmatches
woken-up load address

exceptions** Mem Stage
ordering failures
from store search
youngest store idx Age
load depends on
ROB Logic
br
kill st->ld addr match
xcpt kill request
TLB miss
Store
unbusy†
fwd_std_val fwd_std_idx Data nack
ECC/
SDQ Fmt WB Stage
val data
NACK
br
kill
nack
Writeport
Register
File
22
UC Berkeley Feature Summary
Feature BOOM
ISA RISC-V (RV64G)
Synthesizable √
FPGA √
Parameterized √
floating point √
AMOs+LR/SC √
caches √
VM √
Boots Linux √
Multi-core √
lines of code 9k + 11k
23
Comparison against ARM
UC Berkeley
Category ARM Cortex-A9 RISC-V BOOM-2w
ISA 32-bit ARM v7 64-bit RISC-V v2 (RV64G)

2 wide, 3+1 issue Out-of- 2 wide, 3 issue Out-of-Order
Architecture Order 8-stage 6-stage
% !
+9
Performance 3.59 CoreMarks/MHz 3.91 CoreMarks/MHz
Process TSMC 40GPLUS TSMC 40GPLUS

Area with 32K
caches ~2.5 mm2 ~1.00 mm2
Area efficiency 1.4 CoreMarks/MHz/mm2 3.9 CoreMarks/MHz/mm2
Frequency 1.4 GHz 1.5 GHz

0.5-1.9 W (2 cores + L2) 0.25 W (1 core + L1)
Power @ TSMC 40nm, 0.8-2.0 GHz @ TSMC 45nm, 1 GHz
note:
not to scale
24
UC Berkeley Challenges execuZng SPECint
◾ 12 benchmarks in CINT2006
- 35 workloads wri6en in C or C++ (-‐-‐reference)
- average 2 trillion instrucEons
- ~24T instrucEons total
- (2 hours for a 3 GHz Intel processor at CPI=1)
- (7 years for a sw simulator @ 100 KIPS)
◾ RISC-‐V
- requires libc (riscv64-‐unknown-‐linux-‐gnu*)
- run on Linux
- haps://github.com/ccelio/Speckle
- useful for generaEng portable SPEC directories
25
UC Berkeley Preliminary Data
26
UC Berkeley SPEC Score
◾ TODO :( astar.1
◾ need more DRAM
◾ need DRAM emulaEon
◾ will take ~1 day with cluster
h264.1 bzip2.2
27
UC Berkeley How to use BOOM...
28
UC Berkeley Repositories
◾ BOOM
- h6ps://github.com/ucb-‐bar/riscv-‐boom
- just the BOOM core code
◾ Rocket-‐chip
- h6ps://github.com/ucb-‐bar/rocket-‐chip
- the rest of the SoC
- chisel, rocket, uncore, juncEons, riscv-‐tools, fpga-‐zynq, etc.
- generates C++ emulator, verilog for FPGA
- h6p://riscv.org/tutorial-‐hpca2015/riscv-‐rocket-‐chip-‐tutorial-‐bootcamp-‐hpca2015.pdf
(for more informaEon)
29
UC Berkeley BOOM Quick-‐start
$ export ROCKETCHIP_ADDONS="boom"
$ git clone https://github.com/ucb-‐bar/rocket-‐chip.git
$ cd rocket-‐chip
$ git checkout boom
$ git submodule update -‐-‐init
$ cd riscv-‐tools
$ git submodule update -‐-‐init -‐-‐recursive riscv-‐tests
$ cd ../emulator; make run CONFIG=BOOMCPPConfig
◾ "BOOM-‐chip" is (currently) a branch of rocket-‐chip
◾ "make run" builds and runs riscv-‐tests suite
◾ there are many different CONFIGs available!
- rocket-‐chip/src/main/scala/PrivateConfigs.scala
- rocket-‐chip/boom/src/main/scala/configs.scala
- slightly different configs for different targets (CPP, FPGA,
VLSI)
30
UC Berkeley How do you verify and debug BOOM?
◾ parameterizaEon makes this tough!
◾ tested with:
◾ riscv-‐tests
- (assembly funcEonal tests + bare-‐metal micro-‐benchmarks)
◾ CoreMark + riscv-‐pk
◾ SPEC + Linux
◾ riscv-‐torture
31
UC Berkeley riscv-‐torture
◾ now open-‐source!
- h6ps://github.com/ucb-‐bar/riscv-‐torture
◾ torture generates random tests to stress the core
pipeline
◾ runs test on Spike and your processor
- architectural register state dumped to memory on program
terminaEon
- diff state
- if error found, finds smallest version of test that exhibits an
error
32
UC Berkeley riscv-‐torture quick-‐start
Run Rocket-chip (BOOM-chip?)
$ git clone https://github.com/ucb-‐bar/rocket-‐chip.git
$ cd rocket-‐chip
$ cd riscv-‐tools
$ git submodule update -‐-‐init -‐-‐recursive riscv-‐tests
$ cd ../emulator; make run CONFIG=BOOMCPPConfig
Run Torture
$ cd rocket-‐chip
$ git clone https://github.com/ucb-‐bar/riscv-‐torture.git
$ cd riscv-‐torture
$ vim Makefile # change RTL_CONFIG=BOOMCPPConfig
$ make igentest # test that torture works, gen a single test
$ make cnight # run C++ emulator overnight
33
UC Berkeley Commit Logging
◾ BOOM can generate commit logs
- (priv level, PC, inst, wb raddr, wb data)
◾ Spike supports generaEng commit logs!
- configure Spike with "-‐-‐enable-‐commitlog" flag
- outputs to stderr
◾ only semi-‐automated
- valid differences from Spike and hw
- e.g., spinning on LR+SC
# modify the build.sh in riscv-tools

build_project riscv-isa-sim --prefix=$RISCV --with-fesvr=$RISCV --enable-commitlog
34
UC Berkeley What documentaZon is available?
◾ A design document is in progress
- h6ps://github.com/ccelio/riscv-‐boom-‐doc
◾ Wiki
- h6ps://github.com/ucb-‐bar/riscv-‐boom/wiki
35
UC Berkeley Future Plans
◾ tape-‐out this year
◾ uncached loads/stores
◾ update to Chisel 3.0
◾ SPEC score
- improve branch predicEon
- memory ordering speculaEon
◾ finish documentaEon
◾ build a community of Baby Boomers?
36
UC Berkeley Conclusion
◾ BOOM is RV64G, runs SPEC on Linux on an FPGA
◾ BOOM is ~10k loc and 4 person-‐years of work
◾ excellent plaOorm for prototyping new ideas
◾ now open-‐source!
◾ we'll conEnue to support and improve BOOM
branch
redirects
fpu
Out-of-order processing dcache

BTB BP bypass
issue
unit
lsu
NPC I$ Dc/Rn IS/RF EX WB
rob
Fetch alu imul uncore

BHT Buffer MUL/ rename
DIV alu
WB Com muldiv
rob
regfile
D$ Fmt WB regread icache
37
UC Berkeley QuesZons?
38
UC Berkeley Funding Acknowledgements
◾ Research par*ally funded by DARPA Award Number HR0011-‐12-‐2-‐0016, the Center for
Future Architecture Research, a member of STARnet, a Semiconductor Research
Corpora*on program sponsored by MARCO and DARPA, and ASPIRE Lab industrial
sponsors and affiliates Intel, Google, Huawei, Nokia, NVIDIA, Oracle, and Samsung.
◾ Approved for public release; distribu*on is unlimited. The content of this presenta*on
does not necessarily reflect the posi*on or the policy of the US government and no official
endorsement should be inferred.
◾ Any opinions, findings, conclusions, or recommenda*ons in this paper are solely those of
the authors and does not necessarily reflect the posi*on or the policy of the sponsors.
39
UC Berkeley Extra Slides
40
UC Berkeley Synthesis Results
Core Area (um^2)
700000
BOOM Other
600000
Imul
500000 FPU
FetchBuffer
BusyTable
400000
Freelist
Br Predictor
300000 Rocket LSU
ROB
200000 Register File
RRd Stage (bypasses)
100000 Rename Stage (maptables)
Issue Unit
0
No FPU FPU BOOM-1w BOOM-2w BOOM-4w
UC Berkeley Synthesis Results
Core Area (um^2) Tile Area (um^2)
700000 1200000
D$ (16 KB)
600000 1000000 I$ (16 KB)
500000 Core
800000
400000
600000
300000
400000
200000
200000
100000
0 0
No FPU BOOM-1w BOOM-4w Rck-I Rck-G BOOM-1wBOOM-2wBOOM-4w
Issue Unit Rename Stage (maptables)

RRd Stage (bypasses) Register File
ROB LSU
Br Predictor Freelist
BusyTable FetchBuffer
FPU Imul
Other
UC Berkeley Industry Comparisons
dge CoreMark/MHz
6.00 Bri
Ivy
15
x-‐ A -‐ 4w
r te O M
5.00 Co BO
-‐ 2w
O M -‐ A 9
4.00 BO tex
CoreMark/MHz
or
C
74k 8
3.00 P S -‐ A
MI r tex ket
-‐ A 5
Co c ex
Ro r t
Co
2.00
1.00
0.00
out-‐of-‐order in-‐order
processors processors
UC Berkeley Industry Comparisons
CoreMark/
Processor Core Area Freq (MHz) IPC
MHz
4 8 x
Intel Xeon E5 2668 (Ivy) ~12 mm2@22nm 5.6 3,300 1.96
ARM Cortex-A15 2
2.8 mm @28nm 4.72 2,116 1.5
BOOM-4wide 1.1 mm2@45nm 4.7 1,000 1.5
BOOM-2wide 0.8 mm2@45nm 3.91 1,500 1.26
ARM Cortex-A9 2.5 mm2@40nm 3.59 1,400 1.27
MIPS 74K 2.5 mm2@65nm 2.5 1,600 -
Rocket (RV64G) 0.5 mm2@45nm 2.32 1,500 0.76
ARM Cortex-A5 0.5 mm2@40nm 2.13 - -
UC Berkeley Ivy Bridge Tile Comparison
I$ D$ (32k)
LLC Data
Ivy Bridge-‐EP Tile
Exe BOOM-2w Chip (32kB/32kB + 256kB caches)
Issue
Uncore
Exe
Regfile Ren (32kb/32kB + 256kB caches) ~12nm @ 22nm
Uncore
1.7mm2 @ 45nm
ROB LLC Data (256k)
Rename
bpd I$ (32k)
I$ D$ (32k) 2-wide BOOM layout.

BOOM-‐2w Chip
LLC Data
Exe
Uncore
Regfile
scaled to 0.4mm2 @ 22nm

Issue Ren
Exe
45
Uncore
LLC Data (256k)
preliminary results
ROB Rename
bpd I$ (32k)
2-wide BOOM layout.

The Berkeley Out - of - Order Machine (Boom!) : An Open - Source Industry - Compeeeve, Synthesizable, Parameterized Risc - V Processor

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

The Berkeley Out - of - Order Machine (Boom!) : An Open - Source Industry - Compeeeve, Synthesizable, Parameterized Risc - V Processor

Uploaded by

Copyright:

Available Formats

The

Berkeley Out-­‐of-­‐Order Machine (BOOM!):

◾ Great for ...

Performance! (and easy to program!)

*slightly out-of-date uncore

div Quad-issue (9r,4w) bypassing

exe_units += Module(new ALUExeUnit(is_branch_unit = true)) ALU

• Abstract FunctionalUnit Pipelined UnPipelined

– describes common IO

resolution, branch kills FPTo

◾ Runs on FPGA dcache

ALU LSU ALU

- global history Fetch

- SRAM (1 r/w port) BPD

LOAD/STORE retry load address (needs translation)

woken-up load address

TLB miss or !ready TLB store address

br page fault paddr TLB load address

woken-up load address

ISA RISC-V (RV64G)

ISA 32-bit ARM v7 64-bit RISC-V v2 (RV64G)

Process TSMC 40GPLUS TSMC 40GPLUS

Area efficiency 1.4 CoreMarks/MHz/mm2 3.9 CoreMarks/MHz/mm2

Frequency 1.4 GHz 1.5 GHz

# modify the build.sh in riscv-tools

Out-of-order processing dcache

Fetch alu imul uncore

D$ Fmt WB regread icache

Issue Unit Rename Stage (maptables)

BOOM-4wide 1.1 mm2@45nm 4.7 1,000 1.5

BOOM-2wide 0.8 mm2@45nm 3.91 1,500 1.26

ARM Cortex-A9 2.5 mm2@40nm 3.59 1,400 1.27

MIPS 74K 2.5 mm2@65nm 2.5 1,600 -

Rocket (RV64G) 0.5 mm2@45nm 2.32 1,500 0.76

ARM Cortex-A5 0.5 mm2@40nm 2.13 - -

I$ D$ (32k) 2-wide BOOM layout.

scaled to 0.4mm2 @ 22nm

2-wide BOOM layout.

You might also like

Berkeley Out-‐of-‐Order Machine (BOOM!):