Scalable Memory Architecture for Soft-core

Processors
Tiago T. Jost, Gabriel L. Nazar, Luigi Carro
Instituto de Informática
Universidade Federal do Rio Grande do Sul
Porto Alegre, Brazil
{ttjost, glnazar, carro}@inf.ufrgs.br

Abstract—Restrictions over memory performance have
always had a great impact on soft-core processors. The reduced
number of ports on FPGAs' block RAMs may limit the
exploitation of parallelism on soft-core processors that are
implemented on top of these devices. Multiple memory ports on
FPGAs are cumbersome and do not scale well, having a high cost
in area and power consumption when implemented. In order to
mitigate the impact of the memory bottleneck on such devices, we
propose a scalable memory architecture for soft-cores. We make
use of software-managed memories to build a memory system
capable of improving performance and instruction-level
parallelism (ILP) on soft-core processors. Results show that our
architecture overcomes the limited parallelism realized on a
dual-ported processor, reducing execution time by 16.5%. These
improvements come with no area costs, as the processor is kept
with the same total memory. Automated code transformations
implemented within the LLVM compiler keep changes in
application code to a minimum. We also show that our
architecture scales better when boosting the number of
functional units in the system.
Keywords—FPGA; BRAM; soft-core; compiler; memory
architecture
I. INTRODUCTION
Memory bandwidth has long been a limiting factor in
performance for many applications. Even when exploiting
access locality through cache memories to minimize this
limitation, multi-ported memories are known to introduce
significant costs [1]. When developing applications in Field
Programmable Gate Arrays (FPGAs), providing an efficient
on-chip memory hierarchy may become even harder, as we are
restricted to fixed resources. Modern FPGAs may include tens
of megabytes of on-chip memory, distributed over numerous
independent banks typically referred to as block RAMs
(BRAMs). Each BRAM has its own memory ports and address
space, giving an enormous bandwidth to access these internal
memories. High bandwidth is achieved when data is fetched in
parallel from multiple uncorrelated BRAMs, i.e., there is no
need for a unique address space. Nevertheless, from a software
programmer’s point of view, memory is seen as a sizable
storage with a unique address space. Hence, the traditional
FPGA memory model is not particularly suited to meet
software’s requirements. FPGAs’ flexibility allows the
combination of multiple BRAMs aiming at providing a unified
and larger address space, with the downside of significantly
increasing area and compromising memory bandwidth. Such
This work is sponsored by CNPq (Conselho Nacional de
Desenvolvimento Científico e Tecnológico, Brazil).
constraints might reduce instruction-level parallelism (ILP) and
performance on soft-cores that are implemented on the top of
FPGAs [2], [3].
Due to area restrictions on FPGAs, soft-core processors
lean towards less complexity regarding their designs when
compared to hardwired processors, usually with the absence of
register renaming or branch prediction. Memory constraints
may also add an extra complication due to the limited number
of ports in BRAMs. The use of BRAMs to design multi-ported
memories that can be specially used for soft-core processors
does not scale well, as area increases and frequency decreases
significantly as the number of ports increases [4]. The number
of memory ports is, therefore, a limiting factor for increasing
performance in many applications.
In this paper, we propose a scalable memory architecture
(SMA) for boosting soft-core processors through the use of
multiple software-managed memories (SMMs). We have
implemented an LLVM-based compiler that automates code
generation for the memory architecture. We have found no
work that focuses primarily on using software-managed
memories to increase performance and ILP on soft-cores. Some
previous works [5], [6] make use of software-controlled
memories, called scratchpad memories, as a replacement for
caches with the purpose of reducing energy consumption;
others [7], [8] focus on enhancing the performance of
embedded processors, although due to a reduction of cache
misses and not an increase in parallelism. The major difference
between scratchpads from previous works and our memories is
how address spaces are treated. Scratchpads typically use the
same address space of the memory hierarchy, i. e., a range of
addresses is scratchpad-addressable while others are dealt
through the cache. On the other hand, our software-managed
memories have unique address spaces, with no correlation
among them whatsoever.
FPGA’s restrictions impose a new challenge on increasing
bandwidth on soft-core processors. Our approach avoids high
costs in area and latency while providing parallel memory ports
and, in consequence, ILP improvements. We demonstrate how
our memory architecture improves performance using a soft-
core very long instruction word (VLIW) processor through a
set of benchmarks, such as matrix multiplication, sum of
absolute differences (SAD) and x264. We also show that our
architecture scales better when boosting the number of
functional units in the system.

978-1-5090-5142-7/16/$31.00 2016
c IEEE 396

Authorized licensed use limited to: Indraprastha Institute of Information Technology. Downloaded on January 15,2021 at 13:17:24 UTC from IEEE Xplore. Restrictions apply.
 The rest of the paper is organized as follows. Section II
presents related work. Section III describes hardware and
software modifications required for our technique. Section IV
presents the results, and section V concludes the paper.
II. RELATED WORK
Software-controlled memories are an attractive option for
substituting caches on embedded processors due to their
simpler hardware. There is no need for any kind of hardware
coherency mechanism, as software is responsible for their
control. They can, therefore, provide considerable area and
latency improvements over caches. Our technique uses
BRAMs as software-controlled memories to boost scalability
with regard to parallel memory accesses. Hence, our work
encounters many similarities with two different topics of
research: (1) the use of software-controlled memories as a
replacement for high-consuming caches [5], [8], (2) efficient
ways of using memories on FPGAs [4], [9]–[12].
The use of software-managed memories in replacement of
caches has been vastly explored in the past decade. The work
in [5] shows an average reduction of 20-44% in energy
consumption on instruction scratchpads, while Avisar et. al. [8]
uses an automatic compiler method for statically allocating
program data to scratchpad memories. In contrast to previous
works [7], [8] that primarily focused on performance
improvements due to decreasing miss penalty on the memory
hierarchy, we implement a scalable memory architecture with
the combination of software-managed memories and memory
cache. We aim at reusing data to improve performance and ILP
in soft-core processors, showing that multiple memories can
also help increasing performance on such processors by
enabling multiple accesses at once. Moreover, if one considers
the use of FPGA for the implementation of processors,
additional care must be taken due to the restriction of BRAMs,
which are typically dual-ported.
Recent researches have provided various contributions on
the use of BRAMs. Adler et. al. [10] propose Logic-based
Environment for Application Programming (LEAP), an
automatic memory management tool for reconfigurable logic.
The main idea is to use LEAP scratchpads to provide an easy
way of managing memory on FPGAs. Thus, developers can
primarily focus on developing their core algorithms and less on
managing memory. Yang et. al. [9] and Winterstein et. al. [12]
extend LEAP scratchpads to automate the construction
application-specific memory hierarchies. Therefore, every
application has an optimized memory hierarchy that best fits its
characteristics. CoRAM++ [11] provides a convenient
approach for designing efficient complex data structures in
memory without penalizing access performance. It uses an
application-level interface that model data structures, such as
multi-dimensional arrays, linked lists, and simple data patterns.
All works mentioned above help to address the mismatch
between the needs of applications and memory mapping on
FPGAs. The study in [13] demonstrates that memory-aware
designs show a significant energy improvement comparing to
non-optimal designs by over a factor of 2. Moreover, the work
in [14] presents a comparison of performance and energy
consumption between FPGAs and graphics processing units
2016 IEEE 34th International Conference on Computer Design (ICCD)
Authorized licensed use limited to: Indraprastha Institute of Information Technology. Downloaded on January 15,2021 at 13:17:24 UTC from IEEE Xplore. Restrictions apply.
(GPUs) and illustrates how memory bandwidth can have an
impact on those results. Our approach is orthogonal to
techniques like [10] and [11], i.e., they could even be coupled
to improve performance even more.
The work in [4] introduces a new approach for creating
multi-ported memories in FPGAs that efficiently combines
BRAMs while achieving significant performance
improvements over other methods. It replicates memories
according to the number of read/write ports. A 8-read/4-write
memory, for example, can be implemented using eight 2-
read/1-write memories. It also uses live value tables (LVT) to
identify where the most recently copy of the data is located.
Although this approach is better than others, it still requires at
least doubling the area to build multi-ported memories with
more than two ports.
III. HARDWARE AND SOFTWARE REQUIREMENTS
Software-managed memories are particularly beneficial
when reuse can be guaranteed, i. e., one needs to assure that
temporal and spatial locality are presented in the program. Our
approach aims at using these memories in soft-cores,
processors that struggle with limited memory bandwidth due to
how FPGAs traditionally handle large address spaces. FPGAs
are very fast and effective when handling multiple address
spaces. However, large memory spaces are often cumbersome
when considering more than two ports in FPGA’s BRAMs.
Our main goal is to mitigate memory complexity in soft-
cores by creating a memory architecture that explores the use
of multiple BRAMs in parallel. That way, single-ported
memories are mapped to single-ported BRAMs and no
overhead in adding new ports is necessary. The rest of this
section details hardware changes and software implementation
required for the technique.
A. Hardware
Minimum hardware changes are necessary to support
SMMs in our system. Figure 1 shows an example of how they
are placed within a hypothetical 4-issue VLIW processor with
a 5-stage pipeline. A memory architecture is created with the
placement of a software-managed memory within each lane.
With the exception of Lane0, which can also access the regular
data memory hierarchy through the L1 data cache (MEM block
in Figure 1), each lane can only access its own memory’s
content. The main difference between SMMs and caches relies
on who controls them. The former is controlled by software
and requires no extra hardware for tag storage and checking
while the latter is hardware-controlled and the programmer has
little control over where data is placed in memory.
Because lanes can only operate in one memory, there is no
need to specify which memory SMM instructions should
access. Each lane only sees its own memory, and the address
spaces are independent of one another. Thereby, we avoid the
costs of having multi-ported memories, while still providing
parallelism for the processor. SMM instructions operate in the
same way as loads and stores: one register as the base register,
an immediate offset that will be added to the base register, and
a second register that will hold the value that must be stored to
or was loaded from memory.
397

Fig. 1. 4-issue VLIW processor with software-managed memories
B. Software
In order to minimize the programmer’s effort, the LLVM
compiler was extended to support SMMs. The compiler is able
to guarantee the correct and efficient code generation through a
series of data flow analysis and transformations. Users can
decide which variable is placed within the memories through
code annotation. Compiler, then, modifies the code by
replacing memory instructions with instructions that
manipulate SMMs. We aim at using these memories primarily
to store vector and matrix variables, since they offer more
possibility of parallelism.
A preamble code is also a requirement for most
applications. This piece of code, generated by the compiler, is
responsible for fetching values that should reside in the SMMs
from the regular memory hierarchy, i.e., through the L1 data
cache. The unveiling of ILP relies on using multiple software-
managed memories in parallel. Once data is loaded into the
SMMs, it can be accessed with high bandwidth due to the
multiple independent ports. So, the compiler’s goal is
maximizing the usage of these memories while still reasoning
about parallel operations.
IV. EXPERIMENTAL RESULTS
For a performance evaluation, we have developed an
LLVM [15] backend for a VLIW processor and added the
technique as part of the framework. LLVM has proven to be
very modular, which made a perfect fit for our work. Code
optimizations are in the form of passes, that may either collect
information or transform the program. We have added a new
pass that analyzes and transforms code right before register
allocation takes place. That way, the standard LLVM heuristics
deal with register allocation after the addition of new code.
Code transformation is performed in virtual registers, which
ease the process of inserting and modifying code.
Our VLIW processor implements the VEX instruction set
architecture (ISA) [16], developed by HP. The company
provides a handful of tools [17] to help use the architecture,
including a customizable simulation system that was used for
simulation. We have modified it and added new instructions to
handle SMMs. The tool already provides a built-in level-one
cache simulator. Hence, a complete simulation platform is
included.
398 2016 IEEE 34th International Conference on Computer Design (ICCD)
Authorized licensed use limited to: Indraprastha Institute of Information Technology. Downloaded on January 15,2021 at 13:17:24 UTC from IEEE Xplore. Restrictions apply.
We have compared our proposed solution measuring
execution time between processors with different
configurations. Processors that implement the technique are
denoted as SMA processors, while those that make no use of
software-managed memories are the regular processors. We
firstly consider 8-issue processors. Six benchmarks were used
to measure gains: a discrete Fourier transform (DFT), three
matrix multiplication with different matrixes sizes, the sum of
absolute differences (SAD), and an x264 video encoder
algorithm. For a fairer comparison, we have reduced the size of
our level-one data cache on the SMA processors proportional
to the size of all SMMs together. SMMs are 2 KBytes of size
each, for a total of 16 Kbytes. These dimensions are in tune
with the BRAM blocks offered by current state-of-the-art
FPGAs [18]. Thus, SMA processors have 16 KBytes of data
cache, while regular processors have 32 Kbytes. This is a
conservative approach as the extra 16 KBytes of cache of the
baseline still require additional tag storage, which is not
necessary for the SMAs.
Figure 2 illustrates the comparison between 8-issue dual-
ported processors, as BRAMs in FPGAs typically have this
characteristic. Results show how the SMA processor reduces
execution time over the baseline for all benchmarks. The use
of software-managed memories has shown to reduce execution
time by an average of 16.5%. As the number of memory ports
increases, for instance, with four or even eight ports, the
software-managed memories become more dispensable, since
the memory hierarchy can already provide the system with a
high throughput. As discussed, however, such caches are very
costly, especially on the FPGA substrate. Still, for the dual-
ported case, the SMA processor provides performance
improvements over the baseline, with execution time reduction
between 10% and 25%.
Our last measurement consisted in verifying whether our
memory architecture helps boosting performance when
increasing the number of issues on the processor. Figure 3
depicts the performance comparison for 16-issue dual-ported
processors. The average improvement over the baseline went
from 16.5% to 20.3%. The improvements offered by the
proposed technique, when compared to those observed for 8-
1,1
1,0
0,9
Normalized execution time
0,8
0,7
0,6
0,5
0,4
0,3
0,2
0,1
0,0
DFT Matrix Matrix Matrix32x32 sad x264
10x10 16x16
Regular - 2 Mem SMA - 2 Mem
Fig. 2. Comparison between 8-issue dual-ported processors. Average execution
time reduction of 16.5%.

1,1
1,0
Normalized execution time
0,9
0,8
0,7
0,6
0,5
0,4
0,3
0,2
0,1
0,0
DFT Matrix Matrix
10x10 16x16
Regular - 16i - 2 Mem
Fig. 3. Comparison between 16-issue dual-ported processors. Average
execution time reduction of 20.3%.
issue processors, were greater in four out of six benchmarks.
One benchmark, Matrix 10x10, showed virtually the same
ratio. Matrix 16x16, on the other hand, went from 19.1%
reduction to 28.8%. These results indicate that, when the
application exhibits high ILP, the traditional memory
hierarchy is limiting factor that can be alleviated with
independent SMMs. Moreover, DFT has shown a performance
reduction for both 16-issue dual-ported processors when
comparing to the 8-issue counterpart. The register file cannot
accommodate all live variables at the same time. Thus, some
of the variables are spilled to memory and performance is
reduced. Because the SMA processor exploits more
parallelism, more variables are kept alive at a certain point in
the program, introducing more spilled code and reducing the
difference between the 16-issue processors.
V. CONCLUSION
This work proposed a scalable memory architecture for
soft-cores through the use of software-managed memories. We
have developed an LLVM-based backend that analyzes and
transforms code to operate such memories in cooperation with
the memory hierarchy. Results have shown execution time
reduction of 16.5% for the dual-ported processor. Our
architecture also shows promising gains when the number of
issues is boosted to 16, scaling better than the baseline when
increasing the issue width. The same area of the baseline
processor was kept for all cases.
Future work will focus on eliminating the need for code
annotation from the user part. The SMA is not restricted to
VLIW processors, although special care will have to be taken
when using this technique on other ILP-capable architectures,
e.g., superscalar processors. Extending the technique for such
cases is a relevant future work. We also plan on extending the
work to reason about energy consumption.
REFERENCES
[1] Y. Tatsumi and H. J. Mattausch, “Fast quadratic increase of multiport-
storage-cell area with port number,” Electron. Lett., vol. 35, no. 25, pp.
2185–2187, 1999.
2016 IEEE 34th International Conference on Computer Design (ICCD)
Authorized licensed use limited to: Indraprastha Institute of Information Technology. Downloaded on January 15,2021 at 13:17:24 UTC from IEEE Xplore. Restrictions apply.
[2] J. G. Tong, I. D. L. Anderson, and M. A. S. Khalid, “Soft-core
processors for embedded systems,” in Microelectronics, 2006. ICM’06.
International Conference on, 2006, pp. 170–173.
[3] S. Wong, T. Van As, and G. Brown, “$ρ$-VEX: A reconfigurable and
extensible softcore VLIW processor,” in ICECE Technology, 2008. FPT
2008. International Conference on, 2008, pp. 369–372.
[4] C. E. LaForest and J. G. Steffan, “Efficient multi-ported memories for
FPGAs,” in Proceedings of the 18th annual ACM/SIGDA international
symposium on Field programmable gate arrays, 2010, pp. 41–50.
[5] M. Verma, L. Wehmeyer, and P. Marwedel, “Cache-aware scratchpad
allocation algorithm,” in Proceedings of the conference on Design,
automation and test in Europe-Volume 2, 2004, p. 21264.
[6] S. Steinke, L. Wehmeyer, B.-S. Lee, and P. Marwedel, “Assigning
Matrix32x32 sad x264
program and data objects to scratchpad for energy reduction,” in Design,
Automation and Test in Europe Conference and Exhibition, 2002.
SMA - 16i - 2 Mem Proceedings, 2002, pp. 409–415.
[7] F. Angiolini, F. Menichelli, A. Ferrero, L. Benini, and M. Olivieri, “A
Post-compiler Approach to Scratchpad Mapping of Code,” in
Proceedings of the 2004 International Conference on Compilers,
Architecture, and Synthesis for Embedded Systems, 2004, pp. 259–267.
[8] O. Avissar, R. Barua, and D. Stewart, “An Optimal Memory Allocation
Scheme for Scratch-pad-based Embedded Systems,” ACM Trans.
Embed. Comput. Syst., vol. 1, no. 1, pp. 6–26, Nov. 2002.
[9] H.-J. Yang, K. Fleming, M. Adler, F. Winterstein, and J. Emer,
“Scavenger: Automating the construction of application-optimized
memory hierarchies,” in Field Programmable Logic and Applications
(FPL), 2015 25th International Conference on, 2015, pp. 1–8.
[10] M. Adler, K. E. Fleming, A. Parashar, M. Pellauer, and J. Emer, “Leap
scratchpads: automatic memory and cache management for
reconfigurable logic,” in Proceedings of the 19th ACM/SIGDA
international symposium on Field programmable gate arrays, 2011, pp.
25–28.
[11] G. Weisz and J. C. Hoe, “CoRAM++: Supporting data-structure-specific
memory interfaces for FPGA computing,” in Field Programmable Logic
and Applications (FPL), 2015 25th International Conference on, 2015,
pp. 1–8.
[12] F. Winterstein, K. Fleming, H.-J. Yang, S. Bayliss, and G.
Constantinides, “MATCHUP: memory abstractions for heap
manipulating programs,” in Proceedings of the 2015 ACM/SIGDA
International Symposium on Field-Programmable Gate Arrays, 2015,
pp. 136–145.
[13] E. Kadric, D. Lakata, and A. DeHon, “Impact of Memory Architecture
on FPGA Energy Consumption,” in Proceedings of the 2015
ACM/SIGDA International Symposium on Field-Programmable Gate
Arrays, 2015, pp. 146–155.
[14] B. Betkaoui, D. B. Thomas, and W. Luk, “Comparing performance and
energy efficiency of FPGAs and GPUs for high productivity
computing,” in Field-Programmable Technology (FPT), 2010
International Conference on, 2010, pp. 94–101.
[15] C. Lattner and V. Adve, “LLVM: A Compilation Framework for
Lifelong Program Analysis & Transformation,” in Proceedings of the
International Symposium on Code Generation and Optimization:
Feedback-directed and Runtime Optimization, 2004, p. 75–.
[16] J. A. Fisher, P. Faraboschi, and C. Young, Embedded computing: a
VLIW approach to architecture, compilers and tools. Elsevier, 2005.
[17] Hewlett-Packard Laboratories, “VEX Toolchain.” [Online]. Available:
http://www.hpl.hp.com/downloads/vex/.
[18] Xilinx, “UltraScale Architecture and Product Overview,” vol. 890, pp.
1–31, 2015.
399