FPGA Design with

High Level Synthesis (HLS)

FPGA Design with VIVADO High Level Synthesis

Sections and Lecturers of the Course:

Section 1. Overview of High Level Synthesis Tool

Lecture 1 : HLS Introduction, VIVADO HLS Overview
Lecture 2 : HLS Design Flow, review of C/C++ on HLS and HLS Libraries
Lecture 3 : Lab1: Basic Logic Component: Counter Design in VIVADO HLS

Section 2. Design, Simulation and Implementation with VIVADO HLS

Lecture 1 : VIVADO HLS Design Methodology
Lecture 2 : Simulation with VIVADO HLS
Lecture 3 : Lab 21: Matrix Multiplier Design, Simulation & Synthesizing in Vivado HLS
Lecture 4 : Lab 22: Designing ,Simulating & Synthesizing Frequency Modulator on HLS
Lecture 5 : Lab 23: Designing ,Simulating & Synthesizing Numerically Controller
Oscillator (NCO)

FPGA Design with VIVADO High Level Synthesis

Sections and Lecturers of the Course:

Section 3. Sobel Edge Detection with High Level Synthesis Tool

Lecture 1 : Overview of OpenCV with HLS [OpenCV Libraries for HLS]
Lecture 2 : Sobel Edge Detection Introduction
Lecture 3 : Lab31: Sobel Edge Detection with VIVADO HLS

FPGA Design with VIVADO High Level Synthesis

How the Sobel Edge Detection Works
Lecture 1 : OpenCV with HLS [OpenCV
Libraries for HLS]

-Adam Taylor

FPGA Design with VIVADO High Level Synthesis

The following libraries are included with Vivado HLS: UG902
Name Description

Arbitrary Precision Data Types Integer and fixed-point (ap_cint.h, ap_int.h and systemc.h)

HLS Stream Models for streaming data structures. Designed to obtain best performance and area (hls_stream.h)

Extensive support for the synthesis of the standard C (math.h) and C++ (cmath.h) math libraries. The support includes
floating point and fixed-point functions: abs, atan, atanf, atan2, atan2, ceil, ceilf, copysign, copysignf, cos, cosf, coshf,
HLS Math
expf, fabs, fabsf, floorf, fmax, fmin, logf, fpclassify, isfinite, isinf, isnan, isnormal, log, log10, modf, modff, recip, recipf,
round, rsqrt, rsqrtf, 1/sqrt, signbit, sin, sincos, sincosf, sinf, sinhf, sqrt, tan, tanf, trunc

Video library to implement several aspects of modeling video design in C++ with video Functions, specific data types,
memory line buffer and memory window (hls_video.h). Through a data type hls::Mat, Vivado HLS is also compatible with
existing OpenCV functions: AXIvideo2cvMat, AXIvideo2CvMat, AXIvideo2IplImage, cvMat2AXIvideo, CvMat2AXIvideo,
cvMat2hlsMat, CvMat2hlsMat, CvMat2hlsWindow, hlsMat2cvMat, hlsMat2CvMat, hlsMat2IplImage,
HLS Video hlsWindow2CvMat, IplImage2AXIvideo, IplImage2hlsMat, AbsDiff, AddS, AddWeighted, And, Avg, AvgSdv, Cmp, CmpS,
CornerHarris, CvtColor, Dilate, Duplicate, EqualizeHist, Erode, FASTX, Filter2D, GaussianBlur, Harris, HoughLines2,
Integral, InitUndistortRectifyMap, Max, MaxS, Mean, Merge, Min, MinMaxLoc, MinS, Mul, Not, PaintMask, PyrDown,

PyrUp, Range, Remap, Reduce, Resize, Set, Scale, Sobel , Split, SubRS, SubS, Sum, Threshold, Zero

HLS IP Integrate the LogiCORE IP FFT and FIR Compiler (hls_fft.h, hls_fir.h, ap_shift_reg.h)
HLS Linear Algebra Support for the following functions: cholesky, cholesky_inverse, matrix_multiply, qrf, qr_inverse, svd
(hls_linear_algebra.h)
Support for the following functions: atan2, awgn, cmpy, convolution_encoder, nco, qam_demod, qam_mod, sqrt,
HLS DSP
viterbi_decoder (hls_dsp.h)
Figure: HLS Functions of Video Library (UG902 & Saha Souvik, 2017)
Lecture 2 : Sobel Edge Detection Introduction
Sobel Introduction: From XAPP890
➢ Sobel edge detection is a classical algorithm in the field of image and video processing for the
extraction of object edges.
➢ Edge detection using Sobel operators works on the premise of computing an estimate of the first
derivative of an image to extract edge information [Ref 4].
➢ By computing the x and y direction derivatives of a specific pixel against a neighborhood of
surrounding pixels, it is possible to extract the boundary between two distinct elements in an image.
➢ Due to the computational load of calculating derivatives using squaring and square root operators,
fixed coefficient masks have been adopted as a suitable approximation in computing the derivative at
a specific point. In the case of Sobel, the masks used are shown in Table 1.
How Sobel Works?
• The Sobel algorithm functions by identifying
edges in a image and emphasizing them
such that they can be easily identified.
• Typically this will create a grey scale image,
where the edges are identified as shades of grey
/ white.
• Sobel edge detection works by detecting a
change in the gradient of the image in both
the horizontal and vertical directions.
• To do this two convolution filters are applied to
the original image, the results of these
convolution filters are then combined to
determine the magnitude of the gradient.

https://www.hackster.io/adam-taylor/fpga-
Sobel convolution filters, Source: Adam Taylor
based-edge-detection-using-hls-192ad2
Lecture 2 : Sobel Edge Detection Introduction
Sobel Introduction: From XAPP890
Lecture 2 : Sobel Edge Detection Introduction
Sobel Function in OpenCV:

FPGA Design with VIVADO High Level Synthesis

Lecture 2 : Sobel Edge Detection Introduction
• Sobel Function in HLS OpenCV: from http://www.wiki.xilinx.com/HLS+Sobel
Lecture 2 : Sobel Edge Detection Introduction
HLS Video Library: from www.wiki.xilinx.com/HLS+Video+Library
• HLS Video Library is a C/C++ library provided with Vivado HLS to help
accelerate computer vision/image processing appliations on FPGA. It
includes commonly used data structures, OpenCV interfaces, AXI4-
Stream I/O, and video processing functions.
• HLS Video Library uses OpenCV libraries as reference model, most
video processing functions has the similar interface and equaivalent
behavior with corresponding OpenCV functions.
• The pre-built OpenCV libraries (with FFmpeg support) are also shipped
with Vivado HLS on different platforms, so users are able to use
OpenCV directly without extra effort.
FPGA Design with VIVADO High Level Synthesis
Lecture 2 : Sobel Edge Detection Introduction
HLS Video Library: from www.wiki.xilinx.com/HLS+Video+Library
• There are two header files that can help users to start using HLS Video Library:
#include <hls_video.h>
• include this file in top design to access all specific video data structures and
image/video processing functions, they are all synthesizable. When using the
synthesizable part of HLS Video Library, it is required to use the hls namespace
or hls:: scoped.
#include <hls_opencv.h>
• include this file in testbench to access pre-built OpenCV libraries in Vivado HLS and
the its interface functions to HLS Video Library. Most of them are not
synthesizable.
• For more, review: XAPP1167, UG902 and OpenCV Installation

FPGA Design with VIVADO High Level Synthesis

Lecture 2 : Sobel Edge Detection Introduction

HLS Design Flow:

1. Implementing Sobel Edge Detection in C/C++ OpenCV with HLS

i. AXI to RGB_IMAGE stream
ii. Grayscaling Resources
Provided
iii. Gaussian Blur Noise Reduction by
iv. Calculate gradients in x and y direction using Sobel filter XAPP890,
XAPP1167
v. RGB_IMAGE to AXI stream
and Zynq
TRD

FPGA Design with VIVADO High Level Synthesis

HLS Design Flow
2. Designing Accelerator from HLS Design
i. Synthesizing HLS Design
ii. Creating Testbench and Simulating HLS Design with Testbench on C/C++
iii. Export HLS Design into IP XACT format which is supported by VIVADO IP integrator.

FPGA Design with VIVADO High Level Synthesis

Working Module of Sobel In Vivado HLS

(Saha Souvik, 2017)

FPGA Design with VIVADO High Level Synthesis
1. Implementing Canny Edge Detection in C/C++
OpenCV with HLS: 5 Stage Algorithm
// AXI to RGB_IMAGE stream
hls::AXIvideo2Mat( input, src );
// Grayscaling
hls::CvtColor<HLS_RGB2GRAY>(
src, src_bw );
// Gaussian Blur Noise Reduction
hls::GaussianBlur<5,5>( src_bw,
src_blur, 1.4, 1.4 );
// Calculate gradients in x and
y direction using Sobel filter
hls::Sobel<1,0,3>( src1,sobel_g
);
// Calculate gradient magnitude
and direction

FPGA Design with VIVADO High Level Synthesis

 Sobel Edge Detection Algorithm on HLS

C Programming Code Snippet

-Library File
-Main Program

//the X order is 1 & Y order is 0

//so the X image
//Derivative will calculated.

//ksize=3, which is 3x3 kernel

hls::Sobel<1,0,3>(img1, img2);
Sobel Edge Detection Algorithm Testbench

C Programming Testbench
For VIVADO HLS
• So, Here is the output of the
sobel Edge Detection
algorithm.
• We can observe from the
image that this algorithm
identifies edges and most of
the false edges generated
from the noise are removed
Figure : HLS Output of Sobel Edge Detection Algorithm
from the image.

FPGA Design with VIVADO High Level Synthesis

HLS Optimization Methodology

FPGA Design with VIVADO High Level Synthesis

 Pragma Directives: Optimization Methodology
• For instance, the inner loops for computing the Sobel gradients and absolute difference can be fully
unrolled because there is no data dependencies between each computation.
• Unrolling a loop can be enabled by inserting the #HLS pragma unroll directive. By using loop
unrolling will the tool allocate more resource for multiple operators to achive parallelism.
• By contrast, the higher level loops can be optimized by using the #HLS pragma pipeline directive.
Unlike loop unrolling, pipelining will induce operator resource sharing through every iteration of the
higher level loop.
• In addition, Vivado HLS also provides the option to collapse nested loops to a single loop to reduce
latency and improve resource sharing. This can be achieved by inserting the
#HLS pragma loop_flatten directive.
• Moreover, sequential function calls in a top level function can be automatically pipelined by using
the #HLS pragma dataflow. It can potentially reduce the latency between execution of functions
and improve total throughput if there are multiple function calls in the top level function.

FPGA Design with VIVADO High Level Synthesis

HLS Design References:
Thesis:
1. Implementation Of Canny Edge Detection Using HLS(HIGH LEVEL SYNTHESIS), Darpan Daru ,2016
2. Hardware Acceleration of Computer Vision Application, Soran Al-Naqshbandi, 2016
3. Rapid Prototyping of an FPGA-Based Video Processing System, Zhun Shi , 2016
4. Gaussian Based Sobel Edge Detection Using SOC(SYSTEM ON CHIP), Souvik Saha,2016

Papers:
1. High Level Synthesis of Canny Edge Detection Algorithm on Zynq Platform, Hanaa M. Abdelgawad,

FPGA Design with VIVADO High Level Synthesis

Mona Safar, Ayman M. Wahba, 2015
2. A Survey and Evaluation of FPGA High-Level Synthesis Tools, Razvan Nane,2015

Xilinx Application Notes

1. Xapp1167
2. Xapp890
3. Zynq TRD
Other Reference Design
1. Logic Brick Zynq TRD Ported to ZedBoard
Websites
1. http://www.wiki.xilinx.com/HLS+Video+Library
Key Documents
For the most current links to Vivado High-Level Synthesis resources, use the
Design Hub View in Vivado Document Navigator and select "High-Level
Synthesis".

Name Description

UG1197 UltraFast High-Level

Productivity Design Methodology Methodology guide
Guide

WP416 Vivado Design Suite Vivado Design Suite Backgrounder

High-Level Synthesis Tutorial

UG871 Vivado Design Suite Tutorial

UG902 Vivado Design Suite User High-Level Synthesis User Guide

Guide

UG958 Vivado Design Suite

Model-based DSP Design using System Generator
Reference Guide

FPGA Design with VIVADO High Level Synthesis

Application Notes

XAPP599 Floating Point Design with Vivado HLS

XAPP745 Processor Control of Vivado HLS Designs

Implementing Memory Structures for Video

XAPP793
Processing

Zynq All Programmable SoC Sobel Filter

XAPP890
Implementation

Accelerating OpenCV Applications with Zynq-7000

XAPP1167
AP SoC using Vivado HLS Video Libraries

FPGA Design with VIVADO High Level Synthesis

Thank You
FPGA Design with VIVADO High Level Synthesis