Developing Video Applications on Spartan3A DSP FPGAs

Video Starter Kit Tutorial Lab 3

Create a Simulink Edge Detector

Video Starter Kit Lab

April 15, 2008

www.xilinx.com

Build Video Application

This Page Intentionally Left Blank

Developoing Video Applications on Xilinx FPGAs

Page 2 of 13

April 23rd, 2008

Build Video Application

VSK Tutorial Lab D Overview

In this part of the tutorial you will learn how a high-level Simulink executable specification can be used with
model-based design techniques to create an FPGA hardware design. A series of design will be created that
demonstrate how a frame based Simulink design is serialized and turned into a streaming design suitable for
implementation in an FPGA.
The objective of this lab is to provide an introduction to the Xilinx Simulink to FPGA design methodology. For
more detailed training on System and the Xilinx DSP Blockset please visit the Xilinx e-learning web site and
check out the free System Generator e-learning training class at:
http://www.xilinx.com/support/training/free-courses.htm

Objectives

Introduce users to the model based design flow for video applications targeting Xilinx FPGAs
Show how frame based executable specifications can be used for serial streaming hardware designs
Introduce users to Xilinx System Generator for DSP

Developoing Video Applications on Xilinx FPGAs

Page 3 of 13

April 23rd, 2008

Build Video Application

Part 1: Create a Simulink Simplified Executable Specification

Procedure
1.

Copy the file edge_detection_of_gamma_image.mdl from the Lab2 folder into the Lab3 folder and
rename this file edge_detect_hw_arch_def.mdl.

2.

Launch Simulink and set the working directory to the Lab3 folder and open the edge_detect_hw_arch.mdl
file. Re-simulate the design to make sure it still functions correctly.
Note: In this part of the lab we need to convert the image frame of data into a serial pixel stream that enters
the hardware and then take the serial stream output from the hardware and build back up a frame of data that
can be used by the abstract Simulink Imaging and Video Blockset

3.

Add the following blocks to the Simulink diagram

-

4.

Signal Processing Blockset/Math Functions/Matrices and Linear Algebra/Matrix Operations/Transpose

Signal Processing Blockset/Signal Management/Signal Attributes/Convert 2D to 1D
Signal Processing Blockset/Signal Management/Signal Attributes/Frame Conversion
Signal Processing Blockset/Signal Management/Buffers/Unbuffer

Once added connect after the Color Space Converter subsystem as shown below:

Next we will convert the serial data stream back to a frame. Then we can verify the logic is working
correctly.

Developoing Video Applications on Xilinx FPGAs

Page 4 of 13

April 23rd, 2008

Build Video Application

5.

Add the following blocks to the Simulink diagram

-

Signal Processing Blockset/Math Functions/Matrices and Linear Algebra/Matrix Operations/Transpose

Signal Processing Blockset/Signal Management/Signal Attributes/Convert 1D to 2D
Signal Processing Blockset/Math Functions/Matrices and Linear Algebra/Matrix Operations/Sub Matrix
Signal Processing Blockset/Signal Management/Buffers/Buffer
Simulink/Signal Attributes/Rate Transition
Simulink/Commonly Used Blocks/Data Type Conversion
Simulink/Video and Image Blockset/Video Viewer Wrong directory and block

And connect the blocks as shown below.

6.

Push the four frames to serial blocks into a subsystem and rename the subsystem Frame to Serial. Then
push the four serial to frame blocks into a subsystem called Serial to Frame. Again this will help keep the
diagram manageable.

7.

Change the output datatype in the From Multimedia File block to uint8
We now need to setup some parameters on the second subsystem to specify the image size. It makes sense to
set these parameters as variables in the model properties call backs because they will be used in several
places.

Developoing Video Applications on Xilinx FPGAs

Page 5 of 13

April 23rd, 2008

Build Video Application

8.

Execute the pulldown menu File -> Model Properties and set the following 3 variables in the InitFcn field
of the model callbacks. These variables will be set to the workspace and we can reference them in the
properties fields of the Simulink blocks. Our frame size is 240 rows by 360 columns executes at 15 frames
per second. Once done OK the form.

9.

Push down into the second subsystem that re-creates the image and double click on the Rate Transition
block and set the Output Port Sample Time to 1/(row*col*fps).

Note: Inclusion of the Rate Transition block is technically not necessary for this exercise as the output
sample times will be correctly calculated by the 4 remaining blocks. However this block is needed later for a
lab that involves hardware co-simulation. It doesnt hurt to put this in now and then we have a serial-toframe conversion subsystem that will support all our future development efforts.
10. Double click on the buffer block and set the following parameter. The buffer block needs to know how big
a frame is. Once done OK for form.

Developoing Video Applications on Xilinx FPGAs

Page 6 of 13

April 23rd, 2008

Build Video Application

11. Double-click on the Convert from 1D to 2D block and set the following parameters. Simulink needs to know
the size of the matrix you wish to create. Once done OK the form.

12. Once done simulate the design you should see a black and white version of image because we only have the
red video stream connected. We have demonstrated that we have successfully converted the image into a
streaming format then converted the streaming image back into the original frame.

We are now ready to begin architecting a Sobel Edge Detector on streaming data!

Developoing Video Applications on Xilinx FPGAs

Page 7 of 13

April 23rd, 2008

Build Video Application

Part 3: Define a hardware Architecture using Simulink

In this part of the exercise we are going to create a streaming model of our color space converter and edge
detector that represents the hardware architecture we wish to use.
The Y output of an RGB to YCrCb color space converter can be implemented using the following equation:
Y = 0.299R + 0.587G + 0.114B
The block diagram below shows the hardware architecture for a modified Sobel edge detector that does an
approximation on the X and Y gradient using rows of data but not columns. The filter coefficients are displayed
above each instance of the 3-tap FIR blocks.
Filter Coefficients for Sobel XY Edge Detector

For thresholding well compare the summation to the value 100 and if it is greater set the output to 255 (white) or
if is smaller set the output to 0 (black). This will give us clear edges for the gradients.

Procedure
1.

Add the following blocks to the diagram

Block
Embedded MATLAB Function
Integer Delay
Digital Filter
add
abs
Constant
Switch
Data Type Conversion

Library
Simulink/User-Defined Functions
Simulink/Discrete
Signal Processing Blockset/Filtering/Filter
Designs
Simulink/Math Operations
Simulink/Math Operations
Simulink/Sources
Simulink/Signal Routing
Simulink/Commonly Used Blocks

Developoing Video Applications on Xilinx FPGAs

Page 8 of 13

Qty
1
2
3
2
1
2
1
1

April 23rd, 2008

Build Video Application

2.

Double click on the Embedded MATLAB function and modify the MATLAB function as shown below: The
embedded MATLAB function block provides a simple way to implement the color space converter that we
need for this design.

3.

Connect the blocks as shown below:

4.

Double-click on the Integer Delay block and set the delay = col. Remember we set this as a Model
Properties Callback InitFcn variable. We will use this simple delay to model a line buffer

5.

Double-click on each of the 3 Digital Filters and set the coefficients to the corresponding values shown in the
block diagram at the beginning of Part 3.

6.

Set the constant values to 255 and 0

7.

Set the Switch Threshold field = 90.

8.

Set the Convert block to output double datatypes. This Simulink digital filter requires a double as input.

9.

Once done push all the logic for this edge detector into a subsystem this will help keep the top-level
diagram manageable

Developoing Video Applications on Xilinx FPGAs

Page 9 of 13

April 23rd, 2008

Build Video Application

The Edge Detector subsystem should be connected at the top level as shown below.

10. Set the simulation period to 1/fps or 1/15. The video source executes at 15 frames per second and for
each Simulink simulation cycle will process 15 frames of data. For hardware development purposes we can
limit this to one frame for now. Since we set the fps variable in the callbacks we can reference it here.

11. Once complete Simulate the design You will notice some slight differences between the Video and Imaging
Blockset Sobel Edge Detector and the hardware accurate model we created from Simulink this is primarily
due the specification of a lower threshold value. It is set for the lighting in the building rather then the
outdoor lighting of the video. The basic results should look acceptable. If you wish you can adjust the
threshold value specified in the switch block on the Simulink edge detector to get a more exact image.

Developoing Video Applications on Xilinx FPGAs

Page 10 of 13

April 23rd, 2008

Build Video Application

Part 4: Define a hardware Architecture using System Generator

In this part of the exercise we are going to design the exact same Edge Detector using System Generator and the
Xilinx DSP Blockset. This should be a straightforward process as System Generator has blocks that correspond to
the ones used in the Simulink model

Procedure
1.

Add the following blocks to the Diagram

Block
System Generator
Gateway In
Gateway Out
FIR Compiler 4.0
AddSub
Constant
Relational
Mux
Delay
Cmult
Convert

Library
Xilinx Blockset/Index
Xilinx Blockset/Index
Xilinx Blockset/Index
Xilinx Blockset/Index
Xilinx Blockset/Index
Xilinx Blockset/Index
Xilinx Blockset/Index
Xilinx Blockset/Index
Xilinx Blockset/Index
Xilinx Blockset/Index
XilinxBlockset/Index

Qty
1
1
1
3
5
3
1
1
3
3
1

2.

Open the library called lab3_lib.mdl and add two of the ABS blocks. The ABS block doesnt exist in the
Xilinx Blockset. An ABS block has been saved to a library to save time. This ABS block is actually provided
in one of the other VSK reference designs it is in the DVI_Pass_Through_Demo. If you wish you can get
it from there as well.

3.

First we will create the color space converter Wire up the design as shown in the diagram below.

4.

Double-click on the Convert block and set the output datatype to Unsigned [8 0]

5.

Set the constant values of the Cmult blocks to .299 for the R input, .587 for the G input and .114 for the B
input. Also set the latency equal to 1. You dont need to modify the fixed point values but you may if you
wish the default setting is provides more bits than are necessary. The Spartan-3A DSP supports arithmetic
operations up to 18 bits with a hard DSP48A slice.

Developoing Video Applications on Xilinx FPGAs

Page 11 of 13

April 23rd, 2008

Build Video Application

6.

Once done with the CSC push the logic into a subsystem (cntrl <g>) and connect the edge detector as
shown below

7.

Set the parameters on the AddSub and Delay blocks to have a latency of 1. This will force a pipeline register
into the block to improve performance.

8.

Double-click on the Gateway In blocks and set the following parameters:

Note: System Generator requires that the actual sample rate be specified on the Gateway In and System
Generator blocks. In this model a frame of data is being input to the system with a sample rate of 1/fps.
Because this Frame is decomposed into a vector of data that gets passed into System Generator than the
system generator design must run at a much higher rate it is row*col faster to be exact. This is
accomplished by setting the Sample Period field = 1/(row*col*fps). This is an important concept to
understand when using System Generator with the Video and Imaging Blockset
Developoing Video Applications on Xilinx FPGAs

Page 12 of 13

April 23rd, 2008

Build Video Application

9.

Set the Coefficients on each of the FIR Compiler 4.0 blocks to the same values as the Simulink Digital Filter
blocks. There 3 different coefficients. Below is an example of one of them..

10. On the FIR Compiler Implementation Tab select the Transpose_Multiply_Acculate architecture.

11. Set the constant blocks to the values of 0, 255 and 90

12. Set the System Generator Block Simulink system period to 1/(row*col*fps) as shown below:

13. Once the design is wired together and the block parameters are set push the System Generator blocks into a
subsystem as you did in the previous lab. Your diagram should look something like this:

14. Once done Simulate the design. If you get an error about continues states double check all the sample rate
settings. This means that two sample rates have come together somewhere in the Simulink diagram where
they are not supposed to. You should see results that are similar as before.

End of Lab 3

Developoing Video Applications on Xilinx FPGAs

Page 13 of 13

April 23rd, 2008