Lab Workbook 错误!

使用“开始”选项卡将 Heading 1 应用于要在此处显示的文字。

AI Quantizer and AI Compiler – TensorFlow2 and

PyTorch
Vitis AI Development Environment 2.5

Abstract
This lab illustrates the Vitis™ AI quantizer and Vitis AI compiler flow for the TensorFlow2 and
PyTorch frameworks. You will explore the required options for these tools and then run the
tools.
This lab should take approximately 45-60 minutes.
CloudShare users: You may face some delays in running this lab depending on your region. This
lab is supported only from the Amsterdam region—other regions are not supported.

CloudShare Users Only

You are provided with three attempts to access a lab, and the time allotted to complete each lab
is twice the time expected to complete the lab. Once the timer starts, you cannot pause the
timer. Each lab attempt will reset the previous attempt—that is, your work from a previous
attempt is not saved.

Objectives
After completing this lab, you will be able to:
• Describe the AI quantizer flow and AI compiler flow for the TensorFlow2 and PyTorch
frameworks
• Quantize a pre-trained model
• Compile the quantized model for the DPUCZDX8G architecture
• Verify the files generated after quantization and compilation for the TensorFlow 2 and
PyTorch model networks

Introduction
The Vitis AI toolchain provides an innovative workflow to deploy deep learning inference
applications on the DPU with the following four steps:
• Quantize the neural network model
• Compile the neural network model
• Program with the Vitis AI programming interface
• Run and evaluate the deployed DPU application

www.amd.com 1
© Copyright 2022 Advanced Micro Devices, Inc.
错误!使用“开始”选项卡将 Heading 1 应用于要在此处显示的文字。 Lab Workbook

This lab focuses on quantization and compilation.

Model Quantization
The Vitis AI quantizer supports the TensorFlow (both 1.x and 2.x) and PyTorch frameworks. The
Vitis AI quantizers are named vai_q_<framework name>.
Post-training quantization (PTQ) requires only a small set of unlabeled images to analyze the
distribution of activations. The running time of quantized calibration varies from a few seconds
to several minutes, depending on the size of the neural network.
Generally, there is some drop in accuracy after quantization. However, for some networks (such
as MobileNet), the accuracy loss might be large. In this situation, quantization-aware training
(QAT) can be leveraged to improve the accuracy of the quantized models.
QAT requires the original training dataset. Several epochs of finetuning are needed and the time
varies from several minutes to several hours. It is recommended to use small learning rates when
performing QAT.
For PTQ, the cross-layer equalization algorithm is implemented. Cross-layer equalization can
improve calibration performance, especially for networks including depth-wise convolution.
Note: For the Vitis AI development environment 1.4 onwards, the term "quantize calibration" is
replaced with "post-training quantization", and "quantize finetuning" is replaced with
"quantization-aware training."

Frameworks Versions Post-Training Quantization- Fast Finetuning

Quantization Aware Training (Advanced
(PTQ) (QAT) Calibration)

TensorFlow 1.x Based on 1.15 Yes Yes No

TensorFlow 2.x Supports 2.3 Yes Yes Yes

PyTorch Supports 1.2 – 1.9 Yes Yes Yes

Compiling
The Vitis AI compiler provides a unified interface to all the necessary tools for developing DPUs.
Each compiler maps a network model into a highly optimized DPU instruction sequence.
The XIR-based compiler takes the quantized TensorFlow or PyTorch model as the input. The
TensorFlow model is transformed to XIR format by xcompiler. However, for PyTorch, the
quantizer will output an XIR format xmodel.
First, it transforms the input models into the XIR format as the foundation of the following
processes. Most of the variations among different frameworks are eliminated and transferred to
a unified representation in XIR.
The compiler then applies various optimizations to the graph and breaks up the graph into
several subgraphs based on whether the operation can be executed on the DPU. More

2 www.amd.com
© Copyright 2022 Advanced Micro Devices, Inc.
Lab Workbook 错误!使用“开始”选项卡将 Heading 1 应用于要在此处显示的文字。

architecture-aware optimizations are applied for each subgraph, as required. For the DPU
subgraph, the compiler generates the instruction stream and attaches to it.
Finally, the optimized graph with the necessary information and instructions for VART is
serialized into a compiled xmodel file.

DPU Name Hardware Platform

DPUCZDX8G Zynq® UltraScale+™ MPSoC

DPUCAHX8H Alveo™ U50LV, U55C Data Center accelerator cards

DPUCADF8H Alveo U200, U250 Data Center accelerator cards

DPUCVDX8G Versal® ACAP VCK190 (Versal AI Core Series) evaluation kit

DPUCVDX8H Versal ACAP VCK5000 evaluation kit

You can find the arch.json files for those platforms in /opt/vitis_ai/compiler/arch.
You will focus on the AI quantizer and AI compiler for TensorFlow2 and PyTorch in this lab.
Understanding the Lab Environment
The labs and demos provided in this course are designed to run on a Linux platform.
One environment variable is required: TRAINING_PATH, which points to where the lab files are
located. This variable comes configured in the CloudShare/CustEd_VM environments.
Some tools can use this environment variable directly (that is, $TRAINING_PATH will be
expanded), and some tools require manual expansion (/home/xilinx/training for the
CloudShare/CustEd_VM environments). The lab instructions describe what to do for each tool.
Both the Vivado Design Suite and the Vitis platform offer a Tcl environment that is used by
many labs. When the tool is launched, it starts with a clean Tcl environment with none of the
procs or variables remaining from a previous launch of the tools.
If you sourced a Tcl script or manually set any Tcl variables and you closed the tool, when you
reopen the tool, you will need to re-source the Tcl script and set any variables that the lab
requires. This is also true of terminal windows—any variable settings will be cleared when a new
terminal opens.

www.amd.com 3
© Copyright 2022 Advanced Micro Devices, Inc.
错误!使用“开始”选项卡将 Heading 1 应用于要在此处显示的文字。 Lab Workbook

Nomenclature
Formal nomenclature is used to explain how different arguments are used. The following are
some of the more commonly used symbols:

Symbol Description Example Explanation

<text> Indicates a cd <dir> <dir> represents the name of the

field directory. The < and > symbols are
NOT entered. If the directory to
change to is XYZ, then you would
enter cd XYZ into the environment.

[text] Indicates an ls [ | more] This could be interpreted as ls

optional <Enter> or ls | more <Enter>.
argument The first instance lists the files in
the current Linux directory, the
second lists the files in the current
Linux directory, but additionally
runs the output through the more
tool, which paginates the output.
Here, the pipe symbol (|) is a Linux
operator.

| Indicates cmd <ZCU104 | VCK190> The cmd command takes a single

choices argument, which could be ZCU104
OR VCK190. You would enter either
cmd ZCU104 or cmd VCK190.

General Flow
Step 1: Step 2: Step 3: Step 4:
Reviewing Running Reviewing Running
the TF2 the TF2 the PyTorch the PyTorch
Arguments & Quantizer & Arguments & Quantizer &
Scripts Compiler Scripts Compiler

4 www.amd.com
© Copyright 2022 Advanced Micro Devices, Inc.
Lab Workbook 错误!使用“开始”选项卡将 Heading 1 应用于要在此处显示的文字。

Reviewing the TensorFlow2 AI Quantizer Arguments and Training

Scripts Step 1
Here is a high-level outline of the overall model quantization flow:

Figure 1-1: Vitis AI Quantizer Flow

The Vitis AI quantizer takes a floating-point model as input, performs

pre-processing (folding batch norms and removing nodes not required for
inference), and then quantizes the weights/biases and activations to the given bit
width.
The quantizer runs several iterations to capture activation statistics and improve
the accuracy of quantized models. These statistics are used to calibrate the
activations. A calibration image dataset input is required for this process.
Generally, the quantizer works well with 100–1000 calibration images. This is
because there is no need for back propagation; the unlabeled dataset is
sufficient.
After calibration, the quantized model is transformed into a DPU-deployable
model (named deploy_model.pb for vai_q_tensorflow, model_name.h5
for vai_q_tensorflow2, or model_name.xmodel for vai_q_pytorch),
which follows the data format for a DPU.
This model can then be compiled by the Vitis AI compiler and deployed to the
DPU. The quantized model cannot be taken in by the standard version of the
TensorFlow or PyTorch framework.

www.amd.com 5
© Copyright 2022 Advanced Micro Devices, Inc.
错误!使用“开始”选项卡将 Heading 1 应用于要在此处显示的文字。 Lab Workbook

1-1. Copy the lab files to the docker location.

1-1-1. Click the Terminal icon( ) in the toolbar or press <Ctrl + Alt + T> to open a Linux
terminal window.
1-1-2. Enter the following command to copy the files to the specified directory:
[host]$ cp -rf $TRAINING_PATH/vai_q_c_tf2_pt /home/xilinx/Vitis-AI/
vai_q_c_tf2_pt
Hint: If you are using the CustEd VM or CloudShare, the tilde symbol ('~') represents
/home/xilinx. You can use this as a shortcut to reduce your typing.
You will be using the following scripts and text file for running the TF2 quantizer and
compiler.
o 1_tf2_quantize.sh
▪ This script calls the vitis-ai quantizer for TensorFlow2.
o 4_tf2_compile_for_mpsoc.sh
▪ This script calls the vitis-ai compiler for TensorFlow2.
o val.txt
▪ This is the evaluation image list file used during quantization.

1-2. Review the 1_tf2_quantize.sh script.

1-2-1. Enter the following command to review the quantizer arguments in the quantizer script:
[host]$ gedit /home/xilinx/Vitis-AI/vai_q_c_tf2_pt/lab/
1_tf2_quantize.sh
Note: The train_eval_h5.py Python script quantizes the trained model. This script is
typically created by the developer for training the model but is provided to you here as
an example.
1-2-2. Compare the arguments from the script to the meanings of the arguments provided
here:

Quantizer Argument Description

model Specifies the TensorFlow2 floating-point network model

h5 file

quantize true enables quantization

quantize_output_dir Generated output location

eval_only true signifies evaluation only

eval_images Set it to true for evaluation images

6 www.amd.com
© Copyright 2022 Advanced Micro Devices, Inc.
Lab Workbook 错误!使用“开始”选项卡将 Heading 1 应用于要在此处显示的文字。

Quantizer Argument Description

eval_images_path Location to the evaluation images directory

eval_images_list Images list

label_offset Set to 1

gpus Selects GPU or CPU

1-2-3. After completing your review, press <Ctrl + Q> to close the editor.

You will be using the tf2_resnet50_imagenet_224_224_7.76G_2.5 pre-trained

model from the Vitis AI Model Zoo
(https://github.com/Xilinx/Vitis-AI/tree/master/model_zoo).
The pre-trained model is provided for you in the /home/xilinx/Vitis-AI/
vai_q_c_tf2_pt/lab directory.
Note: Typically, you would visit the Xilinx GitHub
(https://github.com/Xilinx/Vitis-AI/tree/master/model_zoo/model-list) and navigate to
the model that you want to use. Within that branch is a YAML file that contains various
download links supporting multiple platforms. You would then download that image and
continue.
You will find the pre-trained floating model under the directory
tf2_resnet50_imagenet_224_224_7.76G_2.5 > float.

Field Name Field Value Meaning

Model resnet50 Selects ResNet50 architecture

Framework tf2 TensorFlow2

Input size 224_224 Height x width in pixels

OPS per image 7.76G Number of operations per second

Training set ImageNet training Dataset used for training the model

Validation set ImageNet validation Dataset used for verifying the model

Vitis AI tool version 2.5 Release version

Images and labels from ImageNet have been downloaded and placed in the
vai_q_c_tf2_pt/images directory so that calibration and evaluation during the
quantization phase can be conducted.

www.amd.com 7
© Copyright 2022 Advanced Micro Devices, Inc.
错误!使用“开始”选项卡将 Heading 1 应用于要在此处显示的文字。 Lab Workbook

1-3. Review the train_eval_h5.py training script.

Usually, this script comes from the developer for training the model. This
script is specific to the model—in this case, it is for ResNet50. The user will
need to import the Vitis AI quantizer.
1-3-1. Enter the following command to review the training script that has been updated with
the Vitis AI quantizer:
[host]$ gedit /home/xilinx/Vitis-AI/vai_q_c_tf2_pt/lab/
tf2_resnet50_imagenet_224_224_7.76G_2.5/code/com/train_eval_h5.py
o Note the variable declarations beginning with flags.DEFINE_string on or about
line 34:
▪ flags.DEFINE_string(<NAME>, <VALUE>, <DESCRIPTION>)
▪ flags.DEFINE_bool(<NAME>, <True/False>, <DESCRIPTION>)
▪ flags.DEFINE_integer(<NAME>, <INTEGER>, <DESCRIPTION>)
▪ flags.DEFINE_float(<NAME>, <FLOAT>, <DESCRIPTION>)
o The function get_input_data() reads the input data and returns the training data
(train_data) and evaluation data (eval_data):
▪ def get_input_data(num_epochs=1):
o The function main() represents the starting point into this program (on or near line
112)—following the logic:
▪ If save_whole_model is True, then the file just includes weights and the whole
model is saved to an h5 file.
▪ At near line no. 119, if eval_images is set to False, it will read the input data.
eval_images is set to True in the 1_tf2_quantize.sh script.
▪ At near line no. 127, the model variable is updated with the weights of the
trained model, which is passed (resnet50.h5) in the 1_tf2_quantize.sh
script.
▪ At near line no. 129, read the images and labels and store in the img_paths and
labels list.
▪ At near line no. 133, verify the quantize flag, which is set to True in the
1_tf2_quantize.sh script.
▪ From line no. 135 to 141, the user will need to include in their training script:
➢ Importing the Vitis AI quantizer
➢ Calling the Vitis AI quantizer by passing the trained model and calibration
dataset
➢ Saving the quantized model
1-3-2. After completing your review, press <Ctrl + Q> to close the editor.

8 www.amd.com
© Copyright 2022 Advanced Micro Devices, Inc.
Lab Workbook 错误!使用“开始”选项卡将 Heading 1 应用于要在此处显示的文字。

Running the AI Quantizer and AI Compiler for TensorFlow2 Step 2

You will now run the Vitis AI quantizer for TensorFlow2 using the training script,
verify the generated output, and then run the Vitis AI compiler
(vai_c_tensorflow2) for TensorFlow2.
Certain CPUs cannot run TensorFlow2 as this framework uses the avx flag.
If you are not running in the CloudShare environment, you must confirm that
your CPU can run TensorFlow by performing the verification shown below.

2-1. [All users other than CloudShare] Verify if your machine supports the avx
flag.
2-1-1. [Optional] Press <Ctrl + Alt + T> to open a Linux terminal window.
2-1-2. Enter the following command (1):
[host] # lscpu
Alternatively, you can enter:
lscpu | grep -c "avx"
This counts the number of times avx is exactly matched in the output of the lscpu
command. As long as it returns at least one match, then you can continue.
2-1-3. Locate the Flags field at the end of the report (2).

If you see avx listed among the flags, then your processor supports TensorFlow2 (3).

Figure 1-2: Locaing the avx Flag

Note: If your machine does NOT support this flag, you will see an "Illegal
instruction (core dumped)" message.
If your machine cannot run TensorFlow2, skip to the "Reviewing the PyTorch AI Quantizer
Arguments and Training Scripts" step, where you can continue with PyTorch.
2-1-4. If you opened a new terminal window to run this test, enter the following to close this
terminal window:
exit

www.amd.com 9
© Copyright 2022 Advanced Micro Devices, Inc.
错误!使用“开始”选项卡将 Heading 1 应用于要在此处显示的文字。 Lab Workbook

2-2. Load the CPU image from the Docker hub.

2-2-1. If a terminal window is not available, press <Ctrl + Alt + T> to open a new terminal
window.
2-2-2. Enter the following commands to load the CPU image from the Docker hub:
[host]$ cd ~/Vitis-AI
[host]$ sudo ./docker_run.sh xilinx/vitis-ai-cpu:2.5

Note that this approach specifically loads Vitis AI environment 2.5. This lab is tested against
this version and *MAY* work with other versions. When developing your own projects, you
can use the latest tag to pull the most recent version of the tools.
Supported tags can be found at
https://hub.docker.com/r/xilinx/vitis-ai-cpu/tags?page=1&ordering=last_updated.
For example: [host]$ ./docker_run.sh xilinx/vitis-ai-cpu:<Docker_tag>
Selecting the latest version: [host]$ ./docker_run.sh
xilinx/vitis-ai-cpu:latest

Note: Accept the terms and agreements by pressing any key until you are told to
specifically click the 'y' key to accept the final condition.

Figure 1-3: Vitis AI Docker Opening View

It is worthwhile noting that the environment uses its own directory hierarchy, which is
now /workspace.

10 www.amd.com
© Copyright 2022 Advanced Micro Devices, Inc.
Lab Workbook 错误!使用“开始”选项卡将 Heading 1 应用于要在此处显示的文字。

2-3. Activate the TensorFlow2 conda environment.

2-3-1. Enter the following command to activate the TensorFlow2 conda environment:

Vitis-AI /workspace > conda activate vitis-ai-tensorflow2

Notice the change to the prompt:
(vitis-ai-tensorflow2) Vitis-AI /workspace >
The Vitis AI environment uses its own hierarchical workspace, and both the framework
and path are shown as part of the prompt.
2-3-2. Change the directory to the lab subdirectory:
(vitis-ai-tensorflow2) Vitis-AI /workspace > cd vai_q_c_tf2_pt/lab

Note: From this point forward, the rather lengthy prompt will be shown as [VAI]>.

2-3-3. Run the Vitis AI quantizer for Tensorflow2:

[VAI]> sh 1_tf2_quantize.sh
The quantizer tool now starts running. The execution time usually depends on the
calibration dataset and machine performance.
It should take approximately 3 minutes if you are using the VM. If you are using
CloudShare, it will take approximately 5-8 minutes.
After the quantization process completes, you should see messages similar to the
following.

Figure 1-4: TensorFlow2 Quantization Completion

Note: You may see warnings about not being able to load dynamic libraries. These
messages relate to CUDA drivers that are used in GPU implementations and can be
safely ignored here.
After successful execution of the quantizer script, the quantized file quantized.h5 is
generated in the output directory
(.../tf2_resnet50_imagenet_224_224_7.76G_2.5/vai_q_output).
For TensorFlow2.x, the quantizer generates the quantized model in the h5 format.

www.amd.com 11
© Copyright 2022 Advanced Micro Devices, Inc.
错误!使用“开始”选项卡将 Heading 1 应用于要在此处显示的文字。 Lab Workbook

2-4. Review the 4_tf2_compile_for_mpsoc.sh script.

2-4-1. Since the current terminal is running the Vitis AI environment, press <Ctrl + Alt + T> to
open a new terminal window.
2-4-2. Enter the following command to review the compiler arguments in the compiler script:
[host]$ gedit /home/xilinx/Vitis-AI/vai_q_c_tf2_pt/lab/
4_tf2_compile_for_mpsoc.sh
The detailed commands used in the script are as shown below.
vai_c_tensorflow2 -m ${TF2_NETWORK_PATH}/vai_q_output/quantized.h5 \
-a
/opt/vitis_ai/compiler/arch/DPUCZDX8G/ZCU102/arch.json \
-o ${TF2_NETWORK_PATH}/vai_c_output \
-n resnet50_tf2 \
The Vitis AI compiler for the Tensorflow2 arguments are listed here:

Quantizer Argument Description

-m MODEL h5 model file

-a ARCH Architecture JSON file

-o OUTPUT_DIR Output directory location to store the generated output

-n NET_NAME Prefix-name for the outputs

-e OPTIONS Extra options

2-4-3. After completing your review, press <Ctrl + Q> to close the editor.
2-4-4. [Optional] Enter exit in the terminal window to close the terminal.

2-5. Run the AI compiler tool by executing the 4_tf2_compile_for_mpsoc.sh

script.
2-5-1. Return to the terminal running the Vitis AI environment.
2-5-2. Enter the following command to run the Vitis AI compiler for TensorFlow2 (1):
[VAI] > sh 4_tf2_compile_for_mpsoc.sh
When a network model is compiled, the required options should be specified to the Vitis
AI compiler. Once compilation is successful, the Vitis AI compiler will generate the
xmodel file. These files are located under the folder specified by output_dir.

12 www.amd.com
© Copyright 2022 Advanced Micro Devices, Inc.
Lab Workbook 错误!使用“开始”选项卡将 Heading 1 应用于要在此处显示的文字。

You should see the following output after compilation has completed.

Figure 1-5: Results of TensorFlow2 Compilation

The compiler creates three files in the OUTPUT_DIR directory (2):

o Compiled xmodel: resnet50_tf2.xmodel
o For run time: meta.json
o Checksum: md5sum.txt
The compiled xmodel is used to program the DPU.
2-5-3. Enter the following command to only exit the Vitis AI tool:
[VAI] > exit
Note: If your terminal is unresponsive after exiting the Vitis AI tool, clicking the "X" in the
upper-right corner to close the terminal window. If this is your situation, you will
instructed to open a new terminal window in the next step.

www.amd.com 13
© Copyright 2022 Advanced Micro Devices, Inc.
错误!使用“开始”选项卡将 Heading 1 应用于要在此处显示的文字。 Lab Workbook

Reviewing the PyTorch AI Quantizer Arguments and Training

Scripts Step 3
You will now review the scripts available to you for running the quantizer for
PyTorch.

3-1. Review the 5_pytorch_quantize.sh script.

3-1-1. If you closed your terminal window, press <Ctrl + Alt + T> to open a new terminal
window.
3-1-2. Enter the following command to review the quantizer arguments in the quantizer script:
[host]$ gedit /home/xilinx/Vitis-AI/vai_q_c_tf2_pt/lab/
5_pytorch_quantize.sh
The detailed commands used in the script are as shown below.
python resnet50_quantize.py --quant_mode calib
python resnet50_quantize.py --quant_mode test
Note: The script reference to resnet50_quantize.py is used to quantize the trained
model. Usually, this script comes from the trainer who has created it for training the
model.
The argument --quant_mode is set to calib for quantization, and after calibration,
--quant_mode is set to test to evaluate the quantized model.
3-1-3. After completing your review, press <Ctrl + Q> to close the editor.

3-2. Review the resnet50_quantize.py training script.

Usually, this script comes from the trainer who has created it for training the
model. This script is specific to the model—in this case, it is for ResNet50.
The user will need to import the Vitis AI quantizer.
3-2-1. Enter the following command to review the training script that has been updated with
the Vitis AI quantizer:
[host]$ gedit /home/xilinx/Vitis-AI/vai_q_c_tf2_pt/lab/
resnet50_quantize.py
o PyTorch quantize is imported near line 56.
o The script checks to see if a GPU is available on your system. If GPU is not found,
then the CPU is selected, and an informative message is displayed (lines 64-68).

14 www.amd.com
© Copyright 2022 Advanced Micro Devices, Inc.
Lab Workbook 错误!使用“开始”选项卡将 Heading 1 应用于要在此处显示的文字。

o The argument list is parsed, and the necessary information is extracted in the lines
following the call to argparse.ArgumentParser():
▪ --data_dir: Dataset directory, when quant_mode=calib, it is for calibration,
and when quant_mode=test, it is for evaluation.
▪ --model_dir: Trained model file path.
▪ --subset_len: subset_len to evaluate the model, using the whole
validation dataset if it is not set.
▪ --quant_mode: Quantization mode {float, calib, test}:
➢ float: No quantization, evaluate float model.
➢ calib: Quantize.
➢ test: Evaluate the quantized model.
▪ --finetune: Finetune the model before calibration.
o The main() program starts at line no. 293:
▪ At near line no. 295, the model name is set to resnet50.
▪ At near line no. 299, verify the quant_mode argument that is passed as calib in
the 5_pytorch_quantize.sh script.
▪ At near line no. 311, the function quantization has been called with the
following arguments:
➢ title: resnet50 with optimization
➢ model_name: resnet50
➢ file_path: pt_resnet50.pth
➢ quant_mode: calib
➢ finetune: Set to default
3-2-2. After completing your review, press <Ctrl + Q> to close the editor.

www.amd.com 15
© Copyright 2022 Advanced Micro Devices, Inc.
错误!使用“开始”选项卡将 Heading 1 应用于要在此处显示的文字。 Lab Workbook

Running the AI Quantizer and AI Compiler for PyTorch Step 4

You will now run the Vitis AI quantizer for PyTorch using the training script, verify
the generated output, and then run the Vitis AI compiler (vai_c_xir) for
PyTorch.

4-1. Load the CPU image from the Docker hub.

4-1-1. If a terminal window is not available, press <Ctrl + Alt + T> to open a new terminal
window.
4-1-2. Enter the following commands to load the CPU image from the Docker hub:
[host]$ cd ~/Vitis-AI
[host]$ sudo ./docker_run.sh xilinx/vitis-ai-cpu:2.5

Note that this approach specifically loads Vitis AI environment 2.5. This lab is tested against
this version and *MAY* work with other versions. When developing your own projects, you
can use the latest tag to pull the most recent version of the tools.
Supported tags can be found at
https://hub.docker.com/r/xilinx/vitis-ai-cpu/tags?page=1&ordering=last_updated.
For example: [host]$ ./docker_run.sh xilinx/vitis-ai-cpu:<Docker_tag>
Selecting the latest version: [host]$ ./docker_run.sh
xilinx/vitis-ai-cpu:latest

Note: Accept the terms and agreements by pressing any key until you are told to
specifically click the 'y' key to accept the final condition.

Figure 1-6: Vitis AI Docker Opening View

It is worthwhile noting that the environment uses its own directory hierarchy, which is
now /workspace.

16 www.amd.com
© Copyright 2022 Advanced Micro Devices, Inc.
Lab Workbook 错误!使用“开始”选项卡将 Heading 1 应用于要在此处显示的文字。

4-2. Activate the PyTorch conda environment.

4-2-1. Enter the following command to activate the PyTorch conda environment:

Vitis-AI /workspace > conda activate vitis-ai-pytorch

The terminal output should be similar to the following:
(vitis-ai-pytorch) Vitis-AI /workspace >
The Vitis AI environment uses its own hierarchical workspace, and both the framework
and path are shown as part of the prompt.
Note: From this point forward, the rather lengthy prompt will be shown as [VAI]>.

4-2-2. Change the directory to the lab subdirectory:

[VAI] > cd vai_q_c_tf2_pt/lab
4-2-3. Run the Vitis AI quantizer script for PyTorch:
[VAI] > sh 5_pytorch_quantize.sh
The quantizer tool now starts running. The execution time usually depends on the
calibration dataset and machine performance and whether there is a GPU available.
It will take approximately 4 minutes to complete if you are using the VM. If you are using
CloudShare, it should take approximately 5-6 minutes.
After the quantization process completes, you should see messages similar to the
following.

Figure 1-7: PyTorch Quantizer Results

Two things to note:

o A GPU was not found, so the code is using the CPU (1).
o A successful conversion completion message is displayed (2).

www.amd.com 17
© Copyright 2022 Advanced Micro Devices, Inc.
错误!使用“开始”选项卡将 Heading 1 应用于要在此处显示的文字。 Lab Workbook

After successful execution of the quantizer script, the quantized file

ResNet_int.xmodel is generated in the output directory
(/home/xilinx/Vitis-AI/vai_q_c_tf2_pt/pt_resnet50/vai_q_output).
For PyTorch, the quantizer NNDCT outputs the quantized model in the XIR format
directly. Use vai_c_xir to compile it.

4-3. Review the 6_pytorch_compile_for_mpsoc.sh script.

4-3-1. Since the current terminal is running the Vitis AI environment, press <Ctrl + Alt + T> to
open a new terminal window.
4-3-2. Enter the following command to review the compiler arguments in the compiler script:
[host]$ gedit /home/xilinx/Vitis-AI/vai_q_c_tf2_pt/lab/
6_pytorch_compile_for_mpsoc.sh
The detailed commands used in the script are as shown below.
#!/bin/sh
TARGET=MPSOC
NET_NAME=resnet50
DEPLOY_MODEL_PATH=vai_q_output
ARCH=/opt/vitis_ai/compiler/arch/DPUCZDX8G/ZCU102/arch.json
vai_c_xir -x pt_resnet50/vai_q_output/ResNet_int.xmodel \
-o pt_resnet50/vai_c_output/${TARGET} \
-n pt_resnet50 \
-a ${ARCH}
The Vitis AI compiler for the PyTorch arguments are listed here:

Quantizer Argument Description

-x MODEL xmodel file

-o OUTPUT_DIR Output directory location to store the generated output

-n NET_NAME Prefix name for the outputs

-a ARCH Architecture JSON file

-e OPTIONS Extra options

4-3-3. After completing your review, press <Ctrl + Q> to close the editor.
4-3-4. [Optional] Enter exit in the terminal window to close the terminal.

18 www.amd.com
© Copyright 2022 Advanced Micro Devices, Inc.
Lab Workbook 错误!使用“开始”选项卡将 Heading 1 应用于要在此处显示的文字。

4-4. Run the AI compiler tool by executing the

6_pytorch_compile_for_mpsoc.sh script.
4-4-1. Return to the terminal running the Vitis AI environment.
4-4-2. Enter the following command to run the Vitis AI compiler for PyTorch (1):
[VAI] > sh 6_pytorch_compile_for_mpsoc.sh
When a network model is compiled, the required options should be specified to the Vitis
AI compiler. Once compilation is successful, the Vitis AI compiler will generate the
xmodel file. These files are located under the folder specified by output_dir.
You should see the following output after compilation has completed:

Figure 1-8: PyTorch Compilation Results

The compiler creates three files in the OUTPUTPATH directory (2):

o Compiled xmodel: pt_resnet50.xmodel
o For run time: meta.json
o Checksum: md5sum.txt
The compiled xmodel is used to program the DPU.
4-4-3. Enter the following command to exit the Vitis AI tool:
[VAI] > exit
4-4-4. Enter exit in any open terminal window to close the terminal.

Summary
You just completed exploring the process of quantizing, compiling, and verifying TensorFlow2
and PyTorch models. These are mandatory steps in the development process of deploying
trained models.
Other labs will continue the development and deployment process.

www.amd.com 19
© Copyright 2022 Advanced Micro Devices, Inc.