Vitis

Introduction

Xilinx Vitis is a platform for high-performance deep learning inference on Xilinx FPGA device.

Quantization Scheme

8bit per-tensor symmetric linear quantization with power of two scales.

For weights/biases:

\[\begin{equation} q = \mathtt{clamp}(\lfloor x * s \rceil, lb, ub) \end{equation}\]

For activations:

\[\begin{split}\begin{equation} \begin{aligned} q &= \mathtt{clamp}(\lceil x * s \rceil, lb, ub), \text{ where } x*s-\lfloor x*s\rfloor = 0.5 \text{ and } x < 0 \\ q &= \mathtt{clamp}(\lfloor x * s \rceil, lb, ub), \text{ else}. \end{aligned} \end{equation}\end{split}\]

where \(s\) is scaling factor to quantize a number from floating range to integer range, \(lb\) and \(ub\) are bounds of integer range, and [lb, ub] = [-128, 127].

Deploy on Vitis

Deployment:

We provide the example to deploy the quantized EOD model to Vitis, which is winner solution for the Low Power Computer Vision Challenge 2021 (LPCV2021).

  • First quantize model in EOD.

    1python -m eod train -e --config configs/det/yolox/yolox_fpga_quant_vitis.yaml --nm 1 --ng 1 --launch pytorch 2>&1 | tee log_qat_mqbench

  • Second export the quantized model to ONNX [mqbench_qmodel.onnx] and [mqbench_qmodel_deploy_model.onnx].

    1python -m eod quant_deploy --config configs/det/yolox/yolox_fpga_quant_vitis.yaml --ckpt [model_save_path] --input_shape [input_shape] 2>&1 | tee log.delpoy.txt

  • Third build Docker from Dockerfile, convert ONNX to xmodel [mqbench_qmodel_deploy_model.onnx_int.xmodel].

    1python -m mq.dep.convert_xir -Q [mqbench_qmodel.onnx] -C [mqbench_qmodel_deploy_model.onnx] -N [model_name]

  • Fourth compile xmodel to deployable model [mqbench_qmodel.xmodel].

    1vai_c_xir -x [mqbench_qmodel_deploy_model.onnx_int.xmodel] -a [new_arch.json] -o [output_path] -n [model_name]