Technology Document Guide of TensorRT

Technology Document Guide of TensorRT

Abstract

本示例支持指南概述了GitHub和产品包中包含的所有受支持的TensorRT 7.2.1示例。TensorRT示例在推荐程序、机器翻译、字符识别、图像分类和对象检测等领域有特殊帮助。

有关TensorRT开发文档，请参阅TensorRT归档文件。

1. 1.    Introduction

下面的示例展示了如何在许多用例中使用TensorRT，同时突出显示接口的不同功能。

1.1.  Getting Started With C++ Samples

You can find the C++ samples in the /usr/src/tensorrt/samples package directory as well as on GitHub. The following C++ samples are shipped with TensorRT.

• “Hello World” For TensorRT
• Building A Simple MNIST Network Layer By Layer
• Importing The TensorFlow Model And Running Inference
• “Hello World” For TensorRT From ONNX
• Building And Running GoogleNet In TensorRT
• Building An RNN Network Layer By Layer
• Performing Inference In INT8 Using Custom Calibration
• Performing Inference In INT8 Precision
• Adding A Custom Layer To Your Network In TensorRT
• Object Detection With Faster R-CNN
• Object Detection With A TensorFlow SSD Network
• Movie Recommendation Using Neural Collaborative Filter (NCF)
• Movie Recommendation Using MPS (Multi-Process Service)
• Object Detection With SSD
• “Hello World” For Multilayer Perceptron (MLP)
• Specifying I/O Formats Using The Reformat Free I/O APIs
• Adding A Custom Layer That Supports INT8 I/O To Your Network In TensorRT
• Digit Recognition With Dynamic Shapes In TensorRT
• Neural Machine Translation (NMT) Using A Sequence To Sequence (seq2seq) Model
• Object Detection And Instance Segmentation With A TensorFlow Mask R-CNN Network
• Object Detection With A TensorFlow Faster R-CNN Network
• Algorithm Selection API Usage Example Based On sampleMNIST In TensorRT¹

Getting Started With C++ Samples

每个C++样本包括一个GitHub中的README.md文件，该文件提供有关示例如何工作的详细信息、示例代码以及有关如何运行和验证其输出的分步说明。

Running C++ Samples on Linux

如果使用Debian文件安装TensorRT，在构建C++示例之前，首先复制/usr/src/tensorrt到新目录。如果使用tar文件安装了TensorRT，那么示例位于{TAR_EXTRACT_PATH}/samples中。要生成所有示例，然后运行其中一个示例，请使用以下命令：

$ cd <samples_dir>

$ make -j4

$ cd ../bin

$ ./<sample_bin>

Running C++ Samples on Windows

Windows上的所有C++样本都作为VisualStudio解决方案文件提供。若要生成示例，请打开其相应的VisualStudio解决方案文件并生成解决方案。输出可执行文件将在（ZIP_EXTRACT_PATH）\bin中生成。然后可以直接或通过visual studio运行可执行文件。

1.2.  Getting Started With Python Samples

可以在/usr/src/tensorrt/samples/python包目录中找到Python示例。以下Python示例随TensorRT一起提供。

• Introduction To Importing Caffe, TensorFlow And ONNX Models Into TensorRT Using Python
• “Hello World” For TensorRT Using TensorFlow And Python
• “Hello World” For TensorRT Using PyTorch And Python
• Adding A Custom Layer To Your TensorFlow Network In TensorRT In Python
• Object Detection With The ONNX TensorRT Backend In Python
• Object Detection With SSD In Python
• INT8 Calibration In Python
• Refitting An Engine In Python
• TensorRT Inference Of ONNX Models With Custom Layers In Python

Getting Started With Python Samples

每个Python示例都包含README.md文件。请参阅

/usr/src/tensorrt/samples/python/<sample-name>/README.md文件获取有关示例如何工作的详细信息、示例代码以及有关如何运行和验证其输出的分步说明。

Running Python Samples

要运行其中一个Python示例，该过程通常包括两个步骤：

1. Install the sample requirements:

1. python<x> -m pip install -r requirements.txt

where python<x> is either python2 or python3.

1. Run the sample code with the data directory provided if the TensorRT sample data is not in the default location. For example:

python<x> sample.py [-d DATA_DIR]

For more information on running samples, see the README.md file included with the sample.