L4T Multimedia API Reference

28.1 Release

 All Data Structures Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Groups Pages
multimedia_api/ll_samples/docs/l4t_mm_video_dec_tensorrt.md
Go to the documentation of this file.
1 Copyright (c) 2016-2017, NVIDIA CORPORATION. All rights reserved.
2 
3 @page l4t_mm_vid_decode_trt 04_video_dec_trt
4 @{
5 
6  - [Overview](#overview)
7  - [Building and Running](#build_and_run)
8  - [Flow](#flow)
9  - [Key Structure and Classes](#key)
10  - [Key Thread](#key_thread)
11  - [Programming Notes](#notes)
12  - [Command Line Options](#options)
13 
14 - - - - - - - - - - - - - - -
15 <a name="overview">
16 ## Overview ##
17 
18 This sample demonstrates the simplest way to use NVIDIA<sup>&reg;</sup>
19 TensorRT<sup>&tm;</sup> to decode video and save the bounding box information to
20 the `result.txt` file. TensorRT was previously known as GPU Inference Engine
21 (GIE).
22 
23 This samples does not require a Camera or display.
24 
25 <a name="build_and_run">
26 - - - - - - - - - - - - - - -
27 ## Building and Running ##
28 
29 #### Prerequisites ####
30 * You have followed Steps 1-3 in @ref mmapi_build.
31 * You have installed the following:
32  - CUDA
33  - TensorRT (previously known as GPU Inference Engine (GIE))
34  - OpenCV4Tegra
35 
36 
37 ### To build:
38 * Enter:
39 
40  $ cd $HOME/tegra_multimedia_api/samples/04_video_dec_trt
41  $ make
42 
43 ### To run
44 * Enter:
45 
46  $ ./video_dec_trt <in-file> <in-format> [options]
47 
48 ### Notes
49 
50 1. The result saves the normalized rectangle within [0,1].
51 2. `02_video_dec_cuda` can verify the result and scale the rectangle parameters
52  with the following command:
53 
54  $ ./video_dec_cuda <in-file> <in-format> --bbox-file result.txt
55 3. Supports in-stream resolution changes.
56 4. The default deploy file is:
57 
58  ../../data/Model/GoogleNet_one_class/GoogleNet_modified_oneClass_halfHD.prototxt
59  and the model file is:
60 
61  ../../data/Model/GoogleNet_one_class/GoogleNet_modified_oneClass_halfHD.caffemodel
62  5. The batch size can be changed in `GoogleNet_modified_oneClass_halfHD.prototxt`, but is limited
63  by the available system memory.
64  6. The following command must be executed before running with a new batch size:
65 
66  $ rm trtModel.cache
67 
68 <a name="flow">
69 - - - - - - - - - - - - - - -
70 ## Flow
71 
72 The data pipeline is as follow:
73 
74  Input video file -> Decoder -> VIC -> TensorRT Inference -> Plain text file with Bounding Box info
75 
76 #### Operation Flow ####
77 
78 The sample does the following:
79 
80 1. Encodes the input video stream.
81 2. Performs one-channel video decodeVIC, which does the following:
82  - Converts the buffer layout from block linear to pitch linear.
83  - Scales the image resolution to what's required by TensorRT.
84 3. Uses TensorRT performs object identification and adds a bounding box to identified
85  object of original frame.
86 4. Converts YUV to RGB format and save it in a file.
87 
88 ![](l4t_mm_video_dec_tensorrt.jpg)
89 
90 The block diagram contains not only the pipeline, but also the memory sharing information among different engines, which can be a reference for the other samples.
91 
92 
93 - - - - - - - - - - - - - - -
94 <a name="key">
95 ## Key Structure and Classes ##
96 
97 This sample uses the following key structures and classes:
98 
99 The global structure `context_t` manages all the resources in the application.
100 
101 |Element|Description|
102 |---|---|
103 |NvVideoDecoder|Contains all video decoding-related elements and functions.|
104 |NvVideoConverter|Contains elements and functions for video format conversion.|
105 |EGLDisplay|The EGLImage used for CUDA processing.|
106 |conv_output_plane_buf_queue|Output plane queue for video conversion.|
107 |TRT_Context|Provide a series of interfaces to load Caffemodel and do inference.|
108 
109 
110 <a name="key_thread">
111 ## Key Thread ##
112 
113 |Member|Description|
114 |---|---|
115 |decCaptureLoop|Get buffers from dec capture plane and push to converter, and handle resolution change.|
116 |Conv outputPlane dqThread|Return the buffers dequeued from converter output plane to decoder capture plane.|
117 |Conv captuerPlane dqThread|Get buffers from conv capture plane and push to the TensorRT buffer queue.|
118 |trtThread|CUDA process and doing inference.|
119 
120 <a name="notes">
121 ## Programming Notes ##
122 
123 
124 - The result saves the normalized rectangle within [0,1].
125 - Can use 02_video_dec_cuda sample to display and verify the result and scale the rectangle parameters with the following command:
126 
127  $ ./video_dec_cuda <in-file> <in-format> --bbox-file result.txt
128 
129 - Supports in-stream resolution changes.
130 - The default deploy file is
131 
132  GoogleNet-modified.prototxt
133 
134  The default model file is
135 
136  GoogleNet-modified-online_iter_30000.caffemodel
137 
138  In this directory:
139 
140  $SDKDIR/data/model
141 
142 - The batch size can be changed in GoogleNet-modified.prototxt.
143  The batch size is limited by the memory.
144  The biggest batch size for Ivanet is less than 40.
145 - End-of-stream (EOS) process:
146 
147  a. Completely read the file.
148 
149  b. Push a null `v4l2buf` to decoder.
150 
151  c. Decoder waits for all output plane buffers to return.
152 
153  d. Set `get_eos`:
154 
155  decCap thread exit
156 
157  e. End the TensorRT thread.
158 
159  f. Send EOS to the converter:
160 
161  conv output plane dqThread callback return false
162  conv output plane dqThread exit
163  conv capture plane dqThread callback return false
164  conv capture plane dqThread exit
165 
166  g. Delete the decoder:
167 
168  deinit output plane and capture plane buffers
169 
170  h. Delete the converter:
171 
172  unmap capture plane buffers
173 
174 
175 <a name="options">
176 ## Command Line Options ##
177 
178  ./video_dec_trt <in-file> <in-format> [options]
179 
180 |Option|Description|
181 |--|--|
182 |`--`trt-deployfile|Sets deploy file name.|
183 |`--`trt-modelfile|Sets the model file name.|
184 |`--`trt-float32 <int>|Specifies to use float16 or not[0-2], where <int> is one of the following:<ul><li>0 use default</li><li>1 float16</li><li>2 float32</li></ul>|
185 |`--`trt-enable-perf|Enables performance measurement.|
186 
187 @}
188 
189 
190