/deepstream-dev

nvinfer Configuration File Reference

Overview

The nvinfer GStreamer plugin uses a configuration file to define model parameters, preprocessing settings, and postprocessing options. This document provides a complete reference for all configuration parameters.

Configuration File Formats

nvinfer supports two configuration file formats:

Format 1: YAML Format (.yml or .yaml) - Recommended

property:
  gpu-id: 0
  net-scale-factor: 0.00392156862745098
  onnx-file: /path/to/model.onnx
  batch-size: 1
  # ... more properties

class-attrs-all:
  topk: 20
  pre-cluster-threshold: 0.2

Format 2: INI-style Text Format (.txt)

[property]
gpu-id=0
net-scale-factor=0.00392156862745098
onnx-file=/path/to/model.onnx
batch-size=1
# ... more properties

[class-attrs-all]
topk=20
pre-cluster-threshold=0.2

Key Syntax Differences

Property Section Reference

The property section contains core inference configuration.

Model Definition

Usage Example (YAML):

property:
  onnx-file: /opt/nvidia/deepstream/deepstream/samples/models/Primary_Detector/resnet18_trafficcamnet_pruned.onnx
  model-engine-file: /opt/nvidia/deepstream/deepstream/samples/models/Primary_Detector/resnet18_trafficcamnet_pruned.onnx_b1_gpu0_fp16.engine
  labelfile-path: /opt/nvidia/deepstream/deepstream/samples/models/Primary_Detector/labels.txt

model-engine-file — Purpose and Naming Convention

Purpose: The first time nvinfer runs with an ONNX model, TensorRT builds an optimised engine file. This serialisation step can take minutes. By specifying model-engine-file, you tell nvinfer to load an already-built engine directly, skipping the ONNX-to-engine conversion on subsequent runs and dramatically reducing startup time.

Agent guidance: When generating nvinfer config files, always include model-engine-file alongside onnx-file. This avoids expensive re-compilation every time the pipeline starts. The engine file is specific to the batch size, GPU, and precision — if any of these change, a new engine must be generated (i.e. the first run without a matching engine file will trigger generation automatically). The engine-cache location must be writable.

Naming convention: TensorRT engine files follow the pattern:

<onnx-filename>_b<batch-size>_gpu<gpu-id>_<precision>.engine

Examples by batch size:

# batch-size: 1
property:
  batch-size: 1
  model-engine-file: /opt/nvidia/deepstream/deepstream/samples/models/Primary_Detector/resnet18_trafficcamnet_pruned.onnx_b1_gpu0_fp16.engine

# batch-size: 4
property:
  batch-size: 4
  model-engine-file: /opt/nvidia/deepstream/deepstream/samples/models/Primary_Detector/resnet18_trafficcamnet_pruned.onnx_b4_gpu0_fp16.engine

# batch-size: 16 (e.g. secondary classifier)
property:
  batch-size: 16
  model-engine-file: /opt/nvidia/deepstream/deepstream/samples/models/Secondary_VehicleMake/resnet18_vehiclemakenet_pruned.onnx_b16_gpu0_fp16.engine

INI-style equivalent:

[property]
batch-size=4
model-engine-file=/opt/nvidia/deepstream/deepstream/samples/models/Primary_Detector/resnet18_trafficcamnet_pruned.onnx_b4_gpu0_fp16.engine

Processing Configuration

Usage Example (YAML):

property:
  gpu-id: 0
  batch-size: 4
  process-mode: 1
  network-mode: 2  # FP16
  interval: 0
  gie-unique-id: 1

Network Input Configuration

Agent guidance — infer-dims and dynamic ONNX models: Many popular model frameworks (Ultralytics YOLO, HuggingFace, etc.) export ONNX models with dynamic axes by default. These models have symbolic dimension names (e.g., batch, height, width) instead of fixed integers, which TensorRT reads as -1. Without infer-dims, TensorRT's setDimensions call fails because all dimensions must be >= 0. Always add infer-dims when the ONNX model has dynamic input shapes.

Usage Example (YAML) — static-shape model (infer-dims optional):

property:
  net-scale-factor: 0.00392156862745098  # 1/255
  offsets: 0;0;0
  model-color-format: 0  # RGB
  maintain-aspect-ratio: 1

Usage Example (YAML) — dynamic-shape ONNX model (infer-dims required):

property:
  net-scale-factor: 0.00392156862745098  # 1/255
  model-color-format: 0  # RGB
  infer-dims: 3;640;640  # REQUIRED for dynamic ONNX models
  maintain-aspect-ratio: 1

Usage Example (INI) — dynamic-shape ONNX model:

[property]
net-scale-factor=0.00392156862745098
model-color-format=0
infer-dims=3;640;640
maintain-aspect-ratio=1

Geometry contract for masks and boxes: When postprocessing produces boxes or masks that will be used outside the model-input grid, declare the model-input, pipeline/mux, and final output coordinate spaces. If maintain-aspect-ratio or symmetric-padding is enabled, account for the resize scale and padding before projecting model output into the target space. Identify whether nvinfer or the custom parser owns each transform, and apply the resize/padding inversion exactly once. For a full-frame mask, project the box into the declared frame-mask grid before compositing its bbox-local mask; do not treat pipeline-space rectangles as source-frame rectangles.

Detection Configuration

Usage Example (YAML):

property:
  num-detected-classes: 4
  cluster-mode: 2  # NMS

Oriented bounding boxes (OBB) — rotation_angle: nvinfer supports oriented bounding boxes via NvDsInferObjectDetectionInfo.rotation_angle. If you are using an OBB model, the angle output by the model can be directly assigned to rotation_angle in your custom bbox parser. If you are not using an OBB model, set rotation_angle = 0. In C++, NvDsInferObjectDetectionInfo obj{}; value-initializes the struct and zero-initializes all fields, including rotation_angle; plain NvDsInferObjectDetectionInfo obj; does not and can leave rotated-box metadata uninitialized.

Example (C++):

NvDsInferObjectDetectionInfo obj{};
// ... fill classId, confidence, left/top/width/height ...
obj.rotation_angle = is_obb_model ? angle_from_model : 0.0f;

Custom Instance-Segmentation Parsers

Use this path only for a model that produces detections with one mask per detected object. It is distinct from semantic segmentation, which produces a full-frame class map.

property:
  network-type: 3
  custom-lib-path: /path/to/libcustom_parser.so
  parse-bbox-instance-mask-func-name: NvDsInferParseCustomInstanceMask
  output-instance-mask: 1

• Use parse-bbox-instance-mask-func-name, not the ordinary parse-bbox-func-name, and implement the instance-mask parser ABI provided by the DeepStream SDK in the target image.
• After thresholding and NMS, emit each retained box, class ID, confidence, and matching bbox-local object mask together.
• When mask and final-box resolutions differ, resize or crop the mask into the final bbox-local geometry.
• output-instance-mask=1 exposes parsed object masks to DeepStream metadata and OSD. It does not create a frame-level output mask.

Secondary GIE Configuration (process-mode: 2)

Usage Example (YAML) - Secondary classifier:

property:
  gpu-id: 0
  onnx-file: /path/to/classifier.onnx
  batch-size: 16
  process-mode: 2
  network-mode: 2
  network-type: 1
  gie-unique-id: 2
  operate-on-gie-id: 1
  operate-on-class-ids: 0
  classifier-async-mode: 1
  classifier-threshold: 0.51
  classifier-type: vehicletype

Tensor Output Configuration

Usage Example (YAML):

property:
  output-tensor-meta: 1  # Enable tensor output for custom postprocessing

Scaling Configuration

Class Attributes Sections

Class attributes sections configure detection parameters per class or for all classes.

class-attrs-all (All Classes)

Applies to all detected classes.

IMPORTANT — camelCase key: The DBSCAN minimum cluster size parameter is minBoxes (camelCase). Do NOT use min-boxes (kebab-case) — it is not recognized and will produce an "unknown key" warning at runtime.

Usage Example (YAML) - NMS clustering:

class-attrs-all:
  topk: 20
  nms-iou-threshold: 0.5
  pre-cluster-threshold: 0.2

Usage Example (YAML) - DBSCAN clustering:

class-attrs-all:
  detected-min-w: 4
  detected-min-h: 4
  minBoxes: 3
  eps: 0.7
  dbscan-min-score: 0.5

class-attrs-N (Per-Class)

Override attributes for specific class ID N.

class-attrs-0:
  topk: 30
  nms-iou-threshold: 0.4
  pre-cluster-threshold: 0.3

class-attrs-1:
  topk: 10
  nms-iou-threshold: 0.6
  pre-cluster-threshold: 0.5

Complete Configuration Examples

Example 1: Primary Detector (YAML)

# Primary detector using ResNet18 TrafficCamNet
property:
  gpu-id: 0
  net-scale-factor: 0.00392156862745098
  onnx-file: /opt/nvidia/deepstream/deepstream/samples/models/Primary_Detector/resnet18_trafficcamnet_pruned.onnx
  model-engine-file: /opt/nvidia/deepstream/deepstream/samples/models/Primary_Detector/resnet18_trafficcamnet_pruned.onnx_b1_gpu0_fp16.engine
  labelfile-path: /opt/nvidia/deepstream/deepstream/samples/models/Primary_Detector/labels.txt
  batch-size: 1
  process-mode: 1
  model-color-format: 0
  network-mode: 2
  num-detected-classes: 4
  interval: 0
  gie-unique-id: 1
  cluster-mode: 2

class-attrs-all:
  topk: 20
  nms-iou-threshold: 0.5
  pre-cluster-threshold: 0.2

class-attrs-0:
  topk: 20
  nms-iou-threshold: 0.5
  pre-cluster-threshold: 0.4

Example 2: Primary Detector (INI-style)

# Primary detector using ResNet18 TrafficCamNet
[property]
gpu-id=0
net-scale-factor=0.00392156862745098
onnx-file=/opt/nvidia/deepstream/deepstream/samples/models/Primary_Detector/resnet18_trafficcamnet_pruned.onnx
model-engine-file=/opt/nvidia/deepstream/deepstream/samples/models/Primary_Detector/resnet18_trafficcamnet_pruned.onnx_b1_gpu0_fp16.engine
labelfile-path=/opt/nvidia/deepstream/deepstream/samples/models/Primary_Detector/labels.txt
batch-size=1
process-mode=1
model-color-format=0
network-mode=2
num-detected-classes=4
interval=0
gie-unique-id=1
cluster-mode=2

[class-attrs-all]
topk=20
nms-iou-threshold=0.5
pre-cluster-threshold=0.2

[class-attrs-0]
topk=20
nms-iou-threshold=0.5
pre-cluster-threshold=0.4

Example 3: Secondary Classifier (YAML)

# Secondary classifier for vehicle make
property:
  gpu-id: 0
  net-scale-factor: 1.0
  onnx-file: /opt/nvidia/deepstream/deepstream/samples/models/Secondary_VehicleMake/resnet18_vehiclemakenet_pruned.onnx
  model-engine-file: /opt/nvidia/deepstream/deepstream/samples/models/Secondary_VehicleMake/resnet18_vehiclemakenet_pruned.onnx_b16_gpu0_fp16.engine
  labelfile-path: /opt/nvidia/deepstream/deepstream/samples/models/Secondary_VehicleMake/labels.txt
  batch-size: 16
  process-mode: 2
  model-color-format: 1
  network-mode: 2
  network-type: 1
  gie-unique-id: 2
  operate-on-gie-id: 1
  operate-on-class-ids: 0
  classifier-async-mode: 1
  classifier-threshold: 0.51
  classifier-type: vehiclemake

Example 4: Tensor Output for Custom Postprocessing (YAML)

# Enable tensor output for custom postprocessing
property:
  gpu-id: 0
  net-scale-factor: 0.00392156862745098
  onnx-file: /path/to/custom_model.onnx
  batch-size: 1
  process-mode: 1
  model-color-format: 0
  network-mode: 2
  num-detected-classes: 4
  gie-unique-id: 1
  output-tensor-meta: 1
  cluster-mode: 4  # No clustering, use custom postprocessing

class-attrs-all:
  pre-cluster-threshold: 0.1

Common Pitfalls

Pitfall 1: Wrong Section Name

Wrong (using model: instead of property:):

model:
  onnx-file: /path/to/model.onnx
  batch-size: 1

Correct:

property:
  onnx-file: /path/to/model.onnx
  batch-size: 1

Pitfall 2: Missing Colons in YAML

Wrong:

property
  gpu-id: 0

Correct:

property:
  gpu-id: 0

Pitfall 3: Wrong Indentation

Wrong:

property:
gpu-id: 0
batch-size: 1

Correct:

property:
  gpu-id: 0
  batch-size: 1

Pitfall 4: Using YAML syntax in INI file

Wrong (YAML in .txt file):

property:
  gpu-id: 0

Correct (INI format in .txt file):

[property]
gpu-id=0

Pitfall 5: Incorrect process-mode for Secondary GIE

Wrong (using process-mode=1 for secondary):

property:
  process-mode: 1
  operate-on-gie-id: 1  # Won't work with process-mode=1

Correct:

property:
  process-mode: 2  # Must be 2 for secondary GIE
  operate-on-gie-id: 1

Pitfall 6: Missing infer-dims for Dynamic ONNX Models

Wrong (no infer-dims with a dynamic-shape ONNX model):

# Model exported with dynamic=True (e.g., Ultralytics YOLO)
# ONNX input shape: [batch, 3, height, width] — all symbolic
property:
  onnx-file: yolo_model.onnx
  net-scale-factor: 0.00392156862745098
  # Missing infer-dims → TensorRT sees -1 for dynamic dims → engine build fails

Error: IOptimizationProfile::setDimensions: Error Code 3: API Usage Error (Parameter check failed, condition: std::all_of(dims.d, dims.d + dims.nbDims, [](int32_t x) noexcept { return x >= 0; }))

Correct:

property:
  onnx-file: yolo_model.onnx
  net-scale-factor: 0.00392156862745098
  infer-dims: 3;640;640  # C;H;W — tells TensorRT the concrete input dimensions

When to add infer-dims: Whenever the ONNX model was exported with dynamic axes (e.g., dynamic=True in Ultralytics, dynamic batch in other frameworks). If unsure, inspect the model with python -c "import onnx; m = onnx.load('model.onnx'); print(m.graph.input)" and check for symbolic dimension names.

Pitfall 7: Using Legacy is-classifier Instead of network-type

Wrong (legacy key, produces deprecation warning):

property:
  is-classifier: 1

Correct (use network-type in YAML configs):

property:
  network-type: 1  # 0=Detector, 1=Classifier, 2=Segmentation, 3=Instance Segmentation

For primary detectors, simply omit both keys — the default is detector (network-type: 0).

Pitfall 8: Using min-boxes Instead of minBoxes

Wrong (kebab-case — not recognized, produces "unknown key" warning):

class-attrs-all:
  min-boxes: 3

Correct (camelCase):

class-attrs-all:
  minBoxes: 3

Unlike most nvinfer config keys which use kebab-case, minBoxes uses camelCase. This is a legacy naming exception in the parser.

DeepStream Sample Model Paths

DeepStream includes sample models at:

/opt/nvidia/deepstream/deepstream/samples/models/
├── Primary_Detector/
│   ├── resnet18_trafficcamnet_pruned.onnx
│   ├── labels.txt
│   └── cal_trt.bin (INT8 calibration)
├── Secondary_VehicleMake/
│   ├── resnet18_vehiclemakenet_pruned.onnx
│   └── labels.txt
├── Secondary_VehicleTypes/
│   ├── resnet18_vehicletypenet_pruned.onnx
│   └── labels.txt
└── SONYC_Audio_Classifier/
    └── ...

Primary Detector Labels (4 classes):

• 0: Car
• 1: TwoWheeler
• 2: Person
• 3: RoadSign

GObject Properties vs Config File Parameters

Some parameters can be set via GObject properties on the nvinfer element:

pipeline.add("nvinfer", "infer", {
    "config-file-path": "/path/to/config.yml",  # Required
    "batch-size": 4,                             # Overrides config file
    "unique-id": 1,                              # Overrides config file
    "output-tensor-meta": 1,                     # Overrides config file
    "interval": 2                                # Overrides config file
})

Properties settable via GObject (override config file):

• batch-size
• unique-id
• process-mode
• interval
• output-tensor-meta
• input-tensor-meta
• output-instance-mask
• model-engine-file

Properties only in config file:

• net-scale-factor
• onnx-file
• infer-dims
• labelfile-path
• num-detected-classes
• cluster-mode
• All class-attrs-* parameters

Validation Checklist

Before running your pipeline, verify:

• Config file extension matches format (.yml for YAML, .txt for INI)
• Section name is property: (YAML) or [property] (INI)
• Model file path exists and is accessible
• model-engine-file is set and its name matches the current batch-size, gpu-id, and network-mode (precision)
• infer-dims is set if the ONNX model has dynamic input shapes (e.g., exported with dynamic=True)
• num-detected-classes matches your model
• batch-size <= number of streams
• process-mode is correct (1=Primary, 2=Secondary)
• Secondary GIE has operate-on-gie-id set correctly
• gie-unique-id is unique across all nvinfer instances

Related Documentation

• GStreamer Plugins Overview: gstreamer_plugins.md
• Service Maker Python API: service_maker_api.md
• Use Cases & Pipelines: use_cases_pipelines.md
• Best Practices: best_practices.md