NVIDIA JetPack
JetPack
NVIDIA JetPack SDK is the most comprehensive solution for building end-to-end accelerated AI applications. JetPack provides a full development environment for hardware-accelerated AI-at-the-edge development on Nvidia Jetson modules. JetPack includes Jetson Linux with bootloader, Linux kernel, Ubuntu desktop environment, and a complete set of libraries for acceleration of GPU computing, multimedia, graphics, and computer vision. It also includes samples, documentation, and developer tools for both host computer and developer kit, and supports higher level SDKs such as DeepStream for streaming video analytics, Isaac for robotics, and Riva for conversational AI.
More Info refer to https://developer.nvidia.com/embedded/jetpack
- JetPack 5.1.2 includes CUDA 11.4.19, TensorRT 8.5.2, DeepStream 6.2.
- JetPack 6.2 includes CUDA 12.6, TensorRT 10.3, DeepStream 7.1
DeepStream
DeepStream is a complete streaming analytics toolkit based on GStreamer for AI-based multi-sensor processing, video, audio, and image understanding. It’s ideal for vision AI developers, software partners, startups, and OEMs building IVA apps and services. Developers can now create stream processing pipelines that incorporate neural networks and other complex processing tasks such as tracking, video encoding/decoding, and video rendering. DeepStream pipelines enable real-time analytics on video, image, and sensor data.
DeepStream’s multi-platform support gives you a faster, easier way to develop vision AI applications and services. You can even deploy them on-premises, on the edge, and in the cloud with the click of a button.
More Info refer to https://developer.nvidia.com/deepstream-sdk
TensorRT
TensorRT is a high performance deep learning inference runtime for image classification, segmentation, and object detection neural networks. TensorRT is built on CUDA, NVIDIA’s parallel programming model, and enables you to optimize inference for all deep learning frameworks. It includes a deep learning inference optimizer and runtime that delivers low latency and high-throughput for deep learning inference applications.
Applications
Edge AI SDK / Vision Application (input: video file)
| Application (FPS) | Model | EPC-R7300 | AIR-030 | AIR-020R |
|---|---|---|---|---|
| Object Detection | yolov3 | 52 | 59 | 52 |
| Person Detection | sample_ssd_relu6.uff | 58 | 25 | 58 |
| Face Detection | facenet.etlt / faciallandmarks.etlt | 55 | 32 | 56 |
| Pose Estimation | bodypose2D model | 22 | 46 | 22 |
Edge AI SDK / GenAI Application
| Version | Application | Model | Note |
|---|---|---|---|
| v3.0.0 | LLM Chatbot | Llama-2-7b | |
| v3.3.0 | GenAI Chatbot | Gemma3:4b | Refer to Link |
Benchmark
Jetson is used to deploy a wide range of popular DNN models and ML frameworks to the edge with high performance inferencing, for tasks like real-time classification and object detection, pose estimation, semantic segmentation, and natural language processing (NLP).
More Info refer to https://developer.nvidia.com/embedded/jetson-benchmarks
More Info refer to https://github.com/NVIDIA-AI-IOT/jetson_benchmarks
cd /opt/Advantech/EdgeAISuite/Jetson/benchmark
./exec_benchmark.sh
Optimization
Performance
| Platform | Model | CPU | GPU | Optimization | Note |
|---|---|---|---|---|---|
| AIR-020R (Orin-Nano) | Yolov11 | 1 | 50 | How-To | CPU:ONNX FP32 , GPU:TensorRT @FP16 OS:JetPack6.2 |
Accelerator
| Accelerator | Description | Note |
|---|---|---|
| nvv4l2decoder | DeepStream extends the open source V4L2 codec plugins (here called Gst-v4l2) to support hardware-accelerated codecs | Link |
| nvvideoconvert | This plugin performs video color format conversion. It accepts NVMM memory as well as RAW (memory allocated using calloc() or malloc() ), and provides NVMM or RAW memory at the output | Link |
| nvinfer | The Gst-nvinfer plugin does inferencing on input data using NVIDIA® TensorRT™. | Link |
| nvdsosd | This plugin draws bounding boxes, text, arrows, lines, circles and region of interest (RoI) polygons. | Link |
| nv3dsink | This is a hardware-accelerated GStreamer sink plugin primarily designed for high-performance video rendering on the NVIDIA Jetson platform | Link |
Tegrastats Utility
This SDK provides the tegrastats utility, which reports memory usage and processor usage for Tegra-based devices. You can find the utility in your package at the following location.
More Info refer to Link
