准备工作

首先在https://github.com/NVIDIA-AI-IOT/yolo_deepstream下载yolo_deepstream源代码,并按照docker-desktop安装测试DeepStream配置好docker。
按照https://github.com/NVIDIA-AI-IOT/yolo_deepstream/blob/main/deepstream_yolo/README.md编译得到libnvdsinfer_custom_impl_Yolo.so。
LZ编译过程遇到一些问题,通过修改Makefile如下解决:

################################################################################
# SPDX-FileCopyrightText: Copyright (c) 2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: MIT
#
# Permission is hereby granted, free of charge, to any person obtaining a
# copy of this software and associated documentation files (the "Software"),
# to deal in the Software without restriction, including without limitation
# the rights to use, copy, modify, merge, publish, distribute, sublicense,
# and/or sell copies of the Software, and to permit persons to whom the
# Software is furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
# DEALINGS IN THE SOFTWARE.
################################################################################

CC:= g++
NVCC:=/usr/local/cuda-12.5/bin/nvcc

CFLAGS:= -Wall -std=c++11 -shared -fPIC -Wno-error=deprecated-declarations
CFLAGS+= -I/opt/nvidia/deepstream/deepstream/sources/includes/ -I/usr/local/cuda-12.5/include -I/usr/local/cuda-12.8/include

CUFLAGS:= -std=c++14 -shared 
CUFLAGS+= -I/opt/nvidia/deepstream/deepstream/sources/includes/ -I/usr/local/cuda-12.5/include -I/usr/local/cuda-12.8/include 
LIBS:= -lnvinfer_plugin -lnvinfer  -L/usr/local/cuda-12.5/targets/x86_64-linux/lib -lcudart  -lstdc++fs
LFLAGS:= -shared -Wl,--start-group $(LIBS) -Wl,--end-group

INCS:= $(wildcard *.h)
SRCFILES:= nvdsparsebbox_Yolo.cpp\
			nvdsparsebbox_Yolo_cuda.cu

TARGET_LIB:= libnvdsinfer_custom_impl_Yolo.so

TARGET_OBJS:= $(SRCFILES:.cpp=.o)
TARGET_OBJS:= $(TARGET_OBJS:.cu=.o)

all: $(TARGET_LIB)

%.o: %.cpp $(INCS) Makefile
	$(CC) -c -o $@ $(CFLAGS) $<

%.o: %.cu $(INCS) Makefile
	$(NVCC) -c -o $@ --compiler-options '-fPIC' $(CUFLAGS)  $<

$(TARGET_LIB) : $(TARGET_OBJS)
	$(CC) -o $@  $(TARGET_OBJS) $(LFLAGS)

clean:
	rm -rf $(TARGET_LIB) *.o

运行测试

这里我们测试yolov7模型推理。
https://github.com/NVIDIA-AI-IOT/yolo_deepstream/tree/main/yolov7_qat中下载yolov7.onnx模型,放倒yolo_deepstream-main/deepstream_yolo/文件夹下。
并修改deepstream_app_config_yolo.txt的116、117行如下

#config-file=config_infer_primary_yoloV4.txt
config-file=config_infer_primary_yoloV7.txt

此时运行

deepstream-app -c deepstream_app_config_yolo.txt

输出:


** WARN: <parse_tracker:1756>: Unknown key 'enable-batch-process' for group [tracker]
** WARN: <parse_tracker:1756>: Unknown key 'enable-past-frame' for group [tracker]
Unknown or legacy key specified 'is-classifier' for group [property]
Opening in BLOCKING MODE
0:00:00.583776324 12145 0x645ac3ec77c0 INFO                 nvinfer gstnvinfer.cpp:685:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:2123> [UID = 1]: Trying to create engine from model files
0:11:07.412116984 12145 0x645ac3ec77c0 INFO                 nvinfer gstnvinfer.cpp:685:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:2155> [UID = 1]: serialize cuda engine to file: /docker_share/yolo_deepstream-main/deepstream_yolo/yolov7.onnx_b16_gpu0_fp16.engine successfully
INFO: ../nvdsinfer/nvdsinfer_model_builder.cpp:363 [FullDims Engine Info]: layers num: 2
0   INPUT  kFLOAT images          3x640x640       min: 1x3x640x640     opt: 16x3x640x640    Max: 16x3x640x640
1   OUTPUT kFLOAT output          25200x85        min: 0               opt: 0               Max: 0

0:11:08.469031523 12145 0x645ac3ec77c0 INFO                 nvinfer gstnvinfer_impl.cpp:343:notifyLoadModelStatus:<primary_gie> [UID 1]: Load new model:/docker_share/yolo_deepstream-main/deepstream_yolo/config_infer_primary_yoloV7.txt sucessfully
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE

Runtime commands:
        h: Print this help
        q: Quit

        p: Pause
        r: Resume


**PERF:  FPS 0 (Avg)    FPS 1 (Avg)     FPS 2 (Avg)     FPS 3 (Avg)     FPS 4 (Avg)     FPS 5 (Avg)     FPS 6 (Avg)     FPS 7 (Avg)     FPS 8 (Avg)     FPS 9 (Avg)  FPS 10 (Avg)     FPS 11 (Avg)    FPS 12 (Avg)    FPS 13 (Avg)    FPS 14 (Avg)    FPS 15 (Avg)
**PERF:  0.00 (0.00)    0.00 (0.00)     0.00 (0.00)     0.00 (0.00)     0.00 (0.00)     0.00 (0.00)     0.00 (0.00)     0.00 (0.00)     0.00 (0.00)     0.00 (0.00)  0.00 (0.00)      0.00 (0.00)     0.00 (0.00)     0.00 (0.00)     0.00 (0.00)     0.00 (0.00)
** INFO: <bus_callback:291>: Pipeline ready

WARNING from src_elem: No decoder available for type 'audio/mpeg, mpegversion=(int)4, framed=(boolean)true, stream-format=(string)raw, level=(string)2, base-profile=(string)lc, profile=(string)lc, codec_data=(buffer)119056e500, rate=(int)48000, channels=(int)2'.
Debug info: ../gst/playback/gsturidecodebin.c(1003): unknown_type_cb (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin1/GstURIDecodeBin:src_elem
WARNING from src_elem: No decoder available for type 'audio/mpeg, mpegversion=(int)4, framed=(boolean)true, stream-format=(string)raw, level=(string)2, base-profile=(string)lc, profile=(string)lc, codec_data=(buffer)119056e500, rate=(int)48000, channels=(int)2'.
Debug info: ../gst/playback/gsturidecodebin.c(1003): unknown_type_cb (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin3/GstURIDecodeBin:src_elem
WARNING from src_elem: No decoder available for type 'audio/mpeg, mpegversion=(int)4, framed=(boolean)true, stream-format=(string)raw, level=(string)2, base-profile=(string)lc, profile=(string)lc, codec_data=(buffer)119056e500, rate=(int)48000, channels=(int)2'.
Debug info: ../gst/playback/gsturidecodebin.c(1003): unknown_type_cb (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin2/GstURIDecodeBin:src_elem
WARNING from src_elem: No decoder available for type 'audio/mpeg, mpegversion=(int)4, framed=(boolean)true, stream-format=(string)raw, level=(string)2, base-profile=(string)lc, profile=(string)lc, codec_data=(buffer)119056e500, rate=(int)48000, channels=(int)2'.
Debug info: ../gst/playback/gsturidecodebin.c(1003): unknown_type_cb (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin0/GstURIDecodeBin:src_elem
WARNING from src_elem: No decoder available for type 'audio/mpeg, mpegversion=(int)4, framed=(boolean)true, stream-format=(string)raw, level=(string)2, base-profile=(string)lc, profile=(string)lc, codec_data=(buffer)119056e500, rate=(int)48000, channels=(int)2'.
Debug info: ../gst/playback/gsturidecodebin.c(1003): unknown_type_cb (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin4/GstURIDecodeBin:src_elem
WARNING from src_elem: No decoder available for type 'audio/mpeg, mpegversion=(int)4, framed=(boolean)true, stream-format=(string)raw, level=(string)2, base-profile=(string)lc, profile=(string)lc, codec_data=(buffer)119056e500, rate=(int)48000, channels=(int)2'.
Debug info: ../gst/playback/gsturidecodebin.c(1003): unknown_type_cb (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin5/GstURIDecodeBin:src_elem
WARNING from src_elem: No decoder available for type 'audio/mpeg, mpegversion=(int)4, framed=(boolean)true, stream-format=(string)raw, level=(string)2, base-profile=(string)lc, profile=(string)lc, codec_data=(buffer)119056e500, rate=(int)48000, channels=(int)2'.
Debug info: ../gst/playback/gsturidecodebin.c(1003): unknown_type_cb (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin6/GstURIDecodeBin:src_elem
WARNING from src_elem: No decoder available for type 'audio/mpeg, mpegversion=(int)4, framed=(boolean)true, stream-format=(string)raw, level=(string)2, base-profile=(string)lc, profile=(string)lc, codec_data=(buffer)119056e500, rate=(int)48000, channels=(int)2'.
Debug info: ../gst/playback/gsturidecodebin.c(1003): unknown_type_cb (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin7/GstURIDecodeBin:src_elem
WARNING from src_elem: No decoder available for type 'audio/mpeg, mpegversion=(int)4, framed=(boolean)true, stream-format=(string)raw, level=(string)2, base-profile=(string)lc, profile=(string)lc, codec_data=(buffer)119056e500, rate=(int)48000, channels=(int)2'.
Debug info: ../gst/playback/gsturidecodebin.c(1003): unknown_type_cb (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin8/GstURIDecodeBin:src_elem
WARNING from src_elem: No decoder available for type 'audio/mpeg, mpegversion=(int)4, framed=(boolean)true, stream-format=(string)raw, level=(string)2, base-profile=(string)lc, profile=(string)lc, codec_data=(buffer)119056e500, rate=(int)48000, channels=(int)2'.
Debug info: ../gst/playback/gsturidecodebin.c(1003): unknown_type_cb (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin9/GstURIDecodeBin:src_elem
WARNING from src_elem: No decoder available for type 'audio/mpeg, mpegversion=(int)4, framed=(boolean)true, stream-format=(string)raw, level=(string)2, base-profile=(string)lc, profile=(string)lc, codec_data=(buffer)119056e500, rate=(int)48000, channels=(int)2'.
Debug info: ../gst/playback/gsturidecodebin.c(1003): unknown_type_cb (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin10/GstURIDecodeBin:src_elem
WARNING from src_elem: No decoder available for type 'audio/mpeg, mpegversion=(int)4, framed=(boolean)true, stream-format=(string)raw, level=(string)2, base-profile=(string)lc, profile=(string)lc, codec_data=(buffer)119056e500, rate=(int)48000, channels=(int)2'.
Debug info: ../gst/playback/gsturidecodebin.c(1003): unknown_type_cb (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin11/GstURIDecodeBin:src_elem
WARNING from src_elem: No decoder available for type 'audio/mpeg, mpegversion=(int)4, framed=(boolean)true, stream-format=(string)raw, level=(string)2, base-profile=(string)lc, profile=(string)lc, codec_data=(buffer)119056e500, rate=(int)48000, channels=(int)2'.
Debug info: ../gst/playback/gsturidecodebin.c(1003): unknown_type_cb (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin12/GstURIDecodeBin:src_elem
WARNING from src_elem: No decoder available for type 'audio/mpeg, mpegversion=(int)4, framed=(boolean)true, stream-format=(string)raw, level=(string)2, base-profile=(string)lc, profile=(string)lc, codec_data=(buffer)119056e500, rate=(int)48000, channels=(int)2'.
Debug info: ../gst/playback/gsturidecodebin.c(1003): unknown_type_cb (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin13/GstURIDecodeBin:src_elem
Opening in BLOCKING MODE
WARNING from src_elem: No decoder available for type 'audio/mpeg, mpegversion=(int)4, framed=(boolean)true, stream-format=(string)raw, level=(string)2, base-profile=(string)lc, profile=(string)lc, codec_data=(buffer)119056e500, rate=(int)48000, channels=(int)2'.
Debug info: ../gst/playback/gsturidecodebin.c(1003): unknown_type_cb (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin14/GstURIDecodeBin:src_elem
Opening in BLOCKING MODE
WARNING from src_elem: No decoder available for type 'audio/mpeg, mpegversion=(int)4, framed=(boolean)true, stream-format=(string)raw, level=(string)2, base-profile=(string)lc, profile=(string)lc, codec_data=(buffer)119056e500, rate=(int)48000, channels=(int)2'.
Debug info: ../gst/playback/gsturidecodebin.c(1003): unknown_type_cb (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin15/GstURIDecodeBin:src_elem
Opening in BLOCKING MODE
** INFO: <bus_callback:277>: Pipeline running

**PERF:  12.00 (14.80)  12.00 (15.28)   12.40 (15.29)   12.00 (15.28)   12.40 (15.29)   12.40 (15.29)   12.00 (15.28)   12.00 (15.28)   12.40 (15.29)   12.40 (15.29)12.00 (15.28)    12.20 (15.29)   12.40 (15.29)   11.80 (17.63)   12.00 (15.28)   12.00 (15.28)
**PERF:  17.00 (16.01)  17.20 (16.35)   17.20 (16.34)   17.20 (16.35)   17.20 (16.34)   17.20 (16.34)   17.20 (16.35)   17.20 (16.35)   17.20 (16.34)   17.20 (16.34)17.20 (16.35)    17.20 (16.35)   17.20 (16.34)   17.20 (17.37)   17.20 (16.35)   17.20 (16.35)
**PERF:  17.60 (16.58)  17.60 (16.80)   17.60 (16.79)   17.60 (16.80)   17.60 (16.79)   17.60 (16.79)   17.60 (16.80)   17.60 (16.80)   17.60 (16.79)   17.60 (16.79)17.60 (16.80)    17.60 (16.80)   17.60 (16.79)   17.60 (17.46)   17.60 (16.80)   17.60 (16.80)
**PERF:  19.20 (17.27)  19.20 (17.44)   19.20 (17.42)   19.20 (17.44)   19.20 (17.42)   19.20 (17.42)   19.20 (17.44)   19.20 (17.44)   19.20 (17.42)   19.20 (17.42)19.20 (17.44)    19.20 (17.43)   19.20 (17.42)   19.20 (17.93)   19.20 (17.44)   19.20 (17.44)
**PERF:  19.20 (17.67)  19.20 (17.80)   19.20 (17.79)   19.20 (17.80)   19.20 (17.79)   19.20 (17.79)   19.20 (17.80)   19.20 (17.80)   19.20 (17.79)   19.20 (17.79)19.20 (17.80)    19.20 (17.80)   19.20 (17.79)   19.20 (18.20)   19.20 (17.80)   19.20 (17.80)
**PERF:  19.40 (17.97)  19.40 (18.08)   19.40 (18.07)   19.40 (18.08)   19.40 (18.07)   19.40 (18.07)   19.40 (18.08)   19.40 (18.08)   19.40 (18.07)   19.40 (18.07)19.40 (18.08)    19.40 (18.08)   19.40 (18.07)   19.40 (18.41)   19.40 (18.08)   19.40 (18.08)
**PERF:  19.00 (18.12)  19.00 (18.21)   19.00 (18.21)   19.00 (18.21)   19.00 (18.21)   19.00 (18.21)   19.00 (18.21)   19.00 (18.21)   19.00 (18.21)   19.00 (18.21)19.00 (18.21)    19.00 (18.21)   19.00 (18.21)   19.00 (18.50)   19.00 (18.21)   19.00 (18.21)
**PERF:  18.40 (18.15)  18.60 (18.26)   18.60 (18.26)   18.60 (18.26)   18.60 (18.26)   18.60 (18.26)   18.60 (18.26)   18.60 (18.26)   18.60 (18.26)   18.60 (18.26)18.60 (18.26)    18.60 (18.26)   18.60 (18.26)   18.40 (18.49)   18.60 (18.26)   18.40 (18.24)
**PERF:  18.60 (18.20)  18.80 (18.33)   18.80 (18.32)   18.80 (18.33)   18.80 (18.32)   18.80 (18.32)   18.80 (18.33)   18.80 (18.33)   18.80 (18.32)   18.80 (18.32)18.80 (18.33)    18.80 (18.32)   18.80 (18.32)   18.80 (18.52)   18.80 (18.33)   18.80 (18.30)
**PERF:  19.40 (18.33)  19.40 (18.43)   19.40 (18.43)   19.40 (18.43)   19.40 (18.43)   19.20 (18.41)   19.40 (18.43)   19.40 (18.43)   19.40 (18.43)   19.40 (18.43)19.40 (18.43)    19.40 (18.43)   19.40 (18.43)   19.40 (18.61)   19.40 (18.43)   19.40 (18.41)
**PERF:  18.60 (18.35)  18.60 (18.45)   18.60 (18.44)   18.60 (18.45)   18.60 (18.44)   18.60 (18.42)   18.60 (18.45)   18.60 (18.45)   18.60 (18.44)   18.60 (18.44)18.60 (18.45)    18.60 (18.45)   18.60 (18.44)   18.60 (18.61)   18.60 (18.45)   18.60 (18.43)
**PERF:  19.20 (18.42)  19.20 (18.51)   19.20 (18.51)   19.20 (18.51)   19.20 (18.51)   19.20 (18.49)   19.20 (18.51)   19.20 (18.51)   19.20 (18.51)   19.20 (18.51)19.20 (18.51)    19.20 (18.51)   19.20 (18.51)   19.20 (18.66)   19.20 (18.51)   19.20 (18.50)
**PERF:  19.00 (18.47)  19.00 (18.55)   19.00 (18.55)   19.00 (18.55)   19.00 (18.55)   19.00 (18.53)   19.00 (18.55)   19.00 (18.55)   19.00 (18.55)   19.00 (18.55)19.00 (18.55)    19.00 (18.55)   19.00 (18.55)   19.00 (18.69)   19.00 (18.55)   19.00 (18.54)
**PERF:  19.00 (18.51)  19.00 (18.58)   19.00 (18.58)   19.00 (18.58)   19.00 (18.58)   19.00 (18.56)   19.00 (18.58)   19.00 (18.58)   19.00 (18.58)   19.00 (18.58)19.00 (18.58)    19.00 (18.58)   19.00 (18.58)   19.00 (18.71)   19.00 (18.58)   19.00 (18.57)
**PERF:  18.60 (18.51)  18.60 (18.59)   18.60 (18.58)   18.60 (18.59)   18.60 (18.58)   18.60 (18.57)   18.60 (18.59)   18.60 (18.59)   18.60 (18.58)   18.60 (18.58)18.60 (18.59)    18.60 (18.58)   18.60 (18.58)   18.60 (18.71)   18.60 (18.59)   18.60 (18.57)
nvstreammux: Successfully handled EOS for source_id=12
nvstreammux: Successfully handled EOS for source_id=4
nvstreammux: Successfully handled EOS for source_id=8
nvstreammux: Successfully handled EOS for source_id=2
nvstreammux: Successfully handled EOS for source_id=9
nvstreammux: Successfully handled EOS for source_id=11
nvstreammux: Successfully handled EOS for source_id=5
nvstreammux: Successfully handled EOS for source_id=6
nvstreammux: Successfully handled EOS for source_id=10
nvstreammux: Successfully handled EOS for source_id=7
nvstreammux: Successfully handled EOS for source_id=14
nvstreammux: Successfully handled EOS for source_id=1
nvstreammux: Successfully handled EOS for source_id=3
nvstreammux: Successfully handled EOS for source_id=15
nvstreammux: Successfully handled EOS for source_id=13
nvstreammux: Successfully handled EOS for source_id=0
** INFO: <bus_callback:334>: Received EOS. Exiting ...

Quitting
App run successful

运行成功。

Logo

腾讯云面向开发者汇聚海量精品云计算使用和开发经验,营造开放的云计算技术生态圈。

更多推荐