Posts made by zhengbicheng

zhengbicheng

I'm looking forward to it.

zhengbicheng

@jugg 解码和播放运行第一条命令应该就有了？

zhengbicheng

@jugg 硬解码CPU占用能到50?

zhengbicheng

@jugg https://mirrors.tuna.tsinghua.edu.cn/help/ubuntu-ports/ 换成这个源试试

zhengbicheng

@MSUMMERS999 Maybe you need to try burning the image again?

zhengbicheng

@qwe 具体报什么错误，截图看看

zhengbicheng

Introduction

YOLOv5 is the world's most loved vision AI, representing Ultralytics open-source research into future vision AI methods, incorporating lessons learned and best practices evolved over thousands of hours of research and development.

FastDeploy is an open source and very easy to use warehouse, which can quickly deploy YOLOv5 on coolpi-4B.

Model Speed Table

In addition to YOLOv5, we also support the deployment of YOLOX, YOLOv7, and PPYOLOE. The following speeds are the end-to-end speeds of the model:

Model Name	Whether to Quantify	Speed（ms）	Download Address
yolov5-s-relu	是	70	https://bj.bcebos.com/paddlehub/fastdeploy/rknpu2/yolov5-s-relu.zip
yolov7-tiny	是	58	https://bj.bcebos.com/paddlehub/fastdeploy/rknpu2/yolov7-tiny.zip
yolox-s	是	130	https://bj.bcebos.com/paddlehub/fastdeploy/rknpu2/yolox-s.zip
ppyoloe	是	141	https://bj.bcebos.com/paddlehub/fastdeploy/rknpu2/ppyoloe_plus_crn_s_80e_coco.zip

C++ Demo

It is difficult to develop AI applications using C++, but it can be matched with other useful libraries on CoolPI-4B. For your convenience, we provide a demo of C++.

// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include "fastdeploy/vision.h"

void RKNPU2Infer(const std::string& model_file, const std::string& image_file) {
  auto option = fastdeploy::RuntimeOption();
  option.UseRKNPU2();

  auto format = fastdeploy::ModelFormat::RKNN;

  auto model = fastdeploy::vision::detection::RKYOLOV5(
      model_file, option,format);

  auto im = cv::imread(image_file);

  fastdeploy::vision::DetectionResult res;
  fastdeploy::TimeCounter tc;
  tc.Start();
  if (!model.Predict(im, &res)) {
    std::cerr << "Failed to predict." << std::endl;
    return;
  }
  auto vis_im = fastdeploy::vision::VisDetection(im, res,0.5);
  tc.End();
  tc.PrintInfo("RKYOLOV5 in RKNN");
  std::cout << res.Str() << std::endl;
  cv::imwrite("vis_result.jpg", vis_im);
  std::cout << "Visualized result saved in ./vis_result.jpg" << std::endl;
}

int main(int argc, char* argv[]) {
  if (argc < 3) {
    std::cout
        << "Usage: infer_demo path/to/model_dir path/to/image run_option, "
           "e.g ./infer_model ./picodet_model_dir ./test.jpeg"
        << std::endl;
    return -1;
  }

  RKNPU2Infer(argv[1], argv[2]);

  return 0;
}

Python Demo

In addition to using C++, you can also use FastDeploy's Python API for rapid development.

# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import fastdeploy as fd
import cv2
import os


def parse_arguments():
    import argparse
    import ast
    parser = argparse.ArgumentParser()
    parser.add_argument(
        "--model_file", required=True, help="Path of rknn model.")
    parser.add_argument(
        "--image", type=str, required=True, help="Path of test image file.")
    return parser.parse_args()


if __name__ == "__main__":
    args = parse_arguments()

    model_file = args.model_file
    params_file = ""

    # 配置runtime，加载模型
    runtime_option = fd.RuntimeOption()
    runtime_option.use_rknpu2()

    model = fd.vision.detection.RKYOLOV5(
        model_file,
        runtime_option=runtime_option,
        model_format=fd.ModelFormat.RKNN)

    # 预测图片分割结果
    im = cv2.imread(args.image)
    result = model.predict(im)
    print(result)

    # 可视化结果
    vis_im = fd.vision.vis_detection(im, result, score_threshold=0.5)
    cv2.imwrite("visualized_result.jpg", vis_im)
    print("Visualized result save in ./visualized_result.jpg")

Result

input image

输入图片

output image

zhengbicheng

@retroman https://github.com/yanyitech/coolpi_4B_docs

zhengbicheng

@retroman CoolPI Forum is an open forum where developers from all countries communicate. We have created Github Docs Repo. If you have other documents that we need to sort out, please refer to issues.

zhengbicheng

@ccjjww1222 您误会我的意思了，我的意思是你可以用电脑利用转接器连接Cool-pi开发板的串口。使用串口输入账号密码后就是命令行了。

zhengbicheng

@ccjjww1222 不是可以串口连接吗？不可能电脑屏幕也没有把。。。

zhengbicheng

@bbsvs2000 目标追踪？指deepsort一类的吗？

zhengbicheng

Brief Introduction

With the improvement of AI computing power, Face Recognition algorithms have gradually shifted from machine learning to deep learning. Through this tutorial, you will learn how to use FastDeploy to quickly implement Face Recognition on Coolpi.

What is ArcFace

ArcFace is a human face recognition model. Please refer to InsightFace Github Repo for detailed parameters and model introduction. For developers' testing, models exported by InsightFace are provided below. Developers can download and use them directly. The accuracy metric is sourced from the model description in InsightFace. Refer to the introduction in InsightFace for more details.

Model	Size	Accuracy (AgeDB_30)
CosFace-r18	92MB	97.7
CosFace-r34	131MB	98.3
CosFace-r50	167MB	98.3
CosFace-r100	249MB	98.4
ArcFace-r18	92MB	97.7
ArcFace-r34	131MB	98.1
ArcFace-r50	167MB	-
ArcFace-r100	249MB	98.4
ArcFace-r100_lr0.1	249MB	98.4

How to transform the model

If you need to convert your ONNX model to RKNN model, FastDeploy also provides the corresponding method. You only need
to modify the configuration file of the model in the tools/rknpu2/config to achieve rapid model conversion

git clone https://github.com/PaddlePaddle/FastDeploy.git
wget https://bj.bcebos.com/paddlehub/fastdeploy/ms1mv3_arcface_r18.onnx

python -m paddle2onnx.optimize --input_model ./ms1mv3_arcface_r18/ms1mv3_arcface_r18.onnx \
                               --output_model ./ms1mv3_arcface_r18/ms1mv3_arcface_r18.onnx \
                               --input_shape_dict "{'data':[1,3,112,112]}"
                               
python  /Path/To/FastDeploy/tools/rknpu2/export.py \
        --config_path tools/rknpu2/config/arcface_unquantized.yaml \
        --target_platform rk3588

Workflow

# Download CoolPI-AI
git clone https://github.com/yanyitech/coolpi-ai.git

# Go to face detection demo
cd example/face_recognition

# Build and make
mkdir build
cd build
cmake .. -DFASTDEPLOY_INSTALL_DIR=${PWD}/../../../third_party/fastdeploy-develop
make -j8

# 下载官方转换好的ArcFace模型文件和测试图片
wget https://bj.bcebos.com/paddlehub/fastdeploy/ms1mv3_arcface_r18.onnx
wget https://bj.bcebos.com/paddlehub/fastdeploy/rknpu2/face_demo.zip
unzip face_demo.zip

# CPU推理
./infer_arcface_demo ms1mv3_arcface_r100.onnx face_0.jpg face_1.jpg face_2.jpg 0
# RKNPU推理
./infer_arcface_demo ms1mv3_arcface_r100.onnx face_0.jpg face_1.jpg face_2.jpg 1

Navigation

zhengbicheng

Brief Introduction

With the improvement of AI computing power, Face Detection algorithms have gradually shifted from machine learning to deep learning. Through this tutorial, you will learn how to use FastDeploy to quickly implement face detection on Coolpi.

What is SCRFD

SCRFD is an efficient high accuracy face detection approach which initially described in Arxiv, and accepted by ICLR-2022. In order to facilitate you to quickly understand the parameters of the model, we only give the parameters of common models. If you need to view the detailed data of the model, please go to InsightFace's official Github.

Name	Easy	Medium	Hard	FLOPs	Params(M)
SCRFD_500M_KPS	90.97	88.44	69.49	500M	0.57
SCRFD_2.5G_KPS	93.80	92.02	77.13	2.5G	0.82
SCRFD_10G_KPS	95.40	94.01	82.80	10G	4.23

Workflow

# Download CoolPI-AI
git clone https://github.com/yanyitech/coolpi-ai.git

# Go to face detection demo
cd example/face_detection

# Build and make
mkdir build
cd build
cmake .. -DFASTDEPLOY_INSTALL_DIR=${PWD}/../../../third_party/fastdeploy-develop
make -j8

# Download model and picture
wget https://bj.bcebos.com/paddlehub/fastdeploy/rknpu2/scrfd_500m_bnkps_shape640x640_rknpu2.zip
unzip scrfd_500m_bnkps_shape640x640_rknpu2.zip
wget https://raw.githubusercontent.com/DefTruth/lite.ai.toolkit/main/examples/lite/resources/test_lite_face_detector_3.jpg

# Run
./infer_with_scrfd scrfd_500m_bnkps_shape640x640_rknpu2/scrfd_500m_bnkps_shape640x640_rk3588_quantized.rknn \
              test_lite_face_detector_3.jpg \
              1

How to transform the model

If you need to convert your ONNX model to RKNN model, FastDeploy also provides the corresponding method. You only need to modify the configuration file of the model in the tools/rknpu2/config to achieve rapid model conversion

git clone https://github.com/PaddlePaddle/FastDeploy.git
wget  https://bj.bcebos.com/paddlehub/fastdeploy/rknpu2/scrfd_500m_bnkps_shape640x640.zip
unzip scrfd_500m_bnkps_shape640x640.zip
python  /Path/To/FastDeploy/tools/rknpu2/export.py \
        --config_path tools/rknpu2/config/scrfd_quantized.yaml \
        --target_platform rk3588

Result

Input Data

Output Data

Navigation

zhengbicheng

Brief Introduction

FastDeploy is an easy-to-use and high performance AI model deployment toolkit for Cloud, Mobile and Edge with out-of-the-box and unified experience, end-to-end optimization for over 150+ Text, Vision, Speech and Cross-modal AI models. Including image classification, object detection, image segmentation, face detection, face recognition, keypoint detection, matting, OCR, NLP, TTS and other tasks to meet developers' industrial deployment needs for multi-scenario, multi-hardware and multi-platform.

We spent a lot of time on the adaptation of FastDeploy for Coolpi. The model we have adapted can be used in Detection, Face Detection, Face Recognition, Face Alignment, Segmentation, OCR and other fields. These models will greatly improve Coolpi's ability in edge computing.

Model List

In order to facilitate you to choose a reasonable model to apply to your project, we have provided the running speed of the model.

Task	Model Name	Model Version	Whether to Quantify	RKNN Speed(ms)
Classification	ResNet	ResNet50_vd	No	33
Detection	Picodet	Picodet-s	No	112
Detection	PaddleDetection Yolov8	yolov8-n	No	100
Detection	PPYOLOE	ppyoloe-s	Yes	141
Detection	RKYOLOV5	YOLOV5-S-Relu	Yes	57
Detection	RKYOLOX	yolox-s	Yes	130
Detection	RKYOLOV7	yolov7-tiny	Yes	58
Segmentation	Unet	Unet-cityscapes	No	-
Segmentation	PP-HumanSegV2Lite	portrait	Yes	43
Segmentation	PP-HumanSegV2Lite	human	Yes	43
Face Detection	SCRFD	SCRFD-2.5G-kps-640	Yes	42
Face FaceRecognition	InsightFace	ms1mv3_arcface_r18	Yes	12

Demo

Contribution

Thanks to Baidu FastDeploy team for making such a good tool so that we can easily deploy AI model.

zhengbicheng

@yihuan1984 rk系列的板子解码应该不是用gpu吧，应该是mpp

zhengbicheng

@kingpin1cn 这个你可以尝试一下，因为我不太了解opencv 。但是肯定和 npu 是没有啥关系的

zhengbicheng

@kingpin1cn 不行哦，只支持RKNN框架

zhengbicheng

@1103854485 详细安装文档参考https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/faq/rknpu2/install_rknn_toolkit2.md

zhengbicheng

@1103854485 rknn-tookit-lite是用于3399Pro这一类RK一代NPU上的python库。RK3588是rknn-tookit-lite2，最高版本就是1.4.0，不需要更新。只需要把RK1代的转换仓库从rknn_toolkit改成rknn_toolkit2转换一下模型就能跑了