https://github.com/kendryte/nncase
Open deep learning compiler stack for Kendryte AI accelerators ✨
https://github.com/kendryte/nncase
ai-compiler compiler deep-learning k210 k230 k510 kendryte llm neural-network
Last synced: about 1 month ago
JSON representation
Open deep learning compiler stack for Kendryte AI accelerators ✨
- Host: GitHub
- URL: https://github.com/kendryte/nncase
- Owner: kendryte
- License: apache-2.0
- Created: 2018-04-04T12:11:19.000Z (about 7 years ago)
- Default Branch: master
- Last Pushed: 2024-10-29T11:29:16.000Z (6 months ago)
- Last Synced: 2024-10-29T11:42:45.403Z (6 months ago)
- Topics: ai-compiler, compiler, deep-learning, k210, k230, k510, kendryte, llm, neural-network
- Language: C#
- Homepage:
- Size: 56.8 MB
- Stars: 747
- Watchers: 29
- Forks: 181
- Open Issues: 18
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- edge-ai - nncase - Open deep learning compiler stack for Kendryte K210 AI accelerator. (Uncategorized / Uncategorized)
README
[](https://github.com/kendryte/nncase) [](https://gitee.com/kendryte/nncase) [](https://github.com/kendryte/nncase/releases)
[切换中文](docs/readme_ZH.md)
`nncase` is a neural network compiler for AI accelerators.
Telegram: [nncase community](https://t.me/joinchat/PPcEPZMLaTViNDI1)
Technical Discussion QQ Group: 790699378 . Answer: 人工智能
---
## K230
- [Usage](./docs/USAGE_v2_EN.md)
- [FAQ](./docs/FAQ_EN.md)
- [Example](./examples/user_guide/k230_simulate-EN.ipynb)
- [Colab run](https://colab.research.google.com/drive/1m8TTree096m5VHmq-Uc60gXyltVCgnRb?usp=sharing)
- [ *Version relationship between `nncase` and `K230_SDK`* ](https://developer.canaan-creative.com/k230/dev/zh/03_other/K230_SDK_nncase%E7%89%88%E6%9C%AC%E5%AF%B9%E5%BA%94%E5%85%B3%E7%B3%BB.html#k230sdknncase)
- [update nncase runtime library in SDK](https://developer.canaan-creative.com/k230/dev/zh/03_other/K230_SDK%E6%9B%B4%E6%96%B0nncase%E8%BF%90%E8%A1%8C%E6%97%B6%E5%BA%93%E6%8C%87%E5%8D%97.html)
### Install
- Linux：
```shell
pip install nncase nncase-kpu
```
- Windows:
```shell
1. pip install nncase
2. Download `nncase_kpu-2.x.x-py2.py3-none-win_amd64.whl` in below link.
3. pip install nncase_kpu-2.x.x-py2.py3-none-win_amd64.whl
```
All version of `nncase` and `nncase-kpu` in [Release](https://github.com/kendryte/nncase/releases).
### Supported operators
- [TFLite ops](./docs/tflite_ops.md)
- [Caffe ops](./docs/caffe_ops.md)
- [ONNX ops](./docs/onnx_ops.md)
### benchmark test
kind model shape quant_type(If/W) nncase_fps tflite_onnx_result accuracy info
Image Classification
mobilenetv2 [1,224,224,3] u8/u8 600.24 top-1 = 71.3%
top-5 = 90.1% top-1 = 71.1%
top-5 = 90.0% dataset(ImageNet 2012, 50000 images)
tflite
resnet50V2 [1,3,224,224] u8/u8 86.17 top-1 = 75.44%
top-5 = 92.56% top-1 = 75.11%
top-5 = 92.36% dataset(ImageNet 2012, 50000 images)
onnx
yolov8s_cls [1,3,224,224] u8/u8 130.497 top-1 = 72.2%
top-5 = 90.9% top-1 = 72.2%
top-5 = 90.8% dataset(ImageNet 2012, 50000 images)
yolov8s_cls(v8.0.207)
Object Detection
yolov5s_det [1,3,640,640] u8/u8 23.645 bbox
mAP50-90 = 0.374
mAP50 = 0.567 bbox
mAP50-90 = 0.369
mAP50 = 0.566dataset(coco val2017, 5000 images)
yolov5s_det(v7.0 tag, rect=False, conf=0.001, iou=0.65)
yolov8s_det [1,3,640,640] u8/u8 9.373 bbox
mAP50-90 = 0.446
mAP50 = 0.612
mAP75 = 0.484 bbox
mAP50-90 = 0.404
mAP50 = 0.593
mAP75 = 0.45dataset(coco val2017, 5000 images)
yolov8s_det(v8.0.207, rect = False)
Image Segmentation
yolov8s_seg [1,3,640,640] u8/u8 7.845 bbox
mAP50-90 = 0.444
mAP50 = 0.606
mAP75 = 0.484
segm
mAP50-90 = 0.371
mAP50 = 0.578
mAP75 = 0.396 bbox
mAP50-90 = 0.444
mAP50 = 0.606
mAP75 = 0.484
segm
mAP50-90 = 0.371
mAP50 = 0.579
mAP75 = 0.397 dataset(coco val2017, 5000 images)
yolov8s_seg(v8.0.207, rect = False, conf_thres = 0.0008)
Pose Estimation
yolov8n_pose_320 [1,3,320,320] u8/u8 36.066 bbox
mAP50-90 = 0.6
mAP50 = 0.843
mAP75 = 0.654
keypoints
mAP50-90 = 0.358
mAP50 = 0.646
mAP75 = 0.353 bbox
mAP50-90 = 0.6
mAP50 = 0.841
mAP75 = 0.656
keypoints
mAP50-90 = 0.359
mAP50 = 0.648
mAP75 = 0.357 dataset(coco val2017, 2346 images)
yolov8n_pose(v8.0.207, rect = False)
yolov8n_pose_640 [1,3,640,640] u8/u8 10.88 bbox
mAP50-90 = 0.694
mAP50 = 0.909
mAP75 = 0.776
keypoints
mAP50-90 = 0.509
mAP50 = 0.798
mAP75 = 0.544 bbox
mAP50-90 = 0.694
mAP50 = 0.909
mAP75 = 0.777
keypoints
mAP50-90 = 0.508
mAP50 = 0.798
mAP75 = 0.54 dataset(coco val2017, 2346 images)
yolov8n_pose(v8.0.207, rect = False)
yolov8s_pose [1,3,640,640] u8/u8 5.568 bbox
mAP50-90 = 0.733
mAP50 = 0.925
mAP75 = 0.818
keypoints
mAP50-90 = 0.605
mAP50 = 0.857
mAP75 = 0.666 bbox
mAP50-90 = 0.734
mAP50 = 0.925
mAP75 = 0.819
keypoints
mAP50-90 = 0.604
mAP50 = 0.859
mAP75 = 0.669 dataset(coco val2017, 2346 images)
yolov8s_pose(v8.0.207, rect = False)
### Demo
|[eye gaze](https://developer.canaan-creative.com/modelDetail?id=142) | [space_resize](https://developer.canaan-creative.com/modelDetail?id=141) | [face pose](https://developer.canaan-creative.com/modelDetail?id=125)|
| --- | --- | --- |
| 
| 
| 
|
---
## K210/K510
- [Usage](https://github.com/kendryte/nncase/blob/release/1.0/docs/USAGE_EN.md)
- [FAQ](https://github.com/kendryte/nncase/blob/release/1.0/docs/FAQ_EN.md)
- [Example](https://github.com/kendryte/nncase/blob/release/1.0/examples/user_guide/)
### Supported operators
- [TFLite ops](https://github.com/kendryte/nncase/blob/release/1.0/docs/tflite_ops.md)
- [Caffe ops](https://github.com/kendryte/nncase/blob/release/1.0/docs/caffe_ops.md)
- [ONNX ops](https://github.com/kendryte/nncase/blob/release/1.0/docs/onnx_ops.md)
---
## Features
- Supports multiple inputs and outputs and multi-branch structure
- Static memory allocation, no heap memory acquired
- Operators fusion and optimizations
- Support float and quantized uint8 inference
- Support post quantization from float model with calibration dataset
- Flat model with zero copy loading
---
## Architecture
---
## Build from source
**It is recommended to install nncase directly through `pip`. At present, the source code related to k510 and K230 chips is not open source, so it is not possible to use `nncase-K510` and `nncase-kpu` (K230) directly by compiling source code.**
If there are operators in your model that `nncase` does not yet support, you can request them in the issue or implement them yourself and submit the PR. Later versions will be integrated, or contact us to provide a temporary version.
Here are the steps to compile `nncase`.
```shell
git clone https://github.com/kendryte/nncase.git
cd nncase
mkdir build && cd build
# Use Ninja
cmake .. -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=./install
ninja && ninja install
# Use make
cmake .. -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=./install
make && make install
```
---
## Resources
### Canaan developer community
[Canaan developer community](https://developer.canaan-creative.com/resource) contains all resources related to K210, K510, and K230.
- 资料下载 --> Pre-compiled images available for the development boards corresponding to the three chips.
- 文档 --> Documents corresponding to the three chips.
- 模型库 --> Examples and code for industrial, security, educational and other scenarios that can be run on the K210 and K230.
- 模型训练 --> The model training platform for K210 and K230 supports the training of various scenarios.
### Bilibili
- [Canaan AI tutorial and application demonstration](https://space.bilibili.com/677429436)
### K210 related repo
- [K210_Yolo_framework](https://github.com/zhen8838/K210_Yolo_framework)
- [Shts!'s Blog (Japanese)](https://www.shtsno24.tokyo/2020/03/nncase-v020.html)
- [Examples](https://github.com/kendryte/canmv_examples/tree/main/01-K210)
### K230 related repo
- C: [K230_SDK](https://github.com/kendryte/k230_sdk)
- [Documents](https://github.com/kendryte/k230_docs)
- [K230 end-to-end tutorial](https://github.com/kendryte/K230_training_scripts)
- MicroPython: [Canmv_k230](https://github.com/kendryte/k230_canmv)
- [Documents](https://github.com/kendryte/k230_canmv_docs)
- [Examples](https://github.com/kendryte/canmv_examples/tree/main/02-K230)
---