Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/seeed-studio/modelassistant

Seeed SenseCraft Model Assistant is an open-source project focused on embedded AI. 🔥🔥🔥
https://github.com/seeed-studio/modelassistant

arduino deep-learning esp32 image-classification jetson ncnn object-detection onnx openmmlab pytorch raspberry-pi tflite tinyml yolov5

Last synced: 1 day ago
JSON representation

Seeed SenseCraft Model Assistant is an open-source project focused on embedded AI. 🔥🔥🔥

Host: GitHub
URL: https://github.com/seeed-studio/modelassistant
Owner: Seeed-Studio
Created: 2022-10-18T01:19:55.000Z (over 2 years ago)
Default Branch: main
Last Pushed: 2025-01-08T07:39:04.000Z (16 days ago)
Last Synced: 2025-01-15T15:21:00.869Z (9 days ago)
Topics: arduino, deep-learning, esp32, image-classification, jetson, ncnn, object-detection, onnx, openmmlab, pytorch, raspberry-pi, tflite, tinyml, yolov5
Language: Python
Homepage: https://sensecraftma.seeed.cc/
Size: 25.7 MB
Stars: 410
Watchers: 26
Forks: 49
Open Issues: 0
Metadata Files:
- Readme: README.md

Awesome Lists containing this project

README

SenseCraft Model Assistant by Seeed Studio

[![docs-build](https://github.com/Seeed-Studio/ModelAssistant/actions/workflows/docs-build.yml/badge.svg)](https://github.com/Seeed-Studio/ModelAssistant/actions/workflows/docs-build.yml)
![GitHub Release](https://img.shields.io/github/v/release/Seeed-Studio/ModelAssistant)
[![license](https://img.shields.io/github/license/Seeed-Studio/ModelAssistant.svg)](https://github.com/Seeed-Studio/ModelAssistant/blob/main/LICENSE)
[![Average time to resolve an issue](http://isitmaintained.com/badge/resolution/Seeed-Studio/ModelAssistant.svg)](http://isitmaintained.com/project/Seeed-Studio/ModelAssistant "Average time to resolve an issue")
[![Percentage of issues still open](http://isitmaintained.com/badge/open/Seeed-Studio/ModelAssistant.svg)](http://isitmaintained.com/project/Seeed-Studio/ModelAssistant "Percentage of issues still open")

Documentation |
Installation |
Colab |
Model Zoo |
Deploy -
简体中文

## Introduction

**S**eeed **S**ense**C**raft **M**odel **A**ssistant is an open-source project focused on providing state-of-the-art AI algorithms for embedded devices. It is designed to help developers and makers to easily deploy various AI models on low-cost hardwares, such as microcontrollers and single-board computers (SBCs).

**Real-world deploy examples on MCUs with less than 0.3 Watts power consumption.*

### 🤝 User-friendly

SSCMA provides a user-friendly platform that allows users to easily perform training on collected data, and to better understand the performance of algorithms through visualizations generated during the training process.

### 🔋 Models with low computing power and high performance

SSCMA focuses on end-side AI algorithm research, and the algorithm models can be deployed on microprocessors, similar to [ESP32](https://www.espressif.com.cn/en/products/socs/esp32), some [Arduino](https://arduino.cc) development boards, and even in embedded SBCs such as [Raspberry Pi](https://www.raspberrypi.org).

### 🗂️ Supports multiple formats for model export

[TensorFlow Lite](https://www.tensorflow.org/lite) is mainly used in microcontrollers, while [ONNX](https://onnx.ai) is mainly used in devices with Embedded Linux. There are some special formats such as [TensorRT](https://developer.nvidia.com/tensorrt), [OpenVINO](https://docs.openvino.ai) which are already well supported by OpenMMLab. SSCMA has added TFLite model export for microcontrollers, which can be directly converted to [TensorRT](https://developer.nvidia.com/tensorrt), [UF2](https://github.com/microsoft/uf2) format and drag-and-drop into the device for deployment.

## Features

We have optimized excellent algorithms from [OpenMMLab](https://github.com/open-mmlab) for real-world scenarios and made implementation more user-friendly, achieving faster and more accurate inference. Currently we support the following directions of algorithms:

### 🔍 Anomaly Detection

In the real world, anomalous data is often difficult to identify, and even if it can be identified, it requires a very high cost. The anomaly detection algorithm collects normal data in a low-cost way, and anything outside normal data is considered anomalous.

### 👁️ Computer Vision

Here we provide a number of computer vision algorithms such as **object detection, image classification, image segmentation and pose estimation**. However, these algorithms cannot run on low-cost hardwares. SSCMA optimizes these computer vision algorithms to achieve good running speed and accuracy in low-end devices.

### ⏱️ Scenario Specific

SSCMA provides customized scenarios for specific production environments, such as identification of analog instruments, traditional digital meters, and audio classification. We will continue to add more algorithms for specified scenarios in the future.

## What's New

SSCMA is always committed to providing the cutting-edge AI algorithms for best performance and accuracy, along with the community feedbacks, we keeps updating and optimizing the algorithms to meet the actual needs of users, here are some of the latest updates:

### 🔥 RTMDet, VAE, QAT

We have added the RTMDet algorithm for real-time multi-object detection, VAE for anomaly detection, and QAT for quantization-aware training. These algorithms are optimized for low-cost hardwares and can be deployed on microcontrollers.

![RTMDet COCO Benchmark](docs/images/rtmdet_coco_eval.png)

We also optimized the training process for these algorithms, now the training process is much more faster than before.

### YOLOv8, YOLOv8 Pose, Nvidia Tao Models and ByteTrack

With [SSCMA-Micro](https://github.com/Seeed-Studio/SSCMA-Micro), now you can deploy the latest [YOLOv8](https://github.com/ultralytics/ultralytics), YOLOv8 Pose, [Nvidia TAO Models](https://docs.nvidia.com/tao/tao-toolkit/text/model_zoo/cv_models/index.html) on microcontrollers. we also added the [ByteTrack](https://github.com/ifzhang/ByteTrack) algorithm to enable real-time object tracking on low-cost hardwares.

### Swift YOLO

We implemented a lightweight object detection algorithm called Swift YOLO, which is designed to run on low-cost hardware with limited computing power. The visualization tool, model training and export command-line interface has refactored now.

### Meter Recognition

Meter is a common instrument in our daily life and industrial production, such as analog meters, digital meters, etc. SSCMA provides meter recognition algorithms that can be used to identify the readings of various meters.

## The SSCMA Toolchains

SSCMA provides a complete toolchain for users to easily deploy AI models on low-cost hardwares, including:

- [SSCMA-Model-Zoo](https://sensecraft.seeed.cc/ai/#/model) SSCMA Model Zoo provides a series of pre-trained models for different application scenarios for you to use. The source code for this web is [hosted here](https://github.com/Seeed-Studio/sscma-model-zoo).
- [SSCMA-Web-Toolkit, which is now renamed to SenseCraft AI](https://sensecraft.seeed.cc/ai/#/home) A web-based tool that makes trainning and deploying machine learning models (with a focus on vision models by now) fast, easy, and accessible to everyone.
- [SSCMA-Micro](https://github.com/Seeed-Studio/SSCMA-Micro) A cross-platform framework that deploys and applies SSCMA models to microcontrol devices.
- [Seeed-Arduino-SSCMA](https://github.com/Seeed-Studio/Seeed_Arduino_SSCMA) Arduino library for devices supporting the SSCMA-Micro firmware.
- [Python-SSCMA](https://github.com/Seeed-Studio/python-sscma) A Python library for interacting with microcontrollers using SSCMA-Micro, and for higher-level deep learning applications.

## Acknowledgement

SSCMA is a united effort of many developers and contributors, we would like to thank the following projects and organizations for their contributions which SSCMA referenced to implement:

- [OpenMMLab](https://openmmlab.com/)
- [ONNX](https://github.com/onnx/onnx)
- [NCNN](https://github.com/Tencent/ncnn)
- [TinyNN](https://github.com/alibaba/TinyNeuralNetwork)

## License

This project is released under the [Apache 2.0 license](LICENSE).