https://github.com/fastmachinelearning/ml4fg
Machine Learning on frame grabbers for ultra-low latency in situ inference
https://github.com/fastmachinelearning/ml4fg
dl dnn fpga hls imaging inference machine-learning ml vivado vivado-hls
Last synced: 15 days ago
JSON representation
Machine Learning on frame grabbers for ultra-low latency in situ inference
- Host: GitHub
- URL: https://github.com/fastmachinelearning/ml4fg
- Owner: fastmachinelearning
- Created: 2023-11-12T20:13:08.000Z (over 1 year ago)
- Default Branch: master
- Last Pushed: 2024-11-15T22:18:30.000Z (5 months ago)
- Last Synced: 2025-03-27T15:52:14.962Z (about 1 month ago)
- Topics: dl, dnn, fpga, hls, imaging, inference, machine-learning, ml, vivado, vivado-hls
- Language: Jupyter Notebook
- Homepage:
- Size: 182 MB
- Stars: 5
- Watchers: 0
- Forks: 4
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Machine Learning for Frame Grabbers
### Getting Started
A tutorial and reference design for machine learning inference on FPGA-based frame grabber devices in high-throughput imaging applications. This tutorial leverages the hls4ml package and the CustomLogic toolkit to deploy neural networks to Euresys frame grabber devices. Refer to ```hls4ml-frame-grabber-tutorial.ipynb``` to get started. See ```part9_FOLO_frame_grabbers_advanced_features.ipynb``` for a more advanced guide on implementing a YOLO-style model and taking advantage of the suite of optimizations hls4ml provides.
To install ```hls4ml_frame_grabber``` conda environment
- ```conda env create -f environment.yml```
- ```conda activate hls4ml_frame_grabber```
- ```ipython kernel install --user --name=hls4ml_frame_grabber```
### Medium article
We have also released ```hls4ml-frame-grabber-tutorial.ipynb``` in a medium article format which can be found [here](https://medium.com/@forelliryan/deploying-neural-networks-for-in-situ-inference-on-frame-grabber-fpgas-in-high-speed-imaging-6201557fdabc).
### Acknowledgement
Primary development was completed by Fermi National Accelerator Laboratory, Northwestern University, and Drexel University. This work resulted from the implementation described in this paper: https://arxiv.org/abs/2312.00128. Deepest thanks to Euresys and the Columbia University HBT-EP group for their assistance and contribution to this development.