Projects in Awesome Lists tagged with batch-inference
A curated list of projects in awesome lists tagged with batch-inference .
https://github.com/ttanzhiqiang/onnx_tensorrt_project
Support Yolov5(4.0)/Yolov5(5.0)/YoloR/YoloX/Yolov4/Yolov3/CenterNet/CenterFace/RetinaFace/Classify/Unet. use darknet/libtorch/pytorch/mxnet to onnx to tensorrt
batch-inference centerface centernet classify darknet libtorch mxnet onnx-tensorrt pytorch retinaface unet yolor yolov4 yolov5 yolox
Last synced: 20 Mar 2025
https://github.com/milenkovicm/torchfusion
Torchfusion is a very opinionated torch inference on datafusion.
batch-inference datafusion inference machine-learning pytorch rust sql torch userdefined-functions
Last synced: 08 Oct 2025
https://github.com/milenkovicm/lightfusion
LightGBM Inference on Datafusion
batch-inference datafusion inference lightgbm machine-learning rust sql udf userdefined-functions
Last synced: 06 May 2026
https://github.com/artain-ai/ignite-ms
Fast self-hosted embedding engine for search, RAG, and reindexing workloads on NVIDIA GPUs. Built in Rust + TensorRT for teams that care about scale, cost, and control.
batch-inference batch-processing cuda embeddings gpu high-performance huggingface machine-learning multi-gpu nlp rag rust self-hosted semantic-search tensorrt text-embeddings vector-search
Last synced: 04 Jun 2026
https://github.com/kimmmmyy223/llm-batch
🚀 Process JSON data in batches with `llm-batch`, leveraging sequential or parallel modes for efficient interaction with LLMs.
aws batch-inference bedrock deep-learning distributed-computing dynamic-batching flask language-model large-language-models llm-agent llm-inference nlp ops python rabbitmq ray-data react vllm
Last synced: 08 Apr 2026
