Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

awesome-production-machine-learning

A curated list of awesome open source libraries to deploy, monitor, version and scale your machine learning
https://github.com/EthicalML/awesome-production-machine-learning

Last synced: 19 days ago
JSON representation

Filtered by generative-ai. Clear filters

Adversarial Robustness

  • Factool - NLP/factool.svg?style=social) - Factool is a tool augmented framework for detecting factual errors of texts generated by large language models.

Agentic Framework

  • LangGraph - ai/langgraph.svg?style=social) - LangGraph is a library for building stateful, multi-actor applications with LLMs, used to create agent and multi-agent workflows.
  • TensorZero - TensorZero is an open-source framework for building production-grade LLM applications. It unifies an LLM gateway, observability, optimization, evaluations, and experimentation.

Data Annotation and Synthesis

  • SDV - dev/SDV.svg?style=social) - Synthetic Data Vault (SDV) is a Synthetic Data Generation ecosystem of libraries that allows users to easily learn single-table, multi-table and timeseries datasets to later on generate new Synthetic Data that has the same format and statistical properties as the original dataset.

Data Pipeline

  • Instill VDP - ai/instill-core.svg?style=social) - Instill VDP (Versatile Data Pipeline) aims to streamline the data processing pipelines from inception to completion.

Deployment and Serving

  • BentoML - BentoML is an open source framework for high performance ML model serving.
  • OpenVINO - OpenVINO is an open-source toolkit for optimizing and deploying AI inference.
  • Vercel AI - Vercel AI is a TypeScript toolkit designed to help you build AI-powered applications using popular frameworks like Next.js, React, Svelte, Vue and runtimes like Node.js.
  • AirLLM - AirLLM optimizes inference memory usage, allowing 70B large language models to run inference on a single 4GB GPU card without quantization, distillation and pruning.
  • Jina-serve - ai/serve.svg?style=social) - Jina-serve is a framework for building and deploying AI services that communicate via gRPC, HTTP and WebSockets.

Evaluation and Monitoring

  • Evidently - Evidently is an open-source framework to evaluate, test and monitor ML and LLM-powered systems.
  • FMBench - samples/foundation-model-benchmarking-tool.svg?style=social) - FMBench is a tool for running performance benchmarks for any Foundation Model (FM) deployed on any AWS Generative AI service, be it Amazon SageMaker, Amazon Bedrock, Amazon EKS, or Amazon EC2.
  • OpenLLMetry - OpenLLMetry provides developers with deep visibility into Large Language Model applications through performance monitoring, execution tracing, and debugging capabilities.

Industry Strength Information Retrieval

  • Cognita - Cognita is a RAG framework for building modular and production-ready applications.
  • fastRAG - fastRAG is a research framework for efficient and optimized retrieval augmented generative pipelines, incorporating state-of-the-art LLMs and Information Retrieval.

Industry Strength Natural Language Processing

  • Haystack - ai/haystack.svg?style=social) - Haystack is an open source NLP framework to interact with your data using Transformer models and LLMs (GPT-3 and alike). Haystack offers production-ready tools to quickly build ChatGPT-like question answering, semantic search, text generation, and more.
  • LangChain - ai/langchain.svg?style=social) - LangChain assists in building applications with LLMs through composability.
  • Burr - Burr helps you develop applications that make decisions (chatbot, agent, simulation). It comes with production-ready features (telemetry, persistence, deployment, etc.) and the open-source, free, and local-first Burr UI.

Industry Strength Visualisation

  • Transformer Explainer - explainer.svg?style=social) - Transformer Explainer is an interactive visualization tool designed to help anyone learn how Transformer-based models like GPT work.

Model Training and Orchestration

  • NeMo - NVIDIA NeMo is a scalable and cloud-native generative AI framework built for researchers and PyTorch developers working on Large Language Models (LLMs), Multimodal Models (MMs), Automatic Speech Recognition (ASR), Text to Speech (TTS), and Computer Vision (CV) domains. It is designed to help you efficiently create, customize, and deploy new generative AI models by leveraging existing code and pre-trained model checkpoints.

Neural Search and Retrieval

  • BeyondLLM - Beyond LLM offers an all-in-one toolkit for experimentation, evaluation, and deployment of RAG systems, simplifying the process with automated integration, customizable evaluation metrics, and support for various LLMs tailored to specific needs, ultimately aiming to reduce LLM hallucination risks and enhance reliability.

Privacy and Safety

  • AI Gateway - ai/gateway.svg?style=social) - The AI Gateway is a blazing fast AI Gateway with integrated guardrails.
  • Awesome RAG Production - Curated list of production-grade tools and best practices for building scalable RAG systems.