Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
awesome-llmops
An awesome & curated list of best LLMOps tools for developers
https://github.com/tensorchord/awesome-llmops
- Alpaca - lab/stanford_alpaca.svg?style=flat-square) |
- BELLE - tune by 34B Chinese Character Corpus, based on LLaMA and Alpaca. |  |
- Bloom - science Open-access Multilingual Language Model |  |
- dolly - square) |
- Falcon 40B - 40B-Instruct is a 40B parameters causal decoder-only model built by TII based on Falcon-40B and finetuned on a mixture of Baize. It is made available under the Apache 2.0 license. | |
- FastChat (Vicuna) - T5. |  |
- Gemma
- GLM-6B (ChatGLM) - Trained Model, quantization of ChatGLM-130B, can run on consumer-level GPUs. |  |
- ChatGLM2-6B - 6B is the second-generation version of the open-source bilingual (Chinese-English) chat model [ChatGLM-6B](https://github.com/THUDM/ChatGLM-6B). |  |
- GLM-130B (ChatGLM) - Trained Model (ICLR 2023) |  |
- GPT-NeoX - neox.svg?style=flat-square) |
- Luotuo - Alpaca-LoRA. |  |
- Mixtral-8x7B-v0.1 - 8x7B Large Language Model (LLM) is a pretrained generative Sparse Mixture of Experts. | |
- StableLM - AI/StableLM.svg?style=flat-square) |
- disco-diffusion - diffusion.svg?style=flat-square) |
- midjourney
- segment-anything (SAM) - anything.svg?style=flat-square) |
- stable-diffusion - to-image diffusion model |  |
- stable-diffusion v2 - Resolution Image Synthesis with Latent Diffusion Models |  |
- bark - based text-to-audio model created by Suno. Bark can generate highly realistic, multilingual speech as well as other audio - including music, background noise and simple sound effects. |  |
- whisper - Scale Weak Supervision |  |
- Alpaca-LoRA-Serve - LoRA as Chatbot service |  |
- CTranslate2 - square) |
- Clip-as-a-service - ai/clip-as-service.svg?style=flat-square) |
- DeepSpeed-MII - latency and high-throughput inference possible, powered by DeepSpeed. |  |
- Faster Whisper - whisper.svg?style=flat-square) |
- FlexGen - oriented scenarios. |  |
- Flowise - square) |
- llama.cpp - square) |
- Infinity - embeddings |  |
- Modelz-LLM - llm.svg?style=flat-square) |
- Ollama - square) |
- TensorRT-LLM - LLM.svg?style=flat-square) |
- text-generation-inference - generation-inference.svg?style=flat-square) |
- text-embeddings-inference - embedding models |  |
- vllm - throughput and memory-efficient inference and serving engine for LLMs. |  |
- whisper.cpp - square) |
- x-stable-diffusion - time inference for Stable Diffusion - 0.88s latency. Covers AITemplate, nvFuser, TensorRT, FlashAttention. |  |
- BentoML - square) |
- Jina - ai/jina.svg?style=flat-square) |
- Mosec - to-use Python interface. |  |
- TFServing - performance serving system for machine learning models. |  |
- Torchserve - square) |
- Triton Server (TRTIS) - inference-server/server.svg?style=flat-square) |
- langchain-serve - ai/langchain-serve.svg?style=flat-square) |
- lanarky - grade LLM applications |  |
- ray-llm - RayLLM |  |
- Xinference - source language models, speech recognition models, and multimodal models, whether in the cloud, on-premises, or even on your laptop. |  |
- Plexiglass - labs/plexiglass?style=flat-square) |
- Azure OpenAI Logger - openai-logger?style=flat-square) |
- Deepchecks - square) |
- Evidently - square) |
- Fiddler AI - production to production. Ship more ML and LLMs into production, and monitor ML and LLM metrics like hallucination, PII, and toxicity. |  |
- Giskard - AI/giskard.svg?style=flat-square) |
- Great Expectations - expectations/great_expectations.svg?style=flat-square) |
- whylogs - square) |
- agenta - AI/agenta.svg?style=flat-square) |
- AI studio - square) |
- Arize-Phoenix - ai/phoenix.svg?style=flat-square) |
- BudgetML - square) |
- CometLLM - ml/comet-llm.svg?style=flat-square) |
- deeplake - square) |
- Dify - source framework aims to enable developers (and even non-developers) to quickly build useful applications based on large language models, ensuring they are visual, operable, and improvable. |  |
- Dstack - effective LLM development in any cloud (AWS, GCP, Azure, Lambda, etc). |  |
- Embedchain - square) |
- Fiddler AI - production to production. Ship more ML and LLMs into production, and monitor ML and LLM metrics like hallucination, PII, and toxicity. |  |
- Glide - Native LLM Routing Engine. Improve LLM app resilience and speed. |  |
- GPTCache - square) |
- Haystack - answering and more. |  |
- Izlo
- Keywords AI
- langchain - square) |
- LangFlow - and-drop components and a chat interface. |  |
- Langfuse - square) |
- LangKit - of-the-box LLM telemetry collection library that extracts features and profiles prompts, responses and metadata about how your LLM is performing over time to find problems at scale. |  |
- LiteLLM 🚅 - square) |
- Literal AI - modal LLM observability and evaluation platform. Create prompt templates, deploy prompts versions, debug LLM runs, create datasets, run evaluations, monitor LLM metrics and collect human feedback. | |
- LlamaIndex - square) |
- LLMApp - time LLM-enabled data pipelines with few lines of code. |  |
- LLMFlows - answering systems, and agents. |  |
- LLMonitor - and-play integration into LangChain. |  |
- magentic - powered functionality. |  |
- Manag.ai - in-one prompt management and observability platform. Craft, track, and perfect your LLM prompts with ease. | |
- Mirascope - fast, efficient development and ensuring quality in LLM-based applications |  |
- OpenLIT - native GenAI and LLM Application Observability tool and provides OpenTelmetry Auto-instrumentation for monitoring LLMs, VectorDBs and Frameworks. It provides valuable insights into token & cost usage, user interaction, and performance related metrics. |  |
- Parea AI - controlled enhanced prompt playground. |  |
- Pezzo 🕹️ - source LLMOps platform built for developers and teams. In just two lines of code, you can seamlessly troubleshoot your AI operations, collaborate and manage your prompts in one place, and instantly deploy changes to any environment. |  |
- PromptHub - technical team members. Test, version, collaborate, deploy, and monitor, all from one place. | |
- promptfoo - source tool for testing & evaluating prompt quality. Create test cases, automatically check output quality and catch regressions, and reduce evaluation cost. |  |
- Prompteams - time APIs. Have GitHub style with repos, branches, and commits (and commit history). | |
- prompttools - source tools for testing and experimenting with prompts. The core idea is to enable developers to evaluate prompts using familiar interfaces like code and notebooks. In just a few lines of codes, you can test your prompts and parameters across different models (whether you are using OpenAI, Anthropic, or LLaMA models). You can even evaluate the retrieval accuracy of vector databases. |  |
- TreeScale - enhanced APIs seamlessly using tools for prompt optimization, semantic querying, version management, statistical evaluation, and performance tracking. As a part of the developer friendly API implementation TreeScale offers Elastic LLM product, which makes a unified API Endpoint for all major LLM providers and open source models. | |
- TrueFoundry - prem) Infra including deploying, Fine-tuning, tracking Prompts and serving Open Source LLM Models with full Data Security and Optimal GPU Management. Train and Launch your LLM Application at Production scale with best Software Engineering practices. | |
- ReliableGPT 💪 - square) |
- Portkey - efficient apps. | |
- Vellum
- Weights & Biases (Prompts) - first W&B MLOps platform. Utilize W&B Prompts for visualizing and inspecting LLM execution flow, tracking inputs and outputs, viewing intermediate results, securely managing prompts and LLM chain configurations. | |
- Wordware - hosted IDE where non-technical domain experts work with AI Engineers to build task-specific AI agents. It approaches prompting as a new programming language rather than low/no-code blocks. | |
- xTuring - tuning. |  |
- ZenML - io/zenml.svg?style=flat-square) |
- AquilaDB - NN search. |  |
- Awadb - ai/awadb.svg?style=flat-square) |
- Chroma - core/chroma.svg?style=flat-square) |
- Infinity - embeddings |  |
- Lancedb - friendly, serverless vector database for AI applications. Easily add long-term memory to your LLM apps! |  |
- Marqo - ai/marqo.svg?style=flat-square) |
- Milvus - io/milvus.svg?style=flat-square) |
- Pinecone - performance vector search applications. Developer-friendly, fully managed, and easily scalable without infrastructure hassles. | |
- pgvector - source vector similarity search for Postgres. |  |
- pgvecto.rs - square) |
- Qdrant - square) |
- txtai - powered semantic search applications |  |
- Vald - square) |
- Vearch - based vector retrieval |  |
- VectorDB - no more, no less. |  |
- Vellum - of-box support for OCR, text chunking, embedding model experimentation, metadata filtering, and production-grade APIs. | |
- Weaviate - tolerance and scalability of a cloud-native database, all accessible through GraphQL, REST, and various language clients. |  |
- CodeGeeX - square) |
- CodeGen - source model for program synthesis. Trained on TPU-v4. Competitive with OpenAI Codex. |  |
- CodeT5 - square) |
- Continue - source autopilot for software development—bring the power of ChatGPT to VS Code |  |
- fauxpilot - source alternative to GitHub Copilot server |  |
- tabby - hosted AI coding assistant. An opensource / on-prem alternative to GitHub Copilot. |  |
- code server - server.svg?style=flat-square) |
- conda - agnostic, system-level binary package manager and ecosystem. |  |
- Docker - source project created by Docker to enable and accelerate software containerization. |  |
- envd - square) |
- Jupyter Notebooks - based notebook environment for interactive computing. |  |
- Kurtosis - container environments. |  |
- Wordware - hosted IDE where non-technical domain experts work with AI Engineers to build task-specific AI agents. It approaches prompting as a new programming language rather than low/no-code blocks. | |
- alpaca-lora - tune LLaMA on consumer hardware |  |
- finetuning-scheduler - tuning schedules. |  |
- Flyflow - devs/flyflow.svg?style=flat-square) |
- LMFlow - square) |
- Lora - rank adaptation to quickly fine-tune diffusion models. |  |
- peft - of-the-art Parameter-Efficient Fine-Tuning. |  |
- p-tuning-v2 - tuning on small/medium-sized models and sequence tagging challenges. [(ACL 2022)](https://arxiv.org/abs/2110.07602) |  |
- QLoRA - bit finetuning task performance. |  |
- TRL - square) |
- Accelerate - GPU, TPU, mixed-precision. |  |
- Apache MXNet - aware Dataflow Dep Scheduler. |  |
- axolotl - tuning of various AI models, offering support for multiple configurations and architectures. |  |
- Caffe - square) |
- ColossalAI - scale model training system with efficient parallelization techniques. |  |
- DeepSpeed - square) |
- Horovod - square) |
- Jax - performance machine learning research. |  |
- Kedro - source Python framework for creating reproducible, maintainable and modular data science code. |  |
- Keras - team/keras.svg?style=flat-square) |
- LightGBM - square) |
- MegEngine - to-use deep learning framework, with auto-differentiation. |  |
- metric-learn - learn-contrib/metric-learn.svg?style=flat-square) |
- MindSpore - ai/mindspore.svg?style=flat-square) |
- Oneflow - centered and open-source deep learning framework. |  |
- PaddlePaddle - square) |
- PyTorch - square) |
- PyTorch Lightning - AI/lightning.svg?style=flat-square) |
- XGBoost - square) |
- scikit-learn - learn/scikit-learn.svg?style=flat-square) |
- TensorFlow - square) |
- VectorFlow - square) |
- Aim - to-use and performant open-source experiment tracker. |  |
- ClearML - Magical CI/CD to streamline your ML workflow. Experiment Manager, MLOps and Data-Management. |  |
- Comet - ml/comet-examples.svg?style=flat-square) |
- Guild AI - square) |
- MLRun - square) |
- Kedro-Viz - Viz is an interactive development tool for building data science pipelines with Kedro. Kedro-Viz also allows users to view and compare different runs in the Kedro project. |  |
- LabNotebook - square) |
- Sacred - square) |
- Weights & Biases - powered applications, featuring W&B Prompts for LLM execution flow visualization, input and output monitoring, and secure management of prompts and LLM chain configurations. |  |
- Fiddler AI - production to production. Ship more ML and LLMs into production, and monitor ML and LLM metrics like hallucination, PII, and toxicity. |  |
- Maniford - agnostic visual debugging tool for machine learning. |  |
- netron - square) |
- OpenOps - Bring multiple data streams into one dashboard.
- TensorBoard - square) |
- TensorSpace - trained deep learning models from TensorFlow, Keras, TensorFlow.js. |  |
- dtreeviz - square) |
- Zetane Viewer - square) |
- Zeno - ml/zeno.svg?style=flat-square) |
- FastEdit - square) |
- ArtiVC - square) |
- Dolt - square) |
- DVC - Git for Data & Models - ML Experiments Management. |  |
- Delta-Lake - io/delta.svg?style=flat-square) |
- Pachyderm - square) |
- Quilt - organizing data hub for S3. |  |
- JuiceFS - square) |
- LakeFS - like capabilities for your object storage. |  |
- Lance - ai/lance.svg?style=flat-square) |
- Piperider - square) |
- LUX - org/lux.svg?style=flat-square) |
- Featureform - square) |
- FeatureTools - square) |
- Upgini - to-use features from public and community shared data sources and enriches your training dataset with only the accuracy improving features |  |
- Feast - dev/feast.svg?style=flat-square) |
- Comet - ml/comet-examples.svg?style=flat-square) |
- ClearML - Magical CI/CD to streamline your ML workflow. Experiment Manager, MLOps and Data-Management. |  |
- Hopsworks - tuning and serving LLMs. Hopsworks includes both a feature store and vector database for RAG. |  |
- OpenLLM - tune, serve, deploy, and monitor any LLMs with ease. |  |
- MLflow - square) |
- MLRun - square) |
- ModelFox - square) |
- Kserve - square) |
- Kubeflow - square) |
- PAI - square) |
- Polyaxon - square) |
- Primehub - square) |
- OpenModelZ - click machine learning deployment (LLM, text-to-image and so on) at scale on any cluster (GCP, AWS, Lambda labs, your home lab, or even a single machine). |  |
- Seldon-core - core.svg?style=flat-square) |
- Starwhale - tuning. |  |
- TrueFoundry - tune and serve LLM Models on a company’s own Infrastructure with Data Security and Optimal GPU and Cost Management. Launch your LLM Application at Production scale with best DevSecOps practices. | |
- Weights & Biases - powered applications, featuring W&B Prompts for LLM execution flow visualization, input and output monitoring, and secure management of prompts and LLM chain configurations. |  |
- Airflow - square) |
- aqueduct - Source Platform for Production Data Science |  |
- Argo Workflows - workflows.svg?style=flat-square) |
- Flyte - native workflow automation platform for complex, mission-critical data and ML processes at scale. |  |
- Hamilton - inc/hamilton.svg?style=flat-square) |
- Kubeflow Pipelines - square) |
- LangFlow - and-drop components and a chat interface. |  |
- Metaflow - life data science projects with ease! |  |
- Ploomber - square) |
- Prefect - square) |
- VDP - source unstructured data ETL tool to streamline the end-to-end unstructured data processing pipeline. |  |
- ZenML - io/zenml.svg?style=flat-square) |
- Kueue - native Job Queueing. |  |
- PAI - sourced by Microsoft). |  |
- Slurm - square) |
- Volcano - sh/volcano.svg?style=flat-square) |
- Yunikorn - weight, universal resource scheduler for container orchestrator systems. |  |
- Comet - ml/comet-examples.svg?style=flat-square) |
- dvc - Data Version Control - Git for Data & Models |  |
- ModelDB - square) |
- MLEM - square) |
- ormb - square) |
- ONNX-MLIR - mlir.svg?style=flat-square) |
- TVM - square) |
- octoml-profile - profile is a python library and cloud service designed to provide the simplest experience for assessing and optimizing the performance of PyTorch models on cloud hardware with state-of-the-art ML acceleration technology. |  |
- scalene - performance, high-precision CPU, GPU, and memory profiler for Python |  |
- Archai - square) |
- autoai - square) |
- AutoGL - square) |
- AutoGluon - square) |
- automl-gs - gs.svg?style=flat-square) |
- autokeras - team/autokeras.svg?style=flat-square) |
- Auto-PyTorch - PyTorch.svg?style=flat-square) |
- auto-sklearn - in replacement for a scikit-learn estimator. |  |
- Dragonfly - square) |
- Determined - ai/determined.svg?style=flat-square) |
- DEvol (DeepEvolution) - square) |
- EvalML - square) |
- FEDOT - itmo/FEDOT.svg?style=flat-square) |
- FLAML - us/research/publication/flaml-a-fast-and-lightweight-automl-library/)). |  |
- Goptuna - bata/goptuna.svg?style=flat-square) |
- HpBandSter - square) |
- HPOlib2 - square) |
- Hyperband - square) |
- Hypernets - square) |
- Hyperopt - square) |
- hyperunity - box hyperparameter optimisation. |  |
- Intelli - square) |
- Katib - native project for automated machine learning (AutoML). |  |
- Keras Tuner - team/keras-tuner.svg?style=flat-square) |
- learn2learn - learning Framework for Researchers. |  |
- Ludwig - square) |
- MOE - square) |
- Model Search - square) |
- NASGym - of-concept OpenAI Gym environment for Neural Architecture Search (NAS). |  |
- NNI - parameter tuning. |  |
- Optuna - square) |
- Pycaret - source, low-code machine learning library in Python that automates machine learning workflows. |  |
- REMBO - dimensions via random embedding. |  |
- RoBO - square) |
- scikit-optimize(skopt) - based optimization with a `scipy.optimize` interface. |  |
- Spearmint - square) |
- TPOT - source software packages. |  |
- Torchmeta - Learning library for PyTorch. |  |
- Vegas - noah/vega.svg?style=flat-square) |
- FeatherCNN - square) |
- Forward - square) |
- NCNN - performance neural network inference framework optimized for the mobile platform. |  |
- PocketFlow - square) |
- TensorFlow Model Optimization - optimization.svg?style=flat-square) |
- TNN - square) |
- EasyFL - to-use Federated Learning Platform |  |
- FATE - square) |
- FedML - scale cross-silo federated learning, cross-device federated learning on smartphones/IoTs, and research simulation. |  |
- Flower - square) |
- Harmonia - source project aiming at developing systems/infrastructures and libraries to ease the adoption of federated learning (abbreviated to FL) for researches and production usage. |  |
- TensorFlow Federated - square) |
- Awesome Argo - argo.svg?style=flat-square) |
- Awesome AutoDL - depth analysis) |  |
- Awesome AutoML - related research, tools, projects and other resources |  |
- Awesome AutoML Papers - automl-papers.svg?style=flat-square) |
- Awesome-Code-LLM - LLM for research. |  |
- Awesome Federated Learning Systems - paper.svg?style=flat-square) |
- Awesome Federated Learning - organized from Arxiv (mostly) |  |
- awesome-federated-learning - federated-learning.svg?style=flat-square) |
- Awesome Open MLOps - open-mlops.svg?style=flat-square) |
- Awesome Production Machine Learning - production-machine-learning.svg?style=flat-square) |
- Awesome Tensor Compilers - tensor-compilers.svg?style=flat-square) |
- kelvins/awesome-mlops - mlops.svg?style=flat-square) |
- visenger/awesome-mlops - An awesome list of references for MLOps |  |
- currentslab/awesome-vector-search - vector-search.svg?style=flat-square) |
- pleisto/flappy - Ready LLM Agent SDK for Every Developer |  |
Programming Languages
Keywords
machine-learning
97
python
54
deep-learning
53
mlops
43
llm
38
data-science
36
pytorch
34
ai
26
tensorflow
22
ml
21
automl
21
kubernetes
21
llmops
17
large-language-models
15
chatgpt
15
inference
15
hyperparameter-optimization
14
vector-database
14
keras
13
gpu
11
vector-search
11
llama
11
gpt
11
neural-architecture-search
11
openai
10
golang
9
artificial-intelligence
9
docker
9
llms
9
hyperparameter-tuning
8
scikit-learn
8
developer-tools
8
data-engineering
8
workflow
8
go
8
nearest-neighbor-search
8
transformers
8
generative-ai
7
chatbot
7
model-serving
7
rag
7
cloud-native
7
fine-tuning
7
neural-network
7
distributed-systems
7
federated-learning
7
embeddings
7
language-model
7
langchain
7
jupyter
6