Application Framework

• Helicone - commit/helicone/helicone?color=green)
• OpenLIT - native LLM Observability, GPU Monitoring, Guardrails, Evaluations, Prompt Management, Vault, Playground. 🚀💻 Integrates with 50+ LLM Providers, VectorDBs, Agent Frameworks and GPUs.   
• PostHog - source web & product analytics, session recording, feature flagging and A/B testing that you can self-host. Get started - free.   
• phoenix - ai/phoenix.svg?style=flat&color=green)  
• DeepEval - ai/deepeval.svg?style=flat&color=green)  
• Evidently - source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics.   
• lunaary - ai/lunary.svg?style=flat&color=green)  
• Weave - powered applications, built by Weights & Biases.   
• Neurolink - provider support, and professional CLI. Build, test, and deploy AI applications with multiple ai providers.   
• Langfuse - commit/langfuse/langfuse?color=green)
• ragas - commit/explodinggradients/ragas?color=green)

Agent Framework

• Agno - source framework for building multi-agent systems with memory, knowledge and reasoning.   
• AutoGPT - gravitas/autogpt.svg?style=flat&color=green)  
• crewAI - playing, autonomous AI agents. By fostering collaborative intelligence, CrewAI empowers agents to work together seamlessly, tackling complex tasks.   
• kagent - dev/kagent.svg?style=flat&color=green)   
• PydanticAI - ai.svg?style=flat&color=green)  
• Semantic Kernel - edge LLM technology quickly and easily into your apps.   
• LangGraph - ai/langgraph.svg?style=flat&color=green)  
• OpenAI Agents SDK - agent workflows.   
• fast-agent - agent.svg?style=flat&color=green)  
• CAMEL - agent framework. Finding the Scaling Law of Agents.   
• Suna - Open Source Generalist AI Agent   
• Swarm - agent orchestration. Managed by OpenAI Solution team.    
• Qwen-Agent - Agent.svg?style=flat&color=green)  
• Agent Development Kit (ADK) - source, code-first Python toolkit for building, evaluating, and deploying sophisticated AI agents with flexibility and control.   
• autogen - commit/microsoft/autogen?color=green)
• Flowise - commit/flowiseai/flowise?color=green)
• MetaGPT - Agent Framework: First AI Software Company, Towards Natural Language Programming.   

RAG

• RAG-Anything - Anything: All-in-One RAG Framework"   
• GraphRAG - based Retrieval-Augmented Generation (RAG) system.   
• LightRAG - Augmented Generation"   
• quivr - commit/QuivrHQ/quivr?color=green)
• RAGFlow - source RAG (Retrieval-Augmented Generation) engine based on deep document understanding.   

Workflow

• Haystack - ready LLM applications. Connect components (models, vector DBs, file converters) to pipelines or agents that can interact with your data. With advanced retrieval methods, it's best suited for building RAG, question answering, semantic search or conversational agent chatbots.   
• LangChain - aware reasoning applications.   
• Inference - commit/roboflow/inference?color=green) 
• Dify - ready platform for agentic workflow development.   
• FastGPT - based platform built on the LLMs, offers a comprehensive suite of out-of-the-box capabilities such as data processing, RAG retrieval, and visual AI workflow orchestration, letting you easily develop and deploy complex question-answering systems without the need for extensive setup or configuration.   
• LlamaIndex - powered agents over your data.   

Agent

• OpenManus - commit/mannaandpoem/openmanus?color=green)
• Claudia - Create custom agents, manage interactive Claude Code sessions, run secure background agents, and more.   

Evolutionary Framework

• AIDE ML - Driven Exploration in the Space of Code. The machine Learning engineering agent that automates AI R&D.   
• OpenEvolve - source implementation of AlphaEvolve   
• ShinkaEvolve - Ended and Sample-Efficient Program Evolution   
• AI-Researcher - Researcher: Autonomous Scientific Innovation" -- A production-ready version: https://novix.science/chat   

Workflow

• Seldon-Core - core.svg?style=flat&color=green)  
• ZenML - io/zenml.svg?style=flat&color=green)  
• Flyte - commit/flyteorg/flyte?color=green)
• Kubeflow - commit/kubeflow/kubeflow?color=green)
• Metaflow - commit/netflix/metaflow?color=green)
• MLflow - commit/mlflow/mlflow?color=green)
• Polyaxon - commit/polyaxon/polyaxon?color=green)
• Ray - project/ray.svg?style=flat&color=green)  

FineTune

• MLX-VLM - VLM is a package for inference and fine-tuning of Vision Language Models (VLMs) on your Mac using MLX.   
• Axolotl - ai-cloud/axolotl.svg?style=flat&color=green)  
• LLaMa-Factory - Tuning of 100+ LLMs & VLMs (ACL 2024).   
• torchtune - training library.   
• Transformer Lab - tune, and evaluate large language models on your own computer.   
• EasyLM - training, finetuning, evaluating and serving LLMs in JAX/Flax.   
• LMFlow - commit/optimalscale/lmflow?color=green)
• maestro - tuning process for multimodal models: PaliGemma 2, Florence-2, and Qwen2.5-VL.   
• Swift - parameter to finetune 450+ LLMs (Qwen2.5, InternLM3, GLM4, Llama3.3, Mistral, Yi1.5, Baichuan2, DeepSeek-R1, ...) and 150+ MLLMs (Qwen2.5-VL, Qwen2-Audio, Llama3.2-Vision, Llava, InternVL2.5, MiniCPM-V-2.6, GLM4v, Xcomposer2.5, Yi-VL, DeepSeek-VL2, Phi3.5-Vision, GOT-OCR2, ...).   
• unsloth - R1 & Reasoning LLMs 2x faster with 70% less memory! 🦥   

Evaluation

• lm-evaluation-harness - shot evaluation of language models.   
• opik - ready dashboards.   
• AgentBench - commit/thudm/agentbench?color=green)
• ASQI Engineer - run containerised AI tests and map to score cards!   
• LiveBench - Free LLM Benchmark   
• terminal-bench - institute/terminal-bench.svg?style=flat&color=green)  
• OpenCompass - 4,LLaMa2, Qwen,GLM, Claude, etc) over 100+ datasets.   
• LongBench - commit/thudm/longbench?color=green)

Alignment

• OpenRLHF - to-use, Scalable and High-performance RLHF Framework (70B+ PPO Full Tuning & Iterative DPO & LoRA & RingAttention & RFT).   
• Self-RLHF - alignment/safe-rlhf.svg?style=flat&color=green)  

Framework

• AXLearn - commit/apple/axlearn?color=green)
• DLRover - machine-learning/dlrover.svg?style=flat&color=green)  
• Candle - commit/huggingface/candle?color=green)
• Ludwig - code framework for building custom LLMs, neural networks, and other AI models.   
• MaxText - commit/google/maxtext?color=green)
• MLX - explore/mlx.svg?style=flat&color=green)  
• ColossalAI - commit/hpcaitech/ColossalAI?color=green)

LLM Router

• LLMRouter - Source Library for LLM Routing   
• LiteLLM - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq].   
• AI Gateway - ai/gateway.svg?style=flat&color=green)  
• bifrost - commit/maximhq/bifrost?color=green)
• vLLM Semantic Router - of-Models Router for Efficient LLM Inference   
• RouteLLM - save LLM costs without compromising quality.   

Inference Engine

• web-llm - performance In-browser LLM Inference Engine.    
• transformers.js - of-the-art Machine Learning for the web. Run 🤗 Transformers directly in your browser, with no need for a server!    
• Xinference - source language models, speech recognition models, and multimodal models, whether in the cloud, on-premises, or even on your laptop.   
• LMDeploy - commit/internlm/lmdeploy?color=green)
• MInference - context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to 10x for pre-filling on an A100 while maintaining accuracy.    
• zml - commit/zml/zml?color=green)
• TinyGrad - commit/tinygrad/tinygrad?color=green)
• OpenVINO - commit/openvinotoolkit/openvino?color=green)
• Cortex.cpp - commit/janhq/cortex.cpp?color=green)
• DeepSpeed-MII - latency and high-throughput inference possible, powered by DeepSpeed.   
• llama-box - box.svg?style=flat&color=green)  
• OpenLLM - source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.   
• Text Generation Inference - generation-inference.svg?style=flat&color=green)  
• vLLM - throughput and memory-efficient inference and serving engine for LLMs.   
• MLC LLM - ai/mlc-llm.svg?style=flat&color=green)  
• MLServer - model serving and more.   
• Ollama - R1, Phi-4, Gemma 3, and other large language models.   
• LoRAX - LoRA inference server that scales to 1000s of fine-tuned LLMs.    
• Llumnix - instance LLM serving.   
• Nvidia Dynamo - dynamo/dynamo.svg?style=flat&color=green)  
• Petals - style. Fine-tuning and inference up to 10x faster than offloading   
• Ratchet - platform browser ML framework.    
• SGLang - project/sglang.svg?style=flat&color=green)  
• Triton Inference Server - inference-server/server.svg?style=flat&color=green)  

Inference Platform

• llm-d - d is a Kubernetes-native high-performance distributed LLM inference framework   
• Mooncake - ai/mooncake.svg?style=flat&color=green)  
• OME - grade management and serving of Large Language Models (LLMs)   
• llmaz - commit/inftyai/llmaz?color=green)
• AIBrix - efficient and pluggable Infrastructure components for GenAI inference.   
• BentoML - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!   
• KubeAI - to-text.   
• beta9 - cloud/beta9.svg?style=flat&color=green)  
• Kaito - model inference and fine-tuning, with GPU auto-provisioning, container-based hosting, and CRD-based orchestration.   
• Kserve - commit/kserve/kserve?color=green)
• Modular - commit/modular/modular?color=green)

AI Gateway

• Envoy AI Gateway - gateway.svg?style=flat&color=green)  
• Kong - Native API Gateway and AI Gateway.   
• kgateway - Native API Gateway and AI Gateway.   
• gateway-api-inference-extension - sigs/gateway-api-inference-extension.svg?style=flat&color=green)  
• APISIX - Native API Gateway and AI Gateway with extensive plugin system and AI capabilities.   
• LLM Semantic Router - of-Models (MoM) System with Privacy Preservation and Prompt Guard. The semantic router intelligently directs OpenAI compliant API requests to the most suitable backend models based on semantic understanding of request content.   
• agentgateway - commit/agentgateway/agentgateway?color=green)
• Higress - commit/alibaba/higress?color=green)

Output

• BAML - commit/boundaryml/baml?color=green)
• Instructor - ai/instructor.svg?style=flat&color=green)  
• Outlines - ai/outlines.svg?style=flat&color=green)  
• XGrammar - ai/xgrammar.svg?style=flat&color=green)  

Benchmark

• genai-bench - bench is a powerful benchmark tool designed for comprehensive token-level performance evaluation of large language model (LLM) serving systems.   
• Inference Benchmark - Hypercomputer/inference-benchmark.svg?style=flat&color=green)  
• Inference Perf - sigs/inference-perf.svg?style=flat&color=green)  

• Nanoflow - oriented high-performance serving framework for LLMs |
• TensorRT-LLM - LLM) |  |  | TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs.||

Middleware

• Checkpoint Engine - engine is a simple middleware to update model weights in LLM inference engines   
• LMCache - Context LLM By Smart KV Cache Optimizations.    
• kvcached - project/kvcached.svg?style=flat&color=green)  

Simulator

• Vidur - scale simulation framework for LLM inference   

Code Agent

• Open SWE - Source Asynchronous Coding Agent   
• SWE-agent - agent takes a GitHub issue and tries to automatically fix it, using your LM of choice. It can also be employed for offensive cybersecurity or competitive coding challenges. [NeurIPS 2024]   
• Tabby - hosted AI coding assistant.   
• Continue - source IDE extensions and hub of models, rules, prompts, docs, and other building blocks.   

Tool

• Browser Use - use/browser-use.svg?style=flat&color=green)  
• Mem0 - commit/mem0ai/mem0?color=green)
• OpenAI CUA - cua-sample-app.svg?style=flat&color=green)  
• Graphiti - Time Knowledge Graphs for AI Agents.   

AI Terminal

• Stakpak - commit/stakpak/agent?color=green)
• Gemini CLI - source AI agent that brings the power of Gemini directly into your terminal.   
• kubectl-ai - ai.svg?style=flat&color=green)  
• Codex - commit/openai/codex?color=green) 
• aider - AI/aider.svg?style=flat&color=green)  
• OpenCode - commit/sst/opencode?color=green)
• Crush - commit/charmbracelet/crush?color=green)

Database

• weaviate - source vector database that stores both objects and vectors, allowing for the combination of vector search with structured filtering with the fault tolerance and scalability of a cloud-native database​.   
• deeplake - time to PyTorch/TensorFlow.   
• Faiss - commit/facebookresearch/faiss?color=green)
• chroma - native open-source embedding database.   
• milvus - performance, cloud-native vector database built for scalable vector ANN search.   

Code Assistant

• Auto-dev - powered coding wizard（AI 驱动编程助手）with multilingual support 🌐, auto code generation 🏗️, and a helpful bug-slaying assistant 🐞! Customizable prompts 🎨 and a magic Auto Dev/Testing/Document/Agent feature 🧪 included! 🚀.   
• Codefuse-chatbot - Agent Framework, working with DevOps Toolkits, Code&Doc Repo RAG, etc.   
• Cody - commit/sourcegraph/cody?color=green)
• Sweep - commit/sweepai/sweep?color=green)

AI Agent

• Magentic-UI - centered web agent   
• goose - install, execute, edit, and test with any LLM   
• Tongyi Deep Research - source DeepResearch Agent   
• OpenManus - commit/FoundationAgents/openmanus?color=green)

Chatbot

• 5ire - platform desktop AI assistant, MCP client. It compatible with major service providers, supports local knowledge base and tools via model context protocol servers.   
• NextChat - commit/chatgptnextweb/nextchat?color=green)
• FastChat - sys/fastchat.svg?style=flat&color=green)  
• Gradio - app/gradio.svg?style=flat&color=green)  
• LLM - line   
• Lobe Chat - an open-source, modern-design AI chat framework. Supports Multi AI Providers( OpenAI / Claude 3 / Gemini / Ollama / DeepSeek / Qwen), Knowledge Base (file upload / knowledge management / RAG ), Multi-Modals (Plugins/Artifacts) and Thinking. One-click FREE deployment of your private ChatGPT/ Claude / DeepSeek application.   
• AnythingLLM - in-one Desktop & Docker AI application with built-in RAG, AI agents, No-code agent builder, MCP compatibility, and more.   
• Chat SDK - featured, hackable Next.js AI chatbot built by Vercel   
• Chatbot UI - ui.svg?style=flat&color=green)  
• Cherry Studio - r1.   
• Open WebUI - friendly AI Interface (Supports Ollama, OpenAI API, ...).   
• Jan - commit/janhq/jan?color=green)
• opcode - Create custom agents, manage interactive Claude Code sessions, run secure background agents, and more.   
• PrivateGPT - ai/private-gpt.svg?style=flat&color=green)  

Evolve Agent

• AgentEvolver - Evolving Agent System   
• EvoAgentX - Evolving Ecosystem of AI Agents   

Observation

• OpenLLMetry - source observability for your LLM application, based on OpenTelemetry.   
• wandb - tune models, and manage models from experimentation to production.   

Sandbox

• Daytona - Generated Code.   
• E2B - dev/E2B.svg?style=flat&color=green)  

Scheduling

• KAI Scheduler - Scheduler.svg?style=flat&color=green)  
• Project-HAMi - HAMi/HAMi.svg?style=flat&color=green)  

Management

• NVIDIA GPU Operator - operator.svg?style=flat&color=green)  

MCP Server

• MCPMarket
• mcp-directory - directory.svg?style=flat&color=green)  
• awesome-mcp-servers - mcp-servers.svg?style=flat&color=green)  
• BaiLian MCP
• Cline MCP Marketplace
• Docker MCP Catalog - quality MCP servers as Docker Images, spanning database solutions, developer tools, productivity platforms, and API integrations.
• Higress MCP Marketplace
• MCPMarket
• ModelScope MCP

MCP Client

• awesome-mcp-clients - mcp-clients.svg?style=flat&color=green)