awesome-llm-tools
A curated directory of essential LLM tools: Local inference engines, agent frameworks, RAG pipelines, vector databases, and fine-tuning kits.
https://github.com/sam-blackfly/awesome-llm-tools
Last synced: 11 days ago
JSON representation
-
10. Model Preparation & Quantization
-
Data & Alignment Tools
- mergekit
- bitsandbytes - bit optimizers and 4-bit quantization | Quantization Base |
- AutoAWQ
- ExLlamaV2
- outlines - ai/outlines?style=flat-square) | Apache-2.0 |
-
-
11. Benchmarks & Leaderboards
-
Data & Alignment Tools
- LMSYS Chatbot Arena - based leaderboard |
- Open LLM Leaderboard - source LLMs across multiple benchmarks |
- MMLU
- HumanEval
- SWE-bench
- LiveCodeBench - free evaluation of LLMs for code |
-
-
12. Miscellaneous
-
Data & Alignment Tools
- llmfit - square) | MIT |
- selfhostllm - hosted LLM inference |  | MIT |
- gitingest - friendly text dump; replace "hub" with "ingest" in any GitHub URL |  | MIT |
- Repomix - friendly formats; similar to gitingest but focused on Node ecosystem |  | MIT |
- Stenography - ai/stenography?style=flat-square) | Proprietary |
- Phoenix - source AI observability platform for tracing, evaluating, and monitoring LLM apps |  | Apache-2.0 |
- OpenLIT - source GenAI and LLM observability platform native to OpenTelemetry |  | Apache-2.0 |
- Openllmetry - source observability for LLM applications based on OpenTelemetry |  | Apache-2.0 |
- MLflow - source platform for the machine learning lifecycle, including LLM tracking |  | Apache-2.0 |
- Trigger.dev - running AI workflows and dependable agents |  | Apache-2.0 |
- context7 - to-date code documentation for LLMs and AI code editors |  | MIT |
- deepwiki-open - powered wiki generator for GitHub/Gitlab/Bitbucket repositories |  | MIT |
- cai - square) | MIT |
- speakr - hosted web application for transcribing audio recordings |  | MIT |
- presenton - source AI presentation generator and API |  | MIT |
- Gorilla - Connected APIs; enables LLMs to use 1600+ APIs |  | Apache-2.0 |
- LlamaHub - hub?style=flat-square) | MIT |
- OmniGen2 - square) | MIT |
- 4o-ghibli-at-home - hosted AI photo stylizer for performance and privacy |  | MIT |
- Observer - agents that observe, log and react to system events |  | MIT |
- gabber - dev/gabber?style=flat-square) | MIT |
- promptcat - dependency prompt manager/catalog in a single HTML file |  | MIT |
- heretic - e-w/heretic?style=flat-square) | MIT |
- llama-swap - swap?style=flat-square) | MIT |
-
-
1. Local Agents
-
AI Browsers
- Perplexity Comet - native browser with built-in search intelligence and agentic web interaction | Web / Desktop |
- Dia
- Browserbase
- ChatGPT Atlas - aware tasks and autonomous browsing within ChatGPT | Web |
- Page Assist
- nanobrowser - source Chrome extension for AI-powered web automation |
- Skyvern - based workflows | Python / Playwright | Open-source |
-
Audio / Voice Agents
- faster-whisper - based Whisper reimplementation; 4× faster with less memory | ✅ | Speech-to-Text |
- whisper.cpp - to-Text |
- Piper - to-speech engine | ✅ | Text-to-Speech |
- Kokoro-FastAPI - compatible API | ✅ | Text-to-Speech |
- Coqui TTS - voice & multilingual | ✅ | Text-to-Speech |
- SpeechBrain - in-one speech toolkit (ASR, TTS, speaker ID, emotion recognition) | ✅ | Multi-task |
- Home Assistant Wyoming - word) | ✅ | Voice Pipeline |
- Speecht5 (HuggingFace) - text pre-training framework | ✅ | ASR + TTS |
- Vapi - latency conversational agents with natural speech | ❌ Cloud | API / Voice |
- Retell AI - time conversational AI for developers to build natural-sounding voice agents | ❌ Cloud | API / Voice |
-
Autonomous Agents
- OpenFang - built agents, MCP & A2A | Rust | Self-hosted |
- NullClaw - optimised | Zig | Self-hosted |
- ZeroClaw - swap any component | Rust | Self-hosted |
- SuperAgent
- ix - 4 agent platform; provides a full-featured UI for agent orchestration | Python | Agent OS |
- waggledance.ai - Validate-Solve approach | Python | Goal Solving |
- GPT Pilot
- Davika - source |
- n8n - agent workflow automation; combines AI nodes with 400+ business process integrations | Node.js | Low-code |
- MetaGPT - agent framework that assigns roles (Product Manager, Architect, Coder) to agents | Python | Software engineering |
- ChatDev
- GPT Prompt Engineer
- Sauna - compounding assistant that learns taste and detects patterns proactively | Python | Personal assistant |
- AgentMail
-
Coding Agents & IDE Extensions
- Continue - source local AI coding assistant |
- Aider
- Cline - compat | Autonomous coding agent in VS Code |
- KiloCode - source; supports many local & cloud LLMs |
- OpenCode - based TUI agent with LSP, Git-context; 26k+ stars |
- Antigravity - editor Agent | VS Code / JetBrains | ✅ Via API | Google DeepMind AI coding assistant |
- Copilot (GitHub) - only |
- Cody (Sourcegraph)
- Tabby - hosted | VS Code / JetBrains / Vim | ✅ Self-hosted | Self-hosted coding assistant server |
- Fauxpilot - hosted | VS Code (Copilot compat) | ✅ Self-hosted | Open-source GitHub Copilot alternative |
- Void - source Cursor alternative |
- Ghostwriter - powered pair programmer |
- Amazon Q
- Tabnine - grade local/cloud AI code completions |
- TurboPilot - hosted | VS Code (Copilot compat) | ✅ Local | Self-hosted Copilot clone using llama.cpp to run Salesforce Codegen |
- Lovable - stack app generation from natural language |
- Warp - native terminal with natural language commands, agent mode, and team sharing |
- Warp Oz - step tasks end-to-end with full shell context |
- Roo-Code - editor Agent | VS Code | ✅ OpenAI-compat | Autonomous coding agent formerly known as Roo-Cline |
- crush
- Plandex - compat | Long-running terminal agent for complex tasks |
- SWE-agent
- OpenHands (OpenDevin) - compat | Full autonomous software engineering agent |
- humanlayer - in-the-loop layer for AI agents to solve hard problems in complex codebases |
- ProxyAI - source AI copilot for JetBrains IDEs |
- Claude Code
- Jules
- Browser-Use - compat | Enable AI to control your browser; web automation agent |
- Neovim + llm.nvim
- Windsurf
-
Computer Use & OS Agents
- OmniParser - based GUI agent screen parsing tool | Python / Model |
- cua - Use AI agents | Docker |
- self-operating-computer
- Agent-S
- openwork - source alternative to Claude Cowork powered by OpenCode | Desktop |
- mobile-use
-
General-Purpose Local Agents
- Open WebUI - rich self-hosted web UI for Ollama / OpenAI APIs | Tools, RAG, image gen built-in |
- AnythingLLM - in-one local AI app with agents, RAG, tools | Desktop + Docker |
- LibreChat - provider AI chat with agents, plugins, RAG | Self-hosted |
- SillyTavern - based AI interactions | Self-hosted |
- Lobe Chat
- ChatBox - friendly desktop client app for AI models (Ollama, OpenAI, Claude, etc.) | Windows/macOS/Linux |
- PrivateGPT - ready private document Q&A agent | 100% local |
- Dash (Agno) - learning data / text-to-SQL agent grounded in 6 layers of context; inspired by OpenAI's internal impl | Built on Agno framework |
- visual-explainer
-
Low-code & Visual Flow
-
Research & Knowledge Agents
- Perplexica - powered search engine. An OpenSource alternative to Perplexity AI | Self-hosted |
- DocsGPT - powered chat for documentation search & assistance | Web |
- Chaindesk - code platform for semantic search and documents retrieval | Web |
- Paper QA
- DB-GPT
- gpt-researcher
- SurfSense - hosted |
- open-notebook - source implementation of NotebookLM with local model support | Web |
- RD-Agent
- local-deep-researcher
- maestro - powered research application designed to streamline complex tasks | Web |
- Phind - powered search engine for developers; answers complex technical questions with sources | Web |
- Metaphor - scale context | Web |
- You.com
- Komo - powered search engine focused on quick, relevant results | Web |
-
-
2. Libraries & Frameworks
-
Go
- LangChainGo
- Eino - agent coordination | High-performance Go services |
- adk-go - agent orchestration, tool use, Gemini/Vertex AI integration | Google ADK Go agents |
- go-openai
- ollama-go
-
Java
- Semantic Kernel - language AI SDK | Enterprise, .NET / Java ecosystem |
- Spring AI
- LangChain4j
- Quarkus AI
- Apache Camel AI
- DJL (Deep Java Library) - agnostic deep learning library for Java; run HuggingFace & PyTorch models | Java ML inference |
- openai-java
-
JavaScript / TypeScript
- LangChain.js - stack JS LLM apps |
- LlamaIndex.TS
- VoltAgent - native open-source agentic framework | TypeScript-first developers |
- Mastra
- Vercel AI SDK - stack AI toolkit for Next.js / React | AI-powered web UIs |
- GenKit (Firebase)
- Bee Agent Framework - source TypeScript agent framework | Enterprise TypeScript agents |
- openai
- anthropic
- @google/genai
- ollama-js
- ai - first unified API for LLMs, RSC support, tool calling | Universal LLM SDK for JS/TS |
- genkit
- @google/adk
-
Kotlin
- Koog - Kotlin agent framework; multiplatform (JVM, Android, iOS, WASM), fault-tolerant with retries & state restore | Enterprise Kotlin / multiplatform agents |
- Kotlin OpenAI client
-
Python
- LangChain
- LangGraph - based agent orchestration (state machines) from LangChain team | Deterministic multi-agent workflows |
- LlamaIndex - heavy, enterprise knowledge systems |
- AutoGen - agent conversation framework | Collaborative multi-agent problem solving |
- CrewAI - based multi-agent orchestration; fast & standalone | Specialized crew workflows |
- Pydantic AI - safe agent framework by Pydantic team | Production-grade, type-safe pipelines |
- Agno (Phidata) - modal agent framework | Multi-modal agents with memory & storage |
- Haystack - based NLP / RAG framework for production | Production NLP & search pipelines |
- Smolagents
- Atomic Agents - injection-based agent framework | Composable, testable agents |
- CAMEL-AI - playing communicative multi-agent framework; research and multi-agent systems | Multi-agent Communication |
- OpenAI Agents SDK - native agent workflows |
- Google ADK - agnostic framework with multi-agent orchestration, rich tool ecosystem, built-in eval | Google Cloud / Gemini agents |
- aisuite - provider prototyping |
- swarm - agent orchestration | Learning multi-agent patterns |
- guidance
- outlines
- Multi-Agent Orchestrator - turn conversations | Enterprise multi-agent systems |
- rowboat - powered multi-agent builder with no-code UI, powered by OpenAI Agents SDK | Rapid agent prototyping |
- Medusa
- LMQL
- Griptape
- Embedchain
- SymbolicAI - symbolic framework for building applications with LLMs at the core | Hybrid AI |
- Ludwig - code framework for building custom AI models like LLMs and other deep neural networks | Low-code ML |
- transformers - tuning for every major architecture | Foundation library |
- accelerate - GPU / mixed-precision |
- sentence-transformers - to-use library for state-of-the-art sentence, text, and image embeddings | Embeddings / semantic search |
- tiktoken
- openai
- anthropic
- google-genai
- google-adk
- llm
-
Rust
- Rig - ready Rust library; 20+ providers, 10+ vector stores, WASM support | Performance-critical Rust agents |
- LLM-Chain - step LLM workflows in Rust; prompt templates, chains, tool calling | Type-safe Rust pipelines |
- candle - ready, no Python dependency | Rust-native ML inference |
- async-openai - safe | OpenAI API client |
- ollama-rs
- aichat - in-one Rust CLI with shell integration, RAG, agents, and 20+ LLM providers | CLI / scripting |
-
Programming Languages
Categories
1. Local Agents
97
2. Libraries & Frameworks
68
8. Inference Engines
36
12. Miscellaneous
24
6. RAG Pipelines
17
9. Fine-Tuning
17
7. Vector Databases
12
5. Agent Memory
12
4. Context Optimization
12
3. Prompt Optimization
11
11. Benchmarks & Leaderboards
6
10. Model Preparation & Quantization
5
License
1
Sub Categories
Rust
70
Data & Alignment Tools
42
Python
34
Coding Agents & IDE Extensions
30
Desktop / Local
19
Server / Production
17
Research & Knowledge Agents
15
JavaScript / TypeScript
14
Autonomous Agents
14
Training Frameworks
11
Audio / Voice Agents
10
General-Purpose Local Agents
9
Java
7
AI Browsers
7
Computer Use & OS Agents
6
Low-code & Visual Flow
6
Go
5
Kotlin
2
Keywords
llm
86
ai
70
openai
55
python
32
rag
31
chatgpt
30
agents
24
langchain
22
llms
21
llama
21
agent
20
machine-learning
20
ai-agents
19
llmops
17
gpt
17
large-language-models
16
typescript
14
pytorch
14
open-source
13
gemini
13
artificial-intelligence
13
gpt-4
12
deep-learning
12
mcp
11
generative-ai
11
vector-database
11
anthropic
11
language-model
10
ollama
10
prompt-engineering
10
deepseek
10
developer-tools
10
javascript
9
nextjs
9
llm-inference
9
genai
9
llm-evaluation
9
claude
9
mlops
8
automation
8
qwen
8
evaluation
8
search
7
semantic-search
7
nodejs
7
mistral
7
llama3
7
chatbot
7
inference
7
golang
7