Awesome-AI-Engineering
A full-stack LLM engineering playbook- practical guides for building, deploying, and evaluating LLM systems and AI agents.
https://github.com/Eric-LLMs/Awesome-AI-Engineering
Last synced: 1 day ago
JSON representation
-
📑 Table of Contents
-
📑 Further Reading / Resources
- Agent-Memory-Paper-List - Liu.
-
🛠️ Hands-on: A Minimal ReAct Agent
-
🛠️ Hands-on Lab & Examples
- LLMs-Lab - Tuning**, **RAG optimization**, **LangChain**, **Prompt Engineering**, **Function-Calling**, **Agent**, etc. — each studied as a standalone module with multiple project implementations.
- DeepDive - grade AI system for personalized learning — agentic tutoring, dual-track memory, hybrid RAG, personal/team cloud storage, and self-hosted LLM infrastructure. Highlights:
-
🛠️ Hands-on Projects & Tools
- Explore Model Context Protocol (MCP) Projects on GitHub - standard Model Context Protocol (MCP) server implementations.*
- AIInfra — AI Infrastructure Reference - source reference covering the full-stack AI infrastructure for LLMs — from AI chips, compute clusters, and high-speed networking, to distributed training, inference optimization, and deployment — a hands-on resource for building high-performance, private AI infrastructure for the enterprise.
- vLLM - throughput, memory-efficient LLM serving engine (PagedAttention, continuous batching, prefix caching) — the de facto standard for high-performance private inference.
- SGLang - generation runtime for LLM inference, complementing vLLM with radix attention and efficient prefix reuse.
- DeepSpeed - tuning models at massive scale.
- Megatron-LM - scale language model training framework — tensor, pipeline, and sequence parallelism, often paired with DeepSpeed for pre-training.
- Unsloth - efficient fine-tuning library — up to 2x faster and 70% less memory for LoRA/QLoRA fine-tuning of LLMs.
- LLaMA-Factory - driven fine-tuning platform supporting LoRA, QLoRA, and full-parameter tuning across many open LLMs.
- HuggingFace PEFT - efficient fine-tuning library — LoRA, QLoRA, and more — widely used for enterprise model customization.
- Volcano - native batch system with advanced GPU scheduling and job queueing — widely adopted for enterprise AI clusters.
- KubeRay - native orchestration.
- LiteLLM - compatible API — model routing, rate limits, budgets, and logging for enterprise private deployments.
- Ray
- Ollama - hosted private deployment option.
- LocalAI - compatible, self-hosted inference server for private model deployment.
- NeMo Guardrails
- PurpleLlama / Llama Guard - safety classifier and Prompt Guard injection detection.
- Garak - teaming.
- LLM Guard
- Microsoft Presidio
- veRL - grade RL post-training framework — supports PPO, GRPO, DAPO, PRIME, and multi-turn tool-calling agents, with vLLM/SGLang rollout and FSDP/Megatron-LM training backends.
- OpenRLHF - performance distributed RLHF framework built on Ray, vLLM, and DeepSpeed — supporting PPO, GRPO, REINFORCE++, and multi-turn agent training.
- TRL - training — the broadest algorithm coverage (SFT, DPO, GRPO, PPO, RLOO) with first-party OpenEnv integration.
- ART
-
🧰 Key Frameworks & Code Samples
- Mem0 - layer memory framework supporting working, factual, and semantic memory types for agent state persistence.
- Letta (MemGPT)
- LangMem
- Amazon Bedrock Samples
- Zep / Graphiti - graph framework — builds a dynamic, time-aware memory graph for agent state with causal event support.
-
🧰 Key Frameworks & Tools
- typescript-sdk
- python-sdk
- servers — Official Reference Implementations
- specification
- registry - driven MCP server registry — an "app store" for discovering standardized MCP servers.
- FastMCP - level Python framework for building MCP servers — define tools and resources in a few lines of code.
-
🧰 Key Open-Source Projects & References
- Claude Code - based agentic coding tool | Agentic coding, terminal-native, full codebase understanding, git workflows |
- Hermes-Agent - improving AI agent with built-in learning loop | Skill creation from experience, cross-session memory, multi-channel (CLI/Telegram/Discord/Slack) |
- OpenClaw - first, any OS/platform | Local-first Gateway, multi-channel messaging, voice support, session & tool management |
- claurst - maintained reference implementation of Claude Code | Internal architecture study, reverse-engineering insights, codebase structure reference |
- OpenAI Codex - source agentic coding CLI | Agentic coding, terminal-native, sandboxed execution, bash tool use |
- Pi - source AI agent toolkit: unified multi-provider LLM API, agent runtime, and an interactive coding agent CLI | Self-extensible coding agent, multi-provider API, terminal UI library, npm supply-chain hardening |
- DeepSeek Harness - source agent harness with a plugin-driven architecture ("everything is a plugin") | Plugin composability via Cordis, Web UI, monorepo, developer preview |
-
🧰 Key Tools, Frameworks & Strategies
- LangSmith
- LangFuse - source alternative for observability, prompt management, and lightweight evaluation. |
- DeepEval
- OpenEvals - built "LLM-as-a-judge" prompts (Conciseness, Correctness, Coherence) compatible with LangSmith. |
- AgentOps
- Promptfoo - teaming. |
- Braintrust - as-a-judge scoring, online evals, and A/B testing. |
- Arize Phoenix - source LLM tracing, embedding analysis, and RAG/agent evaluation. |
- W&B Weave
- OpenAI Evals - source framework for model-graded evals — YAML/JSON config-driven, custom eval classes, and dataset registries. |
-
🔑 Mind Map (Framework Overview)
-
🔑 Mind Map (Key Concepts)
-
📑 Presentation Slides
-
🔗 Related Protocols
- A2A (Agent2Agent Protocol) - to-agent communication — lets agents discover each other's capabilities and collaborate on tasks.
- ACP (Agent Client Protocol)
-
Programming Languages
Categories
Sub Categories
🛠️ Hands-on Projects & Tools
24
🧰 Key Tools, Frameworks & Strategies
10
📑 Presentation Slides
7
🧰 Key Open-Source Projects & References
7
🧰 Key Frameworks & Tools
6
🧰 Key Frameworks & Code Samples
5
🔑 Mind Map (Key Concepts)
5
🛠️ Hands-on Lab & Examples
2
🔗 Related Protocols
2
📑 Further Reading / Resources
1
🔑 Mind Map (Framework Overview)
1
🛠️ Hands-on: A Minimal ReAct Agent
1
Keywords
llm
16
ai
10
llama
6
qwen
6
python
6
transformers
6
reinforcement-learning
5
openai
5
mcp
5
llms
5
llmops
5
large-language-models
5
deepseek
4
rag
4
llm-evaluation
4
anthropic
4
machine-learning
4
langchain
4
mistral
4
chatgpt
4
agent
4
agents
4
pytorch
4
ai-agents
4
deep-learning
3
lora
3
mcp-server
3
mcp-client
3
prompt-engineering
3
inference
3
gemma
3
kubernetes
3
llama3
3
gpt
2
cuda
2
fine-tuning
2
transformer
2
llm-security
2
evals
2
llm-evaluation-framework
2
moe
2
llm-eval
2
evaluation-framework
2
ray
2
diffusion
2
distributed
2
openclaw
2
image-generation
2
bedrock
2
llm-serving
2