awesome-llmops

A curated list of tools, frameworks, platforms, and resources for Large Language Model Operations (LLMOps).
https://github.com/awesomelistsio/awesome-llmops

Last synced: about 5 hours ago
JSON representation

Overview & Learning
- LLMOps Guide (Weights & Biases) - level overview of LLMOps concepts and tools.
- LLMOps Field Guide (Fiddler)
- LangChain Cookbook
- Full Stack Deep Learning
Model Training & Fine-Tuning
- Hugging Face Transformers - trained and fine-tunable LLMs.
- PEFT - Efficient Fine-Tuning methods for LLMs.
- LoRA - tuning strategy for large models.
- Colossal-AI
Evaluation & Benchmarking
Serving & Inference
- vLLM - efficient inference for LLMs with continuous batching.
- TGI (Text Generation Inference) - performance inference server by Hugging Face.
- DeepSpeed MII - latency inference for Hugging Face models.
- Ray Serve
Monitoring & Observability
Prompt Engineering & Management
Data Management
- Weaviate
- Pinecone - augmented generation.
- ChromaDB - source embeddings DB built for LLMs.
- Label Studio - source data labeling for fine-tuning and RAG pipelines.
Security & Safety
- Guardrails AI
- Rebuff - source framework for prompt injection defense.
- Giskard
- OpenAI Moderation API
Platforms & Frameworks
- LangChain - to-end LLM-powered apps.
- LLamaIndex
- RAGStack (Haystack) - augmented generation framework.
- FastChat - tuning chat LLMs.
Tooling Ecosystem
- MLflow
- PromptLayer
- OpenLLM - source platform to deploy and manage LLMs in production.
Related Awesome Lists

Programming Languages

Categories

Sub Categories

Keywords

ecosyste.ms