awesome-llmops
A curated list of tools, frameworks, platforms, and resources for Large Language Model Operations (LLMOps).
https://github.com/awesomelistsio/awesome-llmops
Last synced: 2 days ago
JSON representation
-
Overview & Learning
- LLMOps Guide (Weights & Biases) - level overview of LLMOps concepts and tools.
- LLMOps Field Guide (Fiddler)
- Full Stack Deep Learning
- LangChain Cookbook
-
Model Training & Fine-Tuning
- Hugging Face Transformers - trained and fine-tunable LLMs.
- PEFT - Efficient Fine-Tuning methods for LLMs.
- Colossal-AI
- LoRA - tuning strategy for large models.
-
Monitoring & Observability
-
Data Management
- Weaviate
- Pinecone - augmented generation.
- Label Studio - source data labeling for fine-tuning and RAG pipelines.
- ChromaDB - source embeddings DB built for LLMs.
-
Related Awesome Lists
-
Tooling Ecosystem
- PromptLayer
- OpenLLM - source platform to deploy and manage LLMs in production.
- MLflow
-
Evaluation & Benchmarking
-
Serving & Inference
- vLLM - efficient inference for LLMs with continuous batching.
- TGI (Text Generation Inference) - performance inference server by Hugging Face.
- DeepSpeed MII - latency inference for Hugging Face models.
- Ray Serve
-
Security & Safety
- Rebuff - source framework for prompt injection defense.
- Guardrails AI
- Giskard
- OpenAI Moderation API
-
Prompt Engineering & Management
-
Platforms & Frameworks
- LangChain - to-end LLM-powered apps.
- LLamaIndex
- RAGStack (Haystack) - augmented generation framework.
- FastChat - tuning chat LLMs.
Programming Languages
Categories
Sub Categories
Keywords
llm
7
deep-learning
6
pytorch
5
llmops
5
mlops
5
transformer
4
inference
4
machine-learning
3
language-model
3
fine-tuning
2
llama
2
gpt
2
nlp
2
evaluation-framework
2
llm-eval
2
llm-evaluation
2
llm-serving
2
python
2
prompt-engineering
2
rag
2
bloom
1
vulnerability-scanners
1
falcon
1
testing
1
red-teaming
1
starcoder
1
bentoml
1
llama2
1
llama3-1
1
llama3-2
1
llama3-2-vision
1
llm-inference
1
llm-ops
1
mistral
1
model-inference
1
open-source-llm
1
openllm
1
vicuna
1
amd
1
adapter
1
diffusion
1
lora
1
parameter-efficient-learning
1
transformers
1
bert
1
flax
1
jax
1
language-models
1
model-hub
1
natural-language-processing
1