https://github.com/kiln-ai/kiln
Build, Evaluate, and Optimize AI Systems. Includes evals, RAG, agents, fine-tuning, synthetic data generation, dataset management, MCP, and more.
https://github.com/kiln-ai/kiln
ai chain-of-thought collaboration dataset-generation evals evaluation evaluation-framework fine-tuning machine-learning macos mcp ml ollama openai prompt prompt-engineering python rlhf synthetic-data windows
Last synced: about 20 hours ago
JSON representation
Build, Evaluate, and Optimize AI Systems. Includes evals, RAG, agents, fine-tuning, synthetic data generation, dataset management, MCP, and more.
- Host: GitHub
- URL: https://github.com/kiln-ai/kiln
- Owner: Kiln-AI
- License: other
- Created: 2024-07-23T23:10:13.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2026-10-24T00:29:46.000Z (5 days ago)
- Last Synced: 2026-10-24T14:13:19.297Z (4 days ago)
- Topics: ai, chain-of-thought, collaboration, dataset-generation, evals, evaluation, evaluation-framework, fine-tuning, machine-learning, macos, mcp, ml, ollama, openai, prompt, prompt-engineering, python, rlhf, synthetic-data, windows
- Language: Python
- Homepage: https://kiln.tech
- Size: 60.8 MB
- Stars: 5,229
- Watchers: 34
- Forks: 394
- Open Issues: 57
-
Metadata Files:
- Readme: README.md
- Contributing: .github/CONTRIBUTING.md
- License: LICENSE.txt
- Agents: AGENTS.md
Awesome Lists containing this project
- awesome-ChatGPT-repositories - Kiln - The easiest tool for fine-tuning LLM models, synthetic data generation, and collaborating on datasets. (Prompts)
- awesome-production-machine-learning - Kiln - ai/kiln.svg?style=social) - Kiln is an OSS tool for fine-tuning LLM models, synthetic data generation, and collaborating on datasets. (Deployment and Serving)
- awesome-llm-tools - Kiln - tuning tool with synthetic data generation and dataset collaboration | SFT / Synthetic Data | ✅ | (9. Fine-Tuning / Training Frameworks)
- awesome-local-llm - Kiln - the easiest tool for fine-tuning LLM models, synthetic data generation, and collaborating on datasets (Tools / Training and Fine-tuning)
README
A free app and open-source library to build better AI products.
Highlights •
Evals •
Auto-Optimize •
RAG •
Agents •
Fine-Tuning •
Synthetic Data •
All Docs
## What is Kiln?
Kiln is a workbench for the full AI development loop: evals, optimization, prompts, RAG, fine-tuning, synthetic data, agents, and tools - all working together. The desktop app lets your whole team contribute (PMs, subject-experts, and QA can rate outputs and add data without writing code). The MIT-licensed Python library ships the same tasks to production. Runs locally - bring your own API keys, or go fully offline with Ollama.
## Highlights
### Iterate, optimize, and collaborate
- 🖥️ [**Intuitive app**](https://kiln.tech/download) - Easy-to-use apps for Mac, Windows, and Linux. One-click install.
- 📊 [**Eval Builder**](https://docs.kiln.tech/docs/evaluations) - Auto-generate evals (judge + synthetic eval dataset), and align to your preference in ~10 minutes.
- 🚀 [**Auto-Optimize**](https://docs.kiln.tech/docs/prompts/automatic-prompt-optimizer) - Automatically find the best way to run your AI task, optimizing prompt, model selection, tools, skills, subagents, parameters, and more.
- 💬 [**AI Assistant**](https://docs.kiln.tech) - Your AI data-science partner. Kiln Assistant proposes improvements, optimizes prompts, runs experiments, creates evals, and more.
- 🤝 [**Git-native collaboration**](https://docs.kiln.tech/docs/collaboration) - The app syncs to Git automatically — even for teammates who don't know what Git is.
### Build & ship agents
- 🔍 [**RAG**](https://docs.kiln.tech/docs/documents-and-search-rag) - Drag-and-drop docs (PDF, image, video, audio) to create a RAG. Auto-generated RAG evals from your own documents.
- 🤖 [**Subagents**](https://docs.kiln.tech/docs/agents) - Compose multi-agent hierarchies. Each runs in its own focused context window.
- 🪄 [**Synthetic Data Generation**](https://docs.kiln.tech/docs/synthetic-data-generation) - Generate data for evals or fine-tuning in minutes.
- 🎛️ [**Fine-Tuning**](https://docs.kiln.tech/docs/fine-tuning-guide) - Zero-code fine-tuning across 60+ models (Qwen, Llama, GPT, Gemini, …) on Fireworks, Together, and Vertex. Serverless deployment included.
- 🐍 [**Open Python library**](https://docs.kiln.tech/developers/python-library-quickstart) - Agents built in the app can be deployed to production. MIT open-source.
- 🧰 [**…and more**](https://docs.kiln.tech) - Tools & MCP, Skills, structured outputs, reasoning models, model library (190+ tested).
## App Quickstart
Get started in minutes - one-click install.
Download Kiln Desktop for macOS, Windows, or Linux, then follow the [5-minute quickstart](https://docs.kiln.tech/getting-started/quickstart) to run your first task.
[](https://kiln.tech/download) [](https://kiln.tech/download) [](https://kiln.tech/download)
Prefer to start in code? See the [Python library quickstart](https://docs.kiln.tech/developers/python-library-quickstart).
## Demo
[Watch a 2-minute overview](https://kiln.tech#demo), or our [end-to-end project demo (20 minutes)](https://docs.kiln.tech/docs/end-to-end-project-demo).
## Why Kiln?
Most AI tooling forces a tradeoff: a code-only framework that covers one slice (orchestration *or* evals *or* RAG), or a paid SaaS that locks in your data and can't be extended. Kiln is a free, local-first workbench where a single task and dataset flow through evals, prompt optimization, fine-tuning, RAG, agents, and synthetic data — all in one tool.
- **One dataset, every technique.** Define a task once. Eval it, optimize the prompt, fine-tune a model, generate synthetic data, add RAG — all against the same dataset, with results that compound across stages.
- **Track every axis. Move fast. Don't regress.** Keeping agents running well is hard — a prompt change quietly regresses behavior three steps downstream; a model upgrade improves five things and breaks two. Kiln tracks quality across every dimension you care about, so you iterate without breaking what already works.
- **Optimization, not just evaluation.** Other tools tell you how a prompt scores, but not how to fix it. Kiln's Auto-Optimize searches across hundreds of prompt mutations and models to find what works best for every eval dimension.
- **GUI for the whole team, library for engineers.** Kiln's desktop app lets PMs rate outputs, SMEs add training examples, and QA flag regressions — without a terminal. Engineers ship the same tasks via an MIT-licensed Python library. Data scientists can use the library in notebooks and experiments.
- **Local-first.** Most AI platforms are SaaS-only. Kiln runs entirely on your machine. Bring your own API keys, or go fully offline with Ollama. Your data never leaves your control. Team-sync is provided via Git infrastructure you already own.
- **190+ models tested across every provider.** Skip the guesswork — we've tested every model's capabilities across all major providers. OpenAI, Anthropic, Gemini, Bedrock, Ollama, OpenRouter, Fireworks, Groq, any OpenAI-compatible endpoint, and more. Swap models with confidence.
## Open-source Python Library
Build AI tasks in the app. Deploy with the open-source library. Same engine, same project files, no rewrite. The MIT-licensed `kiln-ai` library is the same library used in the app. Load Kiln projects, run tasks, build fine-tunes, work in notebooks, integrate Pandas/Polars dataframes, and more.
```bash
pip install kiln-ai
```
[📚 Library docs](https://docs.kiln.tech/developers/python-library-quickstart) · [REST API](https://docs.kiln.tech/developers/rest-api) · [PyPI](https://pypi.org/project/kiln-ai/)
## Docs
Full docs at [docs.kiln.tech](https://docs.kiln.tech). Common starting points:
- [Quickstart](https://docs.kiln.tech/getting-started/quickstart) — run your first task in 5 minutes
- [Evals](https://docs.kiln.tech/docs/evaluations)
- [Auto-Optimize](https://docs.kiln.tech/docs/prompts/automatic-prompt-optimizer)
- [RAG](https://docs.kiln.tech/docs/documents-and-search-rag)
- [Agents](https://docs.kiln.tech/docs/agents)
- [Fine-Tuning](https://docs.kiln.tech/docs/fine-tuning-guide)
- [Python Library](https://docs.kiln.tech/developers/python-library-quickstart)
- [End-to-end project demo](https://docs.kiln.tech/docs/end-to-end-project-demo) (20-min video)
## Community
- Chat with the community on [Discord](https://kiln.tech/discord).
- Subscribe to the [newsletter](https://kiln.tech/blog) for new features.
- File issues, request features, or open a discussion on [GitHub](https://github.com/Kiln-AI/Kiln/issues).
## Contributing
See [CONTRIBUTING.md](.github/CONTRIBUTING.md) for development setup and contribution guidelines.
## License & Trademarks
Kiln's core Python library and REST server are [MIT-licensed](libs/core/LICENSE.txt). The desktop app is [source-available](app), free to use, and built on the [fair-code](https://faircode.io) model — so Kiln stays free for individuals while remaining sustainable.
Datasets are open JSON. You own and control your datasets.
[Kiln Pro](https://kiln.tech/pricing) is our service that adds the AI Assistant, Auto-Optimize, and the Eval Builder. It's opt-in, and the core Kiln app remains fully functional without it.
The Kiln name and logos are trademarks of Chesterfield Laboratories Inc.
Copyright 2024 — Chesterfield Laboratories Inc.