Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/dreadnode/rigging
Lightweight LLM Interaction Framework
https://github.com/dreadnode/rigging
agents litellm llms pydantic
Last synced: 3 days ago
JSON representation
Lightweight LLM Interaction Framework
- Host: GitHub
- URL: https://github.com/dreadnode/rigging
- Owner: dreadnode
- License: mit
- Created: 2024-01-29T20:15:08.000Z (9 months ago)
- Default Branch: main
- Last Pushed: 2024-05-22T15:40:30.000Z (6 months ago)
- Last Synced: 2024-05-23T02:14:38.177Z (6 months ago)
- Topics: agents, litellm, llms, pydantic
- Language: Python
- Homepage: https://rigging.dreadnode.io
- Size: 1.14 MB
- Stars: 92
- Watchers: 5
- Forks: 7
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-langchain - Rigging
README
Simplify using LLMs in code
Rigging is a lightweight LLM framework built on Pydantic XML. The goal is to make leveraging language models in production code as simple and effective as possible. Here are the highlights:
- **Structured Pydantic models** can be used interchangably with unstructured text output.
- LiteLLM as the default generator giving you **instant access to a huge array of models**.
- Define prompts as python functions with **type hints and docstrings**.
- Simple **tool calling** abilities for models which don't natively support it.
- Store different models and configs as **simple connection strings** just like databases.
- Chat templating, forking, continuations, generation parameter overloads, stripping segments, etc.
- Async batching and fast iterations for **large scale generation**.
- Metadata, callbacks, and data format conversions.
- Modern python with type hints, async support, pydantic validation, serialization, etc.
```py
import rigging as rg
@rg.prompt(generator_id="gpt-4")
async def get_authors(count: int = 3) -> list[str]:
"""Provide famous authors."""
print(await get_authors())
# ['William Shakespeare', 'J.K. Rowling', 'Jane Austen']
```
Rigging is built by [**dreadnode**](https://dreadnode.io) where we use it daily.
## Installation
We publish every version to Pypi:
```bash
pip install rigging
```
If you want to build from source:
```bash
cd rigging/
poetry install
```
## Supported LLMs
Rigging will run just about any language model:
- Any model from [**LiteLLM**](https://litellm.vercel.app/docs/providers)
- Any model from [**vLLM**](https://docs.vllm.ai/en/latest/models/supported_models.html)
- Any model from [**transformers**](https://huggingface.co/docs/transformers/)
### API Keys
Pass the `api_key` in an generator id or use standard environment variables.
```py
rg.get_generator("gpt-4-turbo,api_key=...")
```
```bash
export OPENAI_API_KEY=...
export MISTRAL_API_KEY=...
export ANTHROPIC_API_KEY=...
...
```
Check out [the docs](https://rigging.dreadnode.io/topics/generators/#api-keys) for more.
## Getting Started
**Check out the guide [in the docs](https://rigging.dreadnode.io/#getting-started)**
1. **Get a generator** using a connection string.
2. Build a **chat** or **completion** pipeline
3. **Run** the pipeline and get the output.
```py
import rigging as rg
# 1 - Get a generator
generator = rg.get_generator("claude-3-sonnet-20240229")
# 2 - Build a chat pipeline
pipeline = generator.chat([
{"role": "system", "content": "Talk like a pirate."},
{"role": "user", "content": "Say hello!"},
])
# 3 - Run the pipeline
chat = await pipeline.run()
print(chat.conversation)
# [system]: Talk like a pirate.
# [user]: Say hello!
# [assistant]: Ahoy, matey! Here be the salty sea dog ready to trade greetings wit' ye. Arrr!
```
Want more?
- Use [structured pydantic parsing](https://rigging.dreadnode.io/#basic-parsing)
- Check out [raw completions](https://rigging.dreadnode.io/topics/completions/)
- Give the LLM [access to tools](https://rigging.dreadnode.io/topics/tools/)
- Play with [generation params](https://rigging.dreadnode.io/topics/generators/#overload-generation-params)
- Use [callbacks in the pipeline](https://rigging.dreadnode.io/topics/callbacks-and-mapping/)
- Scale up with [iterating and batching](https://rigging.dreadnode.io/topics/iterating-and-batching/)
- Save your work with [serialization](https://rigging.dreadnode.io/topics/serialization/)
## Examples
- Basic interactive chat: [**chat.py**](examples/chat.py)
- Jupyter code interpreter: [**jupyter.py**](examples/jupyter.py)
- OverTheWire Bandit Agent: [**bandit.py**](examples/bandit.py)
- Damn Vulnerable Restaurant Agent: [**dvra.py**](examples/dvra.py)
- RAG Pipeline: [**rag.py**](examples/rag.py) (from [kyleavery](https://github.com/kyleavery/))
## Documentation
**[rigging.dreadnode.io](https://rigging.dreadnode.io)** has everything you need.
## Star History
[](https://star-history.com/#dreadnode/rigging&Date)