Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/meistrari/cursive-py
✦ The intuitive python LLM framework
https://github.com/meistrari/cursive-py
Last synced: about 23 hours ago
JSON representation
✦ The intuitive python LLM framework
- Host: GitHub
- URL: https://github.com/meistrari/cursive-py
- Owner: meistrari
- Created: 2023-08-08T22:37:48.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2024-12-02T20:56:10.000Z (about 1 month ago)
- Last Synced: 2025-01-05T11:13:43.402Z (8 days ago)
- Language: Python
- Size: 308 KB
- Stars: 171
- Watchers: 4
- Forks: 17
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
Cursive is a universal and intuitive framework for interacting with LLMs.
## highlights
![]()
✦ **Extensible** - You can easily hook into any part of a completion life cycle. Be it to log, cache, or modify the results.
![]()
✦ **Functions** - Easily describe functions that the LLM can use along with its definition, with any model (currently supporting GPT-4, GPT-3.5, Claude 2, and Claude Instant)
![]()
✦ **Universal** - Cursive aims to bridge as many capabilities between different models as possible. Ultimately, this means that with a single interface, you can allow your users to choose any model.
![]()
✦ **Informative** - Cursive comes with built-in token usage and costs calculations, as accurate as possible.
![]()
✦ **Reliable** - Cursive comes with automatic retry and model expanding upon exceeding context length. Which you can always configure.
## quickstart
1. Install.
```bash
poetry add cursivepy
# or
pip install cursivepy
```
2. Start using.
```python
from cursive import Cursive
cursive = Cursive()
response = cursive.ask(
prompt='What is the meaning of life?',
)
print(response.answer)
```
## usage
### Conversation
Chaining a conversation is easy with `cursive`. You can pass any of the options you're used to with OpenAI's API.
```python
res_a = cursive.ask(
prompt='Give me a good name for a gecko.',
model='gpt-4',
max_tokens=16,
)
print(res_a.answer) # Zephyr
res_b = res_a.conversation.ask(
prompt='How would you say it in Portuguese?'
)
print(res_b.answer) # Zéfiro
```
### Streaming
Streaming is also supported, and we also keep track of the tokens for you!
```python
result = cursive.ask(
prompt='Count to 10',
stream=True,
on_token=lambda partial: print(partial['content'])
)
print(result.usage.total_tokens) # 40
```
### Functions
You can use very easily to define and describe functions, along side with their execution code.
```python
from cursive import cursive_function, Cursive
cursive = Cursive()
@cursive_function()
def add(a: float, b: float):
"""
Adds two numbers.
a: The first number.
b: The second number.
"""
return a + b
res = cursive.ask(
prompt='What is the sum of 232 and 243?',
functions=[add],
)
print(res.answer) # The sum of 232 and 243 is 475.
```
The functions' result will automatically be fed into the conversation and another completion will be made. If you want to prevent this, you can add `pause` to your function definition.
```python
@cursive_function(pause=True)
def create_character(name: str, age: str):
"""
Creates a character.
name: The name of the character.
age: The age of the character.
"""
return {
'name': name,
'age': age,
}
res = cursive.ask(
prompt='Create a character named John who is 23 years old.',
functions=[create_character],
)
print(res.function_result) # { name: 'John', age: 23 }
```
Cursive also supports passing in undecorated functions!
```python
def add(a: float, b: float):
return a + b
res = cursive.ask(
prompt='What is the sum of 232 and 243?',
functions=[add], # this is equivalent to cursive_function(pause=True)(add)
)
if res.function_result:
print(res.function_result) # 475
else:
print(res.answer) # Text answer in case the function is not called
```
### Models
Cursive also supports the generation of Pydantic BaseModels.
```python
from cursive.compat.pydantic import BaseModel, Field # Pydantic V1 API
class Character(BaseModel):
name: str
age: int
skills: list[str] = Field(min_items=2)
res = cursive.ask(
prompt='Create a character named John who is 23 years old.',
function_call=Character,
)
res.function_result # is a Character instance with autogenerated fields
```
### Hooks
You can hook into any part of the completion life cycle.
```python
cursive.on('completion:after', lambda result: print(
result.data.cost.total,
result.data.usage.total_tokens,
))
cursive.on('completion:error', lambda result: print(
result.error,
))
cursive.ask({
prompt: 'Can androids dream of electric sheep?',
})
# 0.0002185
# 113
```
### Embedding
You can create embeddings pretty easily with `cursive`.
```ts
embedding = cursive.embed('This should be a document.')
```
This will support different types of documents and integrations pretty soon.
### Reliability
Cursive comes with automatic retry with backoff upon failing completions, and model expanding upon exceeding context length -- which means that it tries again with a model with a bigger context length when it fails by running out of it.
You can configure this behavior by passing the `retry` and `expand` options to `Cursive` constructor.
```python
cursive = Cursive(
max_retries=5, # 0 disables it completely
expand={
'enable': True,
'defaults_to': 'gpt-3.5-turbo-16k',
'resolve_model': {
'gpt-3.5-turbo': 'gpt-3.5-turbo-16k',
'gpt-4': 'claude-2',
},
},
)
```
## Available Models
OpenAI models
- `gpt-3.5-turbo`
- `gpt-3.5-turbo-16k`
- `gpt-4`
- `gpt-4-32k`
- Any other chat completion model version
###### Credentials
You can pass your OpenAI API key to `Cursive`'s constructor, or set the `OPENAI_API_KEY` environment variable.
Anthropic models
- `claude-2`
- `claude-instant-1`
- `claude-instant-1.2`
- Any other model version
###### Credentials
You can pass your Anthropic API key to `Cursive`'s constructor, or set the `ANTHROPIC_API_KEY` environment variable.
OpenRouter models
OpenRouter is a service that gives you access to leading language models in an OpenAI-compatible API, including function calling!
- `anthropic/claude-instant-1.2`
- `anthropic/claude-2`
- `openai/gpt-4-32k`
- `google/palm-2-codechat-bison`
- `nousresearch/nous-hermes-llama2-13b`
- Any model version from https://openrouter.ai/docs#models
###### Credentials
```python
from cursive import Cursive
cursive = Cursive(
openrouter={
"api_key": "sk-or-...",
"app_title": "Your App Name",
"app_url": "https://appurl.com",
}
)
cursive.ask(
model="anthropic/claude-instant-1.2",
prompt="What is the meaning of life?"
)
```
Cohere models
- `command`
- Any other model version (such as `command-nightly`)
###### Credentials
You can pass your Cohere API key to `Cursive`'s constructor, or set the `COHERE_API_KEY` environment variable.
Replicate models
You can prepend `replicate/` to any model name and version available on Replicate.
###### Example
```python
cursive.ask(
prompt='What is the meaning of life?',
model='replicate/a16z-infra/llama-2-13b-chat:2a7f981751ec7fdf87b5b91ad4db53683a98082e9ff7bfd12c8cd5ea85980a52',
)
```
###### Credentials
You can pass your Replicate API key to `Cursive`'s constructor, or set the `REPLICATE_API_TOKEN` environment variable.
## roadmap
### vendor support
- [x] Anthropic
- [x] Cohere
- [x] Replicate
- [x] OpenRouter
- [ ] Azure OpenAI models
- [ ] Huggingface