https://github.com/inferra/inferra-python-sdk
Official Python SDK for Inferra API access
https://github.com/inferra/inferra-python-sdk
ai api-client async batch-processing inferra language-models llama llm machine-learning mistral openai-compatible python sdk
Last synced: about 1 month ago
JSON representation
Official Python SDK for Inferra API access
- Host: GitHub
- URL: https://github.com/inferra/inferra-python-sdk
- Owner: Inferra
- License: mit
- Created: 2025-02-10T22:29:30.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2025-02-14T20:24:20.000Z (over 1 year ago)
- Last Synced: 2025-05-20T21:37:29.885Z (about 1 year ago)
- Topics: ai, api-client, async, batch-processing, inferra, language-models, llama, llm, machine-learning, mistral, openai-compatible, python, sdk
- Language: Python
- Homepage:
- Size: 89.8 KB
- Stars: 1
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Inferra Python SDK
The official Python SDK for [Inferra.net](https://inferra.net) - Access leading open source AI models with just a few lines of code.
## Installation
```bash
pip install inferra
```
## Quick Start
```python
from inferra import InferraClient
# Initialize client
client = InferraClient(api_key="your-api-key")
# Create a chat completion
response = client.chat.create(
model="meta-llama/llama-3.1-8b-instruct/fp-8",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "What is the meaning of life?"}
],
stream=True
)
# Process streaming response
for chunk in response:
if chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end='', flush=True)
```
## Features
- Full support for Inferra's API
- Async/await support
- Type hints throughout
- Built-in rate limiting and retries
- Streaming support
- Batch processing
- Comprehensive documentation
## Available Models
| Model Name | Price (per 1M tokens) |
|------------|----------------------|
| meta-llama/llama-3.2-1b-instruct/fp-8 | $0.015 |
| meta-llama/llama-3.2-3b-instruct/fp-8 | $0.03 |
| meta-llama/llama-3.1-8b-instruct/fp-8 | $0.045 |
| meta-llama/llama-3.1-8b-instruct/fp-16 | $0.05 |
| mistralai/mistral-nemo-12b-instruct/fp-8 | $0.10 |
| meta-llama/llama-3.1-70b-instruct/fp-8 | $0.30 |
## Development
```bash
# Install development dependencies
pip install -e ".[dev]"
# Run tests
make test
# Run linting
make lint
```