Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/PropsAI/agentserve
An SDK & CLI for hosting and managing AI agents.
https://github.com/PropsAI/agentserve
Last synced: 3 days ago
JSON representation
An SDK & CLI for hosting and managing AI agents.
- Host: GitHub
- URL: https://github.com/PropsAI/agentserve
- Owner: PropsAI
- License: mit
- Created: 2024-10-23T01:44:50.000Z (3 months ago)
- Default Branch: main
- Last Pushed: 2024-10-24T17:19:50.000Z (3 months ago)
- Last Synced: 2024-10-24T18:10:49.977Z (3 months ago)
- Language: Python
- Size: 28.3 KB
- Stars: 1
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome_ai_agents - Agentserve - A framework for hosting and scaling AI agents. (Building / Deployment)
README
```
___ __ ____
/ _ |___ ____ ___ / /_/ __/__ _____ _____
/ __ / _ `/ -_) _ \/ __/\ \/ -_) __/ |/ / -_)
/_/ |_\_, /\__/_//_/\__/___/\__/_/ |___/\__/
/___/
```
# AgentServe
[](https://discord.gg/JkPrCnExSf)
[](https://github.com/PropsAI/agentserve)
AgentServe is a lightweight framework for hosting and scaling AI agents. It is designed to be easy to use and integrate with existing projects and agent / LLM frameworks. It wraps your agent in a REST API and supports optional task queuing for scalability.
Join the [Discord](https://discord.gg/JkPrCnExSf) for support and discussion.
## Goals and Objectives
The goal of AgentServe is to provide the easiest way to take an local agent to production and standardize the communication layer between multiple agents, humans, and other systems.
## Features
- **Standardized:** AgentServe provides a standardized way to communicate with AI agents via a REST API.
- **Framework Agnostic:** AgentServe supports multiple agent frameworks (OpenAI, LangChain, LlamaIndex, and Blank).
- **Task Queuing:** AgentServe supports optional task queuing for scalability. Choose between local, Redis, or Celery task queues based on your needs.
- **Configurable:** AgentServe is designed to be configurable via an `agentserve.yaml` file and overridable with environment variables.
- **Easy to Use:** AgentServe aims to be easy to use and integrate with existing projects and make deployment as simple as possible.
## Requirements
- Python 3.9+
## Installation
To install AgentServe, you can use pip:
```bash
pip install -U agentserve
```
## Getting Started
AgentServe allows you to easily wrap your agent code in a FastAPI application and expose it via REST endpoints. Below are the steps to integrate AgentServe into your project.
### 1. Install AgentServe
First, install the `agentserve` package using pip:
```bash
pip install -U agentserve
```
Make sure your virtual environment is activated if you're using one.
### 2. Create or Update Your Agent
Within your entry point file (e.g. `main.py`) we will import `agentserve` and create an app instance, then decorate an agent function with `@app.agent`. Finally, we will call `app.run()` to start the server.
The agent function should take a single argument, `task_data`, which will be a dictionary of data prequired by your agent.
**Example:**
```python
# main.py
import agentserve
from openai import OpenAI
app = agentserve.app()
@app.agent
def my_agent(task_data):
# Your agent logic goes here
client = OpenAI()
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": task_data["prompt"]}]
)
return response.choices[0].message.content
if __name__ == "__main__":
app.run()
```
In this example:
- We import agentserve and create an app instance using `agentserve.app()`.
- We define our agent function `my_agent` and decorate it with `@app.agent`.
- Within the agent function, we implement our agent's logic.
- We call `app.run()` to start the server.
### 3. Run the Agent Server
To run the agent server, use the following command:
```bash
python main.py
```
### 4. Configure Task Queue (Optional)
By default, AgentServe uses a local task queue, which is suitable for development and testing. If you need more robust queue management for production, you can configure AgentServe to use Redis or Celery.
**Using a Configuration File**
Create a file named agentserve.yaml in your project directory:
```yaml
# agentserve.yaml
task_queue: celery # Options: 'local', 'redis', 'celery'
celery:
broker_url: pyamqp://guest@localhost//
```
**Using Environment Variables**
Alternatively, you can set configuration options using environment variables:
```bash
export AGENTSERVE_TASK_QUEUE=celery
export AGENTSERVE_CELERY_BROKER_URL=pyamqp://guest@localhost//
```
### 5. Start the Worker (if using Celery or Redis)
To start the worker, use the following command:
```bash
agentserve startworker
```
### 6. Test the Agent
With the server and worker (if needed) running, you can test your agent using the available endpoints.
**Synchronous Task Processing**
`POST /task/sync`
```bash
curl -X POST http://localhost:8000/task/sync \
-H "Content-Type: application/json" \
-d '{"input": "Test input"}'
```
**Asynchronously process a task**
`POST /task/async`
```bash
curl -X POST http://localhost:8000/task/async \
-H "Content-Type: application/json" \
-d '{"input": "Test input"}'
```
**Get the status of a task**
`GET /task/status/:task_id`
```bash
curl http://localhost:8000/task/status/1234567890
```
**Get the result of a task**
`GET /task/result/:task_id`
```bash
curl http://localhost:8000/task/result/1234567890
```
## Configuration Options
AgentServe allows you to configure various aspects of the application using a configuration file or environment variables.
#### Using agentserve.yaml
Place an `agentserve.yaml` file in your project directory with the desired configurations.
**Example:**
```yaml
# agentserve.yaml
task_queue: celery # Options: 'local', 'redis', 'celery'
celery:
broker_url: pyamqp://guest@localhost//
redis:
host: localhost
port: 6379
server:
host: 0.0.0.0
port: 8000
queue: # if using local task queue
max_workers: 10 # default
```
#### Using Environment Variables
Set the desired configuration options using environment variables.
You can override configurations using environment variables without modifying the configuration file.
- `AGENTSERVE_TASK_QUEUE`
- `AGENTSERVE_CELERY_BROKER_URL`
- `AGENTSERVE_REDIS_HOST`
- `AGENTSERVE_REDIS_PORT`
- `AGENTSERVE_SERVER_HOST`
- `AGENTSERVE_SERVER_PORT`
- `AGENTSERVE_QUEUE_MAX_WORKERS`
**Example:**
```bash
export AGENTSERVE_TASK_QUEUE=redis
export AGENTSERVE_REDIS_HOST=redis-server-host
export AGENTSERVE_REDIS_PORT=6379
```
### FastAPI Configuration
You can specify FastAPI settings, including CORS configuration, using the `fastapi` key in your `agentserve.yaml` configuration file.
**Example:**
```yaml
# agentserve.yaml
fastapi:
cors:
allow_origins:
- "http://localhost:3000"
- "https://yourdomain.com"
allow_credentials: true
allow_methods:
- "*"
allow_headers:
- "*"
```
#### Using Environment Variables
Alternatively, you can set the desired configuration options using environment variables.
**Example:**
```bash
export AGENTSERVE_CORS_ORIGINS="http://localhost:3000,https://yourdomain.com"
export AGENTSERVE_CORS_ALLOW_CREDENTIALS="true"
export AGENTSERVE_CORS_ALLOW_METHODS="GET,POST"
export AGENTSERVE_CORS_ALLOW_HEADERS="Content-Type,Authorization"
```
## Advanced Usage
### Integrating with Existing Projects
You can integrate AgentServe into your existing projects by importing agentserve and defining your agent function.
**Example:**
```python
# main.py
import agentserve
app = agentserve.app()
@app.agent
def my_custom_agent(task_data):
# Your custom agent logic (e.g. using LangChain, LlamaIndex, etc.)
result = perform_complex_computation(task_data)
return {"result": result}
if __name__ == "__main__":
app.run()
```
### Input Validation
AgentServe allows you to validate the input to your agent function using Pydantic. Simply add an input schema to your agent function.
**Example:**
```python
# main.py
import agentserve
from pydantic import BaseModel
class MyInputSchema(BaseModel):
prompt: str
@app.agent(input_schema=MyInputSchema)
def my_custom_agent(task_data):
# Your custom agent logic
return {"result": "Hello, world!"}
if __name__ == "__main__":
app.run()
```
## Hosting
INSTRUCTIONS COMING SOON
## ROADMAP
- [ ] Add support for streaming responses
- [ ] Add easy instructions for more hosting options (GCP, Azure, AWS, etc.)
- [ ] Add support for external storage for task results
- [ ] Add support for multi model agents
- [ ] Add support for more agent frameworks
## License
This project is licensed under the MIT License.
## Contact
Join the [Discord](https://discord.gg/JkPrCnExSf) for support and discussion.
For any questions or issues, please contact Peter at [email protected].