https://github.com/bastienpo/fastagent
🐆 A tool that lets you quickly deploy your DSPy agents as a server
https://github.com/bastienpo/fastagent
agents chatbot dspy genai
Last synced: 6 days ago
JSON representation
🐆 A tool that lets you quickly deploy your DSPy agents as a server
- Host: GitHub
- URL: https://github.com/bastienpo/fastagent
- Owner: bastienpo
- License: mit
- Created: 2024-10-12T23:52:24.000Z (7 months ago)
- Default Branch: main
- Last Pushed: 2024-10-28T22:40:02.000Z (7 months ago)
- Last Synced: 2024-10-28T23:22:29.941Z (7 months ago)
- Topics: agents, chatbot, dspy, genai
- Language: Makefile
- Homepage:
- Size: 207 KB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# fastagent
fastagent: a tool making it easy to ship your agent to production
[Documentation](https://fastagent.gitbook.io)
---
> [!Warning]
> FastAgent is currently under **active development** and in alpha stage. It currently lacks SSL and CORS security features, as well as comprehensive testing needed for production deployment.
## Installation
You can install the project using pip: (Available on PyPI soon)
```bash
pip install git+https://github.com/bastienpo/fastagent.git
```
## How to use the project
First, create a simple LangChain application.
Let's create an app.py file with the simplest langchain runnable you can make, consisting of just a large language model.
```python
# pip install -qU langchain-mistralai and requires MISTRAL_API_KEY in to be set
from langchain_mistralai import ChatMistralAI
chain = ChatMistralAI(model="ministral-3b-latest")
```
Then, you need to initialize a fastagent configuration file (fastagent.toml) using the following command:
```bash
fastagent init
```
You will need to update the app field in the project section to match the path of your application in the form:
`:` in you case it would be `myapplication.app:chain`
If you decided to use a database in your configuration, you can use the `setup` command to create the tables and setup the database.
```bash
fastagent setup
```
When you are ready to develop or ship, you can start the production server:
```bash
fastagent dev # fastagent run
```
The difference between the `dev` and `run` command is that `dev` will reload the server on code changes and log to the console, while `run` will build a docker image and serve the application.
## Features
This is the current roadmap for the project:
- The project allows to setup a simple server for your agent. It only support langchain and postgresql to store your conversations and authentication data.
- It allows to setup a simple token based authentication and authorization.
- You can configure different middlewares CORS, Authorization and Rate limiting.
## What's next?
> [!Note]
> This project is still under active development and design is subject to change. I'm doing it only as a personal project because I was curious about how to ship an agent to production. Feel free to contribute or give feedback. I'm really open to any suggestion.
Near future:
- Add end-to-end testing of the project for langchain and langgraph.
- Add a permission system.
- Add stateless authentication.
- Add support for more databases (SQLite in particular).
Long term:
- Simplify the integration of monitoring and observability tools (Langfuse, Phoenix)
- Add support for Dspy and other frameworks.
## Resources
Some resources about the dependencies used for the project and thanks to the maintainers of the projects for their work.
- [FastAPI](https://fastapi.tiangolo.com/)
- [Granian](https://granian.dev/)
- [Langserve](https://github.com/Lightning-AI/LitServe)
The project is inspired by [LitServe](https://github.com/Lightning-AI/LitServe).