https://github.com/victorgonbu/eos-ia
EosIA
https://github.com/victorgonbu/eos-ia
Last synced: about 1 month ago
JSON representation
EosIA
- Host: GitHub
- URL: https://github.com/victorgonbu/eos-ia
- Owner: Victorgonbu
- Created: 2025-04-09T07:37:55.000Z (about 1 month ago)
- Default Branch: master
- Last Pushed: 2025-04-10T18:51:52.000Z (about 1 month ago)
- Last Synced: 2025-04-11T01:11:46.940Z (about 1 month ago)
- Language: Python
- Size: 35.2 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# VLLM and FastAPI Project
This project is a containerized application that serves a VLLM (Very Large Language Model) and includes a FastAPI application that consumes the VLLM service.
## Project Structure
```
vllm-fastapi-project
├── inference
│ ├── app
│ │ ├── main.py # Entry point for the FastAPI application
│ │ └── __init__.py # Marks the directory as a Python package
│ ├── Dockerfile # Dockerfile for building the FastAPI application
│ └── requirements.txt # Python dependencies for the FastAPI application
├── vllm_server
│ ├── Dockerfile # Dockerfile for building the VLLM server
│ └── config.json # Configuration settings for the VLLM server
├── docker-compose.yml # Docker Compose configuration for the services
└── README.md # Project documentation
```
## Setup Instructions
1. **Clone the repository:**
```
git clone
cd vllm-fastapi-project
```
2. **Build the Docker images:**
```
docker-compose build
```
3. **Run the services:**
```
docker-compose up
```
4. **Access the FastAPI application:**
Open your browser and navigate to `http://localhost:8000` to access the FastAPI application.
## Usage
- The FastAPI application provides endpoints to interact with the VLLM service. You can find the API documentation at `http://localhost:8000/docs`.
## Notes
- Ensure that Docker and Docker Compose are installed on your machine before running the project.
- Modify the `config.json` file in the `vllm_server` directory to adjust the VLLM server settings as needed.