https://github.com/flozi00/o1-server
https://github.com/flozi00/o1-server
fastapi llm o1 openai openai-api reasoning
Last synced: 7 days ago
JSON representation
- Host: GitHub
- URL: https://github.com/flozi00/o1-server
- Owner: flozi00
- License: apache-2.0
- Created: 2025-01-06T10:10:03.000Z (9 months ago)
- Default Branch: main
- Last Pushed: 2025-01-07T17:36:14.000Z (9 months ago)
- Last Synced: 2025-01-16T03:47:09.007Z (9 months ago)
- Topics: fastapi, llm, o1, openai, openai-api, reasoning
- Language: Python
- Homepage:
- Size: 37.1 KB
- Stars: 3
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Step-by-Step Reasoning API
This project provides an API that leverages large language models (LLMs) to perform complex reasoning tasks through a unique step-by-step approach. It uses a feedback-driven system to ensure accuracy and logical consistency. The API is built with FastAPI and is compatible with OpenAI's chat completion endpoint.
## Features
- **Step-by-Step Reasoning:** Breaks down complex problems into smaller, manageable steps.
- **Feedback Mechanism:** Integrates a rewarder model that provides feedback on each step, ensuring correctness.
- **Logical/Analytical and Creative Task Handling:** Adapts the problem-solving strategy based on the nature of the task.
- **OpenAI Compatibility:** Designed to work with models compatible with the OpenAI API interface.
- **Streaming Support:** Offers streaming responses for real-time interaction.
## Architecture
The core of the system is based on two main components:
1. **Reasoning Agent:** This component generates the step-by-step solution based on the provided prompt and system instructions.
2. **Rewarder Agent:** This component evaluates each step generated by the reasoning agent and provides feedback.
The system iteratively generates and evaluates steps until a complete and correct solution is reached.
### System Message (`SYS_MSG`)
The `SYS_MSG` defines the framework for problem-solving. It outlines the process for both logical/analytical and creative tasks.
**For Logical/Analytical Tasks:**
- **Problem Decomposition:** Clearly state the problem, list given information, constraints, missing information, and the expected output format.
- **Component Analysis:** Analyze primary, secondary, and implicit components, including their properties and dependencies.
- **Relationship Mapping:** Define direct, indirect, hierarchical, causal, and temporal relationships between components.
- **Comprehension Levels:** Understand the problem at literal, contextual, and abstract levels.
- **Solution Development:** List assumptions, detail solution steps with validation points, and define boundary conditions.
- **Solution Verification:** Ensure logical consistency, completeness, handle edge cases, and check for contradictions.
**For Creative Tasks:**
- Focus on originality and innovation.
- Provide clear reasoning for creative choices.
### Rewarder Message (`REWARDER`)
The `REWARDER` message defines the role of the rewarder agent, which is to critically rate each step provided by the user. The rewarder only responds with one of three options:
- **Correct:** If the step is logically sound.
- **Wrong:** If the step contains an error.
- **Too much information in one step. Split this up into smaller parts and think again:** If the step contains multiple facts or ideas that should be separated.
## Setup
### Prerequisites
- Python 3.9+
- An OpenAI-compatible API endpoint (e.g., a self-hosted LLM with an OpenAI-compatible interface).
- An API key for your LLM provider.
**Set environment variables:**
Create a `.env` file in the project root directory and add the following:
```
OAI_BASE_URL=
OAI_KEY=
LLM_NAME=
```
Replace the placeholders with your actual API base URL, API key, and the name of the LLM you want to use.
### Running the API
1. **Start the FastAPI server using uvicorn:**
```bash
uvicorn app:app --reload
```
This will start the API server on `http://0.0.0.0:8000`.