https://github.com/versionhq/exp-agent-performance
AI agents - performance comparison using major LLMs.
https://github.com/versionhq/exp-agent-performance
agentic-ai deepseek-r1 gemini gpt-4o pydantic qwen2-5 versionhq
Last synced: 6 months ago
JSON representation
AI agents - performance comparison using major LLMs.
- Host: GitHub
- URL: https://github.com/versionhq/exp-agent-performance
- Owner: versionHQ
- License: mit
- Created: 2025-02-03T03:09:52.000Z (8 months ago)
- Default Branch: main
- Last Pushed: 2025-02-12T14:11:18.000Z (8 months ago)
- Last Synced: 2025-04-14T12:05:02.203Z (6 months ago)
- Topics: agentic-ai, deepseek-r1, gemini, gpt-4o, pydantic, qwen2-5, versionhq
- Language: Python
- Homepage: https://pypi.org/project/versionhq/
- Size: 10.7 KB
- Stars: 2
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Overview
Experiment on AI agent performance using `versionhq` and `pydantic`.
**Visit:**
- [PyPI](https://pypi.org/project/versionhq/)
- [Github](https://github.com/versionHQ/multi-agent-system)
- [Docs](https://docs.versi0n.io)
- [Process (Medium story)](https://medium.com/@kuriko-iwai/48d42fc57b71)
## Key Features
Generate multi-agent systems based on the task complexity, execute tasks, and evaluate output based on the given criteria.
Agents are model-agnostic, and can handle and share RAG tools, knowledge, memory, and callbacks among other agents.
## Quick Start
1. Install `uv` package manager:
For MacOS:
```
brew install uv
```
For Ubuntu/Debian:
```
sudo apt-get install uv
```
2. Install dependencies:
```
uv venv
source .venv/bin/activate
uv pip install -r requirements.txt
```
* In case of AssertionError/module mismatch, run Python version control using `.pyenv`
```
pyenv install 3.12.8
pyenv global 3.12.8 (optional: `pyenv global system` to get back to the system default ver.)
uv python pin 3.12.8
echo 3.12.8 > .python-version
```
3. Set up environment variables:
Create `.env` file in the project root and add the following:
```
OPENAI_API_KEY=your-openai-api-key
GEMINI_API_KEY=your-gemini-api-key
OPENROUTER_API_KEY=your-openrouter-api-key
```
4. Run:
```
uv run main.py
```
## Customizing
- To add or refine an agent, use `src/agents.py`.
- To add or refine a task, use `src/tasks.py`.
## Results
(Feb 3, 2025)
## Trouble Shooting
Common issues and solutions:
- API key errors: Ensure all API keys in the `.env` file are correct and up to date. Make sure to add `load_dotenv()` on the top of the python file to apply the latest environment values.
- Database connection issues: Check if the Chroma DB is properly initialized and accessible.
- Memory errors: If processing large contracts, you may need to increase the available memory for the Python process.
- Issues related to the Python version: Use Python 3.12.x as default by running `uv venv --python 3.12.8` and `uv python pin 3.12.8`.
- Issues related to dependencies: `uv cache clean`, `uv venv`, and run `uv pip install -r requirements.txt -v`.
- `reportMissingImports` error from pyright after installing the package: This might occur when installing new libraries while VSCode is running. Open the command pallete (ctrl + shift + p) and run the Python: Restart language server task.