https://github.com/manikanda-kumar/vllm-vs-openrouter
Evaluate the inference between vLLM and OpenRouter for Code generations and Opencode coding agent use.
https://github.com/manikanda-kumar/vllm-vs-openrouter
code-gen eval gpt-oss inference opencode python vllm
Last synced: about 1 month ago
JSON representation
Evaluate the inference between vLLM and OpenRouter for Code generations and Opencode coding agent use.
- Host: GitHub
- URL: https://github.com/manikanda-kumar/vllm-vs-openrouter
- Owner: manikanda-kumar
- Created: 2025-10-26T07:32:21.000Z (8 months ago)
- Default Branch: main
- Last Pushed: 2025-10-29T19:56:46.000Z (8 months ago)
- Last Synced: 2025-11-05T11:03:15.114Z (7 months ago)
- Topics: code-gen, eval, gpt-oss, inference, opencode, python, vllm
- Language: Python
- Homepage:
- Size: 156 KB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# vLLM vs OpenRouter Evaluations
This repository contains two complementary evaluation flows for comparing model behavior across local vLLM and OpenRouter providers.
## Code Generation Evaluation
- Location: `codegen/`
- Primary entrypoint: `streamlit run codegen/app.py`
- Utilities: `model_service.py`, `code_evaluation.py`, `code_ingestion.py`
- Reference: [CODE_EVALUATION.md](codegen/CODE_EVALUATION.md) (@CODE_EVALUATION.md)
## Opencode Agent Evaluation
- Location: `opencode/`
- Primary entrypoint: `streamlit run opencode/opencode_app.py`
- Dashboards and CLIs: `opencode_app.py`, `test_opencode.py`, `run_opencode_eval.py`
- Configuration and artifacts: `opencode_test_config.json`, `opencode_test_results.*`
- Reference: [OPENCODE_EVALUATION.md](opencode/OPENCODE_EVALUATION.md) (@OPENCODE_EVALUATION.md)
Each evaluation can be run independently; see the linked documents for full setup and usage guidance.