{"id":50938702,"url":"https://github.com/transcentlin/api-probe-platform","last_synced_at":"2026-06-17T12:00:28.253Z","repository":{"id":365263010,"uuid":"1271280605","full_name":"transcentlin/API-Probe-Platform","owner":"transcentlin","description":"先进的大模型 API 多服务商性能与兼容性评估平台 | An advanced, multi-provider LLM API performance benchmark and evaluation platform.","archived":false,"fork":false,"pushed_at":"2026-06-16T14:52:40.000Z","size":303,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-16T16:24:41.631Z","etag":null,"topics":["api-benchmark","api-evaluation","deepseek","fastapi","llm-benchmark","llm-evaluation","llm-testing","model-evaluation","ollama","openai-compatibility","python","react-dashboard","tool-calling"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/transcentlin.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-06-16T14:04:05.000Z","updated_at":"2026-06-16T14:53:27.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/transcentlin/API-Probe-Platform","commit_stats":null,"previous_names":["transcentlin/api-probe-platform"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/transcentlin/API-Probe-Platform","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/transcentlin%2FAPI-Probe-Platform","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/transcentlin%2FAPI-Probe-Platform/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/transcentlin%2FAPI-Probe-Platform/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/transcentlin%2FAPI-Probe-Platform/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/transcentlin","download_url":"https://codeload.github.com/transcentlin/API-Probe-Platform/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/transcentlin%2FAPI-Probe-Platform/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34447266,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-17T02:00:05.408Z","response_time":127,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api-benchmark","api-evaluation","deepseek","fastapi","llm-benchmark","llm-evaluation","llm-testing","model-evaluation","ollama","openai-compatibility","python","react-dashboard","tool-calling"],"created_at":"2026-06-17T12:00:17.353Z","updated_at":"2026-06-17T12:00:28.200Z","avatar_url":"https://github.com/transcentlin.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# API 平台测试系统 (API Probe Platform)\n\n[![GitHub Stars](https://img.shields.io/github/stars/transcentlin/API-Probe-Platform?style=social)](https://github.com/transcentlin/API-Probe-Platform/stargazers)\n[![GitHub Release](https://img.shields.io/github/v/release/transcentlin/API-Probe-Platform?include_prereleases\u0026style=flat-square)](https://github.com/transcentlin/API-Probe-Platform/releases)\n[![License](https://img.shields.io/badge/License-MIT-blue?style=flat-square)](LICENSE)\n[![Python 3.8+](https://img.shields.io/badge/Python-3.8%2B-3776AB?style=flat-square\u0026logo=python\u0026logoColor=white)](https://www.python.org/)\n[![React](https://img.shields.io/badge/React-Vite-61DAFB?style=flat-square\u0026logo=react\u0026logoColor=white)](https://react.dev/)\n[![FastAPI](https://img.shields.io/badge/FastAPI-009688?style=flat-square\u0026logo=fastapi\u0026logoColor=white)](https://fastapi.tiangolo.com/)\n\n\u003e **先进的大模型 API 多服务商性能与兼容性评估平台** | An advanced, multi-provider LLM API performance benchmark and evaluation platform.\n\n一个面向 LLM API 服务商的全方位性能、能力及兼容性评估与测试平台。系统提供了统一的 CLI 命令行入口以及直观的 Web 可视化仪表盘，旨在全方位评估并对比各大主流大模型（LLM）API 提供商在核心能力维度上的表现。\n\n---\n\n## 🌟 核心功能\n\n- **多维度能力评估探针 (Probes)**：针对 LLM 核心能力进行深度测评：\n  - **Reasoning**：推理与长链思考能力\n  - **Tool Calling**：工具/函数调用准确率与参数解析\n  - **JSON Mode**：输出结构化 JSON 数据的遵从度\n  - **Streaming**：流式传输的响应首包延迟与稳定性\n  - **Vision**：多模态/视觉理解评估\n  - **Basic Chat \u0026 Web Search**：基础对话响应延迟及联网搜索评估\n- **无泄漏的零配置安全管理**：采用高强度的本地配置文件加密机制（如 `platforms.enc`），配合系统环境变量注入，从源头上彻底规避 API 密钥和敏感凭证在 Git 提交历史中泄露的风险。\n- **统一的双入口操作**：支持便捷的一键 CLI 自动化评估运行，并提供基于 React + Vite 的精美 Web 监控控制台，实时查看测试进度与对比结果。\n- **详尽的报告生成与解析**：自动将测试原始日志解析为结构化的评分卡与可视化对比报告。\n\n---\n\n## 🛠️ 技术栈\n\n| 层级 | 技术 |\n|---|---|\n| **后端/核心引擎** | Python 3.x · FastAPI · HTTPX · PyYAML · Cryptography |\n| **前端控制台** | React (Vite) · Vanilla CSS · 现代深色主题 · 微交互动效 |\n| **存储** | 轻量级本地文件数据库 · 结构化 JSON 存储 |\n\n---\n\n## 🚀 快速开始\n\n### 准备条件\n\n- Python 3.8+\n- Node.js 18+ (用于 Web 前端运行)\n\n### 安装步骤\n\n1. **克隆项目仓库**：\n   ```bash\n   git clone https://github.com/transcentlin/API-Probe-Platform.git\n   cd API-Probe-Platform\n   ```\n\n2. **配置后端环境**：\n   ```bash\n   pip install -r api_probe_system/requirements.txt\n   ```\n\n3. **配置前端环境**：\n   ```bash\n   cd api_probe_system/frontend\n   npm install\n   ```\n\n### 启动运行\n\n本系统支持 CLI 和 Web 双重入口：\n\n#### 1. 启动 Web 可视化平台（推荐）\n启动 FastAPI 后端服务：\n```bash\npython run_web.py\n```\n启动 React 前端服务：\n```bash\ncd api_probe_system/frontend\nnpm run dev\n```\n\n#### 2. CLI 命令行评估模式\n您也可以直接在终端针对特定模型或能力运行单个探针：\n```bash\npython api_probe_system/run.py --help\n```\n\n---\n\n## 🔒 凭证配置与安全规范\n\n为了保护您的 API 密钥安全，系统在初始化配置时遵循以下规范：\n1. 本地生成纯净的 `platforms.enc` 加密配置载体。\n2. 通过环境变量注入各平台的 API Token。\n3. 相关的解密 Key 必须置于代码库之外，禁止以任何形式上传至 Git。\n\n---\n\n## 🤝 贡献与反馈\n\n欢迎提出 [Issue](https://github.com/transcentlin/API-Probe-Platform/issues) 或 [Pull Request](https://github.com/transcentlin/API-Probe-Platform/pulls)，共同改进本项目。\n\n如果您觉得本项目对您有帮助，请给我们一个 ⭐ Star，这是对开源开发者最大的鼓励！\n\n---\n\n## 📄 许可证\n\n本项目基于 [MIT License](LICENSE) 开源。\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftranscentlin%2Fapi-probe-platform","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftranscentlin%2Fapi-probe-platform","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftranscentlin%2Fapi-probe-platform/lists"}