{"id":50897704,"url":"https://github.com/aptdnfapt/Ai-speedometer","last_synced_at":"2026-07-03T16:01:18.437Z","repository":{"id":316219878,"uuid":"1059748916","full_name":"aptdnfapt/Ai-speedometer","owner":"aptdnfapt","description":" Benchmark AI model speeds with TTFT, tokens/sec, and performance metrics .","archived":false,"fork":false,"pushed_at":"2026-05-13T00:39:04.000Z","size":497,"stargazers_count":19,"open_issues_count":1,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-13T02:32:56.167Z","etag":null,"topics":["ai","ai-sdk","glm","openai","opencode","qwen"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/aptdnfapt.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null}},"created_at":"2025-09-18T22:20:42.000Z","updated_at":"2026-05-13T00:39:08.000Z","dependencies_parsed_at":"2025-11-03T19:01:38.276Z","dependency_job_id":null,"html_url":"https://github.com/aptdnfapt/Ai-speedometer","commit_stats":null,"previous_names":["aptdnfapt/ai-speedometer"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/aptdnfapt/Ai-speedometer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aptdnfapt%2FAi-speedometer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aptdnfapt%2FAi-speedometer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aptdnfapt%2FAi-speedometer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aptdnfapt%2FAi-speedometer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/aptdnfapt","download_url":"https://codeload.github.com/aptdnfapt/Ai-speedometer/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aptdnfapt%2FAi-speedometer/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35092185,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-03T02:00:05.635Z","response_time":110,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","ai-sdk","glm","openai","opencode","qwen"],"created_at":"2026-06-16T01:31:30.075Z","updated_at":"2026-07-03T16:01:18.192Z","avatar_url":"https://github.com/aptdnfapt.png","language":"TypeScript","funding_links":[],"categories":["ai"],"sub_categories":[],"readme":"# Ai-speedometer\n\nA CLI tool for benchmarking AI models across multiple providers with parallel execution and performance metrics. fully opencode Compatible \n\n[![Discord](https://img.shields.io/badge/Discord-Join%20Community-5865F2?style=for-the-badge\u0026logo=discord\u0026logoColor=white)](https://discord.gg/6S7HwCxbMy)\n\nTrack OSS model speeds over time: [ai-speedometer.oliveowl.xyz](https://ai-speedometer.oliveowl.xyz/)\n\n![Ai-speedometer main menu](./pics/main.png)\n\n![Ai-speedometer benchmark](./pics/benchmark.png)\n\n## Packages\n\nThe project is a monorepo split into three packages:\n\n| Package | Description | Runtime |\n|---|---|---|\n| `ai-speedometer` | Full TUI + headless benchmark CLI | Bun 1.0+ |\n| `ai-speedometer-headless` | Headless benchmark only, no TUI deps | Node.js 18+ or Bun |\n| `@ai-speedometer/core` | Shared benchmark engine (private) | — |\n\n## Install\n\n### Standard (TUI + Headless)\nFull interactive terminal UI and headless benchmark support. Requires [Bun](https://bun.sh).\n\n```bash\nbun install -g ai-speedometer\n```\n\n### Headless Only (Lightweight)\nNo TUI dependencies — runs on **Node.js or Bun**. Ideal for CI/CD, Docker, and scripts.\n\n```bash\nnpm install -g ai-speedometer-headless\n# or\nbun install -g ai-speedometer-headless\n```\n\nOr run directly from source:\n\n```bash\nbun run packages/ai-speedometer/src/index.ts\n```\n\n## What It Measures\n\n- **TTFT** (Time to First Token) - How fast the first response token arrives\n- **Total Time** - Complete request duration\n- **Tokens/Second** - Real-time throughput\n- **F1000** (First to 1000) - Time to complete 1000 agentic requests (~300 tokens each), shown in hours. Captures real-world coding assistant performance better than TPS or TTFT alone. Lower is better.\n- **Token Counts** - Input, output, and total tokens used\n\n## Usage\n\n### Interactive TUI (`ai-speedometer` only)\n```bash\nai-speedometer\n# or short alias and just use ! auto configured via opencode dont have to re add provider here . \naispeed\n```\n\n### Headless Benchmark (both packages)\n\n```bash\n# Verified provider\nai-speedometer --bench openai:gpt-4o\nai-speedometer-headless --bench openai:gpt-4o\n\n# With explicit API key\nai-speedometer-headless --bench openai:gpt-4o --api-key \"sk-...\"\n\n# Custom provider\nai-speedometer-headless --bench-custom myprovider:mymodel \\\n  --base-url https://api.example.com \\\n  --api-key \"...\"\n\n# Pretty-print JSON output\nai-speedometer-headless --bench openai:gpt-4o --formatted\n\n# Custom endpoint format\nai-speedometer-headless --bench-custom myprovider:mymodel \\\n  --base-url https://... --endpoint-format chat/completions\n```\n\n## Features\n\n- **opencode config Compatible aka models / provider from opencode works out of the box in this cli no need to re add them**\n- **Monorepo Architecture** - Split into `core`, `ai-speedometer` (TUI), and `ai-speedometer-headless` (dedicated CLI)\n- **Interactive TUI** - Full terminal UI with theming support, menus, search, and live benchmark progress\n- **33 Themes** - Full theme system ported from opencode — dark themes (tokyonight, dracula, catppuccin, kanagawa, rosepine, nord, gruvbox, monokai, synthwave84, and more) and light themes (github, everforest, solarized, flexoki, vercel, mercury)\n- **Live Theme Switcher** - Press `T` anywhere in the app to open a searchable theme picker — changes apply instantly across the entire UI and persist to config\n- **Animated Progress Bar** - Smooth glow/dim sine wave animation during benchmarks, fully theme-aware\n- **F1000 Metric** - First to 1000: measures real-world agentic coding performance (1000 requests × ~300 tokens)\n- **FAQ/Learn Section** - In-app documentation explaining metrics, formulas, and resources\n- **Headless Mode** - Run benchmarks without interactive UI via CLI flags — outputs JSON, perfect for CI/CD\n- **Node.js Compatible Headless** - `ai-speedometer-headless` targets Node.js, no Bun required\n- **REST API Benchmarking** - Works with all OpenAI-compatible providers\n- **Parallel Execution** - Benchmark multiple models simultaneously\n- **Provider Management** - Add verified, custom verified, and custom providers\n\n## Development\n\n```bash\n# Run from source (monorepo)\nbun run packages/ai-speedometer/src/index.ts\n\n# Build all packages\nbun run build\n\n# Specific builds\nbun run build:tui        # -\u003e packages/ai-speedometer/dist/\nbun run build:headless   # -\u003e packages/ai-speedometer-headless/dist/\n\n# Run tests\nbun test\n\n# Typecheck\nbun run typecheck\n```\n\n## Configuration Files\n\nAPI keys and configuration are stored in:\n\n- **Verified + Custom Verified Providers**:\n  - Primary: `~/.local/share/opencode/auth.json`\n  - Backup: `~/.config/ai-speedometer/ai-benchmark-config.json` (verifiedProviders section)\n- **Custom Providers**: `~/.config/ai-speedometer/ai-benchmark-config.json` (customProviders section)\n- **Provider Definitions**: `./custom-verified-providers.json` (bundled at build time)\n- **Theme**: set `\"theme\"` in `ai-benchmark-config.json`, or press `T` inside the TUI to switch live\n  - Dark: `tokyonight` (default), `dracula`, `catppuccin`, `catppuccin-frappe`, `catppuccin-macchiato`, `kanagawa`, `rosepine`, `nord`, `gruvbox`, `monokai`, `one-dark`, `nightowl`, `aura`, `ayu`, `carbonfox`, `cobalt2`, `cursor`, `material`, `matrix`, `opencode`, `orng`, `lucent-orng`, `osaka-jade`, `palenight`, `synthwave84`, `vesper`, `zenburn`\n  - Light: `github`, `everforest`, `solarized`, `flexoki`, `mercury`, `vercel`\n\n## Requirements\n\n- **`ai-speedometer`**: Bun 1.0+ (install from [bun.sh](https://bun.sh))\n- **`ai-speedometer-headless`**: Node.js 18+ or Bun 1.0+\n- API keys for AI providers\n- Terminal with ANSI color support (TUI only)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faptdnfapt%2FAi-speedometer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faptdnfapt%2FAi-speedometer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faptdnfapt%2FAi-speedometer/lists"}