{"id":50559919,"url":"https://github.com/brahmlower/model-metrics-api","last_synced_at":"2026-06-04T11:30:21.176Z","repository":{"id":361754148,"uuid":"1255680073","full_name":"brahmlower/model-metrics-api","owner":"brahmlower","description":"Scrapes AI model metrics from artificialanalysis.ai and serves them via REST and MCP APIs","archived":false,"fork":false,"pushed_at":"2026-06-01T04:42:57.000Z","size":44,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-01T06:25:37.131Z","etag":null,"topics":["ai-models","go","llm","mcp"],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/brahmlower.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-06-01T04:34:22.000Z","updated_at":"2026-06-01T04:43:46.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/brahmlower/model-metrics-api","commit_stats":null,"previous_names":["brahmlower/model-metrics-api"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/brahmlower/model-metrics-api","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brahmlower%2Fmodel-metrics-api","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brahmlower%2Fmodel-metrics-api/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brahmlower%2Fmodel-metrics-api/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brahmlower%2Fmodel-metrics-api/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/brahmlower","download_url":"https://codeload.github.com/brahmlower/model-metrics-api/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brahmlower%2Fmodel-metrics-api/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33903134,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-04T02:00:06.755Z","response_time":64,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai-models","go","llm","mcp"],"created_at":"2026-06-04T11:30:20.525Z","updated_at":"2026-06-04T11:30:21.159Z","avatar_url":"https://github.com/brahmlower.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# model-metrics-api\n\nScrapes model metrics from [artificialanalysis.ai](https://artificialanalysis.ai/leaderboards/models)\nand serves them via a REST API and MCP server.\n\n## Quick start\n\n```bash\ntask build\nbin/scrape-aa serve\n```\n\n```bash\n# All models, sorted by Intelligence Index (default)\ncurl http://localhost:8080/api/v1/models | jq '.models[0:5] | .[].name'\n\n# Anthropic models sorted by coding score\ncurl \"http://localhost:8080/api/v1/models?creator=anthropic\u0026sort_by=coding_index\" | jq '.models[].name'\n\n# Models scoring above 0.5 on GPQA, cheapest first\ncurl \"http://localhost:8080/api/v1/models?bench=gpqa\u0026min=0.5\u0026sort_by=price_input\u0026order=asc\" \\\n  | jq '.models[] | {name, gpqa, price1mInputTokens}'\n\n# Single model\ncurl http://localhost:8080/api/v1/models/claude-sonnet-4-5 \\\n  | jq '{name, intelligenceIndex, price1mInputTokens}'\n```\n\n## Concepts\n\n**Scrape** — one fetch from artificialanalysis.ai, timestamped and stored in SQLite. The server\nscrapes on startup and then on each `--interval` tick. Every scrape is kept, so you can query\nhistorical data.\n\n**Slug** — a stable URL-safe identifier for each model (e.g. `claude-sonnet-4-5`). Slugs are\nconsistent across scrapes, so you can track a model over time.\n\n## Commands\n\n### `scrape` — one-off scrape to JSON\n\n```bash\nbin/scrape-aa scrape [--out models.json]\n```\n\nFetches all models, writes a JSON file, prints the top 5 by Intelligence Index.\n\n### `serve` — daemon with REST + MCP APIs\n\n```bash\nbin/scrape-aa serve [--addr :8080] [--db ./data.db] [--interval 1h]\n```\n\nScrapes immediately on start, then on each interval. Stores history in SQLite.\n\n## REST API\n\nBase: `http://localhost:8080/api/v1`\n\n### Endpoints\n\n| Method | Path | Description |\n| ------ | ---- | ----------- |\n| `GET` | `/scrapes` | List all scrapes (newest first) |\n| `GET` | `/scrapes/latest` | Latest scrape metadata |\n| `POST` | `/scrapes` | Trigger an immediate re-scrape |\n| `GET` | `/models` | Models from latest scrape |\n| `GET` | `/models/{slug}` | Single model by slug (latest scrape) |\n\n### `/models` query parameters\n\nAll parameters are optional and compose freely.\n\n| Parameter | Type | Default | Description |\n| --------- | ---- | ------- | ----------- |\n| `scrape_id` | integer | — | Return models from this specific scrape |\n| `as_of` | RFC3339 string | — | Return models from the closest scrape at or before this time |\n| `creator` | string | — | Filter by creator name (case-insensitive substring) |\n| `bench` | string | — | Filter by benchmark score (see [Benchmark fields](#benchmark-fields)) |\n| `min` | float | — | Minimum benchmark score, inclusive (requires `bench`) |\n| `max` | float | — | Maximum benchmark score, inclusive (requires `bench`) |\n| `sort_by` | string | `intelligence_index` | Sort field: `intelligence_index`, `coding_index`, `price_input`, `price_output`, `name` |\n| `order` | `asc` \\| `desc` | `desc` | Sort direction |\n\n**Examples:**\n\n```bash\n# Models from a specific historical scrape\ncurl \"http://localhost:8080/api/v1/models?scrape_id=3\"\n\n# Models as of a specific point in time\ncurl \"http://localhost:8080/api/v1/models?as_of=2025-01-01T00:00:00Z\"\n\n# Combine filters and sort\ncurl \"http://localhost:8080/api/v1/models?creator=openai\u0026bench=hle\u0026min=0.2\u0026sort_by=price_input\u0026order=asc\"\n\n# Historical model snapshot\ncurl \"http://localhost:8080/api/v1/models/gpt-4o?scrape_id=1\"\n```\n\n### Benchmark fields\n\nPass one of these field names as the `bench` query parameter.\n\n| Field | Benchmark |\n| ----- | --------- |\n| `intelligenceIndex` | AA Intelligence Index (composite) |\n| `codingIndex` | AA Coding Index (composite) |\n| `agenticIndex` | AA Agentic Index (composite) |\n| `gpqa` | GPQA Diamond — graduate-level science questions |\n| `hle` | Humanity's Last Exam |\n| `mmmuPro` | MMMU-Pro — multimodal understanding |\n| `omniscience` | Omniscience — factual knowledge \u0026 non-hallucination |\n| `scicode` | SciCode — scientific coding tasks |\n| `critpt` | CritPT — critical-point reasoning |\n| `gdpvalNormalized` | GDPVal — long-context dialogue |\n| `ifbench` | IFBench — instruction following |\n| `lcr` | LCR — long-context retrieval |\n| `tau2` | TAU-bench v2 — tool use and agentic tasks |\n| `apexAgents` | APEX Agents |\n| `itbenchSre` | ITBench SRE — IT/site-reliability operations |\n| `terminalbenchHard` | TerminalBench Hard — CLI tasks |\n\n### Response shapes\n\n**Scrape object** (returned by `/scrapes` and `POST /scrapes`):\n\n```json\n{\n  \"id\": 4,\n  \"scrapedAt\": \"2025-05-31T12:00:00Z\",\n  \"modelCount\": 183\n}\n```\n\n**Models response** (returned by `GET /models`):\n\n```json\n{\n  \"scrapeId\": 4,\n  \"scrapedAt\": \"2025-05-31T12:00:00Z\",\n  \"models\": [ ...model objects... ]\n}\n```\n\n**Model object** — key fields:\n\n```json\n{\n  \"id\": \"claude-sonnet-4-5\",\n  \"slug\": \"claude-sonnet-4-5\",\n  \"name\": \"Claude Sonnet 4.5\",\n  \"shortName\": \"Sonnet 4.5\",\n  \"modelCreatorName\": \"Anthropic\",\n  \"modelCreatorSlug\": \"anthropic\",\n\n  \"intelligenceIndex\": 76.4,\n  \"codingIndex\": 71.2,\n  \"agenticIndex\": 68.0,\n  \"gpqa\": 0.718,\n  \"hle\": 0.241,\n\n  \"price1mInputTokens\": 3.0,\n  \"price1mOutputTokens\": 15.0,\n  \"cacheHitPrice\": 0.3,\n\n  \"contextWindowTokens\": 200000,\n  \"totalParameters\": null,\n  \"reasoningModel\": false,\n  \"isOpenWeights\": false,\n  \"deprecated\": false,\n\n  \"medianOutputTokensPerSecond\": 98.4,\n  \"medianTimeToFirstTokenSeconds\": 0.61,\n\n  \"inputModalityText\": true,\n  \"inputModalityImage\": true,\n  \"inputModalitySpeech\": false,\n  \"outputModalityText\": true\n}\n```\n\nNullable fields are `null` when the data isn't available for that model. The full model object\ncontains ~60 additional fields for pricing blends, token-count breakdowns, openness metadata,\nand per-domain omniscience scores.\n\n## MCP Server\n\nSSE endpoint: `http://localhost:8080/mcp/sse`\n\nConnect with any MCP-compatible client (Claude Desktop, Continue, etc.) using the SSE transport.\n\n### Tools\n\n#### `list_models`\n\nReturns models from the latest scrape as a JSON array.\n\n| Parameter | Type | Required | Default | Description |\n| --------- | ---- | -------- | ------- | ----------- |\n| `sort_by` | string | no | `intelligence_index` | `intelligence_index`, `coding_index`, `price_input`, `price_output`, `name` |\n| `creator` | string | no | — | Filter by creator name (case-insensitive substring) |\n| `bench` | string | no | — | Filter by benchmark field name (see table above) |\n| `min` | number | no | — | Minimum bench score, inclusive |\n| `max` | number | no | — | Maximum bench score, inclusive |\n| `limit` | integer | no | `50` | Maximum number of results |\n\n#### `get_model`\n\nReturns a single model by slug.\n\n| Parameter | Type | Required | Description |\n| --------- | ---- | -------- | ----------- |\n| `slug` | string | **yes** | Model slug (e.g. `claude-sonnet-4-5`) |\n\nReturns `\"model not found\"` if the slug doesn't exist.\n\n#### `search_models`\n\nCase-insensitive substring search across model name and creator name.\n\n| Parameter | Type | Required | Description |\n| --------- | ---- | -------- | ----------- |\n| `query` | string | **yes** | Search string |\n\n### Resources\n\n| URI | Description |\n| --- | ----------- |\n| `aa://models` | Full model list from latest scrape as JSON array |\n| `aa://models/{slug}` | Single model as JSON (e.g. `aa://models/claude-sonnet-4-5`) |\n\n## Development\n\n```bash\ntask build     # compile to bin/scrape-aa\ntask test      # go test -race ./...\ntask lint      # golangci-lint + govulncheck + nilaway + go-arch-lint\ntask generate  # sqlc generate (after schema/query changes)\n```\n\nRequires\n- [Task](https://taskfile.dev)\n- [golangci-lint](https://golangci-lint.run)\n- [govulncheck](https://pkg.go.dev/golang.org/x/vuln/cmd/govulncheck)\n- [nilaway](https://github.com/uber-go/nilaway)\n- [sqlc](https://sqlc.dev)\n- [go-arch-lint](https://github.com/fe3dback/go-arch-lint)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbrahmlower%2Fmodel-metrics-api","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbrahmlower%2Fmodel-metrics-api","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbrahmlower%2Fmodel-metrics-api/lists"}