{"id":49120457,"url":"https://github.com/veeicwgy/ai-visibility-toolkit","last_synced_at":"2026-04-22T12:01:06.570Z","repository":{"id":351928086,"uuid":"1212438525","full_name":"veeicwgy/ai-visibility-toolkit","owner":"veeicwgy","description":"AI visibility toolkit for developer tools, APIs, SDKs, and open-source projects.","archived":false,"fork":false,"pushed_at":"2026-04-21T08:53:14.000Z","size":510,"stargazers_count":7,"open_issues_count":0,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-04-21T10:44:34.190Z","etag":null,"topics":["content-ops","developer-tools","evaluation","generative-engine-optimization","geo","llm-evaluation","llm-monitoring","monitoring","observability","open-source","prompt-monitoring","python","quality-assurance","query-pool","testing-framework"],"latest_commit_sha":null,"homepage":"https://github.com/veeicwgy/geo-monitor-toolkit#readme","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/veeicwgy.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-04-16T11:32:08.000Z","updated_at":"2026-04-21T08:49:12.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/veeicwgy/ai-visibility-toolkit","commit_stats":null,"previous_names":["veeicwgy/geo-monitor-toolkit","veeicwgy/ai-visibility-toolkit"],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/veeicwgy/ai-visibility-toolkit","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/veeicwgy%2Fai-visibility-toolkit","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/veeicwgy%2Fai-visibility-toolkit/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/veeicwgy%2Fai-visibility-toolkit/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/veeicwgy%2Fai-visibility-toolkit/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/veeicwgy","download_url":"https://codeload.github.com/veeicwgy/ai-visibility-toolkit/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/veeicwgy%2Fai-visibility-toolkit/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32088913,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-21T06:27:27.065Z","status":"ssl_error","status_checked_at":"2026-04-21T06:27:21.250Z","response_time":128,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["content-ops","developer-tools","evaluation","generative-engine-optimization","geo","llm-evaluation","llm-monitoring","monitoring","observability","open-source","prompt-monitoring","python","quality-assurance","query-pool","testing-framework"],"created_at":"2026-04-21T11:00:55.395Z","updated_at":"2026-04-21T11:00:59.957Z","avatar_url":"https://github.com/veeicwgy.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AI Visibility Toolkit\n\n\u003e Monitor how ChatGPT, Claude, Gemini, and other LLMs describe your developer tool, API, SDK, or open-source project.\n\n[![CI](https://github.com/veeicwgy/ai-visibility-toolkit/actions/workflows/ci.yml/badge.svg)](https://github.com/veeicwgy/ai-visibility-toolkit/actions/workflows/ci.yml)\n![Release](https://img.shields.io/github/v/release/veeicwgy/ai-visibility-toolkit)\n![Python](https://img.shields.io/badge/python-3.11-blue)\n![License](https://img.shields.io/github/license/veeicwgy/ai-visibility-toolkit)\n\nPrefer agent workflow? Install the [ClawHub skill](https://clawhub.ai/veeicwgy/ai-visibility-toolkit).\n\n**AI Visibility Toolkit** is a reproducible monitoring and repair workflow for developer tools, APIs, SDKs, and open-source projects.\nIt connects **Query Pool design, answer monitoring, four-metric scoring, repair loops, activation analysis, and T+7/T+14 regression checks** into one practical system.\n\nIt pairs:\n\n- the **`ai-visibility-toolkit` repo** for runnable demos, artifacts, and reporting scripts\n- the **AI Visibility Toolkit skill on ClawHub** for agent-guided monitoring, query design, and repair workflows\n\nFor 中文说明, see [`README.zh-CN.md`](README.zh-CN.md).\n\n## Why this repository exists\n\nIf your team is already asking whether LLMs mention your product, describe it correctly, recommend it positively, or improve after documentation fixes, this repository turns that concern into an executable and reviewable workflow.\n\n| What you need to answer | What this repository gives you |\n|---|---|\n| Do models mention our product at all? | Query Pool + raw responses |\n| When they mention us, is it accurate and positive? | Four-metric scoring framework |\n| Where should we repair the source of truth? | Placement and repair lens |\n| Are we improving installs, API calls, or agent adoption? | Activation metrics + funnel-stage slices |\n| Did our fixes actually improve model answers? | T+7 / T+14 regression checks |\n\n## Try the Skill\n\nIf you want the agent workflow instead of starting with the CLI, install the companion skill on ClawHub and start with one of these prompts:\n\n- `Analyze how ChatGPT and Claude describe my API docs`\n- `Build an AI visibility query pool for my SDK`\n- `Find negative or outdated LLM claims about my project`\n\nThe ClawHub skill is the companion agent layer for this repository:\n\n- Skill page: [ai-visibility-toolkit on ClawHub](https://clawhub.ai/veeicwgy/ai-visibility-toolkit)\n- Repo quick demo: [30-second path](#30-second-path)\n\n## 30-second path\n\nFor a first run, follow this exact order.\n\n```bash\ngit clone https://github.com/veeicwgy/ai-visibility-toolkit.git\ncd ai-visibility-toolkit\nbash install.sh\nmake doctor\nbash quickstart.sh\n```\n\n## What you will get first\n\nAfter the first run, start with these outputs.\n\n| Output | Path | Why it matters |\n|---|---|---|\n| Raw responses | `data/runs/quickstart-run/raw_responses.jsonl` | Review multi-model answer evidence |\n| Score draft | `data/runs/quickstart-run/score_draft.jsonl` | Start manual review and annotation |\n| Weekly report snapshot | `data/runs/sample-run/weekly_report.md` | See the report format a team can consume |\n| Sciverse sample summary | `data/runs/sciverse-sample-run/summary.json` | See a scientific API sample with funnel-stage slices |\n| Sciverse sample weekly report | `data/runs/sciverse-sample-run/weekly_report.md` | See a complete second sample for API and agent scenarios |\n| Leaderboard snapshot | `assets/leaderboard-sample.png` | Understand the default multi-model comparison |\n| Repair trend snapshot | `assets/repair-trend-sample.png` | See how follow-up runs can be visualized over time |\n\n\u003e `quickstart.sh` creates a fresh `quickstart-run` with raw evidence, then replays built-in sample summaries to generate report and chart snapshots. Your first run therefore shows both what a new run looks like and what a mature output package looks like.\n\n## Beginner-first docs\n\nIf you are new to AI visibility monitoring, start with these entry points.\n\n| Document | Purpose |\n|---|---|\n| `docs/for-beginners.md` | 5-minute path: run it once and read the outputs |\n| `docs/getting-started.md` | Long-form onboarding with modes, outputs, and team usage |\n| `docs/activation-metrics.md` | Extend answer visibility into install, API, and agent adoption |\n\n## Choose your first visibility goal\n\nIf you are not sure where to start, pick the path that matches the business outcome you care about most.\n\n| Goal | Start here | Why |\n|---|---|---|\n| Improve mention and recommendation quality | `data/query-pools/mineru-example.json` + `docs/metric-definition.md` | Baseline the 4 core visibility metrics first |\n| Improve downloads and installs | `docs/activation-metrics.md` + `playbooks/developer-tool-surface-priority.md` | Add actionability and source-surface prioritization |\n| Improve API calls and agent invocations | `playbooks/agent-readiness.md` + `data/query-pools/sciverse-api-integration-example.json` | Focus on integration and agent-selection queries |\n| Improve visibility for scientific products | `playbooks/scientific-product-visibility.md` | Use a product model tuned for MinerU, Sciverse API, and research workflows |\n\n## Which mode should you choose\n\n| Your situation | Recommended mode | Entry point |\n|---|---|---|\n| No API key yet and you only want to see the full workflow | Quickstart replay | `bash quickstart.sh` |\n| You already copied answers from external chat products and want to score them | Manual paste mode | `python -m ai_visibility run --manual-responses ...` |\n| You want real, repeatable, multi-model monitoring | API collection mode | `python -m ai_visibility run --query-pool ... --model-config ...` |\n\n## Core commands\n\n| Command | What it does |\n|---|---|\n| `bash install.sh` | Creates `.venv` and installs dependencies |\n| `make doctor` | Checks Python, dependencies, sample files, and output directories |\n| `bash quickstart.sh` | Runs the zero-API-cost beginner demo |\n| `make sample-report` | Rebuilds the MinerU sample report and chart assets |\n| `make sample-report-sciverse` | Rebuilds the Sciverse API sample summary and weekly report |\n| `make sample-reports` | Rebuilds both default sample report packages |\n| `python -m ai_visibility run ...` | Runs custom Query Pool monitoring |\n\n\u003e Compatibility note: a legacy CLI alias remains supported for existing automation.\n\n## Default sample inputs\n\n| File | Purpose |\n|---|---|\n| `data/query-pools/mineru-example.json` | Default Query Pool sample for developer tools |\n| `data/query-pools/sciverse-api-integration-example.json` | Scientific API and agent workflow Query Pool sample |\n| `data/models.sample.json` | Minimal single-model config |\n| `data/models.multi.sample.json` | Default multi-model config |\n| `data/manual.sample.json` | Minimal manual-response sample |\n| `data/manual.multi.sample.json` | Multi-model manual-response sample |\n| `data/runs/sample-run/summary.json` | Complete MinerU sample summary |\n| `data/runs/sciverse-sample-run/summary.json` | Complete Sciverse API sample summary |\n\n## Docs\n\n- Getting started: [`docs/getting-started.md`](docs/getting-started.md)\n- 5-minute beginner path: [`docs/for-beginners.md`](docs/for-beginners.md)\n- Metric definition: [`docs/metric-definition.md`](docs/metric-definition.md)\n- Activation metrics: [`docs/activation-metrics.md`](docs/activation-metrics.md)\n- Benchmark notes: [`benchmark/README.md`](benchmark/README.md)\n- Example case: [`examples/mineru-case-study.md`](examples/mineru-case-study.md)\n- Weekly report template: [`templates/weekly-report.md`](templates/weekly-report.md)\n- Repair validation template: [`templates/repair-validation.md`](templates/repair-validation.md)\n- Agent readiness: [`playbooks/agent-readiness.md`](playbooks/agent-readiness.md)\n- Developer-tool surface priority: [`playbooks/developer-tool-surface-priority.md`](playbooks/developer-tool-surface-priority.md)\n- Scientific product visibility: [`playbooks/scientific-product-visibility.md`](playbooks/scientific-product-visibility.md)\n- Companion skill: [ClawHub skill page](https://clawhub.ai/veeicwgy/ai-visibility-toolkit)\n\n## Repository positioning\n\nThink of this repository as:\n\n\u003e **AI Visibility Workflow for Developer Tools**\n\u003e\n\u003e It focuses on **monitoring, scoring, repair, activation, and regression**, not on generic marketing copy generation.\n\n## Contributing\n\nContributions are welcome.\n\nUseful contributions include:\n\n- new query pool examples\n- benchmark cases\n- runner improvements\n- report improvements\n- schema and validation improvements\n- documentation and onboarding fixes\n\nSee [`CONTRIBUTING.md`](CONTRIBUTING.md) for details.\n\n## License\n\nMIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fveeicwgy%2Fai-visibility-toolkit","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fveeicwgy%2Fai-visibility-toolkit","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fveeicwgy%2Fai-visibility-toolkit/lists"}