{"id":51621142,"url":"https://github.com/ssskay/wiki-gap-finder","last_synced_at":"2026-07-12T20:00:15.356Z","repository":{"id":370544796,"uuid":"1288815294","full_name":"ssskay/wiki-gap-finder","owner":"ssskay","description":"Finds people missing from English Wikipedia and fact-checks every claim into a research dossier — the AI never writes the article, a human does.","archived":false,"fork":false,"pushed_at":"2026-07-09T14:06:27.000Z","size":106,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-07-09T15:17:29.293Z","etag":null,"topics":["claude-code","knowledge-equity","mediawiki","research-tools","wikidata","wikipedia"],"latest_commit_sha":null,"homepage":"https://pypi.org/project/wiki-gap-finder/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ssskay.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-07-04T02:15:36.000Z","updated_at":"2026-07-09T14:08:14.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/ssskay/wiki-gap-finder","commit_stats":null,"previous_names":["ssskay/wiki-gap-finder"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/ssskay/wiki-gap-finder","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ssskay%2Fwiki-gap-finder","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ssskay%2Fwiki-gap-finder/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ssskay%2Fwiki-gap-finder/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ssskay%2Fwiki-gap-finder/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ssskay","download_url":"https://codeload.github.com/ssskay/wiki-gap-finder/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ssskay%2Fwiki-gap-finder/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35401732,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-12T02:00:06.386Z","response_time":87,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["claude-code","knowledge-equity","mediawiki","research-tools","wikidata","wikipedia"],"created_at":"2026-07-12T20:00:14.628Z","updated_at":"2026-07-12T20:00:15.269Z","avatar_url":"https://github.com/ssskay.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Wikipedia Gap Finder v2\n\nA campaign-driven pipeline that finds people from an underrepresented group who\nshould have an English Wikipedia article but don't, checks every fact against real\nsources, and hands a human a research dossier they write the article from.\n\n\u003e **The tool never writes article prose.** English Wikipedia prohibits\n\u003e LLM-generated/-rewritten article content (guideline enacted by RfC, March 2026, with\n\u003e narrow copyedit and translation exceptions). This tool does detective work,\n\u003e verification, and structure only — a human writes every sentence, and the dossier is\n\u003e a map, not a source: the writer opens and reads every cited source themselves. That\n\u003e isn't a limitation to route around; it's the product's integrity claim: *AI found her\n\u003e and gathered the evidence; a human checked every source and wrote her story.*\n\n\u003cpicture\u003e\n  \u003csource media=\"(prefers-color-scheme: dark)\" srcset=\"https://raw.githubusercontent.com/ssskay/wiki-gap-finder/main/docs/images/pipeline-dark.svg\"\u003e\n  \u003cimg alt=\"Four steps: start with names (orgs, books, event lists); find who's missing (no Wikipedia article yet); check every fact (real sources, exact quotes); a human writes every single sentence. The first three are the AI's detective work, the last is the human's.\" src=\"https://raw.githubusercontent.com/ssskay/wiki-gap-finder/main/docs/images/pipeline-light.svg\"\u003e\n\u003c/picture\u003e\n\n## Pipeline\n\n1. **Intake** — names from CSV/txt lists and/or an optional Wikidata SPARQL redlist query.\n2. **Gap check** — per name: enwiki title + fuzzy search, redirect detection, `Draft:`\n   namespace, deletion-log history, and Wikidata sitelinks. Verdicts:\n   `GAP / EXISTS / REDIRECT_ONLY / DRAFT_EXISTS / DELETED_BEFORE / TRANSLATE_CANDIDATE`\n   (⭐ candidates that already exist in another language are the highest-value, since\n   translation is an allowed path).\n3. **Notability triage** *(planned)* — coverage search scored against WP:GNG.\n4. **Source vetting → dossier** — see below.\n\n## Source vetter\n\nThe vetter is two processes across a two-file JSON boundary, so each side is testable\nin isolation:\n\n\u003cpicture\u003e\n  \u003csource media=\"(prefers-color-scheme: dark)\" srcset=\"https://raw.githubusercontent.com/ssskay/wiki-gap-finder/main/docs/images/vetter-loop-dark.svg\"\u003e\n  \u003cimg alt=\"gap_finder --dossier gathers coverage and writes vetting_worklist.json; the Claude vet-sources subagent classifies and chases, writing vetting_verdicts.json (VERIFIED / LEAD / DEAD_END); gap_finder --dossier then renders dossier.md, which a human writes from. The JSON files are the whole contract.\" src=\"https://raw.githubusercontent.com/ssskay/wiki-gap-finder/main/docs/images/vetter-loop-light.svg\"\u003e\n\u003c/picture\u003e\n\n- **Python** (keyless): gathers coverage, tiers each source against Wikipedia's\n  perennial-sources list (WP:RSP), and renders the dossier.\n- **The Claude subagent** (`skills/vet-sources`): classifies reliability and traces\n  breadcrumbs, sorting every claim into one of three buckets:\n\n  | Bucket | Meaning | Dossier section |\n  |---|---|---|\n  | **VERIFIED** | a single reliable source names the subject *and* supports the claim, with a verbatim quote | Verified facts table |\n  | **LEAD** | a breadcrumb is real, but no reliable source names the subject yet | Research leads — chase before writing |\n  | **DEAD_END** | no corroboration found | UNVERIFIED — do not use |\n\n**The never-stitch rule (WP:SYNTH):** the vetter may not promote a LEAD to VERIFIED by\ncombining a subject-naming unreliable source with a fact-confirming reliable source. Two\nhalf-sources stay a LEAD. And the renderer structurally refuses to print any fact backed\nonly by an unreliable source — even if a verdicts file claims otherwise.\n\n\u003cpicture\u003e\n  \u003csource media=\"(prefers-color-scheme: dark)\" srcset=\"https://raw.githubusercontent.com/ssskay/wiki-gap-finder/main/docs/images/never-stitch-dark.svg\"\u003e\n  \u003cimg alt=\"A fan wiki that names her plus a news story that proves the fact but not her is still not a fact — it stays a research lead. Only one reliable source that names her and states the fact becomes a verified fact, quoted in the dossier.\" src=\"https://raw.githubusercontent.com/ssskay/wiki-gap-finder/main/docs/images/never-stitch-light.svg\"\u003e\n\u003c/picture\u003e\n\n## Install\n\n```bash\npip install wiki-gap-finder            # installs the `gap-finder` command\npip install \"wiki-gap-finder[ui]\"      # + Rich-rendered tables\npip install \"wiki-gap-finder[chase]\"   # + the Firecrawl coverage backend\n```\n\nOr clone this repo and run `python3 gap_finder.py ...` directly — same CLI\n(`pip3 install requests PyYAML jsonschema --break-system-packages`).\n\n## Install as a Claude skill\n\nThe source vetter ships as a standalone Claude skill (`vet-sources`) — the\nreasoning half of the pipeline, packaged so Claude can classify sources and\nemit verdicts on its own. Two ways to install it:\n\n- **Download and drop into Claude.** Grab `vet-sources.skill` from the\n  [latest release](https://github.com/ssskay/wiki-gap-finder/releases/latest)\n  and drop the file into Claude (it's a zip with a top-level `vet-sources/`\n  folder — SKILL.md plus the JSON schemas it references).\n- **Clone into your skills directory.** Copy `skills/vet-sources/` into\n  `~/.claude/skills/`:\n\n  ```bash\n  cp -r skills/vet-sources ~/.claude/skills/\n  ```\n\n## Usage\n\nCommands read and write paths relative to the current directory, so run them\nfrom your project root (campaign paths like `input/candidates.csv` resolve\nfrom there, and results land in `output/\u003ccampaign\u003e/`).\n\n```bash\ngap-finder --campaign campaigns/\u003ccampaign\u003e.yaml --check       # stages 1–2\ngap-finder --campaign campaigns/\u003ccampaign\u003e.yaml --check --no-sparql --limit 5\ngap-finder --campaign campaigns/\u003ccampaign\u003e.yaml --report      # table from saved state\ngap-finder --campaign campaigns/\u003ccampaign\u003e.yaml --dossier \"Name Here\"\ngap-finder --campaign ... --dossier \"Name\" --search-backend firecrawl\n```\n\nA campaign is one small YAML — swapping demographics is a new file, zero code\nchanges. One rule for lists naming living people (the spirit of WP:BLPCAT): only\ninclude identity attributes the person has publicly self-identified with, and only\nwhen they're relevant to their public life. Minimal example:\n\n```yaml\nname: my-campaign\ndescription: Who this campaign is about\nintake:\n  name_lists:\n    - \"input/candidates.csv\"   # CSV with a `name` column, or one name per line\n  # optional: a Wikidata SPARQL WHERE-clause body for redlist intake\nsearch_hints:                  # appended to coverage searches\n  - \"activist\"\n```\n\nOther flags: `-v` logs every request.\n\n**Heads-up on the keyless search default:** DuckDuckGo increasingly serves a\nbot challenge to non-browser clients. If coverage search comes back empty, the\ntool now says so loudly — use `--search-backend firecrawl` (with the\n`firecrawl` CLI installed) for dependable coverage gathering.\n\n## Conventions\n\nZero API keys on the default path (MediaWiki + Wikidata + DuckDuckGo are keyless;\nFirecrawl is opt-in). Polite `User-Agent`, ≥1s between requests with exponential\nbackoff, and `maxlag=5` on MediaWiki API calls so the tool yields when servers lag.\n\n```bash\npython3 -m pytest      # 54 tests\n```\n\n## Layout\n\n- `gap_finder.py` — back-compat shim; the CLI lives in `gapfinder/cli.py`\n- `gapfinder/` — the package (cli, contract, rsp, search, worklist, verdicts, dossier)\n- `gapfinder/data/rsp_seed.json` — curated WP:RSP reliability seed (ships in the wheel)\n- `skills/vet-sources/SKILL.md` — the Claude subagent contract\n- `scripts/build_skill.sh` — packages the skill into `dist/vet-sources.skill`\n- `campaigns/` — one YAML per campaign\n- `docs/superpowers/` — design spec and implementation plan\n\n## Releasing\n\nTag the version from `pyproject.toml`, cut a GitHub release, then build and\nattach the skill asset:\n\n```bash\ngit tag -a vX.Y.Z -m \"wiki-gap-finder vX.Y.Z\" \u0026\u0026 git push origin vX.Y.Z\ngh release create vX.Y.Z --title vX.Y.Z --notes \"...\"\nscripts/build_skill.sh                              # → dist/vet-sources.skill\ngh release upload vX.Y.Z dist/vet-sources.skill\n```\n\nMIT licensed.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fssskay%2Fwiki-gap-finder","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fssskay%2Fwiki-gap-finder","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fssskay%2Fwiki-gap-finder/lists"}