{"id":50521963,"url":"https://github.com/nlai741533/efc-plugin","last_synced_at":"2026-06-03T05:00:55.377Z","repository":{"id":361352541,"uuid":"1254143850","full_name":"Nlai741533/EFC-Plugin","owner":"Nlai741533","description":"Fact-check AI-generated research reports — CLI, Claude Code plugin, GitHub Action","archived":false,"fork":false,"pushed_at":"2026-05-30T13:29:59.000Z","size":114,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-30T15:12:43.604Z","etag":null,"topics":["ai-safety","claude-code","fact-checking","github-actions","hallucination","llm","research","verification"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Nlai741533.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-05-30T07:34:32.000Z","updated_at":"2026-05-30T13:30:11.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/Nlai741533/EFC-Plugin","commit_stats":null,"previous_names":["nlai741533/everything-fact-checked","nlai741533/efc-plugin"],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/Nlai741533/EFC-Plugin","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Nlai741533%2FEFC-Plugin","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Nlai741533%2FEFC-Plugin/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Nlai741533%2FEFC-Plugin/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Nlai741533%2FEFC-Plugin/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Nlai741533","download_url":"https://codeload.github.com/Nlai741533/EFC-Plugin/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Nlai741533%2FEFC-Plugin/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33848862,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-03T02:00:06.370Z","response_time":59,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai-safety","claude-code","fact-checking","github-actions","hallucination","llm","research","verification"],"created_at":"2026-06-03T05:00:54.286Z","updated_at":"2026-06-03T05:00:55.365Z","avatar_url":"https://github.com/Nlai741533.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# EFC-Plugin\n\n**Your AI wrote a confident report. Is it actually *true*?**\n\n`EFC-Plugin` is a Claude Code plugin that turns your agent into a rigorous fact-checker — catching hallucinated numbers, fabricated data points, and exaggerated claims *before* they reach your reader.\n\n[![CI](https://github.com/Nlai741533/EFC-Plugin/actions/workflows/ci.yml/badge.svg)](https://github.com/Nlai741533/EFC-Plugin/actions/workflows/ci.yml)\n[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)\n\n\u003e 🪞 This repo eats its own dog food: it is fact-checked against itself before every release. See [`FACTCHECK.md`](FACTCHECK.md).\n\n---\n\n## Why you'll want this\n\nLLM agents are great at producing reports that *look* authoritative. The trouble is the parts that aren't — and they fail in **predictable, repeatable ways**:\n\n| Failure mode | What it looks like |\n|---|---|\n| 🔢 **Unit / scale errors** | $5.3B that should be $530M — a dropped conversion |\n| 📈 **Fabricated interpolation** | A 6-point chart where only 2 points were ever found |\n| 🔀 **Source conflation** | \"GMV\" reported as \"revenue\"; \"trade\" as \"exports\" |\n| 🕰️ **Stale data as current** | 2023 figures presented as 2025 actuals |\n| 🎭 **Attribution laundering** | A blog cited as if it were a regulatory filing |\n\nThese aren't random slips — they're systematic patterns that show up whenever an LLM does research at scale. This plugin gives your agent a **structured protocol** to hunt them down, plus small scripts that automate the tedious first steps.\n\n## See it in action\n\nPoint it at a flawed report and it produces a verdict-by-verdict audit:\n\n```\n\u003e fact-check examples/sample-report.md\n\n❌ Errors Found\n| Claim                         | Reported  | Actual              | Mode             |\n|-------------------------------|-----------|---------------------|------------------|\n| FY2024 revenue                | $4,200B   | $4.2B (per table)   | Unit/scale error |\n| App \"generated $1.2B revenue\" | revenue   | marketplace GMV     | Source conflation|\n\n⚠️ Unverifiable\n| 2019–2023 revenue series      | only FY2024 had a source → likely interpolated     |\n\n🔗 Broken links: 2 of 3 source URLs don't resolve\nOverall reliability: Low — do not publish without correction\n```\n\nTry it yourself: [`examples/sample-report.md`](examples/sample-report.md) is a deliberately flawed (fictional) report with one of each failure mode, and [`examples/expected-fact-check.md`](examples/expected-fact-check.md) shows the target output.\n\n## Quick start\n\nIt's a standard Claude Code plugin — two lines to install:\n\n```\n/plugin marketplace add Nlai741533/EFC-Plugin\n/plugin install fact-check@everything-fact-checked\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eWait, why three different names?\u003c/summary\u003e\n\nThis project wears three hats, so it answers to three names:\n\n| Name | What it is | Where you see it |\n|---|---|---|\n| **`EFC-Plugin`** | the GitHub **repository** | clone URL, `marketplace add` |\n| **`everything-fact-checked`** | the **marketplace** + the PyPI package | `install …@everything-fact-checked`, `pip install` |\n| **`fact-check`** | the **plugin/skill** itself | `install fact-check@…`, the `/fact-check` prompt |\n\nSo `/plugin install fact-check@everything-fact-checked` reads as \"install the\n**fact-check** plugin from the **everything-fact-checked** marketplace.\"\n\u003c/details\u003e\n\nOr kick the tires for a single session, no install:\n\n```bash\ngit clone https://github.com/Nlai741533/EFC-Plugin\nclaude --plugin-dir ./EFC-Plugin\n```\n\n\u003e Verified against Claude Code 2.1.143. There is no `claude skill add` command —\n\u003e use the plugin commands above. (`--plugin-url` exists but expects a packaged\n\u003e `.zip` URL, not a repo page, so it is intentionally not shown here.)\n\nThen just ask, in plain language:\n\n```\nfact-check this report\nverify the numbers in the market analysis\naudit the data in this deliverable\n```\n\n## How it works\n\nThe skill runs a structured 6-step workflow:\n\n1. **Inventory** every specific, checkable claim\n2. **Triage** by risk (P0 critical → P3 cosmetic)\n3. **Verify** P0/P1 claims against *primary* sources\n4. **Cross-check** charts and tables for internal consistency\n5. **Audit** the source list (do the links even resolve?)\n6. **Report** — every verdict backed by a record in a [standard evidence format](skills/fact-check/SKILL.md#standard-evidence-format)\n\nIt also treats all source content as **untrusted data, not instructions** — so a document can't sweet-talk the fact-checker into stamping itself \"verified.\"\n\n### What it is (and isn't)\n\n- ✅ **A disciplined operating procedure** — triage, primary-source preference, chart/table tracing, marketing-claim labeling, a standard evidence format.\n- ✅ **A CLI (`efc`)** for local use and CI — claim extraction, link checking, evidence validation, source-content verification, and full audit.\n- ✅ **A GitHub Action** that auto-checks markdown reports in PRs.\n- ❌ **Not a push-button oracle.** Deciding whether a source *truly* supports a claim is a judgment call — the agent still opens the primary sources. The tools tell you *what to check*, not *whether it's true*.\n\n## CLI (`efc`)\n\nInstall and use:\n\n```bash\npip install .\n# or: pipx install .\n\nefc version                                    # show version\nefc extract report.md                          # inventory claims\nefc extract report.md --json                   # JSON output\nefc links report.md                            # check source URLs\nefc links --no-network report.md               # list URLs only\nefc evidence evidence.json                     # validate evidence records\nefc verify evidence.json                       # verify source content matches claims\nefc audit --no-network report.md               # full audit (claims + links + summary)\nefc audit --json report.md                     # machine-readable audit for CI\n```\n\nExit codes: `0` = clean, `1` = problems found, `2` = usage/IO error.\n\nNo third-party dependencies — standard library only. Tested on Python 3.11–3.13.\n\nEvery verdict can be recorded in a machine-checkable [evidence format](schemas/evidence.schema.json); `efc evidence` enforces the schema *and* the cross-field rules (e.g. a `verified` verdict must cite a well-formed http/https URL and, for P0/P1 claims, a primary or secondary source). See [`examples/evidence-sample.json`](examples/evidence-sample.json).\n\n## Source-content verification\n\n`efc verify` goes beyond link-checking: it fetches cited URLs, extracts visible text, and checks whether the claimed figure or key terms actually appear in the source.\n\n```bash\nefc verify evidence.json                       # verify all records\nefc verify --claim C001 evidence.json          # verify one claim\nefc verify --json evidence.json                # JSON output\n```\n\nVerdicts: `found` ✅ | `not_found` ❌ | `ambiguous` ⚠️ | `skipped` ⏭️ | `fetch_failed` 🔌\n\n\u003e **`found` is necessary, not sufficient.** This is a term-overlap heuristic: a\n\u003e `found` means the page contains the claim's key terms (numbers, years, names),\n\u003e *not* that the page actually supports the claim. A long article can contain\n\u003e \"$4.2B\" and \"2024\" in unrelated sentences. Treat `found` as \"worth a human\n\u003e read,\" `not_found`/`ambiguous` as \"investigate or flag\" — never as a final\n\u003e verdict on truth.\n\n## GitHub Action\n\nFact-check markdown reports automatically in PRs:\n\n```yaml\n- uses: Nlai741533/EFC-Plugin@v0.2.4\n  with:\n    check-links: 'true'\n    fail-on-broken-links: 'true'\n    link-timeout: '10'\n```\n\nThe action extracts claims from `.md` files (excluding docs like README, CHANGELOG, FACTCHECK), checks source links, and posts results to the PR summary.\n\n## 🛠️ Build on it\n\nThis is designed to be a **foundation, not a finished product**. PRs and forks are very welcome. Some directions that would make it more powerful:\n\n- **More extractors** — the inventory already covers figures, percentages, dates, and superlatives; add named entities, quotes, and currency-conversion pairs.\n- **PDF \u0026 table parsing** — extract claims straight from filings and spreadsheets.\n- **Exchange-rate sanity checks** — flag conversions made without a stated rate.\n- **Domain packs** — tuned source hierarchies for finance, science, law, medicine, etc.\n- **Report scoring** — produce a reliability score based on source coverage, P0/P1 verification rate, and broken link rate.\n\nNew to the repo? Good first issues: add a new claim type to `extract_claims.py` (with a test), or add a fixture for a failure mode that isn't covered yet.\n\n## Contributing\n\n```bash\npython3 -m unittest discover -s tests -v\n```\n\nCI validates the plugin/marketplace manifests, runs the test suite, and smoke-tests the scripts on every push. Please add a test with any new behavior — and run the scripts on your own change, in the spirit of the project. 🙂\n\n### Releasing\n\nBefore any doc change ships, the repo is fact-checked against itself: install commands are run against `claude --help`/`--version`, links are checked, and claims are verified against primary sources. The result lives in [`FACTCHECK.md`](FACTCHECK.md). (Round 1 shipped a fabricated `claude skill add` command — this routine exists so that never happens again.)\n\n## License\n\n[MIT](LICENSE) — use it, fork it, ship it. If it saves you from publishing a wrong number, ⭐ the repo so others can find it too.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnlai741533%2Fefc-plugin","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnlai741533%2Fefc-plugin","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnlai741533%2Fefc-plugin/lists"}