{"id":50708493,"url":"https://github.com/fraware/verified-science-agent","last_synced_at":"2026-06-09T13:31:53.987Z","repository":{"id":361452808,"uuid":"1254519105","full_name":"fraware/verified-science-agent","owner":"fraware","description":null,"archived":false,"fork":false,"pushed_at":"2026-05-30T18:02:36.000Z","size":23,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-30T19:06:58.221Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fraware.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-05-30T17:06:49.000Z","updated_at":"2026-05-30T18:02:40.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/fraware/verified-science-agent","commit_stats":null,"previous_names":["fraware/verified-science-agent"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/fraware/verified-science-agent","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fraware%2Fverified-science-agent","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fraware%2Fverified-science-agent/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fraware%2Fverified-science-agent/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fraware%2Fverified-science-agent/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fraware","download_url":"https://codeload.github.com/fraware/verified-science-agent/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fraware%2Fverified-science-agent/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34110011,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-09T02:00:06.510Z","response_time":63,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-06-09T13:31:53.913Z","updated_at":"2026-06-09T13:31:53.979Z","avatar_url":"https://github.com/fraware.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\n\u003cpre\u003e\n ######################################################################################################\n\n __     __        _  __ _          _   ____       _                          _                    _   \n \\ \\   / /__ _ __(_)/ _(_) ___  __| | / ___|  ___(_) ___ _ __   ___ ___     / \\   __ _  ___ _ __ | |_ \n  \\ \\ / / _ \\ '__| | |_| |/ _ \\/ _` | \\___ \\ / __| |/ _ \\ '_ \\ / __/ _ \\   / _ \\ / _` |/ _ \\ '_ \\| __|\n   \\ V /  __/ |  | |  _| |  __/ (_| |  ___) | (__| |  __/ | | | (_|  __/  / ___ \\ (_| |  __/ | | | |_ \n    \\_/ \\___|_|  |_|_| |_|\\___|\\__,_| |____/ \\___|_|\\___|_| |_|\\___\\___| /_/   \\_\\__, |\\___|_| |_|\\__|\n                                                                                 |___/                \n\n ######################################################################################################\n\u003c/pre\u003e\n\n**Evidence-backed scientific AI report infrastructure**\n\nTreat every AI-generated scientific report like a software build artifact —  \ninputs, source records, claims, validation, provenance, and review status.\n\n\u003cbr\u003e\n\n[![CI](https://github.com/fraware/verified-science-agent/actions/workflows/ci.yml/badge.svg)](https://github.com/fraware/verified-science-agent/actions/workflows/ci.yml)\n![Python](https://img.shields.io/badge/python-3.10%20|%203.11%20|%203.12-blue)\n![Version](https://img.shields.io/badge/version-v0.7.2-orange)\n![License](https://img.shields.io/badge/license-MIT-green)\n![Benchmark](https://img.shields.io/badge/benchmark-50%20tasks-brightgreen)\n\n\u003cbr\u003e\n\n[Documentation](docs/README.md) · [API](docs/api.md) · [Benchmark](docs/benchmark.md) · [Changelog](CHANGELOG.md)\n\n\u003c/div\u003e\n\n---\n\n## Overview\n\nVerified Science Agent (VSA) turns scientific questions into inspectable `ScientificReport` JSON artifacts. Retrieval produces evidence. Generation produces claims. Validation checks claims against evidence. Models cannot invent source fields.\n\n**Mission:** Scientific AI reports should be inspectable by engineers, readable by scientists, and shareable with reviewers — with signed or hashable outputs.\n\n```mermaid\nflowchart LR\n  Q[Question] --\u003e P[Subject parser]\n  P --\u003e C[Connectors]\n  C --\u003e E[Evidence ranking]\n  E --\u003e X[Claim extraction]\n  X --\u003e V[Validation]\n  V --\u003e R[Render / audit / review / export]\n```\n\n**Core rule:** retrieval → evidence · generation → claims · validation → proof\n\n---\n\n## Quick start\n\n```bash\ngit clone https://github.com/fraware/verified-science-agent.git\ncd verified-science-agent\npip install -e \".[dev,ui,pdf,signing,api]\"\nmake demo \u0026\u0026 pytest \u0026\u0026 vsa benchmark\n```\n\nOr run the full suite in one step:\n\n```bash\nmake acceptance\n```\n\n`make acceptance` runs a demo build, all tests, and the 50-task evaluation suite.\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eTypical workflow\u003c/strong\u003e\u003c/summary\u003e\n\n```bash\n# Retrieve and build\nvsa retrieve \"BRCA1 c.68_69del\"\nvsa build examples/brca1_input.json --out reports/brca1_report.json --claim-mode rule\n\n# Validate, audit, export\nvsa validate reports/brca1_report.json\nvsa audit reports/brca1_report.json --audit-mode rule --out reports/audit.json\nvsa export reports/brca1_report.json --out-dir reports/bundle --audit-mode rule\nvsa verify-bundle reports/bundle\n\n# Attestation and review\nvsa attest reports/brca1_report.json --out reports/attestation.json --subject-name report.json\nvsa review start reports/brca1_report.json --reviewer you@example.com\nvsa review approve-claim reports/brca1_report.json --reviewer you@example.com --claim C002\nvsa verify-review reports/brca1_report.json\n\n# Render, sign, serve\nvsa render reports/brca1_report.json --format markdown --out reports/brca1_report.md\nvsa sign reports/brca1_report.json\nvsa serve --port 8000\n```\n\n\u003c/details\u003e\n\n```bash\nmake demo                      # build + validate + audit + export + verify-bundle\nstreamlit run ui/app.py        # interactive inspector with credibility warnings\n```\n\n---\n\n## Features\n\n| Area | What you get |\n|------|----------------|\n| **Artifacts** | Canonical `ScientificReport` schema (v1.2.0), provenance hashes, export bundles |\n| **Credibility** | ClinVar ambiguity alerts, metadata-only warnings, AlphaFold predicted-structure labeling |\n| **Verification** | Schema + semantic validation, rule/hybrid audit, SLSA/in-toto attestation |\n| **Review** | Human review workflow with verifiable event chains |\n| **Benchmark** | 50-task evaluation suite with quality checks |\n| **API** | REST server with optional `VSA_API_KEY` auth |\n\n---\n\n## CLI reference\n\n### Build and retrieve\n\n| Command | Description |\n|---------|-------------|\n| `vsa retrieve \"question\"` | Retrieve evidence from databases |\n| `vsa build input.json --out report.json` | Build a full ScientificReport |\n| `vsa extract input.json` | Extract claims (rule or LLM) |\n| `vsa benchmark` | Run 50-task benchmark suite (`--live` for network) |\n\n### Validate and audit\n\n| Command | Description |\n|---------|-------------|\n| `vsa validate report.json` | Schema + semantic validation |\n| `vsa audit report.json` | Scientific audit (rule + optional LLM hybrid) |\n| `vsa compare report_a.json report_b.json` | Diff two reports |\n| `vsa compare-audit audit_a.json audit_b.json` | Diff audit artifacts |\n\n### Export and verify\n\n| Command | Description |\n|---------|-------------|\n| `vsa export report.json --out-dir dir/` | Full bundle: report, audit, provenance, sources/, manifest |\n| `vsa verify-bundle dir/` | Verify manifest hashes and attestation |\n| `vsa attest report.json --out attestation.json` | SLSA/in-toto provenance attestation |\n| `vsa verify-attestation report.json attestation.json` | Verify attestation digest |\n\n### Review\n\n| Command | Description |\n|---------|-------------|\n| `vsa review start report.json --reviewer NAME` | Start human review session |\n| `vsa review approve-claim ... --claim C001` | Approve specific claims |\n| `vsa review verify report.json` | Verify review chain hashes |\n| `vsa verify-review report.json` | Alias for `review verify` |\n\nLegacy flags remain supported: `vsa review report.json --reviewer NAME --approve C001`.\n\n### Render, sign, and serve\n\n| Command | Description |\n|---------|-------------|\n| `vsa render report.json --format markdown\\|html\\|json\\|pdf` | Render report |\n| `vsa hash report.json` | Provenance hash chain |\n| `vsa sign report.json` | Ed25519-sign report provenance hash |\n| `vsa verify-signature report.json` | Verify Ed25519 signature |\n| `vsa serve --port 8000` | Start REST API (requires `[api]` extra) |\n\nREST endpoint parity: [docs/api.md](docs/api.md)\n\n---\n\n## Scientific credibility\n\nVSA enforces policies that make weak evidence hard to miss:\n\n- **ClinVar ambiguity** — ambiguous queries capped to low reliability; `CLINVAR AMBIGUITY ALERT` in report warnings\n- **Metadata-only papers** — `SCIENTIFIC CREDIBILITY WARNING` when all publication evidence is bibliographic only\n- **AlphaFold** — summaries always declare predicted structure; never treated as experimental\n- **Materials Project** — missing API key degrades with an explicit skip warning\n\nWarnings surface in CLI output, markdown/HTML render, validation checks, and the Streamlit UI.\n\n---\n\n## LLM modes (optional)\n\nCopy `.env.example` to `.env` and add API keys. Never commit `.env`.\n\n```bash\nvsa build examples/brca1_input.json --out reports/brca1_report.json --claim-mode auto\nvsa audit reports/brca1_report.json --audit-mode auto\n```\n\nRule-based modes require no API keys and are used in CI:\n\n```bash\nvsa build examples/brca1_input.json --out reports/brca1_report.json --claim-mode rule\nvsa audit reports/brca1_report.json --audit-mode rule\n```\n\nThe LLM auditor evaluates only claim text and cited evidence in the payload — it cannot introduce new sources.\n\n---\n\n## ScientificReport schema\n\nCanonical artifact model (schema **1.2.0**, also accepts **1.0.0** and **1.1.0**):\n\n```\nScientificReport\n├── subject\n├── claims[]\n├── evidence[]\n├── methods[]\n├── provenance\n├── validation_results\n├── human_review\n└── generated_outputs\n```\n\nSchema: `src/vsa/schemas/scientific_report.schema.json` · Field reference: [docs/schema.md](docs/schema.md)\n\n**Domains:** genomics variants, proteins, papers, chemicals, materials, experiments.\n\n---\n\n## Connectors\n\nRead-only connectors with normalized evidence and file caching (`.vsa_cache/`):\n\n| Category | Sources |\n|----------|---------|\n| Literature | OpenAlex, Crossref, PubMed, Europe PMC, Semantic Scholar |\n| Genomics / protein | ClinVar, UniProt, AlphaFold DB |\n| Materials | Materials Project (`MATERIALS_PROJECT_API_KEY`) |\n\nDetails: [docs/connectors.md](docs/connectors.md)\n\n---\n\n## Benchmark\n\n50 evaluation tasks covering genomics, proteins, papers, materials, and edge cases:\n\n```bash\nvsa benchmark\n```\n\nMeasures source recall/precision, citation integrity, evidence validity, review boundaries, contradiction detection, and bundle reproducibility. See [docs/benchmark.md](docs/benchmark.md).\n\n---\n\n## Repository layout\n\n```\nverified-science-agent/\n├── src/vsa/            CLI, pipeline, connectors, validation, API\n├── schemas/            JSON Schema (symlink to package schema)\n├── benchmarks/         50 tasks + offline fixtures\n├── examples/           Input files and good/bad report examples\n├── docs/               Architecture, schema, connectors, API\n├── ui/                 Streamlit inspector\n├── scripts/            Full test suite helper\n└── .github/workflows/  CI and release pipelines\n```\n\nArchitecture: [docs/architecture.md](docs/architecture.md)\n\n---\n\n## Development\n\n```bash\npip install -e \".[dev,signing,api]\"\npytest\nmake demo\n```\n\n---\n\n## Safety notice\n\nResearch infrastructure only. Not a medical device, clinical decision system, or diagnostic platform.\n\n\u003e Research infrastructure output. Not for diagnosis, treatment, or clinical decision-making without qualified expert review.\n\nHuman expert review is required before any clinical use.\n\n---\n\n## License\n\nMIT License — see [LICENSE](LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffraware%2Fverified-science-agent","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffraware%2Fverified-science-agent","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffraware%2Fverified-science-agent/lists"}