{"id":51167073,"url":"https://github.com/navapbc/temp-simpler-grants-vulnerability-dashboard","last_synced_at":"2026-06-26T20:30:24.621Z","repository":{"id":351873388,"uuid":"1212459329","full_name":"navapbc/temp-simpler-grants-vulnerability-dashboard","owner":"navapbc","description":null,"archived":false,"fork":false,"pushed_at":"2026-04-16T20:19:43.000Z","size":10,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-04-16T22:22:59.426Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/navapbc.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-04-16T11:59:04.000Z","updated_at":"2026-04-16T20:19:48.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/navapbc/temp-simpler-grants-vulnerability-dashboard","commit_stats":null,"previous_names":["navapbc/temp-simpler-grants-vulnerability-dashboard"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/navapbc/temp-simpler-grants-vulnerability-dashboard","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/navapbc%2Ftemp-simpler-grants-vulnerability-dashboard","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/navapbc%2Ftemp-simpler-grants-vulnerability-dashboard/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/navapbc%2Ftemp-simpler-grants-vulnerability-dashboard/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/navapbc%2Ftemp-simpler-grants-vulnerability-dashboard/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/navapbc","download_url":"https://codeload.github.com/navapbc/temp-simpler-grants-vulnerability-dashboard/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/navapbc%2Ftemp-simpler-grants-vulnerability-dashboard/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34832916,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-26T02:00:06.560Z","response_time":106,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-06-26T20:30:22.260Z","updated_at":"2026-06-26T20:30:24.613Z","avatar_url":"https://github.com/navapbc.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Vulnerability Analysis — analytics-dbt\n\n\u003e [!note]\n\u003e This readme was [(re)written by Claude](.dev/llm-threads-simpler-grants-gov/claude/2026-05-22T20-00-45.239Z-aac9c009-156f-44cd-bbee-46bb7782f2cd.jsonl) after an initial attempt to have it help clean up the mixed-approach state of the repo. Some of it may now be out of date and could likely use additional updates.\n\ndbt + DuckDB pipeline that turns Grype scan artifacts from GitHub Actions into\nstructured vulnerability lifecycle data. The goal is a durable, query-able\nhistory of which CVEs appeared in each container image, when they were first\nseen, how long they stayed open, and how the count changes week-over-week.\n\nContext: Jira BEN-219, GitHub epic HHS/simpler-grants-gov#9498.\n\n## Data flow\n\n```\nGitHub Actions (weekly)\n  └─ vulnerability-scans.yml  ← anchore-scan job\n       └─ Grype JSON artifact  (90-day retention)\n            \"anchore-grype-json-{app}-{run_id}\"\n\ncollect-grype-artifacts.sh   ← run once locally to download\n  └─ data/grype/{run_id}-{app}.json\n\ndbt run\n  └─ ingest/grype_findings     ← Python model, reads data/grype/*.json\n  └─ staging/stg_vuln_observations\n  └─ marts/vuln_lifecycle      ← first_seen / last_seen / days_open per CVE\n  └─ marts/weekly_rollup       ← new + open counts by week and severity\n```\n\nThe `ingest/workflow_runs` model is a separate microbatch ingest of GitHub\nworkflow run metadata (needs `DBT_ENV_SECRET_GITHUB_TOKEN`). It is not\nrequired by the vulnerability models; it exists to cross-reference run IDs\nwith commit hashes and branch names.\n\n## Quick start\n\n```bash\n# 1. Install dependencies\ncd analytics-dbt\nuv sync\n\n# 2. (Option A) Use the bundled sample data (3 findings, no credentials needed)\nuv run dbt run --vars '{grype_data_path: data/sample/*.json}' \\\n               --exclude workflow_runs\n\n# 2. (Option B) Download real artifacts from the last 90 days\nexport GITHUB_TOKEN=$(gh auth token)\n./collect-grype-artifacts.sh          # populates data/grype/\nuv run dbt run --exclude workflow_runs\n```\n\nAfter a successful run, query results:\n\n```bash\nduckdb target/results.duckdb \\\n  \"select * from vuln_lifecycle order by days_open desc limit 20\"\n```\n\n## Model reference\n\n| Model | Description |\n|-------|-------------|\n| `ingest/grype_findings` | Raw findings from local Grype JSON files. One row per match per run. |\n| `ingest/workflow_runs` | Workflow run metadata from GH API (requires GH token). |\n| `staging/stg_vuln_observations` | Typed, null-filtered view of `grype_findings`. |\n| `marts/vuln_lifecycle` | First/last seen, days open per (CVE, package, version, app). |\n| `marts/weekly_rollup` | Weekly new-and-open counts by severity. |\n\n## Configuration\n\n| dbt var | Default | Purpose |\n|---------|---------|---------|\n| `grype_data_path` | `data/grype/*.json` | Glob for input Grype artifacts |\n| `gh_url` | `https://api.github.com` | GH API base (for `workflow_runs` model) |\n| `gh_repo` | `HHS/simpler-grants-gov` | Repo slug (for `workflow_runs` model) |\n\nOverride vars on the command line: `dbt run --vars '{grype_data_path: s3://...}'`\n\n## Adding S3 persistence (Phase 2)\n\nWhen the S3 bucket is provisioned:\n1. Run `collect-grype-artifacts.sh` and upload the JSON files to the bucket.\n2. Configure the DuckDB S3 secret in `profiles.yml`.\n3. Override `grype_data_path` with an `s3://.../*.json` glob.\n4. Set up a scheduled Lambda or GitHub Action to run `collect-grype-artifacts.sh`\n   + `dbt run` on a nightly cadence.\n\n## Open questions\n\n- **Closed vs open detection**: `last_seen` is the last scan date the CVE appeared. A\n  vulnerability is \"closed\" only after it stops appearing. This is approximate — if there's\n  a gap in scan history, a temporarily-absent CVE will show a premature `last_seen`.\n  A stricter approach would require a run-level \"expected scans\" list.\n- **Dedup within a run**: if the same CVE matches multiple packages, each gets its own row.\n  This is intentional; the lifecycle model groups by `(cve, package, version, app)`.\n- **EPSS scores**: the original notebook tracked EPSS; Grype doesn't include EPSS in its JSON.\n  Could be joined in from the FIRST EPSS API by CVE ID if needed.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnavapbc%2Ftemp-simpler-grants-vulnerability-dashboard","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnavapbc%2Ftemp-simpler-grants-vulnerability-dashboard","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnavapbc%2Ftemp-simpler-grants-vulnerability-dashboard/lists"}