{"id":28985641,"url":"https://github.com/cveproject/cve-ref-archival","last_synced_at":"2026-02-04T05:34:56.666Z","repository":{"id":283167690,"uuid":"950142599","full_name":"CVEProject/cve-ref-archival","owner":"CVEProject","description":null,"archived":false,"fork":false,"pushed_at":"2025-07-11T19:07:50.000Z","size":1459,"stargazers_count":0,"open_issues_count":5,"forks_count":4,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-07-23T06:26:08.617Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/CVEProject.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-03-17T17:39:36.000Z","updated_at":"2025-07-11T19:07:57.000Z","dependencies_parsed_at":null,"dependency_job_id":"cad18e26-5a2e-4bb3-8cba-e06ac7a0d1a6","html_url":"https://github.com/CVEProject/cve-ref-archival","commit_stats":null,"previous_names":["cveproject/cve-ref-archival"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/CVEProject/cve-ref-archival","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CVEProject%2Fcve-ref-archival","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CVEProject%2Fcve-ref-archival/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CVEProject%2Fcve-ref-archival/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CVEProject%2Fcve-ref-archival/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/CVEProject","download_url":"https://codeload.github.com/CVEProject/cve-ref-archival/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CVEProject%2Fcve-ref-archival/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29071155,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-04T03:31:03.593Z","status":"ssl_error","status_checked_at":"2026-02-04T03:29:50.742Z","response_time":62,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-06-24T19:48:52.484Z","updated_at":"2026-02-04T05:34:56.661Z","avatar_url":"https://github.com/CVEProject.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# CVE Program Reference Archiver\n\n## Summary\n\nThis is a pilot program exploring how to archive URL references found in CVE Records.  \n**Note:** This repository is in early development and is subject to change.\n\n## Status: Review Required\n\nWe are transitioning from **Phase 1** to **Phase 2** and would appreciate feedback.\n\n### Open Tasks\n- [ ] Prepare the live environment.\n- [ ] Pilot in the live environment (target: ~2025-07-10).\n- [ ] Refactor and simplify code (currently fragmented across iterations).\n- [ ] Report findings to AWG/QWG.\n\n### Known (Out of Scope)\n1. **Authentication**\n   - This project runs in an isolated environment. API access is only available via SSH to trusted users.\n2. **Process \u0026 Controls**\n   - Archiving is manual and initiated by trusted users. Deletion or modification of archived assets is not currently supported.\n\n---\n\n## Project Overview\n\nThe archiver includes:\n- **Two Node.js services:**\n  1. `scheduler` — Fastify-based HTTP API that queues archive jobs.\n  2. `engine` — Executes archive jobs and manages asset relocation.\n- **Infrastructure components:**\n  1. [`Amazon S3`](link_here) — For storing and delivering archived resources.\n  2. [`PostgreSQL`](link_here) — Tracks jobs, assets, and source domains.\n- **Development stack:**\n  1. [`Visual Studio Code`](https://code.visualstudio.com/) with DevContainers.\n  2. [`Docker Desktop`](https://www.docker.com/products/docker-desktop) for local environments, including:\n     - [`MinIO`](https://min.io/) (S3-compatible object storage).\n     - [`PostgreSQL`](https://www.postgresql.org/) database.\n\n---\n\n## Documentation\n\n- General documentation: [`docs/`](docs/)\n- Rationale and background: [`docs/rationale.md`](docs/rationale.md#background--reference-materials)\n\n---\n\n## Launch Strategy\n\n### Phase 1: Prototype\n- Set up a local, isolated foundation (database, S3).\n- Use ArchiveBox to generate archives and metadata.\n- Upload to a public S3 bucket.\n- Establish basic workflows for submitting CVEs and reviewing operations.\n\n### Phase 2: Pilot\n- Deploy to a shared but isolated environment.\n- Provide access to stakeholders as needed (no public access).\n- Simulate job submissions over time with test plans.\n- Evaluate results and iterate.\n\n### Phase 3: TBD\n_Reserved._\n\n---\n\n## Development\n\n**Contributions welcome!**  \nSee [`docs/overview.md`](docs/overview.md) to get started.\n\nDevelopment is containerized via [DevContainers](https://code.visualstudio.com/docs/devcontainers/containers) to ensure a consistent environment. Recommended setup:\n- VSCode + DevContainer plugin\n- Docker Desktop\n\n---\n\n## Running Locally\n\n1. Clone and open in VSCode.\n2. Open the Command Palette (`Ctrl/Cmd + P`) and run:  \n   `\u003e Dev Containers: Rebuild and Reopen in Container`\n   - This will reopen the project inside the container environment.\n   - Let the `Configuring...` terminal run; it watches and rebuilds on changes.\n3. Use the integrated terminal:\n   - Run both services: `npm run dev`\n   - Run individually: `npm run dev:scheduler` or `npm run dev:engine`\n\n\u003cimg src=\"docs/img/dev-loaded.png\"\u003e\n\n---\n\n## What to Expect\n\nOnce running:\n\n1. **Submit a job:**\n   ```bash\n   curl --location 'http://localhost:8001/api/v1/jobs' \\\n     --header 'Content-Type: application/json' \\\n     --data '{ \"cve\": \"CVE-2025-24070\" }'\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcveproject%2Fcve-ref-archival","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcveproject%2Fcve-ref-archival","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcveproject%2Fcve-ref-archival/lists"}