{"id":21643963,"url":"https://github.com/application-research/estuary-cid-to-bs","last_synced_at":"2026-05-19T02:03:21.587Z","repository":{"id":104226924,"uuid":"574587656","full_name":"application-research/estuary-cid-to-bs","owner":"application-research","description":"Pull Estuary CID from shuttle to local blockstore","archived":false,"fork":false,"pushed_at":"2022-12-05T16:32:00.000Z","size":68,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-03-19T08:49:21.926Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/application-research.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-12-05T16:30:20.000Z","updated_at":"2023-02-15T00:18:19.000Z","dependencies_parsed_at":null,"dependency_job_id":"5b2fd7f5-74a9-465e-a685-8feccbf4c396","html_url":"https://github.com/application-research/estuary-cid-to-bs","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/application-research/estuary-cid-to-bs","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/application-research%2Festuary-cid-to-bs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/application-research%2Festuary-cid-to-bs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/application-research%2Festuary-cid-to-bs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/application-research%2Festuary-cid-to-bs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/application-research","download_url":"https://codeload.github.com/application-research/estuary-cid-to-bs/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/application-research%2Festuary-cid-to-bs/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265402833,"owners_count":23759237,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-25T05:37:09.514Z","updated_at":"2026-05-19T02:03:21.546Z","avatar_url":"https://github.com/application-research.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Estuary CID to Blockstore\n\nA standalone job that queries Estuary CIDs and pulls them to a specified local blockstore.\n\n## Installation\n### Create the DB connection .env file\n\n```\nDB_NAME=\nDB_HOST=\nDB_USER=\nDB_PASS=\nDB_PORT=\n```\n\n### Install run\n```\n// Install\ngo mod tidy\ngo mod download\ngo build -tags netgo -ldflags '-s -w' -o cid-to-bs\n\n// Run\n./cid-to-bs\n```\n\n\n### Output\n\nNumber of workers: 10\nEach worker is assigned with a group of CID(s) to pull\n```\nworker 10 started  job 1 cid {{bafkreic4ykm2yqwayjphykisv2vgugsf4oqxqw25mn6eonmi43zsvt3bui}}\nworker 5 started  job 2 cid {{bafkreib5jsrjvkalimteyrs5fsoi4e6s6r4tdbea33hg27vjdjozusee3q}}\nworker 7 started  job 3 cid {{bafkreigqomn42blxrsf67sd3gxenck46sfm5smo2m7h6cttdf7e225e7nq}}\nworker 6 started  job 4 cid {{bafkreibsbq7rtux5bbzertabqzl56iiv2kxnf6vjy6yfxjfeb4oyi2c2jm}}\nworker 1 started  job 5 cid {{bafybeihn6vn4ppkpck3ffiv2xsdvjzn3kampwizcqqupdf662fuixrc6ny}}\nworker 8 started  job 6 cid {{bafybeibcayare5lx4y5mv4npupzyaodkmmvex3kjzlil72qap7z3m65oge}}\nworker 2 started  job 7 cid {{bafkreifmaz57r7qgtnxxbtyzqjyjh2kb5lgvukge7jczfualoi3hxute4e}}\nworker 3 started  job 8 cid {{bafy2bzaceb4iu6mhxrzvnw6rvlvnpggkr4n7cp5e6fmvo4zd25gnn3uagsw4o}}\nworker 4 started  job 9 cid {{bafy2bzacec6hpnxarhoj2uqa2ssx677pq5ip3k3jm5nrglzkfii4vtvujqkco}}\nworker 9 started  job 10 cid {{bafkreifr2mjxhtj7b4izefndzokmhzj7mi4yvhaiidk6xv5fl46vx2mtby}}\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fapplication-research%2Festuary-cid-to-bs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fapplication-research%2Festuary-cid-to-bs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fapplication-research%2Festuary-cid-to-bs/lists"}