{"id":20575639,"url":"https://github.com/timweri/download-btc-blockchain","last_synced_at":"2026-04-21T01:01:31.266Z","repository":{"id":115853148,"uuid":"482080203","full_name":"timweri/download-btc-blockchain","owner":"timweri","description":"Download data from blockchain.info API without hitting rate limit","archived":false,"fork":false,"pushed_at":"2022-04-17T06:22:51.000Z","size":6,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-06T10:55:45.743Z","etag":null,"topics":["api","bitcoin","blockchain","core","download","proxy","raw"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/timweri.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-04-15T20:44:52.000Z","updated_at":"2023-06-29T01:50:49.000Z","dependencies_parsed_at":null,"dependency_job_id":"e15d03cf-87f8-4e75-a1b6-5a68cd305b5a","html_url":"https://github.com/timweri/download-btc-blockchain","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/timweri/download-btc-blockchain","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timweri%2Fdownload-btc-blockchain","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timweri%2Fdownload-btc-blockchain/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timweri%2Fdownload-btc-blockchain/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timweri%2Fdownload-btc-blockchain/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/timweri","download_url":"https://codeload.github.com/timweri/download-btc-blockchain/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timweri%2Fdownload-btc-blockchain/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32072323,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-20T21:26:33.338Z","status":"ssl_error","status_checked_at":"2026-04-20T21:26:22.081Z","response_time":94,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api","bitcoin","blockchain","core","download","proxy","raw"],"created_at":"2024-11-16T05:42:36.428Z","updated_at":"2026-04-21T01:01:31.212Z","avatar_url":"https://github.com/timweri.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# download-btc-blockchain\n\nEver need to download parsed transaction data without wanting to run Bitcoin Core? \nThis Python script will help you overcome Blockchain.info API Rate Limit using rotating procies and download the Blockchain data you need.\n\nThis script is written to use rotating proxies to constantly send requests to Blockchain.info API specifically.\nHowever, some work is needed to make this run on other API.\n\nThis script allows you to potentially download years of Bitcoin blockchain data in 4 hours.\n\n## How to run\n\n### Dependencies\n\nMake sure your Python is at least 3.6.\nInstall the dependencies:\n\n```sh\npip install -r requirements.txt\n```\n\n### Run the script\n\nThe script takes in a few command line arguments:\n- `--first-hash [required]`: The hash of the block to start from. The script downloads this block, and then continuously downloades the block before.\n- `--output [required]`: Path to the folder to save the downloaded data.\n- `--month`: If specified, the script will only download blocks from this month.\n\nFor example, run the script like so\n```\npython3 download.py --first-hash 01234 --output data\n```\n\nTo find the desired hash, look up Blockchain.info API.\nThere is a API request where you can pass in the time and the API returns all the block hashes of that day.\n\nThe output will be written in files names `YYYY-MM-DD.out`.\nThe downloaded data will be grouped by date.\nExpect each file to be about 30MB-80MB.\n\n### How to run this faster\n\nEach API request requires very little bandwidth.\nThe bottleneck is mostly the API reponse time and how fast your proxies are.\nI used a premium rotating proxy service and it took, on average, 4s per request (per block).\nIn bad cases, the cost can spike up to 20-30s per block.\nThis makes it a very slow way to download, say, 2 years of blockchain data.\n\nTo speed this up, I recommend running multiple instances of the script in parallel.\nFrom my experience, running multiple instances in parallel scale up the download speed linearly.\nI was able to run 18 instances in parallel and the average cost per block was still 4s.\nSo I was able to speed up my download by 18 times, and there was definitely room for more instances.\nOf course, YMMV depending on your internet and hardware.\n\nTo run multiple instances in parallel without them colliding, I limit each instance to download 1 month.\nThis is what the `--month` option is for.\nOnce the current hash doesn't belong to that target month, the download script terminates.\nIt takes around 4 hours to download a month of data.\n\n#### Example usage\n\nYou want to download blocks in year 2021.\nFirst, look up last block hash of each month in 2021\n\nThen, run the script like so:\n```\npython3 download.py --first-hash \u003clast_hash_of_december\u003e --output data --month 12\npython3 download.py --first-hash \u003clast_hash_of_november\u003e --output data --month 11\npython3 download.py --first-hash \u003clast_hash_of_october\u003e --output data --month 10\n...\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftimweri%2Fdownload-btc-blockchain","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftimweri%2Fdownload-btc-blockchain","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftimweri%2Fdownload-btc-blockchain/lists"}