{"id":30839121,"url":"https://github.com/simatwa/throttlebuster","last_synced_at":"2025-09-06T18:55:12.636Z","repository":{"id":309541617,"uuid":"1036594160","full_name":"Simatwa/ThrottleBuster","owner":"Simatwa","description":"Accelerate file downloads by overcoming common throttling restrictions","archived":false,"fork":false,"pushed_at":"2025-09-03T00:28:31.000Z","size":95,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-09-03T02:23:50.690Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Simatwa.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-08-12T09:55:56.000Z","updated_at":"2025-09-03T00:27:03.000Z","dependencies_parsed_at":"2025-08-12T14:22:46.914Z","dependency_job_id":"4adfcade-7385-445d-b935-93201c44b143","html_url":"https://github.com/Simatwa/ThrottleBuster","commit_stats":null,"previous_names":["simatwa/throttlebuster"],"tags_count":11,"template":false,"template_full_name":null,"purl":"pkg:github/Simatwa/ThrottleBuster","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Simatwa%2FThrottleBuster","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Simatwa%2FThrottleBuster/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Simatwa%2FThrottleBuster/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Simatwa%2FThrottleBuster/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Simatwa","download_url":"https://codeload.github.com/Simatwa/ThrottleBuster/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Simatwa%2FThrottleBuster/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":273948884,"owners_count":25196400,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-06T02:00:13.247Z","response_time":2576,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-09-06T18:55:10.991Z","updated_at":"2025-09-06T18:55:12.628Z","avatar_url":"https://github.com/Simatwa.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\n\n\u003ch1\u003eThrottleBuster  \u003cimg alt=\"Project logo\" src=\"https://raw.githubusercontent.com/Simatwa/ThrottleBuster/refs/heads/main/assets/logo.png\" width=\"60px\"/\u003e\u003c/h1\u003e\n\n[![PyPI version](https://badge.fury.io/py/throttlebuster.svg)](https://pypi.org/project/throttlebuster)\n[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/throttlebuster)](https://pypi.org/project/throttlebuster)\n[![PyPI - License](https://img.shields.io/pypi/l/throttlebuster)](https://pypi.org/project/throttlebuster)\n[![Code Coverage](https://raw.githubusercontent.com/Simatwa/ThrottleBuster/refs/heads/main/assets/coverage.svg)]()\n[![Downloads](https://pepy.tech/badge/throttlebuster)](https://pepy.tech/project/throttlebuster)\n[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)\n\u003c/div\u003e\n\nThrottleBuster is a Python library that accelerates file downloads by bypassing common throttling restrictions. It uses asynchronous methods to improve download speeds and is especially useful for handling large files.\n\n## Features\n\n- Concurrent downloading across multiple tasks\n- Fully asynchronous with synchronous support\n- Ready to use commandline tool\n\n\n## Installation\n\n```bash\n$ pip install \"throttlebuster[cli]\"\n```\n\n## Usage\n\nFor testing, you can set up an [Nginx](https://nginx.org) server with the example configuration below:.\n\n```conf\n# Test server\n\n## Create this file in /etc/nginx/sites-enabled or add it to the default nginx.conf\n\nserver {\n    listen 8888;\n    server_name throttlebuster.test;\n\n    location / {\n        limit_rate 500k;  # Limit rate to 500 KB/s\n        root /home/smartwa/y2mate;  # Set your own directory\n        index index.html;\n    }\n}\n\n```\n\n\u003cdetails open\u003e\n\n\u003csummary\u003e\n\n### Developer\n\u003c/summary\u003e\n\n```python\n\nfrom throttlebuster import ThrottleBuster\n\nasync def main():\n    throttlebuster = ThrottleBuster(tasks=4)\n    downloaded_file = await throttlebuster.run(\n        \"http://localhost:8888/test.1.opus\",\n    )\n    print(\n        downloaded_file\n    )\n\nif __name__ == \"__main__\":\n    import asyncio\n\n    asyncio.run(main())\n\n```\n\n#### Custom progress hook\n\n```python\nfrom throttlebuster import DownloadTracker, ThrottleBuster\n\n\nasync def callback_function(data: DownloadTracker):\n    percent = (data.downloaded_size / data.expected_size) * 100\n    print(f\"\u003e Downloading {data.saved_to.name} {percent:.2f}%\", end=\"\\r\")\n\n\nasync def main():\n    throttlebuster = ThrottleBuster(tasks=1)\n    downloaded_file await throttlebuster.run(\n        \"http://localhost:8888/test.1.opus\", progress_hook=callback_function\n    )\n    print(\n        downloaded_file\n    )\n\n\nif __name__ == \"__main__\":\n    import asyncio\n\n    asyncio.run(main())\n\n```\n\n\u003cdetails\u003e\n\n\u003csummary\u003e\n\n#### Synchronous\n\u003c/summary\u003e\n\n```python\nfrom throttlebuster import ThrottleBuster\n\nthrottlebuster = ThrottleBuster()\n\ndownloaed_file = throttlebuster.run_sync(\"http://localhost:8888/test.1.opus\")\n\nprint(\n    downloaded_file\n)\n\n```\n\u003c/details\u003e\n\n\u003c/details\u003e\n\n### Commandline\n\n\u003cdetails\u003e\n\n\u003csummary\u003e\n\n```sh\n$ python -m throttlebuster --help\n```\n\u003c/summary\u003e\n\n```\nUsage: python -m throttlebuster [OPTIONS] COMMAND [ARGS]...\n\n  Accelerate file downloads by overcoming common throttling restrictions\n  envvar-prefix : THROTTLEBUSTER.\n\nOptions:\n  --version  Show the version and exit.\n  --help     Show this message and exit.\n\nCommands:\n  download  Download file using http protocol\n  estimate  Estimate download duration for different tasks\n```\n\n\u003c/details\u003e\n\n#### Download\n\n```sh\n$ python -m throttlebuster download http://localhost:8888/test.1.opus --tasks 14\n```\n\n\u003cdetails\u003e\n\n\u003csummary\u003e\n\n```sh\n$ python -m throttlebuster download --help\n```\n\n\u003c/summary\u003e\n\n```\nUsage: tbust download [OPTIONS] URL\n\n  Download file using http protocol\n\nOptions:\n  -T, --tasks INTEGER RANGE       Number of tasks to carry out the download\n                                  [default: 2; 1\u003c=x\u003c=1000]\n  -Z, --chunk-size INTEGER        Streaming download chunk size in kilobytes\n                                  [default: 256]\n  -D, --dir DIRECTORY             Directory for saving the downloaded file to\n                                  [default:\n                                  /home/smartwa/git/smartwa/throttlebuster]\n  -P, --part-dir DIRECTORY        Directory for temporarily saving the\n                                  downloaded file-parts to  [default:\n                                  /home/smartwa/git/smartwa/throttlebuster]\n  -E, --part-extension TEXT       Filename extension for download parts\n                                  [default: .part]\n  -H, --request-headers TEXT...   Httpx request header - [key value] : default\n  -C, --request-cookies TEXT...   Httpx request cookie - [key value]: default\n  -B, --merge-buffer-size INTEGER RANGE\n                                  Buffer size for merging the separated files\n                                  in kilobytes  [default: 256; 1\u003c=x\u003c=102400]\n  -F, --filename TEXT             Filename for the downloaded content\n  -M, --mode [start|resume|auto]  Whether to start or resume incomplete\n                                  download  [default: auto]\n  -L, --file-size INTEGER         Size of the file to be downloaded\n  -X, --proxy TEXT                Request proxy with schema of any type\n                                  [default: system-set]\n  -R, --timeout-retry-attempts INTEGER\n                                  Number of times to retry download upon read\n                                  request timing out  [default: 10]\n  -K, --colour TEXT               Progress bar display color  [default: cyan]\n  -k, --keep-parts                Whether to retain the separate download\n                                  parts\n  -s, --simple                    Show percentage and bar only in progressbar\n  -t, --test                      Just test if download is possible but do not\n                                  actually download\n  -a, --ascii                     Use unicode (smooth blocks) to fill the\n                                  progress-bar meter\n  -l, --no-leave                  Do not keep traces of the progressbar\n  -z, --disable-progress-bar      Do not show progress_bar\n  -i, --suppress-incompatible-error\n                                  Do no raise error when response headers lack\n                                  Etag\n  --follow-redirects / --no-follow-redirects\n                                  Follow url redirects by the server\n                                  [default: no-follow-redirects]\n  --verify / --no-verify          Verify server certificate  [default: verify]\n  -q, --quiet                     Do not show any interactive information\n  -v, --verbose                   Show more detailed information  [default: 0]\n  --help                          Show this message and exit.\n```\n\n\u003c/details\u003e\n\n\u003e [!TIP]\n\u003e Shortcuts for `$ python -m throttlebuster` are `$ throttlebuster` \u0026 `$ tbust`\n\n#### Estimate\n\n\u003cdetails\u003e\n\n\u003csummary\u003e\n\n```sh\n$ python -m throttlebuster estimate --url http://localhost:8888/miel-martin.webm 260000\n\n```\n\n\u003c/summary\u003e\n\n```\n       337.88 MB at 260.00 KB/s       \n┏━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n┃ Tasks ┃ Duration   ┃ Load per task ┃\n┡━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n│ 20    │ 1.08 Mins  │ 16.89 MB      │\n│ 19    │ 1.14 Mins  │ 17.78 MB      │\n│ 18    │ 1.20 Mins  │ 18.77 MB      │\n│ 17    │ 1.27 Mins  │ 19.88 MB      │\n│ 16    │ 1.35 Mins  │ 21.12 MB      │\n│ 15    │ 1.44 Mins  │ 22.53 MB      │\n│ 14    │ 1.55 Mins  │ 24.13 MB      │\n│ 13    │ 1.67 Mins  │ 25.99 MB      │\n│ 12    │ 1.80 Mins  │ 28.16 MB      │\n│ 11    │ 1.97 Mins  │ 30.72 MB      │\n│ 10    │ 2.17 Mins  │ 33.79 MB      │\n│ 9     │ 2.41 Mins  │ 37.54 MB      │\n│ 8     │ 2.71 Mins  │ 42.24 MB      │\n│ 7     │ 3.09 Mins  │ 48.27 MB      │\n│ 6     │ 3.61 Mins  │ 56.31 MB      │\n│ 5     │ 4.33 Mins  │ 67.58 MB      │\n│ 4     │ 5.41 Mins  │ 84.47 MB      │\n│ 3     │ 7.22 Mins  │ 112.63 MB     │\n│ 2     │ 10.83 Mins │ 168.94 MB     │\n│ 1     │ 21.66 Mins │ 337.88 MB     │\n└───────┴────────────┴───────────────┘\n```\n\n\u003c/details\u003e\n\n\n\u003cdetails\u003e\n\n\u003csummary\u003e\n\n```sh\n$ python -m throttlebuster estimate --help\n```\n\n\u003c/summary\u003e\n\n```\nUsage: python -m throttlebuster estimate [OPTIONS] THROTTLE\n\n  Estimate download duration for different tasks\n\nOptions:\n  -U, --url TEXT             Url to the target file\n  -S, --size INTEGER         Size in bytes of the targeted file\n  -T, --tasks INTEGER RANGE  Tasks amount to base the estimate on : Range\n                             (2-30)  [1\u003c=x\u003c=1000]\n  -j, --json                 Stdout estimates in json format\n  --help                     Show this message and exit.\n```\n\n\u003c/details\u003e\n\nThrottle unit is **bytes**.\n\n## [Motive](https://github.com/Simatwa/moviebox-api/issues/29#issue-3297158834)\n\n\u003cdiv align=\"center\"\u003e\n\n[Back to Top](#throttlebuster)\n\n\u003c/div\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsimatwa%2Fthrottlebuster","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsimatwa%2Fthrottlebuster","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsimatwa%2Fthrottlebuster/lists"}