{"id":37628095,"url":"https://github.com/rishiraj/firerequests","last_synced_at":"2026-01-16T10:52:30.194Z","repository":{"id":257790531,"uuid":"861202119","full_name":"rishiraj/firerequests","owner":"rishiraj","description":"High-performance, asynchronous Python HTTP client library designed for faster file transfers using concurrency, semaphores, and fault-tolerant features.","archived":false,"fork":false,"pushed_at":"2025-05-12T05:43:27.000Z","size":104,"stargazers_count":57,"open_issues_count":0,"forks_count":5,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-09-17T03:55:32.108Z","etag":null,"topics":["aiohttp","asynchronous","asyncio","concurrency","file-transfer","hacktoberfest","http-client"],"latest_commit_sha":null,"homepage":"https://pypi.org/project/firerequests/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rishiraj.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"buy_me_a_coffee":"rishiraj"}},"created_at":"2024-09-22T09:27:39.000Z","updated_at":"2025-09-15T18:27:36.000Z","dependencies_parsed_at":null,"dependency_job_id":"091a0a51-52fc-4067-bc45-ff9b42fe1c63","html_url":"https://github.com/rishiraj/firerequests","commit_stats":null,"previous_names":["rishiraj/firerequests"],"tags_count":11,"template":false,"template_full_name":null,"purl":"pkg:github/rishiraj/firerequests","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rishiraj%2Ffirerequests","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rishiraj%2Ffirerequests/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rishiraj%2Ffirerequests/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rishiraj%2Ffirerequests/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rishiraj","download_url":"https://codeload.github.com/rishiraj/firerequests/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rishiraj%2Ffirerequests/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28478084,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-16T06:30:42.265Z","status":"ssl_error","status_checked_at":"2026-01-16T06:30:16.248Z","response_time":107,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aiohttp","asynchronous","asyncio","concurrency","file-transfer","hacktoberfest","http-client"],"created_at":"2026-01-16T10:52:30.078Z","updated_at":"2026-01-16T10:52:30.172Z","avatar_url":"https://github.com/rishiraj.png","language":"Python","funding_links":["https://buymeacoffee.com/rishiraj","https://www.buymeacoffee.com/rishiraj"],"categories":[],"sub_categories":[],"readme":"# FireRequests 🔥\n\n\u003cp align=\"center\"\u003e\n    \u003ca href=\"https://github.com/rishiraj/firerequests/releases\"\u003e\u003cimg alt=\"GitHub release\" src=\"https://img.shields.io/github/release/rishiraj/firerequests.svg\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://github.com/rishiraj/firerequests\"\u003e\u003cimg alt=\"PyPi version\" src=\"https://img.shields.io/pypi/pyversions/firerequests.svg\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://pepy.tech/project/firerequests\"\u003e\u003cimg src=\"https://static.pepy.tech/badge/firerequests\" alt=\"PyPI Downloads\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://colab.research.google.com/drive/1BIi46kmPQLotG1w9ofTBptlhmnKiKugZ?usp=sharing\"\u003e\u003cimg alt=\"Open In Colab\" src=\"https://colab.research.google.com/assets/colab-badge.svg\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n**FireRequests** is a high-performance, asynchronous HTTP client library for Python, engineered to accelerate your file transfers. By harnessing advanced concepts like semaphores, exponential backoff with jitter, concurrency, and fault tolerance, FireRequests can achieve up to a **10x real-world speedup** in file downloads and uploads compared to traditional synchronous methods and enables scalable, parallelized LLM interactions with providers like OpenAI and Google.\n\n## Features 🚀\n\n- **Asynchronous I/O**: Non-blocking network and file operations using `asyncio`, `aiohttp`, and `aiofiles`, boosting throughput for I/O-bound tasks.\n- **Concurrent Transfers**: Uses `asyncio.Semaphore` to limit simultaneous tasks, optimizing performance by managing system resources effectively.\n- **Fault Tolerance**: Retries failed tasks with exponentially increasing wait times, adding random jitter to prevent network congestion.\n- **Chunked Processing**: Files are split into configurable chunks for parallel processing, significantly accelerating uploads/downloads.\n- **Parallel LLM Calls**: Efficiently handles large-scale language model requests from OpenAI and Google with configurable parallelism.\n- **Compatibility**: Supports environments like Jupyter through `nest_asyncio`, enabling reusable `asyncio` loops for both batch and interactive Jupyter use.\n\n## Installation 📦\n\nInstall FireRequests using pip:\n\n```bash\n!pip install firerequests\n```\n\n## Quick Start 🏁\n\nAccelerate your downloads with just a few lines of code:\n\n### Python Usage\n\n```python\nfrom firerequests import FireRequests\n\nurl = \"https://mirror.clarkson.edu/zorinos/isos/17/Zorin-OS-17.2-Core-64-bit.iso\"\n\nfr = FireRequests()\nfr.download(url)\n```\n\n### Command Line Interface\n\n```bash\n!fr download https://mirror.clarkson.edu/zorinos/isos/17/Zorin-OS-17.2-Core-64-bit.iso\n```\n\n#### Parameters:\n- `urls` (required): The URL to download the file from.\n- `--filenames` (optional): The name to save the downloaded file. Defaults to filename from URL.\n- `--max_files` (optional): The number of concurrent file chunks. Defaults to 10.\n- `--chunk_size` (optional): The size of each chunk in bytes. Defaults to `2 * 1024 * 1024` (2 MB).\n- `--headers` (optional): A dictionary of headers to include in the download request.\n- `--show_progress` (optional): Whether to show a progress bar. Defaults to True for single file downloads, and False for multiple files.\n\n## Real-World Speed Test 🏎️\n\nFireRequests delivers significant performance improvements over traditional download methods. Below is the result of a real-world speed test:\n\n```plaintext\nNormal Download 🐌: 100%|██████████| 3.42G/3.42G [18:24\u003c00:00, 3.10MB/s]\nDownloading on 🔥: 100%|██████████| 3.42G/3.42G [02:38\u003c00:00, 21.6MB/s]\n\n🐌 Download Time: 1104.84 seconds\n🔥 Download Time: 158.22 seconds\n```\n\n\u003e [!TIP]\n\u003e For Hugging Face Hub downloads it is recommended to use `hf_transfer` for maximum speed gains!\n\u003e For more details, please take a look at this [section](https://huggingface.co/docs/huggingface_hub/hf_transfer).\n\n## Advanced Usage ⚙️\n\n### Downloading Files\n\n```python\nfrom firerequests import FireRequests\n\nurls = [\"https://example.com/file1.iso\", \"https://example.com/file2.iso\"]\nfilenames = [\"file1.iso\", \"file2.iso\"]\n\nfr = FireRequests()\nfr.download(urls, filenames, max_files=10, chunk_size=2 * 1024 * 1024, headers={\"Authorization\": \"Bearer token\"}, show_progress=True)\n```\n\n- **`urls`**: The URL or list of URLs of the file(s) to download.\n- **`filenames`**: The filename(s) to save the downloaded file(s). If not provided, filenames are extracted from the URLs.\n- **`max_files`**: The maximum number of concurrent chunk downloads. Defaults to 10.\n- **`chunk_size`**: The size of each chunk in bytes. Defaults to `2 * 1024 * 1024` (2 MB).\n- **`headers`**: A dictionary of headers to include in the download request (optional).\n- **`show_progress`**: Whether to show a progress bar during download. Defaults to `True` for a single file, and `False` for multiple files (optional).\n\n### Uploading Files\n\n```python\nfrom firerequests import FireRequests\n\nfile_path = \"largefile.iso\"\nparts_urls = [\"https://example.com/upload_part1\", \"https://example.com/upload_part2\", ...]\n\nfr = FireRequests()\nfr.upload(file_path, parts_urls, chunk_size=2 * 1024 * 1024, max_files=10, show_progress=True)\n```\n\n- **`file_path`**: The local path to the file to upload.\n- **`parts_urls`**: A list of URLs where each part of the file will be uploaded.\n- **`chunk_size`**: The size of each chunk in bytes. Defaults to `2 * 1024 * 1024` (2 MB).\n- **`max_files`**: The maximum number of concurrent chunk uploads. Defaults to 10.\n- **`show_progress`**: Whether to show a progress bar during upload. Defaults to `True`.\n\n### Comparing Download Speed\n\n```python\nfrom firerequests import FireRequests\n\nurl = \"https://example.com/largefile.iso\"\n\nfr = FireRequests()\nfr.compare(url)\n```\n\n### Generating Text with LLMs\n\nFireRequests allows you to run LLM API calls (like OpenAI or Google) in parallel batches using a decorator. This keeps the library lightweight and lets users supply their own logic for calling APIs. This approach currently doesn't work in Colab.\n\n```python\nfrom firerequests import FireRequests\n\n# Initialize FireRequests\nfr = FireRequests()\n\n# Use the decorator to define your own prompt function\n@fr.op(max_reqs=2, prompts=[\n    \"What is AI?\",\n    \"Explain quantum computing.\",\n    \"What is Bitcoin?\",\n    \"Explain neural networks.\"\n])\ndef generate(system: str = \"Provide concise answers.\", prompt: str = \"\"):\n    # You can use OpenAI, Google, or any other LLM API here\n    from openai import OpenAI\n    import os\n\n    client = OpenAI(api_key=os.environ[\"OPENAI_API_KEY\"])\n    response = client.chat.completions.create(\n        model=\"gpt-4o-mini\",\n        messages=[\n            {\"role\": \"system\", \"content\": system},\n            {\"role\": \"user\", \"content\": prompt}\n        ]\n    )\n    return response.choices[0].message.content\n\n# Call your decorated function\nresponses = generate()\nprint(responses)\n```\n\n## License 📄\n\nThis project is licensed under the Apache License 2.0 - see the [LICENSE](https://github.com/rishiraj/firerequests/blob/main/LICENSE) file for details.\n\nSponsors ❤️\n--------\n\u003ca href=\"https://x.com/soumikRakshit96\" target=\"_blank\"\u003e\u003cimg src=\"https://pbs.twimg.com/profile_images/1791522152954429440/TqGn_kos_400x400.jpg\" height=\"72\"\u003e\u003c/a\u003e\n\nBecome a sponsor and get a logo here. The funds are used to defray the cost of development.\n\n\u003ca href=\"https://www.buymeacoffee.com/rishiraj\"\u003e\u003cimg width=\"200\" alt=\"bmc-button\" src=\"https://github.com/user-attachments/assets/a362b162-c419-4888-bdc9-c33d00d767ad\"\u003e\u003c/a\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frishiraj%2Ffirerequests","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frishiraj%2Ffirerequests","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frishiraj%2Ffirerequests/lists"}