{"id":13492449,"url":"https://github.com/daijro/hrequests","last_synced_at":"2025-05-14T09:06:25.277Z","repository":{"id":179248992,"uuid":"663169815","full_name":"daijro/hrequests","owner":"daijro","description":"🚀 Web scraping for humans","archived":false,"fork":false,"pushed_at":"2024-12-01T02:55:33.000Z","size":283,"stargazers_count":873,"open_issues_count":41,"forks_count":51,"subscribers_count":14,"default_branch":"main","last_synced_at":"2025-05-14T09:04:42.345Z","etag":null,"topics":["forhumans","gevent","grequests","http","humans","playwright","playwright-python","python","python-requests","requests","scraping","tls"],"latest_commit_sha":null,"homepage":"https://daijro.gitbook.io/hrequests/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/daijro.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-07-06T17:56:09.000Z","updated_at":"2025-05-14T00:26:35.000Z","dependencies_parsed_at":null,"dependency_job_id":"b6df5119-73d2-43a6-a845-d1786e2f5732","html_url":"https://github.com/daijro/hrequests","commit_stats":{"total_commits":59,"total_committers":6,"mean_commits":9.833333333333334,"dds":0.0847457627118644,"last_synced_commit":"6785eac8d44d1b56b9cc13e86dbc77e5bf94013a"},"previous_names":["daijro/hrequests"],"tags_count":22,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/daijro%2Fhrequests","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/daijro%2Fhrequests/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/daijro%2Fhrequests/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/daijro%2Fhrequests/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/daijro","download_url":"https://codeload.github.com/daijro/hrequests/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254110374,"owners_count":22016391,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["forhumans","gevent","grequests","http","humans","playwright","playwright-python","python","python-requests","requests","scraping","tls"],"created_at":"2024-07-31T19:01:06.188Z","updated_at":"2025-05-14T09:06:25.251Z","avatar_url":"https://github.com/daijro.png","language":"Python","funding_links":[],"categories":["Python","scraping"],"sub_categories":[],"readme":"\u003cimg src=\"https://i.imgur.com/r8GcQW1.png\" align=\"center\"\u003e\n\u003c/img\u003e\n\n\u003ch2 align=\"center\"\u003ehrequests\u003c/h2\u003e\n\n\u003ch4 align=\"center\"\u003e\n\u003cp align=\"center\"\u003e\n    \u003ca href=\"https://github.com/daijro/hrequests/blob/main/LICENSE\"\u003e\n        \u003cimg src=\"https://img.shields.io/github/license/daijro/hrequests.svg\"\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://python.org/\"\u003e\n        \u003cimg src=\"https://img.shields.io/badge/python-3.8\u0026#8208;3.13-blue\"\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://pypi.org/project/hrequests/\"\u003e\n        \u003cimg alt=\"PyPI\" src=\"https://img.shields.io/pypi/v/hrequests.svg\"\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://github.com/daijro/hrequests/releases\"\u003e\n        \u003cimg alt=\"downloads\" src=\"https://img.shields.io/github/downloads/daijro/hrequests/total.svg?label=downloads\"\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://pepy.tech/project/hrequests\"\u003e\n        \u003cimg alt=\"PyPI\" src=\"https://img.shields.io/pepy/dt/hrequests?label=pypi installs\u0026color=blue\"\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://github.com/ambv/black\"\u003e\n        \u003cimg src=\"https://img.shields.io/badge/code%20style-black-black.svg\"\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://github.com/PyCQA/isort\"\u003e\n        \u003cimg src=\"https://img.shields.io/badge/imports-isort-yellow.svg\"\u003e\n    \u003c/a\u003e\n\u003c/p\u003e\n    Hrequests (human requests) is a simple, configurable, feature-rich, replacement for the Python requests library. \n\u003c/h4\u003e\n\n### ✨ Features\n\n- Seamless transition between HTTP and headless browsing 💻\n- Integrated fast HTML parser 🚀\n- High performance network concurrency with goroutines \u0026 gevent 🚀\n- Replication of browser TLS fingerprints 🚀\n- JavaScript rendering 🚀\n- Supports HTTP/2 🚀\n- Realistic browser header generation using [BrowserForge](https://github.com/daijro/browserforge) 🚀\n- JSON serializing up to 10x faster than the standard library 🚀\n\n### 💻 Browser crawling\n\n- Simple \u0026 uncomplicated browser automation\n- Anti-detect browsing using [Camoufox](https://camoufox.com) and [Patchright](https://github.com/Kaliiiiiiiiii-Vinyzu/patchright) (**new in v0.9.0!**)\n- Human-like cursor movement and typing\n- Full page screenshots\n- Proxy support\n- Headless and headful support\n- No CORS restrictions\n\n### ⚡ More\n\n- High performance ✨\n- HTTP backend written in Go\n- Automatic gzip \u0026 brotli decode\n- Written with type safety\n- 100% threadsafe ❤️\n\n---\n\n### 🏠 Residential Proxy Rotation ($0.49 per GB)\n\nHrequests includes built-in proxy rotation powered by [Evomi](https://evomi.com?utm_source=github\u0026utm_medium=banner\u0026utm_campaign=daijro-hrequests). 🚀\n\n[Evomi](https://evomi.com?utm_source=github\u0026utm_medium=banner\u0026utm_campaign=daijro-hrequests) is a high quality Swiss proxy provider, with residential proxies avaliable in 150+ countries starting at $0.49/GB. For more information on using Evomi in hrequests, see the [Evomi proxy guide](#evomi-proxies).\n\n- 👩‍💻 **24/7 Expert Support**: Evomi will join your Slack Channel\n- 🌍 **Global Presence**: Available in 150+ Countries\n- ⚡ **Low Latency**\n- 🔒 **Swiss Quality \u0026 Privacy**\n- 🎁 **Free Trial**\n- 🛡️ **99.9% Uptime**\n- 🤝 **Special IP Pool selection**: Optimize for fast, quality, or quantity of IPs\n- 🔧 **Easy Integration**: Compatible with most software and programming languages\n\n[![Evomi Banner](https://my.evomi.com/images/brand/cta.png)](https://evomi.com?utm_source=github\u0026utm_medium=banner\u0026utm_campaign=daijro-hrequests)\n\n---\n\n# Installation\n\nInstall via pip:\n\n```bash\npip install -U hrequests[all]\npython -m hrequests install\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eOr, install without headless browsing support\u003c/i\u003e\u003c/summary\u003e\n\n**Ignore the `[all]` option if you don't want headless browsing support:**\n\n```bash\npip install -U hrequests\n```\n\n\u003c/details\u003e\n\n---\n\n# Documentation\n\n**For the latest stable hrequests documentation, check the [Gitbook page](https://daijro.gitbook.io/hrequests/).**\n\n1. [Simple Usage](#simple-usage)\n2. [Sessions](#sessions)\n3. [Concurrent \u0026 Lazy Requests](#concurrent--lazy-requests)\n4. [HTML Parsing](#html-parsing)\n5. [Browser Automation](#browser-automation)\n6. [Evomi Proxies](#evomi-proxies)\n\n\u003chr width=50\u003e\n\n## Simple Usage\n\nHere is an example of a simple `get` request:\n\n```py\n\u003e\u003e\u003e resp = hrequests.get('https://www.google.com/')\n```\n\nRequests are sent through [bogdanfinn's tls-client](https://github.com/bogdanfinn/tls-client) to spoof the TLS client fingerprint. This is done automatically, and is completely transparent to the user.\n\nOther request methods include `post`, `put`, `delete`, `head`, `options`, and `patch`.\n\nThe `Response` object is a near 1:1 replica of the `requests.Response` object, with some additional attributes.\n\n\u003cdetails\u003e\n\u003csummary\u003eParameters\u003c/summary\u003e\n\n```\nParameters:\n    url (Union[str, Iterable[str]]): URL or list of URLs to request.\n    data (Union[str, bytes, bytearray, dict], optional): Data to send to request. Defaults to None.\n    files (Dict[str, Union[BufferedReader, tuple]], optional): Data to send to request. Defaults to None.\n    headers (dict, optional): Dictionary of HTTP headers to send with the request. Defaults to None.\n    params (dict, optional): Dictionary of URL parameters to append to the URL. Defaults to None.\n    cookies (Union[RequestsCookieJar, dict, list], optional): Dict or CookieJar to send. Defaults to None.\n    json (dict, optional): Json to send in the request body. Defaults to None.\n    allow_redirects (bool, optional): Allow request to redirect. Defaults to True.\n    history (bool, optional): Remember request history. Defaults to False.\n    verify (bool, optional): Verify the server's TLS certificate. Defaults to True.\n    timeout (float, optional): Timeout in seconds. Defaults to 30.\n    proxy (str, optional): Proxy URL. Defaults to None.\n    nohup (bool, optional): Run the request in the background. Defaults to False.\n    \u003cAdditionally includes all parameters from `hrequests.Session` if a session was not specified\u003e\n\nReturns:\n    hrequests.response.Response: Response object\n```\n\n\u003c/details\u003e\n\n### Properties\n\nGet the response url:\n\n```py\n\u003e\u003e\u003e resp.url: str\n'https://www.google.com/'\n```\n\nCheck if the request was successful:\n\n```py\n\u003e\u003e\u003e resp.status_code: int\n200\n\u003e\u003e\u003e resp.reason: str\n'OK'\n\u003e\u003e\u003e resp.ok: bool\nTrue\n\u003e\u003e\u003e bool(resp)\nTrue\n```\n\nGetting the response body:\n\n```py\n\u003e\u003e\u003e resp.text: str\n'\u003c!doctype html\u003e\u003chtml itemscope=\"\" itemtype=\"http://schema.org/WebPage\" lang=\"en\"\u003e\u003chead\u003e\u003cmeta charset=\"UTF-8\"\u003e\u003cmeta content=\"origin\" name=\"referrer\"\u003e\u003cm...'\n\u003e\u003e\u003e resp.content: bytes\nb'\u003c!doctype html\u003e\u003chtml itemscope=\"\" itemtype=\"http://schema.org/WebPage\" lang=\"en\"\u003e\u003chead\u003e\u003cmeta charset=\"UTF-8\"\u003e\u003cmeta content=\"origin\" name=\"referrer\"\u003e\u003cm...'\n\u003e\u003e\u003e resp.encoding: str\n'UTF-8'\n```\n\nParse the response body as JSON:\n\n```py\n\u003e\u003e\u003e resp.json(): Union[dict, list]\n{'somedata': True}\n```\n\nGet the elapsed time of the request:\n\n```py\n\u003e\u003e\u003e resp.elapsed: datetime.timedelta\ndatetime.timedelta(microseconds=77768)\n```\n\nGet the response cookies:\n\n```py\n\u003e\u003e\u003e resp.cookies: RequestsCookieJar\n\u003cRequestsCookieJar[Cookie(version=0, name='1P_JAR', value='2023-07-05-20', port=None, port_specified=False, domain='.google.com', domain_specified=True...\n```\n\nGet the response headers:\n\n```py\n\u003e\u003e\u003e resp.headers: CaseInsensitiveDict\n{'Alt-Svc': 'h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000', 'Cache-Control': 'private, max-age=0', 'Content-Encoding': 'br', 'Content-Length': '51288', 'Content-Security-Policy-Report-Only': \"object-src 'none';base-uri 'se\n```\n\n\u003chr width=50\u003e\n\n## Sessions\n\nCreating a new Firefox Session object:\n\n```py\n\u003e\u003e\u003e session = hrequests.Session()  # version randomized by default\n\u003e\u003e\u003e session = hrequests.Session('firefox', version=129)\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eParameters\u003c/summary\u003e\n\n```\nParameters:\n    browser (Literal['firefox', 'chrome'], optional): Browser to use. Default is 'chrome'.\n    version (int, optional): Version of the browser to use. Browser must be specified. Default is randomized.\n    os (Literal['win', 'mac', 'lin'], optional): OS to use in header. Default is randomized.\n    headers (dict, optional): Dictionary of HTTP headers to send with the request. Default is generated from `browser` and `os`.\n    verify (bool, optional): Verify the server's TLS certificate. Defaults to True.\n    timeout (float, optional): Default timeout in seconds. Defaults to 30.\n    proxy (str, optional): Proxy URL. Defaults to None.\n    cookies (Union[RequestsCookieJar, dict, list], optional): Cookie Jar, or cookie list/dict to send. Defaults to None.\n    certificate_pinning (Dict[str, List[str]], optional): Certificate pinning. Defaults to None.\n    disable_ipv6 (bool, optional): Disable IPv6. Defaults to False.\n    detect_encoding (bool, optional): Detect encoding. Defaults to True.\n    ja3_string (str, optional): JA3 string. Defaults to None.\n    h2_settings (dict, optional): HTTP/2 settings. Defaults to None.\n    additional_decode (str, optional): Decode response body with \"gzip\" or \"br\". Defaults to None.\n    pseudo_header_order (list, optional): Pseudo header order. Defaults to None.\n    priority_frames (list, optional): Priority frames. Defaults to None.\n    header_order (list, optional): Header order. Defaults to None.\n    force_http1 (bool, optional): Force HTTP/1. Defaults to False.\n    catch_panics (bool, optional): Catch panics. Defaults to False.\n    debug (bool, optional): Debug mode. Defaults to False.\n```\n\n\u003c/details\u003e\n\nBrowsers can also be created through the `firefox` and `chrome` shortcuts:\n\n```py\n\u003e\u003e\u003e session = hrequests.firefox.Session()\n\u003e\u003e\u003e session = hrequests.chrome.Session()\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eParameters\u003c/summary\u003e\n\n```\nParameters:\n    version (int, optional): Version of the browser to use. Browser must be specified. Default is randomized.\n    os (Literal['win', 'mac', 'lin'], optional): OS to use in header. Default is randomized.\n    headers (dict, optional): Dictionary of HTTP headers to send with the request. Default is generated from `browser` and `os`.\n    verify (bool, optional): Verify the server's TLS certificate. Defaults to True.\n    timeout (float, optional): Default timeout in seconds. Defaults to 30.\n    proxy (str, optional): Proxy URL. Defaults to None.\n    cookies (Union[RequestsCookieJar, dict, list], optional): Cookie Jar, or cookie list/dict to send. Defaults to None.\n    certificate_pinning (Dict[str, List[str]], optional): Certificate pinning. Defaults to None.\n    disable_ipv6 (bool, optional): Disable IPv6. Defaults to False.\n    detect_encoding (bool, optional): Detect encoding. Defaults to True.\n    ja3_string (str, optional): JA3 string. Defaults to None.\n    h2_settings (dict, optional): HTTP/2 settings. Defaults to None.\n    additional_decode (str, optional): Decode response body with \"gzip\" or \"br\". Defaults to None.\n    pseudo_header_order (list, optional): Pseudo header order. Defaults to None.\n    priority_frames (list, optional): Priority frames. Defaults to None.\n    header_order (list, optional): Header order. Defaults to None.\n    force_http1 (bool, optional): Force HTTP/1. Defaults to False.\n    catch_panics (bool, optional): Catch panics. Defaults to False.\n    debug (bool, optional): Debug mode. Defaults to False.\n```\n\n\u003c/details\u003e\n\n`os` can be `'win'`, `'mac'`, or `'lin'`. Default is randomized.\n\n```py\n\u003e\u003e\u003e session = hrequests.chrome.Session(os='mac')\n```\n\nThis will automatically generate headers based on the browser name and OS:\n\n```py\n\u003e\u003e\u003e session.headers\n{'Accept': '*/*', 'Connection': 'keep-alive', 'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_4; rv:60.2.2) Gecko/20100101 Firefox/60.2.2', 'Accept-Encoding': 'gzip, deflate, br', 'Pragma': 'no-cache'}\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eWhy is the browser version in the header different than the TLS browser version?\u003c/summary\u003e\n\nWebsite bot detection systems typically do not correlate the TLS fingerprint browser version with the browser header.\n\nBy adding more randomization to our headers, we can make our requests appear to be coming from a larger number of clients. We can make it seem like our requests are coming from a larger number of clients. This makes it harder for websites to identify and block our requests based on a consistent browser version.\n\n\u003c/details\u003e\n\n### Properties\n\nHere is a simple get request. This is a wrapper around `hrequests.get`. The only difference is that the session cookies are updated with each request. Creating sessions are recommended for making multiple requests to the same domain.\n\n```py\n\u003e\u003e\u003e resp = session.get('https://www.google.com/')\n```\n\nSession cookies update with each request:\n\n```py\n\u003e\u003e\u003e session.cookies: RequestsCookieJar\n\u003cRequestsCookieJar[Cookie(version=0, name='1P_JAR', value='2023-07-05-20', port=None, port_specified=False, domain='.google.com', domain_specified=True...\n```\n\nRegenerate headers for a different OS:\n\n```py\n\u003e\u003e\u003e session.os = 'win'\n\u003e\u003e\u003e session.headers: CaseInsensitiveDict\n{'Accept': '*/*', 'Connection': 'keep-alive', 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:66.0.3) Gecko/20100101 Firefox/66.0.3', 'Accept-Encoding': 'gzip, deflate, br', 'Accept-Language': 'en-US;q=0.5,en;q=0.3', 'Cache-Control': 'max-age=0', 'DNT': '1', 'Upgrade-Insecure-Requests': '1', 'Pragma': 'no-cache'}\n```\n\n### Closing Sessions\n\nSessions can also be closed to free memory:\n\n```py\n\u003e\u003e\u003e session.close()\n```\n\nAlternatively, sessions can be used as context managers:\n\n```py\nwith hrequests.Session() as session:\n    resp = session.get('https://www.google.com/')\n    print(resp)\n```\n\n\u003chr width=50\u003e\n\n## Concurrent \u0026 Lazy Requests\n\n### Nohup Requests\n\nSimilar to Unix's nohup command, `nohup` requests are sent in the background.\n\nAdding the `nohup=True` keyword argument will return a `LazyTLSRequest` object. This will send the request immediately, but doesn't wait for the response to be ready until an attribute of the response is accessed.\n\n```py\nresp1 = hrequests.get('https://www.google.com/', nohup=True)\nresp2 = hrequests.get('https://www.google.com/', nohup=True)\n```\n\n`resp1` and `resp2` are sent concurrently. They will _never_ pause the current thread, unless an attribute of the response is accessed:\n\n```py\nprint('Resp 1:', resp1.reason)  # will wait for resp1 to finish, if it hasn't already\nprint('Resp 2:', resp2.reason)  # will wait for resp2 to finish, if it hasn't already\n```\n\nThis is useful for sending requests in the background that aren't needed until later.\n\nNote: In `nohup`, a new thread is created for each request. For larger scale concurrency, please consider the following:\n\n### Easy Concurrency\n\nYou can pass an array/iterator of links to the request methods to send them concurrently. This wraps around [`hrequests.map`](#map):\n\n```py\n\u003e\u003e\u003e hrequests.get(['https://google.com/', 'https://github.com/'])\n(\u003cResponse [200]\u003e, \u003cResponse [200]\u003e)\n```\n\nThis also works with `nohup`:\n\n```py\n\u003e\u003e\u003e resps = hrequests.get(['https://google.com/', 'https://github.com/'], nohup=True)\n\u003e\u003e\u003e resps\n(\u003cLazyResponse[Pending]\u003e, \u003cLazyResponse[Pending]\u003e)\n\u003e\u003e\u003e # Sometime later...\n\u003e\u003e\u003e resps\n(\u003cResponse [200]\u003e, \u003cResponse [200]\u003e)\n```\n\n### Grequests-style Concurrency\n\nThe methods `async_get`, `async_post`, etc. will create an unsent request. This levereges gevent, making it _blazing fast_.\n\n\u003cdetails\u003e\n\u003csummary\u003eParameters\u003c/summary\u003e\n\n```\nParameters:\n    url (str): URL to send request to\n    data (Union[str, bytes, bytearray, dict], optional): Data to send to request. Defaults to None.\n    files (Dict[str, Union[BufferedReader, tuple]], optional): Data to send to request. Defaults to None.\n    headers (dict, optional): Dictionary of HTTP headers to send with the request. Defaults to None.\n    params (dict, optional): Dictionary of URL parameters to append to the URL. Defaults to None.\n    cookies (Union[RequestsCookieJar, dict, list], optional): Dict or CookieJar to send. Defaults to None.\n    json (dict, optional): Json to send in the request body. Defaults to None.\n    allow_redirects (bool, optional): Allow request to redirect. Defaults to True.\n    history (bool, optional): Remember request history. Defaults to False.\n    verify (bool, optional): Verify the server's TLS certificate. Defaults to True.\n    timeout (float, optional): Timeout in seconds. Defaults to 30.\n    proxy (str, optional): Proxy URL. Defaults to None.\n    \u003cAdditionally includes all parameters from `hrequests.Session` if a session was not specified\u003e\n\nReturns:\n    hrequests.response.Response: Response object\n```\n\n\u003c/details\u003e\n\nAsync requests are evaluated on `hrequests.map`, `hrequests.imap`, or `hrequests.imap_enum`.\n\nThis functionality is similar to [grequests](https://github.com/spyoungtech/grequests). Unlike grequests, [monkey patching](https://www.gevent.org/api/gevent.monkey.html) is not required because this does not rely on the standard python SSL library.\n\nCreate a set of unsent Requests:\n\n```py\n\u003e\u003e\u003e reqs = [\n...     hrequests.async_get('https://www.google.com/', browser='firefox'),\n...     hrequests.async_get('https://www.duckduckgo.com/'),\n...     hrequests.async_get('https://www.yahoo.com/')\n... ]\n```\n\n#### map\n\nSend them all at the same time using map:\n\n```py\n\u003e\u003e\u003e hrequests.map(reqs, size=3)\n[\u003cResponse [200]\u003e, \u003cResponse [200]\u003e, \u003cResponse [200]\u003e]\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eParameters\u003c/summary\u003e\n\n```\nConcurrently converts a list of Requests to Responses.\nParameters:\n    requests - a collection of Request objects.\n    size - Specifies the number of requests to make at a time. If None, no throttling occurs.\n    exception_handler - Callback function, called when exception occurred. Params: Request, Exception\n    timeout - Gevent joinall timeout in seconds. (Note: unrelated to requests timeout)\n\nReturns:\n    A list of Response objects.\n```\n\n\u003c/details\u003e\n\n#### imap\n\n`imap` returns a generator that yields responses as they come in:\n\n```py\n\u003e\u003e\u003e for resp in hrequests.imap(reqs, size=3):\n...    print(resp)\n\u003cResponse [200]\u003e\n\u003cResponse [200]\u003e\n\u003cResponse [200]\u003e\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eParameters\u003c/summary\u003e\n\n```\nConcurrently converts a generator object of Requests to a generator of Responses.\n\nParameters:\n    requests - a generator or sequence of Request objects.\n    size - Specifies the number of requests to make at a time. default is 2\n    exception_handler - Callback function, called when exception occurred. Params: Request, Exception\n\nYields:\n    Response objects.\n```\n\n\u003c/details\u003e\n\n`imap_enum` returns a generator that yields a tuple of `(index, response)` as they come in. The `index` is the index of the request in the original list:\n\n```py\n\u003e\u003e\u003e for index, resp in hrequests.imap_enum(reqs, size=3):\n...     print(index, resp)\n(1, \u003cResponse [200]\u003e)\n(0, \u003cResponse [200]\u003e)\n(2, \u003cResponse [200]\u003e)\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eParameters\u003c/summary\u003e\n\n```\nLike imap, but yields tuple of original request index and response object\nUnlike imap, failed results and responses from exception handlers that return None are not ignored. Instead, a\ntuple of (index, None) is yielded.\nResponses are still in arbitrary order.\n\nParameters:\n    requests - a sequence of Request objects.\n    size - Specifies the number of requests to make at a time. default is 2\n    exception_handler - Callback function, called when exception occurred. Params: Request, Exception\n\nYields:\n    (index, Response) tuples.\n```\n\n\u003c/details\u003e\n\n#### Exception Handling\n\nTo handle timeouts or any other exception during the connection of the request, you can add an optional exception handler that will be called with the request and exception inside the main thread.\n\n```py\n\u003e\u003e\u003e def exception_handler(request, exception):\n...    return f'Response failed: {exception}'\n\n\u003e\u003e\u003e bad_reqs = [\n...     hrequests.async_get('http://httpbin.org/delay/5', timeout=1),\n...     hrequests.async_get('http://fakedomain/'),\n...     hrequests.async_get('http://example.com/'),\n... ]\n\u003e\u003e\u003e hrequests.map(bad_reqs, size=3, exception_handler=exception_handler)\n['Response failed: Connection error', 'Response failed: Connection error', \u003cResponse [200]\u003e]\n```\n\nThe value returned by the exception handler will be used in place of the response in the result list.\n\nIf an exception handler isn't specified, the default yield type is `hrequests.FailedResponse`.\n\n\u003chr width=50\u003e\n\n## HTML Parsing\n\nHTML scraping is based off [selectolax](https://github.com/rushter/selectolax), which is **over 25x faster** than bs4. This functionality is inspired by [requests-html](https://github.com/psf/requests-html).\n\n| Library        | Time (1e5 trials) |\n| -------------- | ----------------- |\n| BeautifulSoup4 | 52.6              |\n| PyQuery        | 7.5               |\n| selectolax     | **1.9**           |\n\nThe HTML parser can be accessed through the `html` attribute of the response object:\n\n```py\n\u003e\u003e\u003e resp = session.get('https://python.org/')\n\u003e\u003e\u003e resp.html\n\u003cHTML url='https://www.python.org/'\u003e\n```\n\n### Parsing page\n\nGrab a list of all links on the page, as-is (anchors excluded):\n\n```py\n\u003e\u003e\u003e resp.html.links\n{'//docs.python.org/3/tutorial/', '/about/apps/', 'https://github.com/python/pythondotorg/issues', '/accounts/login/', '/dev/peps/', '/about/legal/',...\n```\n\nGrab a list of all links on the page, in absolute form (anchors excluded):\n\n```py\n\u003e\u003e\u003e resp.html.absolute_links\n{'https://github.com/python/pythondotorg/issues', 'https://docs.python.org/3/tutorial/', 'https://www.python.org/about/success/', 'http://feedproxy.g...\n```\n\nSearch for text on the page:\n\n```py\n\u003e\u003e\u003e resp.html.search('Python is a {} language')[0]\nprogramming\n```\n\n### Selecting elements\n\nSelect an element using a CSS Selector:\n\n```py\n\u003e\u003e\u003e about = resp.html.find('#about')\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eParameters\u003c/summary\u003e\n\n```\nGiven a CSS Selector, returns a list of\n:class:`Element \u003cElement\u003e` objects or a single one.\n\nParameters:\n    selector: CSS Selector to use.\n    clean: Whether or not to sanitize the found HTML of ``\u003cscript\u003e`` and ``\u003cstyle\u003e``\n    containing: If specified, only return elements that contain the provided text.\n    first: Whether or not to return just the first result.\n    raise_exception: Raise an exception if no elements are found. Default is True.\n    _encoding: The encoding format.\n\nReturns:\n    A list of :class:`Element \u003cElement\u003e` objects or a single one.\n\nExample CSS Selectors:\n- ``a``\n- ``a.someClass``\n- ``a#someID``\n- ``a[target=_blank]``\nSee W3School's `CSS Selectors Reference\n\u003chttps://www.w3schools.com/cssref/css_selectors.asp\u003e`_\nfor more details.\nIf ``first`` is ``True``, only returns the first\n:class:`Element \u003cElement\u003e` found.\n```\n\n\u003c/details\u003e\n\n### Introspecting elements\n\nGrab an Element's text contents:\n\n```py\n\u003e\u003e\u003e print(about.text)\nAbout\nApplications\nQuotes\nGetting Started\nHelp\nPython Brochure\n```\n\nGetting an Element's attributes:\n\n```py\n\u003e\u003e\u003e about.attrs\n{'id': 'about', 'class': ('tier-1', 'element-1'), 'aria-haspopup': 'true'}\n\u003e\u003e\u003e about.id\n'about'\n```\n\nGet an Element's raw HTML:\n\n```py\n\u003e\u003e\u003e about.html\n'\u003cli aria-haspopup=\"true\" class=\"tier-1 element-1 \" id=\"about\"\u003e\\n\u003ca class=\"\" href=\"/about/\" title=\"\"\u003eAbout\u003c/a\u003e\\n\u003cul aria-hidden=\"true\" class=\"subnav menu\" role=\"menu\"\u003e\\n\u003cli class=\"tier-2 element-1\" role=\"treeitem\"\u003e\u003ca href=\"/about/apps/\" title=\"\"\u003eApplications\u003c/a\u003e\u003c/li\u003e\\n\u003cli class=\"tier-2 element-2\" role=\"treeitem\"\u003e\u003ca href=\"/about/quotes/\" title=\"\"\u003eQuotes\u003c/a\u003e\u003c/li\u003e\\n\u003cli class=\"tier-2 element-3\" role=\"treeitem\"\u003e\u003ca href=\"/about/gettingstarted/\" title=\"\"\u003eGetting Started\u003c/a\u003e\u003c/li\u003e\\n\u003cli class=\"tier-2 element-4\" role=\"treeitem\"\u003e\u003ca href=\"/about/help/\" title=\"\"\u003eHelp\u003c/a\u003e\u003c/li\u003e\\n\u003cli class=\"tier-2 element-5\" role=\"treeitem\"\u003e\u003ca href=\"http://brochure.getpython.info/\" title=\"\"\u003ePython Brochure\u003c/a\u003e\u003c/li\u003e\\n\u003c/ul\u003e\\n\u003c/li\u003e'\n```\n\nSelect Elements within Elements:\n\n```py\n\u003e\u003e\u003e about.find_all('a')\n[\u003cElement 'a' href='/about/' title='' class=''\u003e, \u003cElement 'a' href='/about/apps/' title=''\u003e, \u003cElement 'a' href='/about/quotes/' title=''\u003e, \u003cElement 'a' href='/about/gettingstarted/' title=''\u003e, \u003cElement 'a' href='/about/help/' title=''\u003e, \u003cElement 'a' href='http://brochure.getpython.info/' title=''\u003e]\n\u003e\u003e\u003e about.find('a')\n\u003cElement 'a' href='/about/' title='' class=''\u003e\n```\n\nSearching by HTML attributes:\n\n```py\n\u003e\u003e\u003e about.find('il', role='treeitem')\n\u003cElement 'li' role='treeitem' class=('tier-2', 'element-1')\u003e\n```\n\nSearch for links within an element:\n\n```py\n\u003e\u003e\u003e about.absolute_links\n{'http://brochure.getpython.info/', 'https://www.python.org/about/gettingstarted/', 'https://www.python.org/about/', 'https://www.python.org/about/quotes/', 'https://www.python.org/about/help/', 'https://www.python.org/about/apps/'}\n```\n\n\u003chr width=50\u003e\n\n## Browser Automation\n\nHrequests supports both Firefox and Chrome browsers, headless and headful sessions:\n\n\u003e [!WARNING]\n\u003e It is recommended to use Firefox instead. Chrome does not support fingerprint rotation, mocking human mouse movements, or browser extensions.\n\n### Usage\n\nYou can spawn a `BrowserSession` instance by calling it:\n\n```py\n\u003e\u003e\u003e page = hrequests.BrowserSession()  # headless=True by default\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eParameters\u003c/summary\u003e\n\n```\nParameters:\n    session (hrequests.session.TLSSession, optional): Session to use for headers, cookies, etc.\n    resp (hrequests.response.Response, optional): Response to update with cookies, headers, etc.\n    proxy (Union[str, BaseProxy], optional): Proxy to use for the browser. Example: http://1.2.3.4:8080\n    mock_human (bool, optional): Whether to emulate human behavior. Defaults to False.\n    engine (BrowserEngine, optional): Pass in an existing BrowserEngine instead of creating a new one\n    verify (bool, optional): Whether to verify https requests\n    headless (bool, optional): Whether to run the browser in headless mode. Defaults to True.\n    os (Literal['win', 'mac', 'lin'], optional): Generate headers for a specific OS\n    **kwargs: Additional arguments to pass to Playwright (or Camoufox parameters if using Firefox)\n```\n\n\u003c/details\u003e\n\n`BrowserSession` is entirely safe to use across threads.\n\n#### Camoufox Integration\n\nIf you are using a Firefox BrowserSession, you can pass additional parameters to Camoufox by using the `**kwargs` parameter:\n\n```py\n\u003e\u003e\u003e page = hrequests.BrowserSession(window=(1024, 768), block_images=True, addons=['/path/to/addon'], ...)\n```\n\nYou can find a full list of parameters for Camoufox [here](https://camoufox.com/python/usage).\n\n#### Engine\n\nThe `engine` parameter allows you to pass in an existing `BrowserEngine` instance. This can be useful if you want to reuse a Playwright engine to save time on startup. It is completely threadsafe.\n\n```python\n\u003e\u003e\u003e engine = hrequests.BrowserEngine()\n```\n\nUse the same engine for multiple sessions\n\n```python\n\u003e\u003e\u003e page1 = hrequests.BrowserSession(engine=engine)\n\u003e\u003e\u003e page2 = hrequests.BrowserSession(engine=engine)\n```\n\n### Render an existing Response\n\nResponses have a `.render()` method. This will render the contents of the response in a browser page.\n\nOnce the page is closed, the Response content and the Response's session cookies will be updated.\n\n#### Simple usage\n\nRendered browser sessions will use the browser set in the initial request.\n\nYou can set a request's browser with the `browser` parameter in the `hrequests.get` method:\n\n```py\n\u003e\u003e\u003e resp = hrequests.get('https://example.com')\n```\n\nOr by setting the `browser` parameter of the `hrequests.Session` object:\n\n```py\n\u003e\u003e\u003e session = hrequests.Session()\n\u003e\u003e\u003e resp = session.get('https://example.com')\n```\n\n**Example - submitting a login form:**\n\n```py\n\u003e\u003e\u003e session = hrequests.Session()\n\u003e\u003e\u003e resp = session.get('https://www.somewebsite.com/')\n\u003e\u003e\u003e with resp.render(mock_human=True) as page:\n...     page.type('.input#username', 'myuser')\n...     page.type('.input#password', 'p4ssw0rd')\n...     page.click('#submit')\n# `session` \u0026 `resp` now have updated cookies, content, etc.\n```\n\n\u003csummary\u003e\u003cstrong\u003eOr, without a context manager\u003c/strong\u003e\u003c/summary\u003e\n\n```py\n\u003e\u003e\u003e session = hrequests.Session()\n\u003e\u003e\u003e resp = session.get('https://www.somewebsite.com/')\n\u003e\u003e\u003e page = resp.render(mock_human=True)\n\u003e\u003e\u003e page.type('.input#username', 'myuser')\n\u003e\u003e\u003e page.type('.input#password', 'p4ssw0rd')\n\u003e\u003e\u003e page.click('#submit')\n\u003e\u003e\u003e page.close()  # must close the page when done!\n```\n\n\u003c/details\u003e\n\nThe `mock_human` parameter will emulate human-like behavior. This includes easing and randomizing mouse movements, and randomizing typing speed. This functionality is based on [Botright](https://github.com/Vinyzu/botright/).\n\n\u003cdetails\u003e\n\u003csummary\u003eParameters\u003c/summary\u003e\n\n```\nParameters:\n    headless (bool, optional): Whether to run the browser in headless mode. Defaults to False.\n    mock_human (bool, optional): Whether to emulate human behavior. Defaults to False.\n    extensions (Union[str, Iterable[str]], optional): Path to a folder of unpacked extensions, or a list of paths to unpacked extensions\n    engine (BrowserEngine, optional): Pass in an existing BrowserEngine instead of creating a new one\n    **kwargs: Additional arguments to pass to Camoufox (see https://camoufox.com/python/usage)\n```\n\n\u003c/details\u003e\n\n### Properties\n\nCookies are inherited from the session:\n\n```py\n\u003e\u003e\u003e page.cookies: RequestsCookieJar  # cookies are inherited from the session\n\u003cRequestsCookieJar[Cookie(version=0, name='1P_JAR', value='2023-07-05-20', port=None, port_specified=False, domain='.somewebsite.com', domain_specified=True...\n```\n\n### Pulling page data\n\nGet current page url:\n\n```py\n\u003e\u003e\u003e page.url: str\nhttps://www.somewebsite.com/\n```\n\nGet page content:\n\n```py\n\u003e\u003e\u003e page.text: str\n'\u003c!doctype html\u003e\u003chtml itemscope=\"\" itemtype=\"http://schema.org/WebPage\" lang=\"en\"\u003e\u003chead\u003e\u003cmeta content=\"Search the world\\'s information, including webpag'\n\u003e\u003e\u003e page.content: bytes\nb'\u003c!doctype html\u003e\u003chtml itemscope=\"\" itemtype=\"http://schema.org/WebPage\" lang=\"en\"\u003e\u003chead\u003e\u003cmeta content=\"Search the world\\'s information, including webpag'\n```\n\nGet the status of the last navigation:\n\n```py\n\u003e\u003e\u003e page.status_code: int\n200\n\u003e\u003e\u003e page.reason: str\n'OK'\n```\n\nParsing HTML from the page content:\n\n```py\n\u003e\u003e\u003e page.html.find_all('a')\n[\u003cElement 'a' href='/about/' title='' class=''\u003e, \u003cElement 'a' href='/about/apps/' title=''\u003e, ...]\n\u003e\u003e\u003e page.html.find('a')\n\u003cElement 'a' href='/about/' title='' class=''\u003e, \u003cElement 'a' href='/about/apps/' title=''\u003e\n```\n\nTake a screenshot of the page:\n\n```py\n\u003e\u003e\u003e page.screenshot(path='screenshot.png')\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eParameters\u003c/summary\u003e\n\n```\nTake a screenshot of the page\n\nParameters:\n    selector (str, optional): CSS selector to screenshot\n    path (str, optional): Path to save screenshot to. Defaults to None.\n    full_page (bool): Whether to take a screenshot of the full scrollable page. Cannot be used with selector. Defaults to False.\n\nReturns:\n    Optional[bytes]: Returns the screenshot buffer, if `path` was not provided\n```\n\n\u003c/details\u003e\n\n### Navigate the browser\n\nNavigate to a url:\n\n```py\n\u003e\u003e\u003e page.url = 'https://bing.com'\n# or use goto\n\u003e\u003e\u003e page.goto('https://bing.com')\n```\n\nNavigate through page history:\n\n```py\n\u003e\u003e\u003e page.back()\n\u003e\u003e\u003e page.forward()\n```\n\n### Controlling elements\n\nClick an element:\n\n```py\n\u003e\u003e\u003e page.click('#my-button')\n# or through the html parser\n\u003e\u003e\u003e page.html.find('#my-button').click()\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eParameters\u003c/summary\u003e\n\n```\nParameters:\n    selector (str): CSS selector to click.\n    button (Literal['left', 'right', 'middle'], optional): Mouse button to click. Defaults to 'left'.\n    count (int, optional): Number of clicks. Defaults to 1.\n    timeout (float, optional): Timeout in seconds. Defaults to 30.\n    wait_after (bool, optional): Wait for a page event before continuing. Defaults to True.\n```\n\n\u003c/details\u003e\n\nHover over an element:\n\n```py\n\u003e\u003e\u003e page.hover('.dropbtn')\n# or through the html parser\n\u003e\u003e\u003e page.html.find('.dropbtn').hover()\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eParameters\u003c/summary\u003e\n\n```\nParameters:\n    selector (str): CSS selector to hover over\n    modifiers (List[Literal['Alt', 'Control', 'Meta', 'Shift']], optional): Modifier keys to press. Defaults to None.\n    timeout (float, optional): Timeout in seconds. Defaults to 90.\n```\n\n\u003c/details\u003e\n\nType text into an element:\n\n```py\n\u003e\u003e\u003e page.type('#my-input', 'Hello world!')\n# or through the html parser\n\u003e\u003e\u003e page.html.find('#my-input').type('Hello world!')\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eParameters\u003c/summary\u003e\n\n```\nParameters:\n    selector (str): CSS selector to type in\n    text (str): Text to type\n    delay (int, optional): Delay between keypresses in ms. On mock_human, this is randomized by 50%. Defaults to 50.\n    timeout (float, optional): Timeout in seconds. Defaults to 30.\n```\n\n\u003c/details\u003e\n\nDrag and drop an element:\n\n```py\n\u003e\u003e\u003e page.dragTo('#source-selector', '#target-selector')\n# or through the html parser\n\u003e\u003e\u003e page.html.find('#source-selector').dragTo('#target-selector')\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eParameters\u003c/summary\u003e\n\n```\nParameters:\n    source (str): Source to drag from\n    target (str): Target to drop to\n    timeout (float, optional): Timeout in seconds. Defaults to 30.\n    wait_after (bool, optional): Wait for a page event before continuing. Defaults to False.\n    check (bool, optional): Check if an element is draggable before running. Defaults to False.\n\nThrows:\n    hrequests.exceptions.BrowserTimeoutException: If timeout is reached\n```\n\n\u003c/details\u003e\n\n### Check page elements\n\nCheck if a selector is visible and enabled:\n\n```py\n\u003e\u003e\u003e page.isVisible('#my-selector'): bool\n\u003e\u003e\u003e page.isEnabled('#my-selector'): bool\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eParameters\u003c/summary\u003e\n\n```\nParameters:\n    selector (str): Selector to check\n```\n\n\u003c/details\u003e\n\nEvaluate and return a script:\n\n```py\n\u003e\u003e\u003e page.evaluate('selector =\u003e document.querySelector(selector).checked', '#my-selector')\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eParameters\u003c/summary\u003e\n\n```\nParameters:\n    script (str): Javascript to evaluate in the page\n    arg (str, optional): Argument to pass into the javascript function\n```\n\n\u003c/details\u003e\n\n### Awaiting events\n\n```py\n\u003e\u003e\u003e page.awaitNavigation()\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eParameters\u003c/summary\u003e\n\n```\nParameters:\n    timeout (float, optional): Timeout in seconds. Defaults to 30.\n\nThrows:\n    hrequests.exceptions.BrowserTimeoutException: If timeout is reached\n```\n\n\u003c/details\u003e\n\nWait for a script or function to return a truthy value:\n\n```py\n\u003e\u003e\u003e page.awaitScript('selector =\u003e document.querySelector(selector).value === 100', '#progress')\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eParameters\u003c/summary\u003e\n\n```\nParameters:\n    script (str): Script to evaluate\n    arg (str, optional): Argument to pass to script\n    timeout (float, optional): Timeout in seconds. Defaults to 30.\n\nThrows:\n    hrequests.exceptions.BrowserTimeoutException: If timeout is reached\n```\n\n\u003c/details\u003e\n\nWait for the URL to match:\n\n```py\n\u003e\u003e\u003e page.awaitUrl(re.compile(r'https?://www\\.google\\.com/.*'), timeout=10)\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eParameters\u003c/summary\u003e\n\n```\nParameters:\n    url (Union[str, Pattern[str], Callable[[str], bool]]) - URL to match for\n    timeout (float, optional): Timeout in seconds. Defaults to 30.\n\nThrows:\n    hrequests.exceptions.BrowserTimeoutException: If timeout is reached\n```\n\n\u003c/details\u003e\n\nWait for an element to exist on the page:\n\n```py\n\u003e\u003e\u003e page.awaitSelector('#my-selector')\n# or through the html parser\n\u003e\u003e\u003e page.html.find('#my-selector').awaitSelector()\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eParameters\u003c/summary\u003e\n\n```\nParameters:\n    selector (str): Selector to wait for\n    timeout (float, optional): Timeout in seconds. Defaults to 30.\n\nThrows:\n    hrequests.exceptions.BrowserTimeoutException: If timeout is reached\n```\n\n\u003c/details\u003e\n\nWait for an element to be enabled:\n\n```py\n\u003e\u003e\u003e page.awaitEnabled('#my-selector')\n# or through the html parser\n\u003e\u003e\u003e page.html.find('#my-selector').awaitEnabled()\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eParameters\u003c/summary\u003e\n\n```\nParameters:\n    selector (str): Selector to wait for\n    timeout (float, optional): Timeout in seconds. Defaults to 30.\n\nThrows:\n    hrequests.exceptions.BrowserTimeoutException: If timeout is reached\n```\n\n\u003c/details\u003e\n\nScreenshot an element:\n\n```py\n\u003e\u003e\u003e page.screenshot('#my-selector', path='screenshot.png')\n# or through the html parser\n\u003e\u003e\u003e page.html.find('#my-selector').screenshot('selector.png')\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eParameters\u003c/summary\u003e\n\n```\nScreenshot an element\n\nParameters:\n    selector (str, optional): CSS selector to screenshot\n    path (str, optional): Path to save screenshot to. Defaults to None.\n    full_page (bool): Whether to take a screenshot of the full scrollable page. Cannot be used with selector. Defaults to False.\n\nReturns:\n    Optional[bytes]: Returns the screenshot buffer, if `path` was not provided\n```\n\n\u003c/details\u003e\n\n### Adding Firefox extensions\n\nFirefox extensions can be easily imported into a browser session. Some potentially useful extensions include:\n\n- **uBlock Origin** - Ad \u0026 popup blocker (Automatically installed)\n\n- **hektCaptcha** - Hcaptcha solver ([Download](https://github.com/Wikidepia/hektCaptcha-extension))\n\n- **FastForward** - Bypass \u0026 skip link redirects ([Download](https://nightly.link/FastForwardTeam/FastForward/workflows/main/main/FastForward_firefox.zip))\n\n**Note:** Hrequests only supports Firefox extensions.\n\nExtensions are added with the `extensions` parameter:\n\n- This can be an list of absolute paths to unpacked extensions:\n\n  ```py\n  with resp.render(extensions=['C:\\\\extensions\\\\hektcaptcha', 'C:\\\\extensions\\\\fastforward']):\n  ```\n\nHere is an usage example of using a captcha solver:\n\n```py\n\u003e\u003e\u003e resp = hrequests.get('https://accounts.hcaptcha.com/demo', browser='firefox')\n\u003e\u003e\u003e with resp.render(extensions=['C:\\\\extensions\\\\hektcaptcha']) as page:\n...     page.awaitSelector('.hcaptcha-success')  # wait for captcha to finish\n...     page.click('input[type=submit]')\n```\n\n### Requests \u0026 Responses\n\nRequests can also be sent within browser sessions. These operate the same as the standard `hrequests.request`, and will use the browser's cookies and headers. The `BrowserSession` cookies will be updated with each request.\n\nThis returns a normal `Response` object:\n\n```py\n\u003e\u003e\u003e resp = page.get('https://duckduckgo.com')\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eParameters\u003c/summary\u003e\n\n```\nParameters:\n    url (str): URL to send request to\n    params (dict, optional): Dictionary of URL parameters to append to the URL. Defaults to None.\n    data (Union[str, dict], optional): Data to send to request. Defaults to None.\n    headers (dict, optional): Dictionary of HTTP headers to send with the request. Defaults to None.\n    form (dict, optional): Form data to send with the request. Defaults to None.\n    multipart (dict, optional): Multipart data to send with the request. Defaults to None.\n    timeout (float, optional): Timeout in seconds. Defaults to 30.\n    verify (bool, optional): Verify the server's TLS certificate. Defaults to True.\n    max_redirects (int, optional): Maximum number of redirects to follow. Defaults to None.\n\nThrows:\n    hrequests.exceptions.BrowserTimeoutException: If timeout is reached\n\nReturns:\n    hrequests.response.Response: Response object\n```\n\n\u003c/details\u003e\n\nOther methods include `post`, `put`, `delete`, `head`, and `patch`.\n\n### Closing the page\n\nThe `BrowserSession` object must be closed when finished. This will close the browser, update the response data, and merge new cookies with the session cookies.\n\n```py\n\u003e\u003e\u003e page.close()\n```\n\nNote that this is automatically done when using a context manager.\n\nSession cookies are updated:\n\n```py\n\u003e\u003e\u003e session.cookies: RequestsCookieJar\n\u003cRequestsCookieJar[Cookie(version=0, name='MUID', value='123456789', port=None, port_specified=False, domain='.bing.com', domain_specified=True, domain_initial_dot=True...\n```\n\nResponse data is updated:\n\n```py\n\u003e\u003e\u003e resp.url: str\n'https://www.bing.com/?toWww=1\u0026redig=823778234657823652376438'\n\u003e\u003e\u003e resp.content: Union[bytes, str]\n'\u003c!DOCTYPE html\u003e\u003chtml lang=\"en\" dir=\"ltr\"\u003e\u003chead\u003e\u003cmeta name=\"theme-color\" content=\"#4F4F4F\"\u003e\u003cmeta name=\"description\" content=\"Bing helps you turn inform...\n```\n\n#### Other ways to create a Browser Session\n\nYou can use `.render` to spawn a `BrowserSession` object directly from a url:\n\n```py\n# Using a Session:\n\u003e\u003e\u003e page = session.render('https://google.com')\n# Or without a session at all:\n\u003e\u003e\u003e page = hrequests.render('https://google.com')\n```\n\nMake sure to close all `BrowserSession` objects when done!\n\n```py\n\u003e\u003e\u003e page.close()\n```\n\n\u003chr width=50\u003e\n\n## Evomi Proxies\n\nHrequests has a built in residential proxy rotation service powered by [Evomi](https://evomi.com/).\n\n### Creating a proxy\n\nImport the `evomi` module:\n\n```py\n\u003e\u003e\u003e from hrequests.proxies import evomi\n\u003e\u003e\u003e proxy = evomi.ResidentialProxy(username='daijro', key='password')\n```\n\n### Usage\n\nPass proxies into requests:\n\n```\n\u003e\u003e\u003e resp = hrequests.get('https://example.com', proxy=proxy)\n```\n\nUse Evomi proxies with a `Session`:\n\n```python\n# Add the proxy to the session\n\u003e\u003e\u003e session = hrequests.Session(proxy=proxy)\n# All requests made with this session will use the proxy.\n\u003e\u003e\u003e resp = session.get('https://example.com')\n\u003e\u003e\u003e with resp.render() as page:\n...     # Page is rendered with the proxy.\n...     ...\n```\n\nUse Evomi proxies with a `BrowserSession`:\n\n```python\n\u003e\u003e\u003e page = hrequests.BrowserSession(proxy=proxy)\n\u003e\u003e\u003e page.goto('https://example.com')\n```\n\n### Proxy Types\n\nYou can create either a residential, mobile, or datacenter proxy:\n\n#### Residential\n\n```py\n\u003e\u003e\u003e proxy = evomi.ResidentialProxy(username='daijro', key='password')\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eResidentialProxy Parameters\u003c/summary\u003e\n\n```\nInitialize a new Evomi Residential proxy.\n\nParameters:\n    username (str): Your Evomi username\n    key (str): Your Evomi API key\n    country (str, optional): Target country code (e.g., 'US', 'GB')\n    region (str, optional): Target region/state\n    city (str, optional): Target city name\n    continent (str, optional): Target continent name\n    isp (str, optional): Target ISP\n    pool (Literal[\"standard\", \"speed\", \"quality\"], optional): Proxy pool type\n    session_type (Literal[\"session\", \"hardsession\"]): Session persistence type\n        * \"session\": Optimized for success rate, may change IP for stability. Works with lifetime parameter.\n        * \"hardsession\": Maintains same IP for as long as possible. Cannot use lifetime parameter.\n        Defaults to \"session\".\n    auto_rotate (bool): Whether to automatically rotate IPs between requests.\n        Cannot be used with `session_type`.\n    lifetime (int, optional): Duration of the session in minutes (1-120)\n        Only works with `session_type=\"session\"`. Defaults to 40 if not specified.\n    adblock (bool): Whether to enable ad blocking. Defaults to False.\n```\n\n\u003c/details\u003e\n\n#### Mobile\n\n```py\n\u003e\u003e\u003e proxy = evomi.MobileProxy(username='daijro', key='password')\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eMobileProxy Parameters\u003c/summary\u003e\n\n```\nInitialize a new Evomi Mobile proxy.\n\nParameters:\n    username (str): Your Evomi username\n    key (str): Your Evomi API key\n    country (str, optional): Target country code (e.g., 'US', 'GB')\n    continent (str, optional): Target continent name\n    isp (str, optional): Target ISP\n    session_type (Literal[\"session\", \"hardsession\"]): Session persistence type\n        * \"session\": Optimized for success rate, may change IP for stability. Works with lifetime parameter.\n        * \"hardsession\": Maintains same IP for as long as possible. Cannot use lifetime parameter.\n        Defaults to \"session\".\n    auto_rotate (bool): Whether to automatically rotate IPs between requests.\n        Cannot be used with `session_type`.\n    lifetime (int, optional): Duration of the session in minutes (1-120)\n        Only works with `session_type=\"session\"`. Defaults to 40 if not specified.\n```\n\n\u003c/details\u003e\n\n#### Datacenter\n\n```py\n\u003e\u003e\u003e proxy = evomi.DatacenterProxy(username='daijro', key='password')\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eDatacenterProxy Parameters\u003c/summary\u003e\n\n```\nInitialize a new Evomi Datacenter proxy.\n\nParameters:\n    username (str): Your Evomi username\n    key (str): Your Evomi API key\n    country (str, optional): Target country code (e.g., 'US', 'GB')\n    continent (str, optional): Target continent name\n    session_type (Literal[\"session\", \"hardsession\"]): Session persistence type\n        * \"session\": Optimized for success rate, may change IP for stability. Works with lifetime parameter.\n        * \"hardsession\": Maintains same IP for as long as possible. Cannot use lifetime parameter.\n        Defaults to \"session\".\n    auto_rotate (bool): Whether to automatically rotate IPs between requests.\n        Cannot be used with `session_type`.\n    lifetime (int, optional): Duration of the session in minutes (1-120)\n        Only works with `session_type=\"session\"`. Defaults to 40 if not specified.\n```\n\n\u003c/details\u003e\n\n### Parameter Table\n\n| Parameter      | Description                                           | Residential | Mobile | Datacenter |\n| -------------- | ----------------------------------------------------- | ----------- | ------ | ---------- |\n| `continent`    | Continent name                                        | ✔️          | ✔️     | ✔️         |\n| `country`      | Country code                                          | ✔️          | ✔️     | ✔️         |\n| `region`       | Region, state, province, or territory                 | ✔️          | ✔️     |\n| `city`         | City name                                             | ✔️          |        |\n| `isp`          | ISP name                                              | ✔️          | ✔️     |\n| `pool`         | Proxy pool. Takes standard, speed, or quality.        | ✔️          |        |\n| `session_type` | Session persistence type                              | ✔️          | ✔️     | ✔️         |\n| `auto_rotate`  | Whether to automatically rotate IPs between requests. | ✔️          | ✔️     | ✔️         |\n| `lifetime`     | Duration of the session in minutes (1-120)            | ✔️          | ✔️     | ✔️         |\n| `adblock`      | Whether to enable ad blocking                         | ✔️          |        |\n\n### Geo-targetting\n\nSpecify the geographic location of the proxy:\n\n#### Continent\n\nPossible options are `Africa`, `Asia`, `Europe`, `Oceania`, `North America`, and `South America`.\n\n```py\n\u003e\u003e\u003e proxy = evomi.ResidentialProxy(continent='North America', ...)\n```\n\n#### Country\n\nTarget a specific country. Takes two-letter country codes.\n\n```py\n\u003e\u003e\u003e proxy = evomi.ResidentialProxy(country='US', ...)  # United States\n\u003e\u003e\u003e proxy = evomi.ResidentialProxy(country='CA', ...)  # Canada\n```\n\n#### City\n\nTarget a specific city. Residential proxies only.\n\n```py\n\u003e\u003e\u003e proxy = evomi.ResidentialProxy(city='New York', ...)\n\u003e\u003e\u003e proxy = evomi.ResidentialProxy(city='Tokyo', ...)\n```\n\n#### Region\n\nTarget a specific state, province, or territory. Residential and Mobile proxies only.\n\n```py\n\u003e\u003e\u003e proxy = evomi.ResidentialProxy(region='California', ...)\n\u003e\u003e\u003e proxy = evomi.ResidentialProxy(region='Southern Cape', ...)\n```\n\n---\n\n## Thanks\n\nThis project includes code adapted from the following sources:\n\n- **tls-client**\n\n  - Author: bogdanfinn\n  - Repository: https://github.com/bogdanfinn/tls-client\n  - License: BSD-4-Clause license\n  - Used in [bridge/server.go](https://github.com/daijro/hrequests/blob/main/bridge/server.go)\n\n- **Minet**\n\n  - Author: medialab\n  - Repository: https://github.com/medialab/minet\n  - License: GPL-3.0\n  - Inspired the threadsafe implementation of Playwright\n\n- **Patchright**\n  - Author: Vinyzu and Kaliiiiiiiiii\n  - Repository: https://github.com/Kaliiiiiiiiii-Vinyzu/patchright\n  - License: Apache License 2.0\n  - Used for Chrome browser support\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdaijro%2Fhrequests","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdaijro%2Fhrequests","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdaijro%2Fhrequests/lists"}