{"id":31072355,"url":"https://github.com/miskler/human-requests","last_synced_at":"2026-05-18T09:07:33.052Z","repository":{"id":313156984,"uuid":"1049684967","full_name":"Miskler/human-requests","owner":"Miskler","description":"Asynchronous library for browser‑like HTTP scenarios with controlled offline rendering and two‑way state transfer.","archived":false,"fork":false,"pushed_at":"2025-09-13T21:53:43.000Z","size":812,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-09-13T22:09:40.067Z","etag":null,"topics":["anti-bot","antibot","asyncio","curl-cffi","playwright","python3","scaping","scraper"],"latest_commit_sha":null,"homepage":"https://pypi.org/project/human-requests/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Miskler.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-09-03T10:48:40.000Z","updated_at":"2025-09-13T21:53:47.000Z","dependencies_parsed_at":"2025-09-06T10:25:22.262Z","dependency_job_id":"952c42a8-a25b-4381-bd8e-abf71d3884da","html_url":"https://github.com/Miskler/human-requests","commit_stats":null,"previous_names":["miskler/human-requests","miskler/human-requests-core"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Miskler/human-requests","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Miskler%2Fhuman-requests","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Miskler%2Fhuman-requests/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Miskler%2Fhuman-requests/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Miskler%2Fhuman-requests/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Miskler","download_url":"https://codeload.github.com/Miskler/human-requests/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Miskler%2Fhuman-requests/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":275344040,"owners_count":25447898,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-15T02:00:09.272Z","response_time":75,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["anti-bot","antibot","asyncio","curl-cffi","playwright","python3","scaping","scraper"],"created_at":"2025-09-16T00:50:37.182Z","updated_at":"2026-05-18T09:07:33.044Z","avatar_url":"https://github.com/Miskler.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\n# Human Requests\n\n\u003cimg src=\"https://raw.githubusercontent.com/Miskler/human-requests/refs/heads/main/assets/logo.png\" width=\"70%\" alt=\"logo.webp\" /\u003e\n\n*Asynchronous Playwright wrappers for browser-like HTTP scenarios, controlled render flow, and API autotest integration.*\n\n[![Tests](https://miskler.github.io/human-requests/tests-badge.svg)](https://miskler.github.io/human-requests/tests/tests-report.html)\n[![Coverage](https://miskler.github.io/human-requests/coverage.svg)](https://miskler.github.io/human-requests/coverage/)\n[![Python](https://img.shields.io/badge/python-3.10+-blue)](https://python.org)\n[![PyPI - Package Version](https://img.shields.io/pypi/v/human-requests?color=blue)](https://pypi.org/project/human-requests/)\n[![License](https://img.shields.io/badge/license-MIT-blue)](LICENSE)\n[![BlackCode](https://img.shields.io/badge/code%20style-black-black)](https://github.com/psf/black)\n[![mypy](https://img.shields.io/badge/type--checked-mypy-blue?logo=python)](https://mypy.readthedocs.io/en/stable/index.html)\n[![Discord](https://img.shields.io/discord/792572437292253224?label=Discord\u0026labelColor=%232c2f33\u0026color=%237289da)](https://discord.gg/UnJnGHNbBp)\n[![Telegram](https://img.shields.io/badge/Telegram-24A1DE)](https://t.me/miskler_dev)\n\n**[Star us on GitHub](https://github.com/Miskler/human-requests)** | **[Read the Docs](https://miskler.github.io/human-requests/quick_start)** | **[Report a Bug](https://github.com/Miskler/human-requests/issues)**\n\n\u003c/div\u003e\n\n## Features\n\n- Typed wrappers over Playwright primitives:\n  - `HumanBrowser`\n  - `HumanContext`\n  - `HumanPage`\n- `HumanPage.fetch(...)`: execute HTTP requests from page context and get structured `FetchResponse`.\n- `HumanPage.goto_render(...)`: render already available response payloads without duplicate upstream request.\n- Storage helpers:\n  - `HumanContext.local_storage()` for full context snapshot\n  - `HumanPage.local_storage()` for current page origin\n  - `HumanPage.cookies()` convenience alias\n- Fingerprint snapshot collection: `HumanContext.fingerprint(...)`.\n- Built-in pytest autotest plugin for API clients (`@autotest`, hooks, params, dependencies).\n\n## Installation\n\nBase package:\n\n```bash\npip install human-requests\nplaywright install chromium\n```\n\nOptional autotest addon dependencies:\n\n```bash\npip install human-requests[autotest] pytest pytest-anyio pytest-jsonschema-snapshot pytest-subtests\n```\n\nIf you run with Camoufox, install it separately:\n\n```bash\npip install camoufox\ncamoufox fetch\n```\n\n## Quick Start\n\n### Wrap a Playwright browser\n\n```python\nimport asyncio\nfrom playwright.async_api import async_playwright\n\nfrom human_requests import HumanBrowser\n\n\nasync def main() -\u003e None:\n    async with async_playwright() as p:\n        pw_browser = await p.chromium.launch(headless=True)\n        browser = HumanBrowser.replace(pw_browser)\n\n        ctx = await browser.new_context()\n        page = await ctx.new_page()\n\n        await page.goto(\"https://httpbin.org/html\", wait_until=\"domcontentloaded\")\n        print(page.url)\n\n        await browser.close()\n\n\nasyncio.run(main())\n```\n\n### Direct request in page context (`fetch`)\n\n```python\nresp = await page.fetch(\"https://httpbin.org/json\")\nprint(resp.status_code)\nprint(resp.json())\n```\n\n### Render previously fetched response (`goto_render`)\n\n```python\nchallenge = await page.fetch(\"https://example.com/challenge\")\nawait page.goto_render(challenge, wait_until=\"networkidle\")\n```\n\n### Auto Screenshot on Handler Failure\n\nIf you want a screenshot to be taken when errors like `playwright.Error` (including timeouts) occur,\nyou can set `page.on_error_screenshot_path = \"screenshot.png\"` (this setting is page-specific; by default, screenshots are disabled).\n\n### State helpers\n\n```python\ncookies = await page.cookies()\ncontext_storage = await ctx.local_storage()\npage_storage = await page.local_storage()\nprint(len(cookies), context_storage.keys(), page_storage.keys())\n```\n\n### Fingerprint snapshot\n\n```python\nfingerprint = await ctx.fingerprint(origin=\"https://example.com\")\nprint(fingerprint.user_agent)\nprint(fingerprint.browser_name, fingerprint.browser_version)\n```\n\n## API Tree Boilerplate Helper\n\nTo avoid repetitive `_parent` and `__post_init__` wiring in SDK-style clients\n(like `fixprice_api` / `perekrestok_api`), use:\n\n- `ApiChild[ParentType]`\n- `ApiParent`\n- `api_child_field(...)`\n\n```python\nfrom dataclasses import dataclass\nfrom human_requests import ApiChild, ApiParent, api_child_field\n\n\nclass ClassCatalog(ApiChild[\"ShopApi\"]):\n    async def tree(self):\n        ...\n\n\nclass ClassGeolocation(ApiChild[\"ShopApi\"]):\n    async def cities_list(self):\n        ...\n\n\n@dataclass\nclass ShopApi(ApiParent):\n    Catalog: ClassCatalog = api_child_field(ClassCatalog)\n    Geolocation: ClassGeolocation = api_child_field(ClassGeolocation)\n```\n\n`ApiParent` initializes all `api_child_field(...)` values in `__post_init__`\nautomatically, so manual assignments are no longer needed.\n\nNested chains are supported as well (`Root -\u003e Child -\u003e Child`):\n\n```python\n@dataclass\nclass BranchApi(ApiChild[\"RootApi\"], ApiParent):\n    Catalog: ClassCatalog = api_child_field(ClassCatalog)\n\n@dataclass\nclass RootApi(ApiParent):\n    Branch: BranchApi = api_child_field(BranchApi)\n```\n\n## API Autotest Addon (pytest)\n\n`human-requests` ships with a pytest plugin that can auto-run API methods marked with `@autotest` and validate payloads via `schemashot` from `pytest-jsonschema-snapshot`.\nWith `pytest-subtests`, each discovered `@autotest` method and `@autotest_data`\nprovider is shown as a separate subtest inside `test_autotest_api_methods`.\n\nMinimal `pytest.ini`:\n\n```ini\n[pytest]\nanyio_mode = auto\nautotest_start_class = your_package.StartClass\nautotest_typecheck = warn\n```\n\n`autotest_typecheck` modes:\n\n- `off` (default): no runtime type checks for params provider arguments\n- `warn`: emit `RuntimeWarning` on annotation mismatch\n- `strict`: fail test case with `TypeError` on mismatch\n\nMinimal fixtures:\n\n```python\nimport pytest\nfrom your_package import StartClass\n\n\n@pytest.fixture(scope=\"session\")\ndef anyio_backend() -\u003e str:\n    return \"asyncio\"\n\n\n@pytest.fixture(scope=\"session\")\nasync def api() -\u003e StartClass:\n    async with StartClass() as client:\n        yield client\n```\n\nBusiness code only marks methods:\n\n```python\nfrom human_requests import autotest\n\n\nclass Catalog:\n    @autotest\n    async def tree(self):\n        ...\n```\n\nTest layer adds hooks and params:\n\n```python\nfrom human_requests import autotest_depends_on, autotest_hook, autotest_params\nfrom human_requests.autotest import AutotestCallContext, AutotestContext\n\n\n@autotest_hook(target=Catalog.tree)\ndef _capture_category(_resp, data, ctx: AutotestContext) -\u003e None:\n    ctx.state[\"category_id\"] = data[\"items\"][0][\"id\"]\n\n\n@autotest_depends_on(Catalog.tree)\n@autotest_params(target=Catalog.feed)\ndef _feed_params(ctx: AutotestCallContext) -\u003e dict[str, int]:\n    return {\"category_id\": ctx.state[\"category_id\"]}\n```\n\nParent-specific registration is supported:\n\n```python\n@autotest_hook(target=Child.method, parent=ParentA)\ndef _only_for_parent_a(_resp, data, ctx):\n    ...\n```\n\nFor a complete guide, see `docs/source/autotest.rst`.\n\n## Development\n\nSetup:\n\n```bash\ngit clone https://github.com/Miskler/human-requests.git\ncd human-requests\npython -m venv .venv\nsource .venv/bin/activate  # Windows: .venv\\Scripts\\activate\npip install -r requirements.txt\npip install -e .\n```\n\nCommands:\n\n```bash\npytest\nmake lint\nmake type-check\nmake format\nmake docs\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmiskler%2Fhuman-requests","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmiskler%2Fhuman-requests","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmiskler%2Fhuman-requests/lists"}