{"id":27156738,"url":"https://github.com/vishnuprasad-code/cf_clearance_collector","last_synced_at":"2025-04-10T02:20:00.452Z","repository":{"id":286667684,"uuid":"962152485","full_name":"Vishnuprasad-code/cf_clearance_collector","owner":"Vishnuprasad-code","description":"A simple RESTful API that returns the response cookies after passing cloudflare human verificaton.","archived":false,"fork":false,"pushed_at":"2025-04-07T18:22:48.000Z","size":14,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-07T19:32:26.952Z","etag":null,"topics":["python","scraper-api"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Vishnuprasad-code.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-04-07T18:19:29.000Z","updated_at":"2025-04-07T18:37:08.000Z","dependencies_parsed_at":"2025-04-07T19:32:32.659Z","dependency_job_id":"5593e87f-39e6-48ba-8ed0-b4a5bf1755c1","html_url":"https://github.com/Vishnuprasad-code/cf_clearance_collector","commit_stats":null,"previous_names":["vishnuprasad-code/cf_clearance_collector"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Vishnuprasad-code%2Fcf_clearance_collector","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Vishnuprasad-code%2Fcf_clearance_collector/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Vishnuprasad-code%2Fcf_clearance_collector/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Vishnuprasad-code%2Fcf_clearance_collector/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Vishnuprasad-code","download_url":"https://codeload.github.com/Vishnuprasad-code/cf_clearance_collector/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247924417,"owners_count":21019177,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["python","scraper-api"],"created_at":"2025-04-08T20:33:00.220Z","updated_at":"2025-04-08T20:34:00.742Z","avatar_url":"https://github.com/Vishnuprasad-code.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# cf_clearance cookie collector API\n\nA simple RESTful API that returns the response cookies after passing cloudflare human verificaton.\n\n---\n## About the Solver\n\nThis cf_clearance cookie collector API is designed to extract response cookies after passing cloudflare human verificaton using browser automation tool:- [nodriver](https://github.com/ultrafunkamsterdam/nodriver).\n\n- Nodriver is an efficient browser automation tool that helps us pass cloudflare turnstile human verification. The API simply launches the nodriver chrome browser, visit the url given by the user, pass the human verfication, and extract and return the cookies.\n- The API itself is built using **FastAPI**, providing a fast and intuitive web interface for interaction.\n- Containerized using **Docker**, making it easy to deploy.\n\n---\n## API Setup\n\n- git clone https://github.com/Vishnuprasad-code/cf_clearance_collector.git\n- cd cf_clearance_collector\n- docker-compose up --build\n\n## Usage\n- Create Task and fetch task id\n```\nurl = \"http://0.0.0.0:8000/solve_captcha/clouflare/create_task/\"\npayload = {\n    \"url\": \u003curl:str\u003e,  # url of the site with cloudflare protection\n    \"browser_config_args\": \u003cconfig args: Dict\u003e  #  nodriver config args as dict:- (https://ultrafunkamsterdam.github.io/nodriver/nodriver/classes/others_and_helpers.html#config-class)\n}\nresponse = requests.post(\n    url=url,\n    json=payload\n)\nprint(response.status_code, response.json())\ntask_id = response.json().get('task_id')\n'''\n## sample payload\npayload = {\n    \"url\": \"https://nopecha.com/demo/cloudflare\",\n    \"browser_config_args\": {\n        \"headless\": False,\n        \"browser_args\": [\n            \"--proxy-server=\u003cyour-proxy\u003e\"\n        ],\n        \"lang\": \"en-US\",\n    }\n}\n## sample response\n{'state': 'Created Task', 'task_id': '5bf3436c137411f0babe0e554f559473', 'error': None}\nstatus: 201\n'''\n```\n- Start polling for solution using the task id\n```\nwhile(True):\n    url = \"http://0.0.0.0:8000/solve_captcha/clouflare/get_cookie/\"\n    payload = {\n        \"task_id\": task_id\n    }\n    response = requests.post(\n        url=url,\n        json=payload\n    )\n    print(response.json())\n    if response.json().get(\"state\") == \"Finished\":\n        break\n\n    time.sleep(20)\n'''\n## sample responses\n{\"state\":\"Processing\",\"cookies\":{},\"error\":null}\nor\n{\n  \"state\": \"Finished\",\n  \"cookies\": {\n    \"cookies\": [\n      {\n        \"name\": \"cf_clearance\",\n        \"value\": \"1744006437-1.0.1.1-nimO3UKNZ9oA.oS.9GSnUbaYet0dxvCtYOpaKgMWuG8cRC3CrB8UewrdEqjpRtOgoxLBRUdYpP2Ar5DPD3ccgfhwe_WTIrI8wyaQm8d.X.Y\",\n        \"path\": \"/\"\n      }\n    ],\n    \"userAgent\": \"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Safari/537.36\"\n  },\n  \"error\": null\n}\n'''\n```\n- Use the same user agent obtained from the response while using the cookies\n- If you are passing --proxy-server make sure to use the same ip while uisng the cookies\n\n---\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvishnuprasad-code%2Fcf_clearance_collector","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvishnuprasad-code%2Fcf_clearance_collector","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvishnuprasad-code%2Fcf_clearance_collector/lists"}