{"id":15434158,"url":"https://github.com/xewdy444/cf-clearance-scraper","last_synced_at":"2025-05-16T12:00:19.212Z","repository":{"id":58888967,"uuid":"482650970","full_name":"Xewdy444/CF-Clearance-Scraper","owner":"Xewdy444","description":"A simple program for scraping Cloudflare clearance (cf_clearance) cookies from websites issuing Cloudflare challenges to visitors","archived":false,"fork":false,"pushed_at":"2025-05-13T02:29:59.000Z","size":169,"stargazers_count":314,"open_issues_count":16,"forks_count":37,"subscribers_count":9,"default_branch":"zendriver","last_synced_at":"2025-05-13T03:29:31.899Z","etag":null,"topics":["clearance","cloudflare","cookie","nodriver","playwright","scraper","undetected-chromedriver"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Xewdy444.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-04-17T22:53:04.000Z","updated_at":"2025-05-06T14:18:04.000Z","dependencies_parsed_at":"2023-02-08T04:01:12.089Z","dependency_job_id":"990af3ca-e1be-4b69-b4d1-36c235bb8eea","html_url":"https://github.com/Xewdy444/CF-Clearance-Scraper","commit_stats":{"total_commits":133,"total_committers":6,"mean_commits":"22.166666666666668","dds":0.3834586466165414,"last_synced_commit":"c3edc6d6ea37a99af6c6904f6fd1878d8cb2183a"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Xewdy444%2FCF-Clearance-Scraper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Xewdy444%2FCF-Clearance-Scraper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Xewdy444%2FCF-Clearance-Scraper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Xewdy444%2FCF-Clearance-Scraper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Xewdy444","download_url":"https://codeload.github.com/Xewdy444/CF-Clearance-Scraper/tar.gz/refs/heads/zendriver","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254527071,"owners_count":22085917,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["clearance","cloudflare","cookie","nodriver","playwright","scraper","undetected-chromedriver"],"created_at":"2024-10-01T18:37:31.944Z","updated_at":"2025-05-16T12:00:19.124Z","avatar_url":"https://github.com/Xewdy444.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# CF-Clearance-Scraper\n\n## Zendriver Version\nA simple program for scraping Cloudflare clearance (cf_clearance) cookies from websites issuing Cloudflare challenges to visitors. This program works on all Cloudflare challenge types (JavaScript, managed, and interactive). If you would prefer using Playwright, you can check out the [Playwright version](https://github.com/Xewdy444/CF-Clearance-Scraper/tree/playwright).\n\n\n## Clearance Cookie Usage\nIn order to bypass Cloudflare challenges with the clearance cookies, you must make sure of two things:\n\n- The user agent used to fetch the clearance cookie must match the user agent being used within the requests that use the clearance cookie\n- The IP address used to fetch the clearance cookie must match the IP address being used to make the requests that use the clearance cookie\n\n```mermaid\nflowchart\n\tN14e[\"cf_clearance\"]\n\tN14f[\"IP Address\"]\n\tN150[\"User Agent\"]\n\tN14e --\u003e N14f\n\tN14e --\u003e N150\n```\n\n## Installation\nDownload and install [Google Chrome](https://www.google.com/chrome/index.html).\n\nThen, install the Python dependencies:\n\n    $ pip install -r requirements.txt\n\n## Usage\n\u003e [!WARNING]\n\u003e Depending on the user agent used, it may affect your ability to solve the Cloudflare challenge.\n\n```\nusage: main.py [-h] [-f FILE] [-t TIMEOUT] [-p PROXY] [-ua USER_AGENT] [--disable-http2] [--disable-http3] [--headed] [-ac] [-c] [-w] [-a] URL\n\nA simple program for scraping Cloudflare clearance (cf_clearance) cookies from websites issuing Cloudflare challenges to visitors\n\npositional arguments:\n  URL                   The URL to scrape the Cloudflare clearance cookie from\n\noptions:\n  -h, --help            show this help message and exit\n  -f FILE, --file FILE  The file to write the Cloudflare clearance cookie information to, in JSON format\n  -t TIMEOUT, --timeout TIMEOUT\n                        The timeout in seconds to use for solving challenges\n  -p PROXY, --proxy PROXY\n                        The proxy server URL to use for the browser requests\n  -ua USER_AGENT, --user-agent USER_AGENT\n                        The user agent to use for the browser requests\n  --disable-http2       Disable the usage of HTTP/2 for the browser requests\n  --disable-http3       Disable the usage of HTTP/3 for the browser requests\n  --headed              Run the browser in headed mode\n  -ac, --all-cookies    Retrieve all cookies from the page, not just the Cloudflare clearance cookie\n  -c, --curl            Get the cURL command for the request with the cookies and user agent\n  -w, --wget            Get the Wget command for the request with the cookies and user agent\n  -a, --aria2           Get the aria2 command for the request with the cookies and user agent\n```\n\n## Example\n    $ python main.py -f cookies.json https://sergiodemo.com/security/challenge/legacy-challenge\n    [14:24:27] [INFO] Launching headless browser...\n    [14:24:27] [INFO] Going to https://sergiodemo.com/security/challenge/legacy-challenge...\n    [14:24:28] [INFO] Solving Cloudflare challenge [Interactive]...\n    [14:24:31] [INFO] Cookie: cf_clearance=SkyEdEGvKp1BBA2NpRW3Azsw5neMD6sEEqJd6jOCCfs-1736886257-1.2.1.1-cam47ywp3q_yKE1bw0lZ2YS83dnh_BsIHtS7earbsYE.AxQDBtZiifiHvp1nZGRhABaSdjU7XRQpUCVwUSrlJGH8DXr50YR18pNLxBvcEJFO2gPMxr.ZjKze8rWgM9H4rPeET67jzAo_ZRpNP6hGCvdyO62VVCtqDBQDKhKZz9yZQp7YEHK7tchQIteVgu.dUxYdan5_D.R0zewnS382BP0w1AoTf2p40.lQwbhgildEiKG14xACd13V4EEthkZV0dnliwcn35rT3h32ODf50MABQNSQ8WjhZhbLSNOPO_zEhrK9R0Yn4eBuRKvnL9_x9jKvaBPDPAgyiZv_VzFP_g\n    [14:24:31] [INFO] User agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/133.0.0.0 Safari/537.36\n    [14:24:31] [INFO] Writing Cloudflare clearance cookie information to cookies.json...\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxewdy444%2Fcf-clearance-scraper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fxewdy444%2Fcf-clearance-scraper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxewdy444%2Fcf-clearance-scraper/lists"}