{"id":20766568,"url":"https://github.com/mmqnym/pyppeteer-use-case","last_synced_at":"2025-12-24T13:23:08.546Z","repository":{"id":153824460,"uuid":"519484867","full_name":"mmqnym/pyppeteer-use-case","owner":"mmqnym","description":"Show how to do web crawl via pyppeteer","archived":false,"fork":false,"pushed_at":"2022-08-04T21:17:07.000Z","size":15,"stargazers_count":1,"open_issues_count":0,"forks_count":2,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-08T21:41:14.686Z","etag":null,"topics":["crawl","crawler","pyppeteer","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mmqnym.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2022-07-30T10:23:45.000Z","updated_at":"2023-06-07T08:47:07.000Z","dependencies_parsed_at":"2023-08-26T18:20:28.976Z","dependency_job_id":null,"html_url":"https://github.com/mmqnym/pyppeteer-use-case","commit_stats":null,"previous_names":["mmq88/pyppeteer-use-case","mmqnym/pyppeteer-use-case"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mmqnym%2Fpyppeteer-use-case","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mmqnym%2Fpyppeteer-use-case/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mmqnym%2Fpyppeteer-use-case/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mmqnym%2Fpyppeteer-use-case/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mmqnym","download_url":"https://codeload.github.com/mmqnym/pyppeteer-use-case/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243094246,"owners_count":20235478,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["crawl","crawler","pyppeteer","python"],"created_at":"2024-11-17T11:24:46.237Z","updated_at":"2025-12-24T13:23:03.479Z","avatar_url":"https://github.com/mmqnym.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Pyppeteer usecase\n\n## Set up\n\n``` sh\npip install pyppeteer\n```\n\n\u003cbr /\u003e\n\nThis package is used to hide some webdrive attributes\n\n``` sh\npip install pyppeteer_stealth\n```\n\n## Experience\n\n- You can use https://bot.sannysoft.com/ to check if you are being detected as a bot.\n- There are several Chromium arguments for reference in the `Reference` section below.\n- Most of the problems can be solved by the discussions in `puppetter` or the related docs.\n- **Some of the situations detected as bot may not be caused by the code, just don't use the default Chronium download the latest version of chromium, copy-paste that package in your main file and use `executablePath = f'{os.getcwd()}\\chromium\\chrome.exe'` in lunch, most of them can be solved.**\n- You can refer to the `troubleshooting.md` of [Nodejs puppeteer docs](https://github.com/puppeteer/puppeteer/tree/main/docs), which has a record of some common problems.\n\n\u003cbr /\u003e\n\n## Reference\n\n- ##### [Python Asyncio Guide](https://docs.python.org/3/library/asyncio-dev.html)\n- ##### [Chromium Command Line Switches](https://peter.sh/experiments/chromium-command-line-switches/)\n- ##### [Nodejs puppeteer docs](https://github.com/puppeteer/puppeteer/tree/main/docs)\n- ##### [Pyppeteer](https://github.com/pyppeteer/pyppeteer)\n- ##### [Pyppeteer Docs](https://miyakogi.github.io/pyppeteer/)\n- ##### [Hide webdrive attributes](https://github.com/MeiK2333/pyppeteer_stealth)\n\n\u003cbr /\u003e\n\n\u003e ### 中文補充資料 (Guide in Mandarin)\n\n- ##### [Asyncio 從頭入門](https://ithelp.ithome.com.tw/articles/10199385)\n- ##### [Asyncio 架構探討](https://www.ithome.com.tw/voice/138875)\n\n\u003cbr /\u003e\n\n## License\n\n[MIT](LICENSE)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmmqnym%2Fpyppeteer-use-case","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmmqnym%2Fpyppeteer-use-case","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmmqnym%2Fpyppeteer-use-case/lists"}