{"id":17479065,"url":"https://github.com/antoinevastel/bots-zoo","last_synced_at":"2025-08-16T19:09:48.400Z","repository":{"id":88524126,"uuid":"330178489","full_name":"antoinevastel/bots-zoo","owner":"antoinevastel","description":null,"archived":false,"fork":false,"pushed_at":"2024-03-16T10:19:22.000Z","size":469,"stargazers_count":113,"open_issues_count":0,"forks_count":26,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-04-15T07:06:35.568Z","etag":null,"topics":["bot","crawler","crawling","playwright","puppeteer","scraper","scraping","selenium","user-agent","useragent"],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/antoinevastel.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-01-16T14:16:45.000Z","updated_at":"2025-04-01T13:07:05.000Z","dependencies_parsed_at":"2024-03-16T11:45:15.962Z","dependency_job_id":"daff8833-9c3e-42bd-86eb-1242a74a16d8","html_url":"https://github.com/antoinevastel/bots-zoo","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/antoinevastel/bots-zoo","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antoinevastel%2Fbots-zoo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antoinevastel%2Fbots-zoo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antoinevastel%2Fbots-zoo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antoinevastel%2Fbots-zoo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/antoinevastel","download_url":"https://codeload.github.com/antoinevastel/bots-zoo/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antoinevastel%2Fbots-zoo/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":270758015,"owners_count":24640173,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-16T02:00:11.002Z","response_time":91,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bot","crawler","crawling","playwright","puppeteer","scraper","scraping","selenium","user-agent","useragent"],"created_at":"2024-10-18T20:54:18.131Z","updated_at":"2025-08-16T19:09:48.371Z","avatar_url":"https://github.com/antoinevastel.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Bots Zoo\n\n## News:\nI recently created a [website where you can see your browser fingerprint](https://deviceandbrowserinfo.com/info_device) and different fingerprinting-related signals like your IP address, your canvas fingerprint, your HTTP headers, etc. Some information is accessible both through a webpage and through APIs. Don't hesitate to bookmark it as I will add more signals and more content related to bots.\n\n\nFor my different projects, I often have to launch bots using different kinds of browsers (Firefox, Chrome, Headless/not headless) using different automation frameworks (Puppeteer, Selenium, Playwright) in several programming languages. \n\nSince I'm juggling between different frameworks/languages, sometimes it's difficult to remember/find how to set up a particular kind of bot, or how to execute basic commands. \n\nThat's why I've decided to centralize examples of simple bots in this repository. I hope it will also benefit other people.\n\nFor the moment I only have example for the following bots:\n- Playwright (NodeJS): Chromium, Webkit (Safari), Firefox\n- Playwright extra stealth (Nodejs): Chromium (will be updated when it becomes stable)\n- Puppeteer (NodeJS): Chromium, Firefox, Android (emulation), iPhone (emulation)\n- Puppeteer extra stealth (NodeJS): Chromium\n- Pyppeteer stealth (Python): Chromium\n- Selenium (NodeJS): Chromium, Firefox\n- Selenium stealth (Python): Chrome\n- Selenium driverless (Python): Chrome\n- Undetected Chromedriver (Python): Chrome\n- Ferrum (Ruby): Chrome\n- Watir (Ruby): Chrome, Safari (MacOS)\n- Simple HTTP module/library (NodeJS + Cheerio): Sequential, Parallel, Sequential using Nord VPN, HTTP proxies\n- Simple HTTP module/library (Python requests/aiohttp + Beautifulsoup): Sequential, Parallel (x2 implementations)\n- Simple HTTP module/library (Golang standard library + goquery): Sequential, Parallel\n\nI will continue to add other examples, such as Playwright Firefox/WebKit, Selenium Firefox, both in NodeJS but also in other programming languages like Python.\nI will also provide examples for bot frameworks that provide mechanisms against bot detection solutions.\n\nThe **headers** directory contains data related to HTTP headers.\nFor the moment, it contains:\n- A list of ~16K user-agents;\n- **Accept** headers for the main browsers;\n- **Accept-Encoding** headers for the main browsers;\n- **Header names** for the main browsers;\n- **Fetch metadata** request headers.\n\nThe **browser_apis** directory contains data related to JS APIs sometimes used to identify a browser:\n- ```language.txt``` values of navigator.language;\n- ```languages.txt``` values of navigator.languages;\n- ```mimeTypes.txt``` values of navigator.mimeTypes;\n- ```oscpus.txt``` values of navigator.oscpu;\n- ```platforms.txt``` values of navigator.platform;\n- ```plugins.txt``` values of navigator.plugins;\n- ```webGLrenderers.txt``` values of ```gl.getParameter(debugInfo.UNMASKED_RENDERER_WEBGL)```;\n- ```webGLvendors.txt``` values of ```gl.getParameter(debugInfo.UNMASKED_VENDOR_WEBGL)```;\n\n**DON'T contact me** if you want to ask me how to make your bot(s) undetectable. \nYou can find [articles on my website](https://antoinevastel.com/categories.html#Bot-detection-ref), but I won't provide more details since I'm working for a bot detection company.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fantoinevastel%2Fbots-zoo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fantoinevastel%2Fbots-zoo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fantoinevastel%2Fbots-zoo/lists"}