https://github.com/berstend/puppeteer-extra
💯 Teach puppeteer new tricks through plugins.
https://github.com/berstend/puppeteer-extra
automation chrome headless-browser headless-chrome lerna monorepo puppeteer puppeteer-extra puppeteer-plugins recaptcha stealth user-preferences yarn-workspaces
Last synced: 6 months ago
JSON representation
💯 Teach puppeteer new tricks through plugins.
- Host: GitHub
- URL: https://github.com/berstend/puppeteer-extra
- Owner: berstend
- License: mit
- Created: 2018-05-15T20:02:21.000Z (over 7 years ago)
- Default Branch: master
- Last Pushed: 2024-07-18T12:28:02.000Z (over 1 year ago)
- Last Synced: 2025-05-06T20:29:58.480Z (6 months ago)
- Topics: automation, chrome, headless-browser, headless-chrome, lerna, monorepo, puppeteer, puppeteer-extra, puppeteer-plugins, recaptcha, stealth, user-preferences, yarn-workspaces
- Language: JavaScript
- Homepage: https://extra.community
- Size: 16.9 MB
- Stars: 6,805
- Watchers: 116
- Forks: 750
- Open Issues: 262
-
Metadata Files:
- Readme: README.md
- Funding: .github/FUNDING.yml
- License: LICENSE
Awesome Lists containing this project
- awesome-puppeteer - puppeteer-extra - Wrapper to use stealth mode, custom user prefs, etc. (Packages)
- awesome-puppeteer-zh - puppeteer-extra - 包装使用隐身模式,自定义用户首选项等. (包 / 贡献)
- jimsghstars - berstend/puppeteer-extra - 💯 Teach puppeteer new tricks through plugins. (JavaScript)
- StarryDivineSky - berstend/puppeteer-extra - extra 是一个为 Puppeteer 提供扩展功能的插件库，通过模块化插件系统实现浏览器自动化操作的深度定制。该项目核心特性是通过 "plugins" 机制为 Puppeteer 添加新功能，例如浏览器指纹修改、请求拦截、验证码处理等，开发者可以按需加载插件来增强自动化脚本能力。其工作原理基于 Puppeteer 的插件架构，通过注册自定义插件实现对浏览器实例的扩展，支持通过配置文件或代码直接加载插件。项目内置了多个实用插件如 puppeteer-extra-plugin-stealth（隐藏自动化特征）、puppeteer-extra-plugin-viewport（屏幕分辨率管理）等，可帮助开发者更精准地模拟真实用户行为。安装后可通过 `puppeteer-extra.use(plugin)` 语法集成插件，支持动态加载和参数配置，适合需要长期维护的爬虫项目或自动化测试场景。项目还提供了详细的中文文档和示例代码，开发者可快速上手扩展功能。其设计目标是降低 Puppeteer 功能扩展的复杂度，同时保持核心库的轻量化，通过插件机制实现功能的灵活组合，适用于需要应对反爬机制、多浏览器兼容或复杂页面交互的场景。 (网络信息服务 / 网络爬虫)
- awesome-fingerprinting - PuppeteerExtra
README
# puppeteer-extra [](https://github.com/berstend/puppeteer-extra/)
This is the monorepo for [`puppeteer-extra`](./packages/puppeteer-extra), a modular plugin framework for [`puppeteer`](https://github.com/puppeteer/puppeteer). :-)
🌟 **For the main documentation, please head over to the [`puppeteer-extra`](./packages/puppeteer-extra) package.**
We've also recently introduced support for Playwright, if you're interested in that head over to [`playwright-extra`](./packages/playwright-extra).
## Monorepo
Contributing
### Contributing
PRs and new plugins are welcome! The plugin API for `puppeteer-extra` is clean and fun to use. Have a look the [`PuppeteerExtraPlugin`](./packages/puppeteer-extra-plugin) base class documentation to get going and check out the [existing plugins](./packages/) (minimal example is the [anonymize-ua](./packages/puppeteer-extra-plugin-anonymize-ua/index.js) plugin) for reference.
We use a [monorepo](https://github.com/berstend/puppeteer-extra) powered by [Lerna](https://github.com/lerna/lerna#--use-workspaces) (and yarn workspaces), [ava](https://github.com/avajs/ava) for testing, the [standard](https://standardjs.com/) style for linting and [JSDoc](http://usejsdoc.org/about-getting-started.html) heavily to auto-generate markdown [documentation](https://github.com/documentationjs/documentation) based on code. :-)
Lerna
### Lerna
This monorepo is powered by [Lerna](https://github.com/lerna/lerna) and yarn workspaces.
#### Initial setup
```bash
# Install deps
yarn
# Bootstrap the packages in the current Lerna repo.
# Installs all of their dependencies and links any cross-dependencies.
yarn bootstrap
# Build all TypeScript sources
yarn build
```
#### Development flow
```bash
# Install debug in all packages
yarn lerna add debug
# Install fs-extra to puppeteer-extra-plugin-user-data-dir
yarn lerna add fs-extra --scope=puppeteer-extra-plugin-user-data-dir
# Remove dependency
# https://github.com/lerna/lerna/issues/833
yarn lerna exec --concurrency 1 'yarn remove fs-extra; echo 0'
# Run test in all packages
yarn test
# Update JSDoc based documentation in markdown files
yarn docs
# Upgrade project wide deps like puppeteer
# (We keep the devDependency version blurry)
rm -rf node_modules
rm -rf yarn.lock
yarn
yarn lerna bootstrap
# Update deps within packages (interactive)
yarn lernaupdate
# If in doubt :-(
yarn lerna exec "rm -f yarn.lock; rm -rf node_modules; echo 0"
rm -f yarn.lock && rm -rf node_modules && yarn cache clean
# Run tests of specific package
cd packages/puppeteer-extra-plugin-stealth
yarn test
# Run tests of specific stealth evasion
cd packages/puppeteer-extra-plugin-stealth
yarn ava -v ./evasions/user-agent-override/index.test.js
# Test a local monorepo package in an outside folder as it would've been installed from the registry
# Change PACKAGE_DIR to the path of this monorepo and PACKAGE to the package you wish to install
PACKAGE=puppeteer-extra PACKAGE_DIR=/Users/foo/puppeteer-extra/packages && yarn remove $(echo $PACKAGE); true && rm -f $(pwd)/$(echo $PACKAGE)-latest.tgz && yarn --cwd $(echo $PACKAGE_DIR)/$(echo $PACKAGE) pack --filename $(pwd)/$(echo $PACKAGE)-latest.tgz && YARN_CACHE_FOLDER=/tmp/yarn yarn add file:$(pwd)/$(echo $PACKAGE)-latest.tgz && rm -rf /tmp/yarn
```
#### Publishing
```bash
# make sure you're signed into npm before publishing
# yarn publishing is broken so lerna uses npm
npm whoami
# ensure everything is up2date and peachy
yarn
yarn bootstrap
yarn lerna link
yarn build
yarn test
# Phew, let's publish these packages!
# - Will publish all changed packages
# - Will ask for new pkg version per package
# - Will updated inter-package dependency versions automatically
yarn lerna publish
# Fix new dependency version symlinks
yarn bootstrap && yarn lerna link
```
