{"id":28665327,"url":"https://github.com/spider-rs/spider_chrome","last_synced_at":"2025-06-14T14:01:40.943Z","repository":{"id":297840634,"uuid":"998030558","full_name":"spider-rs/spider_chrome","owner":"spider-rs","description":"A concurrent high-level API to control Chrome or Firefox over the DevTools Protocol for Rust","archived":false,"fork":false,"pushed_at":"2025-06-07T19:59:30.000Z","size":0,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-06-07T20:27:33.408Z","etag":null,"topics":["automation","chrome-devtools-protocol"],"latest_commit_sha":null,"homepage":"https://crates.io/crates/spider_chrome","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/spider-rs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE-APACHE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-06-07T18:06:46.000Z","updated_at":"2025-06-07T19:59:40.000Z","dependencies_parsed_at":"2025-06-07T20:40:05.860Z","dependency_job_id":null,"html_url":"https://github.com/spider-rs/spider_chrome","commit_stats":null,"previous_names":["spider-rs/spider_chrome"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/spider-rs/spider_chrome","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spider-rs%2Fspider_chrome","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spider-rs%2Fspider_chrome/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spider-rs%2Fspider_chrome/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spider-rs%2Fspider_chrome/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/spider-rs","download_url":"https://codeload.github.com/spider-rs/spider_chrome/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spider-rs%2Fspider_chrome/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":259654298,"owners_count":22890990,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["automation","chrome-devtools-protocol"],"created_at":"2025-06-13T13:38:24.093Z","updated_at":"2025-06-13T13:38:26.295Z","avatar_url":"https://github.com/spider-rs.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# spider_chrome\n\n[![Crates.io](https://img.shields.io/crates/v/spider_chrome.svg)](https://crates.io/crates/spider_chrome)\n[![Documentation](https://docs.rs/spider_chrome/badge.svg)](https://docs.rs/spider_chrome)\n\nA concurrent high-level API to control Chrome or Firefox over the DevTools Protocol.\n\nThis project is a fork of [chromiumoxide](https://github.com/mattsse/chromiumoxide) that primarily keeps CDP up to date, applies bug fixes, improves emulation, performance, and enables high-concurrency CDP capabilities.\n\n## Usage\n\n```rust\nuse futures::StreamExt;\n\nuse chromiumoxide::browser::{Browser, BrowserConfig};\n\n#[tokio::main]\nasync fn main() -\u003e Result\u003c(), Box\u003cdyn std::error::Error\u003e\u003e {\n    \n   // create a `Browser` that spawns a `chromium` process running with UI (`with_head()`, headless is default) \n   // and the handler that drives the websocket etc.\n    let (mut browser, mut handler) =\n        Browser::launch(BrowserConfig::builder().with_head().build()?).await?;\n    \n   // spawn a new task that continuously polls the handler\n    let handle = tokio::task::spawn(async move {\n        while let Some(h) = handler.next().await {\n            if h.is_err() {\n                break;\n            }\n        }\n    });\n    \n   // create a new browser page and navigate to the url\n    let page = browser.new_page(\"https://en.wikipedia.org\").await?;\n    \n   // find the search bar type into the search field and hit `Enter`,\n   // this triggers a new navigation to the search result page\n   page.find_element(\"input#searchInput\")\n           .await?\n           .click()\n           .await?\n           .type_str(\"Rust programming language\")\n           .await?\n           .press_key(\"Enter\")\n           .await?;\n\n   let html = page.wait_for_navigation().await?.content().await?;\n   \n    browser.close().await?;\n    let _ = handle.await;\n    Ok(())\n}\n```\n\nThe current API still lacks some functionality, but the [`Page::execute`](src/page.rs) function allows sending all `chromiumoxide_types::Command` types (see [Generated Code](README.md#generated-code)). Most `Element` and `Page` functions are basically just simplified command constructions and combinations, like `Page::pdf`:\n\n```rust\npub async fn pdf(\u0026self, params: PrintToPdfParams) -\u003e Result\u003cVec\u003cu8\u003e\u003e {\n     let res = self.execute(params).await?;\n     Ok(base64::decode(\u0026res.data)?)\n }\n```\n\nIf you need something else, the `Page::execute` function allows for writing your own command wrappers. PRs are very welcome if you think a meaningful command is missing a designated function.\n\n### Add spider_chrome to your project\n\n`spider_chrome` comes with support for [`tokio`](https://github.com/tokio-rs/tokio) runtime. \n\n```toml\nspider_chrome = { version = \"2\", default-features = false }\n```\n\nThis configuration is made possible primarily by the websocket crate of choice: [`tokio-tungstenite`](https://github.com/snapview/tokio-tungstenite/tree/master).\n\n## Generated Code\n\nThe [`chromiumoxide_pdl`](chromiumoxide_pdl) crate contains a [PDL parser](chromiumoxide_pdl/src/pdl/parser.rs), which is a rust rewrite of a [python script in the chromium source tree]( https://chromium.googlesource.com/deps/inspector_protocol/+/refs/heads/master/pdl.py) and a [`Generator`](chromiumoxide_pdl/src/build/generator.rs) that turns the parsed PDL files into rust code. The [`chromiumoxide_cdp`](chromiumoxide_cdp) crate only purpose is to invoke the generator during its [build process](chromiumoxide_cdp/build.rs) and [include the generated output](chromiumoxide_cdp/src/lib.rs) before compiling the crate itself. This separation is done merely because the generated output is ~60K lines of rust code (not including all the proc macro expansions). So expect the compiling to take some time.\nThe generator can be configured and used independently, see [chromiumoxide_cdp/build.rs](chromiumoxide_cdp/build.rs).\n\nEvery chrome pdl domain is put in its own rust module, the types for the page domain of the browser_protocol are in `chromiumoxide_cdp::cdp::browser_protocol::page`, the runtime domain of the js_protocol in  `chromiumoxide_cdp::cdp::js_protocol::runtime` and so on.\n\n[vanilla.aslushnikov.com](https://vanilla.aslushnikov.com/) is a great resource to browse all the types defined in the pdl files. This site displays `Command` types as defined in the pdl files as `Method`. `chromiumoxid` sticks to the `Command` nomenclature. So for everything that is defined as a command type in the pdl (=marked as `Method` on [vanilla.aslushnikov.com](https://vanilla.aslushnikov.com/)) `chromiumoxide` contains a type for command and a designated type for the return type. For every command there is a `\u003cname of command\u003eParams` type with builder support (`\u003cname of command\u003eParams::builder()`) and its corresponding return type: `\u003cname of command\u003eReturns`. All commands share an implementation of the `chromiumoxide_types::Command` trait.\nAll Events are bundled in single enum (`CdpEvent`)\n\n## Fetcher\n\nBy default `spider_chrome` will try to find an installed version of chromium on the computer it runs on.\nIt is possible to download and install one automatically for some platforms using the `fetcher`.\n\nTher features are currently a bit messy due to a Cargo bug and will be changed once it is resolved.\nBased on your runtime and TLS configuration you should enable one of the following:\n- `_fetcher-rusttls-tokio`\n- `_fetcher-native-tokio`\n\n```rust\nuse std::path::Path;\n\nuse futures::StreamExt;\n\nuse chromiumoxide::browser::{BrowserConfig};\nuse chromiumoxide::fetcher::{BrowserFetcher, BrowserFetcherOptions};\n\n#[tokio::main]\nasync fn main() -\u003e Result\u003c(), Box\u003cdyn std::error::Error\u003e\u003e {\n    let download_path = Path::new(\"./download\");\n    tokio::fs::create_dir_all(\u0026download_path).await?;\n    let fetcher = BrowserFetcher::new(\n        BrowserFetcherOptions::builder()\n            .with_path(\u0026download_path)\n            .build()?,\n    );\n    let info = fetcher.fetch().await?;\n\n    let config = BrowserConfig::builder()\n        .chrome_executable(info.executable_path)\n        .build()?,\n}\n```\n\n## Known Issues\n\n* The rust files generated for the PDL files in [chromiumoxide_cdp](./chromiumoxide_cdp) don't compile when support for experimental types is manually turned off (`export CDP_NO_EXPERIMENTAL=true`). This is because the use of some experimental pdl types in the `*.pdl` files themselves are not marked as experimental.\n\n## Troubleshooting\n\nQ: A new chromium instance is being launched but then times out.\n\nA: Check that your chromium language settings are set to English. `chromiumoxide` tries to parse the debugging port from the chromium process output and that is limited to english.\n\n## License\n\nLicensed under either of these:\n\n * Apache License, Version 2.0, ([LICENSE-APACHE](LICENSE-APACHE) or\n   https://www.apache.org/licenses/LICENSE-2.0)\n * MIT license ([LICENSE-MIT](LICENSE-MIT) or\n   https://opensource.org/licenses/MIT)\n   \n\n## References\n\n* [chromedp](https://github.com/chromedp/chromedp)\n* [rust-headless-chrome](https://github.com/atroche/rust-headless-chrome) which the launch config, `KeyDefinition` and typing support among others is taken from.\n* [puppeteer](https://github.com/puppeteer/puppeteer)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fspider-rs%2Fspider_chrome","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fspider-rs%2Fspider_chrome","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fspider-rs%2Fspider_chrome/lists"}