{"id":13996018,"url":"https://github.com/lightpanda-io/browser","last_synced_at":"2025-05-13T00:05:07.617Z","repository":{"id":240268330,"uuid":"598667202","full_name":"lightpanda-io/browser","owner":"lightpanda-io","description":"Lightpanda: the headless browser designed for AI and automation","archived":false,"fork":false,"pushed_at":"2025-05-12T15:23:31.000Z","size":3170,"stargazers_count":8850,"open_issues_count":77,"forks_count":212,"subscribers_count":34,"default_branch":"main","last_synced_at":"2025-05-13T00:04:59.213Z","etag":null,"topics":["browser","browser-automation","cdp","headless","playwright","puppeteer","zig"],"latest_commit_sha":null,"homepage":"https://lightpanda.io","language":"Zig","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lightpanda-io.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-02-07T15:19:34.000Z","updated_at":"2025-05-12T19:24:36.000Z","dependencies_parsed_at":"2024-05-17T17:47:23.731Z","dependency_job_id":"27e373f8-5188-42ea-b925-f084f74b6ee5","html_url":"https://github.com/lightpanda-io/browser","commit_stats":{"total_commits":856,"total_committers":4,"mean_commits":214.0,"dds":"0.39135514018691586","last_synced_commit":"1e64513c16acce6c4d58a1b3c32b0e2d3c8201d2"},"previous_names":["lightpanda-io/browsercore","lightpanda-io/browser"],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lightpanda-io%2Fbrowser","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lightpanda-io%2Fbrowser/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lightpanda-io%2Fbrowser/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lightpanda-io%2Fbrowser/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lightpanda-io","download_url":"https://codeload.github.com/lightpanda-io/browser/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253843213,"owners_count":21972873,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["browser","browser-automation","cdp","headless","playwright","puppeteer","zig"],"created_at":"2024-08-09T14:03:44.083Z","updated_at":"2025-05-13T00:05:07.604Z","avatar_url":"https://github.com/lightpanda-io.png","language":"Zig","readme":"\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://lightpanda.io\"\u003e\u003cimg src=\"https://cdn.lightpanda.io/assets/images/logo/lpd-logo.png\" alt=\"Logo\" height=170\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n\u003ch1 align=\"center\"\u003eLightpanda Browser\u003c/h1\u003e\n\n\u003cp align=\"center\"\u003e\u003ca href=\"https://lightpanda.io/\"\u003elightpanda.io\u003c/a\u003e\u003c/p\u003e\n\n\u003cdiv align=\"center\"\u003e\n\n[![License](https://img.shields.io/github/license/lightpanda-io/browser)](https://github.com/lightpanda-io/browser/blob/main/LICENSE)\n[![Twitter Follow](https://img.shields.io/twitter/follow/lightpanda_io)](https://twitter.com/lightpanda_io)\n[![GitHub stars](https://img.shields.io/github/stars/lightpanda-io/browser)](https://github.com/lightpanda-io/browser)\n\n\u003c/div\u003e\n\nLightpanda is the open-source browser made for headless usage:\n\n- Javascript execution\n- Support of Web APIs (partial, WIP)\n- Compatible with Playwright, Puppeteer through CDP (WIP)\n\nFast web automation for AI agents, LLM training, scraping and testing:\n\n- Ultra-low memory footprint (9x less than Chrome)\n- Exceptionally fast execution (11x faster than Chrome)\n- Instant startup\n\n[\u003cimg width=\"350px\" src=\"https://cdn.lightpanda.io/assets/images/github/execution-time.svg\"\u003e\n](https://github.com/lightpanda-io/demo)\n\u0026emsp;\n[\u003cimg width=\"350px\" src=\"https://cdn.lightpanda.io/assets/images/github/memory-frame.svg\"\u003e\n](https://github.com/lightpanda-io/demo)\n\u003c/div\u003e\n\n_Puppeteer requesting 100 pages from a local website on a AWS EC2 m5.large instance.\nSee [benchmark details](https://github.com/lightpanda-io/demo)._\n\n## Quick start\n\n### Install from the nightly builds\n\nYou can download the last binary from the [nightly\nbuilds](https://github.com/lightpanda-io/browser/releases/tag/nightly) for\nLinux x86_64 and MacOS aarch64.\n\n*For Linux*\n```console\ncurl -L -o lightpanda https://github.com/lightpanda-io/browser/releases/download/nightly/lightpanda-x86_64-linux \u0026\u0026 \\\nchmod a+x ./lightpanda\n```\n\n*For MacOS*\n```console\ncurl -L -o lightpanda https://github.com/lightpanda-io/browser/releases/download/nightly/lightpanda-aarch64-macos \u0026\u0026 \\\nchmod a+x ./lightpanda\n```\n\n*For Windows + WSL2*\n\nThe Lightpanda browser is compatible to run on windows inside WSL. Follow the Linux instruction for installation from a WSL terminal.\nIt is recommended to install clients like Puppeteer on the Windows host.\n\n### Dump a URL\n\n```console\n./lightpanda fetch --dump https://lightpanda.io\n```\n```console\ninfo(browser): GET https://lightpanda.io/ http.Status.ok\ninfo(browser): fetch script https://api.website.lightpanda.io/js/script.js: http.Status.ok\ninfo(browser): eval remote https://api.website.lightpanda.io/js/script.js: TypeError: Cannot read properties of undefined (reading 'pushState')\n\u003c!DOCTYPE html\u003e\n```\n\n### Start a CDP server\n\n```console\n./lightpanda serve --host 127.0.0.1 --port 9222\n```\n```console\ninfo(websocket): starting blocking worker to listen on 127.0.0.1:9222\ninfo(server): accepting new conn...\n```\n\nOnce the CDP server started, you can run a Puppeteer script by configuring the\n`browserWSEndpoint`.\n\n```js\n'use strict'\n\nimport puppeteer from 'puppeteer-core';\n\n// use browserWSEndpoint to pass the Lightpanda's CDP server address.\nconst browser = await puppeteer.connect({\n  browserWSEndpoint: \"ws://127.0.0.1:9222\",\n});\n\n// The rest of your script remains the same.\nconst context = await browser.createBrowserContext();\nconst page = await context.newPage();\n\n// Dump all the links from the page.\nawait page.goto('https://wikipedia.com/');\n\nconst links = await page.evaluate(() =\u003e {\n  return Array.from(document.querySelectorAll('a')).map(row =\u003e {\n    return row.getAttribute('href');\n  });\n});\n\nconsole.log(links);\n\nawait page.close();\nawait context.close();\nawait browser.disconnect();\n```\n\n### Telemetry\nBy default, Lightpanda collects and sends usage telemetry. This can be disabled by setting an environment variable `LIGHTPANDA_DISABLE_TELEMETRY=true`. You can read Lightpanda's privacy policy at: [https://lightpanda.io/privacy-policy](https://lightpanda.io/privacy-policy).\n\n## Status\n\nLightpanda is still a work in progress and is currently at a Beta stage.\n\n:warning: You should expect most websites to fail or crash.\n\nHere are the key features we have implemented:\n\n- [x] HTTP loader\n- [x] HTML parser and DOM tree (based on Netsurf libs)\n- [x] Javascript support (v8)\n- [x] Basic DOM APIs\n- [x] Ajax\n  - [x] XHR API\n  - [x] Fetch API\n- [x] DOM dump\n- [x] Basic CDP/websockets server\n\nNOTE: There are hundreds of Web APIs. Developing a browser (even just for headless mode) is a huge task. Coverage will increase over time.\n\nYou can also follow the progress of our Javascript support in our dedicated [zig-js-runtime](https://github.com/lightpanda-io/zig-js-runtime#development) project.\n\n## Build from sources\n\n### Prerequisites\n\nLightpanda is written with [Zig](https://ziglang.org/) `0.14.0`. You have to\ninstall it with the right version in order to build the project.\n\nLightpanda also depends on\n[zig-js-runtime](https://github.com/lightpanda-io/zig-js-runtime/) (with v8),\n[Netsurf libs](https://www.netsurf-browser.org/) and\n[Mimalloc](https://microsoft.github.io/mimalloc).\n\nTo be able to build the v8 engine for zig-js-runtime, you have to install some libs:\n\nFor Debian/Ubuntu based Linux:\n\n```\nsudo apt install xz-utils \\\n    python3 ca-certificates git \\\n    pkg-config libglib2.0-dev \\\n    gperf libexpat1-dev \\\n    cmake clang\n```\n\nFor systems with [Nix](https://nixos.org/download/), you can use the devShell:\n```\nnix develop\n```\n\nFor MacOS, you only need cmake:\n\n```\nbrew install cmake\n```\n\n### Install and build dependencies\n\n#### All in one build\n\nYou can run `make install` to install deps all in one (or `make install-dev` if you need the development versions).\n\nBe aware that the build task is very long and cpu consuming, as you will build from sources all dependencies, including the v8 Javascript engine.\n\n#### Step by step build dependency\n\nThe project uses git submodules for dependencies.\n\nTo init or update the submodules in the `vendor/` directory:\n\n```\nmake install-submodule\n```\n\n**iconv**\n\nlibiconv is an internationalization library used by Netsurf.\n\n```\nmake install-libiconv\n```\n\n**Netsurf libs**\n\nNetsurf libs are used for HTML parsing and DOM tree generation.\n\n```\nmake install-netsurf\n```\n\nFor dev env, use `make install-netsurf-dev`.\n\n**Mimalloc**\n\nMimalloc is used as a C memory allocator.\n\n```\nmake install-mimalloc\n```\n\nFor dev env, use `make install-mimalloc-dev`.\n\nNote: when Mimalloc is built in dev mode, you can dump memory stats with the\nenv var `MIMALLOC_SHOW_STATS=1`. See\n[https://microsoft.github.io/mimalloc/environment.html](https://microsoft.github.io/mimalloc/environment.html).\n\n**v8**\n\nFirst, get the tools necessary for building V8, as well as the V8 source code:\n\n```\nmake get-v8\n```\n\nNext, build v8. This build task is very long and cpu consuming, as you will build v8 from sources.\n\n```\nmake build-v8\n```\n\nFor dev env, use `make build-v8-dev`.\n\n## Test\n\n### Unit Tests\n\nYou can test Lightpanda by running `make test`.\n\n### End to end tests\n\nTo run end to end tests, you need to clone the [demo\nrepository](https://github.com/lightpanda-io/demo) into `../demo` dir.\n\nYou have to install the [demo's node\nrequirements](https://github.com/lightpanda-io/demo?tab=readme-ov-file#dependencies-1)\n\nYou also need to install [Go](https://go.dev) \u003e v1.24.\n\n```\nmake end2end\n```\n\n### Web Platform Tests\n\nLightpanda is tested against the standardized [Web Platform\nTests](https://web-platform-tests.org/).\n\nThe relevant tests cases are committed in a [dedicated repository](https://github.com/lightpanda-io/wpt) which is fetched by the `make install-submodule` command.\n\nAll the tests cases executed are located in the `tests/wpt` sub-directory.\n\nFor reference, you can easily execute a WPT test case with your browser via\n[wpt.live](https://wpt.live).\n\n#### Run WPT test suite\n\nTo run all the tests:\n\n```\nmake wpt\n```\n\nOr one specific test:\n\n```\nmake wpt Node-childNodes.html\n```\n\n#### Add a new WPT test case\n\nWe add new relevant tests cases files when we implemented changes in Lightpanda.\n\nTo add a new test, copy the file you want from the [WPT\nrepo](https://github.com/web-platform-tests/wpt) into the `tests/wpt` directory.\n\n:warning: Please keep the original directory tree structure of `tests/wpt`.\n\n## Contributing\n\nLightpanda accepts pull requests through GitHub.\n\nYou have to sign our [CLA](CLA.md) during the pull request process otherwise\nwe're not able to accept your contributions.\n\n## Why?\n\n### Javascript execution is mandatory for the modern web\n\nIn the good old days, scraping a webpage was as easy as making an HTTP request, cURL-like. It’s not possible anymore, because Javascript is everywhere, like it or not:\n\n- Ajax, Single Page App, infinite loading, “click to display”, instant search, etc.\n- JS web frameworks: React, Vue, Angular \u0026 others\n\n### Chrome is not the right tool\n\nIf we need Javascript, why not use a real web browser? Take a huge desktop application, hack it, and run it on the server. Hundreds or thousands of instances of Chrome if you use it at scale. Are you sure it’s such a good idea?\n\n- Heavy on RAM and CPU, expensive to run\n- Hard to package, deploy and maintain at scale\n- Bloated, lots of features are not useful in headless usage\n\n### Lightpanda is built for performance\n\nIf we want both Javascript and performance in a true headless browser, we need to start from scratch. Not another iteration of Chromium, really from a blank page. Crazy right? But that’s what we did:\n\n- Not based on Chromium, Blink or WebKit\n- Low-level system programming language (Zig) with optimisations in mind\n- Opinionated: without graphical rendering\n","funding_links":[],"categories":["Zig","Network \u0026 Web","Repos","未分类","\u003ca name=\"Zig\"\u003e\u003c/a\u003eZig"],"sub_categories":["Browser"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flightpanda-io%2Fbrowser","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flightpanda-io%2Fbrowser","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flightpanda-io%2Fbrowser/lists"}