{"id":18553700,"url":"https://github.com/websemantics/codepen-puppeteer","last_synced_at":"2025-04-09T23:30:48.840Z","repository":{"id":143798795,"uuid":"102597894","full_name":"websemantics/codepen-puppeteer","owner":"websemantics","description":"Use Puppeteer to download pens from Codepen.io as single html pages","archived":false,"fork":false,"pushed_at":"2022-06-24T00:45:16.000Z","size":2551,"stargazers_count":24,"open_issues_count":0,"forks_count":13,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-06T17:38:51.375Z","etag":null,"topics":["codepen","headless-chrome","puppeteer","web-scraping"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/websemantics.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-09-06T10:57:52.000Z","updated_at":"2024-07-05T10:39:35.000Z","dependencies_parsed_at":null,"dependency_job_id":"2c46baf1-1334-4c0c-aa1a-16c8d0f30e51","html_url":"https://github.com/websemantics/codepen-puppeteer","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/websemantics%2Fcodepen-puppeteer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/websemantics%2Fcodepen-puppeteer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/websemantics%2Fcodepen-puppeteer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/websemantics%2Fcodepen-puppeteer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/websemantics","download_url":"https://codeload.github.com/websemantics/codepen-puppeteer/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248129532,"owners_count":21052589,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["codepen","headless-chrome","puppeteer","web-scraping"],"created_at":"2024-11-06T21:18:01.594Z","updated_at":"2025-04-09T23:30:47.252Z","avatar_url":"https://github.com/websemantics.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"```\n╭─╮     ╭─╮     ╭┬╮     ╭─╮     ╭─╮     ╭─╮     ╭╮╭        ┬    ╭─╮\n│       │ │      ││     ├┤      ├─╯     ├┤      │││        │    │ │\n╰─╯     ╰─╯     ─┴╯     ╰─╯     ┴       ╰─╯     ╯╰╯    o   ┴    ╰─╯\n╭────╮  ╭──╮╭╮  ╭────╮  ╭────╮  ╭──▞─╮   ╭──╮   ╭────╮ ╭────╮ ╭─┬─╮                 \n│  ╭╮│  │  │││  │  ╭╮│  │  ╭╮│  │ `◯ │  ╭╯  ╰╮  │ ─  │ │ ─  │ │   │                 \n│  ╰╯│  │  ╰╯│  │  ╰╯│  │  ╰╯│  │    │  ╰╮  ╭╯  │    │ │    │ │  ╭╯                 \n│   ╭╯  │    │  │   ╭╯  │   ╭╯  │ ───┤   │ ─┤   │ ───┤ │ ───┤ │  │                  \n╰───╯   ╰───┴╯  ╰───╯   ╰───╯   ╰────╯   ╰──╯   ╰────╯ ╰────╯ ╰──╯  \n```\n\u003e Use Puppeteer to download pens from Codepen.io as single html pages.\n\n## Features\n\n- Download example pens as single html pages\n- Easy preview with an index page\n- Built-in error recovery to resume download\n- Skip already downloaded pens\n- Easy to debug using screenshots\n- Custom template pages\n- Easy to follow source code with comments\n- Support for loading external resources (i.e. `jquery`, `google fonts`)\n\n## Usage\n\n- Clone this project locally,\n\n```bash\ngit clone https://github.com/websemantics/codepen-puppeteer\ncd codepen-puppeteer\n```\n\n- Install dependencies (`puppeteer`),\n\n```bash\nnpm i\n```\n\nThere're two commands to interact with,\n\n1. `search` command to download pens matching search query\n\n```bash\npenpet search flexbox\n```\n\nYou can specify start and end page with `-s` and `-e` options\n\n- Browse to `./pens/index.html` to preview full list of downloads\n\n2. `file` command to download provided list of pens\n\n```bash\npenpet file pens.json\n```\n\nFile `pens.json` is provided as an example\n\n2. For examples and more help, use option `-h` with both commands\n\n## Debug\n\nThis project is a proof of concept so you might find problematic pens that wouldn't download fully. Turn the debug flag `-d` with the `file` command to enable screenshots which might help you debug the issue,\n\n```bash\npenpet file pens.json -d\n```\n\n## Hint\n\nI find the following command useful to force quit running `chromium` processes on OSX\n\n```\npkill -f -- \"chromium\"\n```\n\n## Preview Downloads\n\n\u003cimg alt=\"Codepen Puppeteer Preview Page\" width=\"850\" src=\"assets/img/preview.gif\" /\u003e\n\n## Resources\n\n- [Puppeteer - Headless Chrome Node API](https://github.com/GoogleChrome/puppeteer)\n- [Getting started with Puppeteer and Chrome Headless for Web Scraping](https://medium.com/@e_mad_ehsan/getting-started-with-puppeteer-and-chrome-headless-for-web-scrapping-6bf5979dee3e)\n\n## Support\n\nNeed help or have a question? post at [StackOverflow](https://stackoverflow.com/questions/tagged/codepen-puppeteer+websemantics).\n\n*Please don't use the issue trackers for support/questions.*\n\n*Star if you find this project useful, to show support or simply for being awesome :)*\n\n## Contribution\n\nContributions to this project are accepted in the form of feedback, bugs reports and even better - pull requests.\n\n## License\n\n[MIT license](http://opensource.org/licenses/mit-license.php) Copyright (c) Web Semantics, Inc.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwebsemantics%2Fcodepen-puppeteer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwebsemantics%2Fcodepen-puppeteer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwebsemantics%2Fcodepen-puppeteer/lists"}