{"id":21592393,"url":"https://github.com/samuherek/backup-codepen-cli","last_synced_at":"2025-07-17T01:31:25.627Z","repository":{"id":40728924,"uuid":"237645913","full_name":"samuherek/backup-codepen-cli","owner":"samuherek","description":"CLI tool for backing up your codepen pens into files on your computer. (using puppeteer)","archived":false,"fork":false,"pushed_at":"2022-06-25T10:20:59.000Z","size":78,"stargazers_count":2,"open_issues_count":5,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-04-17T01:35:50.960Z","etag":null,"topics":["backup","cli","codepen","puppeteer","typescript"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/samuherek.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-02-01T16:50:20.000Z","updated_at":"2023-08-25T02:25:50.000Z","dependencies_parsed_at":"2022-08-31T08:51:38.967Z","dependency_job_id":null,"html_url":"https://github.com/samuherek/backup-codepen-cli","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/samuherek/backup-codepen-cli","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/samuherek%2Fbackup-codepen-cli","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/samuherek%2Fbackup-codepen-cli/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/samuherek%2Fbackup-codepen-cli/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/samuherek%2Fbackup-codepen-cli/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/samuherek","download_url":"https://codeload.github.com/samuherek/backup-codepen-cli/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/samuherek%2Fbackup-codepen-cli/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265558449,"owners_count":23787937,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["backup","cli","codepen","puppeteer","typescript"],"created_at":"2024-11-24T17:01:01.516Z","updated_at":"2025-07-17T01:31:25.622Z","avatar_url":"https://github.com/samuherek.png","language":"TypeScript","funding_links":[],"categories":["TypeScript"],"sub_categories":[],"readme":"# Backup Codepen.io CLI\n\n\u003e **This tool is still just WIP**. You have to clone it and run with ts-node directly on ./src/index.ts to get it working!\n\nIf you find yourself in a need to backup your codepen pens into real files on your computer, this cli tool can help you out.\n\nThe need behind this tool was for me to backup my private pens and delete them from codepen.\nI find myself to use codepen less and less because my development focused has changed.\nSo.... as much as I like codepen, my \"PRO\" account needs to retire.\n\n## What this tool can do for you\n\n- It can scrape basic info for public pens for specific profile\n- It can scrap pen's files and save them to your disk\n- _there is a helper script to help you get a list of your private pens in `scripts` folder_\n\n## What packages this tool uses\n\nThis CLI uses \"Typescript\" for code and \"Puppeteer\" for scraping. (Puppeteer, is a headless chrome browser which you can control pretend you are a real user).\n\n## How you get it to work\n\n1. Clone this repo\n2. `cd` into the cloned repo\n3. run `npm install`\n4. read the crappy docks below to see how to use it\n\n### Usage\n\n\u003e NOTE: Because this uses puppeteer, things can take a bit of time. Make sure you have more than just a couple of minutes if you have many pens.\n\nFirstly, you need to create a `json` file which has all the necessary data for this tool to know which pens to backup.\nThe minimum JSON structure needs to be in this format, to make it work properly.\n\n```json\n[\n  {\n    \"title\": \"I believe this not required, but it is used to show you progress in terminal\",\n    \"link\": \"https://codepen.io/ ...link to the pen\",\n    \"sourceFiles\": [\n      \"https://codepen.io/ ... link to the pen + the extensions (.html|.css|.js)\",\n      \"...\"\n    ]\n  }\n]\n```\n\n**There are two ways you can speed up the process of collecting the json data.**\n\n#### 1A. Use the script in google developer tool form \"scripts\" folder\n\nBecause you have to sign in to access your private pens, the quickest way to collect the pens, was to run a simple script in your Google Developer Console. This script will save the necessary info about the pens into the \"localStorage\". You can then read them from the local storage and copy/paste into a JSON file to be used for the CLI.\n\n**Here are the steps to follow:**\n\nNavigate to \"./scripts/collect-private-pens.js\". Select and copy the script into clipboard.\nThen navigate to codepen.io, and sign in. Next, navigate to your dashboard select private pens. Next, it's important you select the _List view_ so the pens show up as a list instead of grid. You can use the URL example from below if you don't know where it is.\n\n`https://codepen.io/${your_username}/pens/private?grid_type=list`\n\nYou have to do some manual work now. Open Google Developers Console (ALT + CMD + i).\nPaste the script you copied from the clipboard and hit \"ENTER\".\n\nNow you are ready to run `scrape()` in the console. You should see an output in the console as an \"array\" with some items in it. That means all went well. Now, let's do next pagination page if it exists.\n\nRun `next()` in the console. Now, make sure you wait until the next list of pens loads and appears on the screen (this might take a second). Once they are there, repeat the process of running `scrape()` and `next()` until you go through all your private pens.\n\nAs the last step, we need to copy all the pens saved in your localStorage and put then in a local JSON file.\n\nRun `JSON.stringify(JSON.parse(localStorage.getItem(\"cp\")), null, 2)` in the chrome console. It should output a big formated string with all the data. Now just copy it and paste it to a local JSON file.\n\n#### 1B. Use the CLI with command \"get-public\"\n\nImportant! This only works for collecting public pens, because we don't want to mess with all the authentication mess right now :D.\n\nYou can simply run this command from within the cloned repo:\n\n```cli\nts-node ./src/index.ts get-public -u USERNAME -o PATH_TO_OUTPUT_DIR\n```\n\n`USERNAME` -\u003e That is the username of the profile you want to scrape\n`PATH_TO_OUTPUT_FILE` -\u003e some directory path on your computer where the CLI needs to save this intermediate step with the list of links. It will save a file in there with name `scraped-public-data.json`\n\n#### 2. Intermediate step to get pen's info\n\nNow, when we have all the links collected, let's get extra information for each pen. Info like the preprocessor you used or what other extra \"packages\" are needed to run this pen.\n\nRun this command from within the cloned repo:\n\n```cli\nts-node ./src/index.ts get-source -i PATH_TO_JSON_FILE -o PATH_TO_OUTPUT_DIR\n```\n\n`PATH_TO_JSON_FILE` -\u003e this is the file we created in the step 1. (../scraped-public-data.json)\n`PATH_TO_OUTPUT_DIR` -\u003e this is the directory a new JSON file will be created with extra information about the pen. It will be called `scraped-with-settings-data.json`\n\n#### 3. Finally grab all the codepen files\n\nAfter all the hard work, the easy bit comes now.\nSimply run this command.\n\n```cli\nts-node ./src/index.ts backup -i PATH_TO_JSON_FILE -o PATH_TO_OUTPUT_DIR\n```\n\n`PATH_TO_JSON_FILE` -\u003e Important! This is the file from step 2. (not the one from step 1). (../scraped-with-settings-data.json)\n`PATH_TO_OUTPUT_DIR` -\u003e this it the directory all the codepen files will be put.\n\n#### The output will be\n\nThe final output will look like this, if you followed all the above steps;\n\n```\n/outputDir\n-- /penName/\n----- /index.html // if used in pen\n----- /index.haml // if used in pen\n----- /index.markdown // if used in pen\n----- /index.slim // if used in pen\n----- /index.pug // if used in pend\n----- /style.css // if used in pend\n----- /style.scss // if used in pend\n----- /style.sass // if used in pend\n----- /style.less // if used in pend\n----- /style.stylus // if used in pend\n----- /style.postcss // if used in pend\n----- /script.js // if used in pend\n----- /script.babel // if used in pend\n----- /script.typescript // if used in pend\n----- /script.coffeescript // if used in pend\n----- /script.livescript // if used in pend\n----- /config.json\n```\n\nThe _config.json_ file for each pen will look like this:\n\n```json\n{\n  \"title\": \"Pen title\",\n  \"link\": \"https://codepen.io/username/pen/penId\",\n  \"createdAt\": \"Date\",\n  \"updatedAt\": \"Date\",\n  \"hearts\": \"0\",\n  \"comments\": \"0\",\n  \"views\": \"0\",\n  \"settings\": {\n    \"html\": { \"preprocessor\": \"none\", \"head\": \"\" },\n    \"css\": { \"preprocessor\": \"scss\", \"resources\": [] },\n    \"js\": {\n      \"preprocessor\": \"babel\",\n      \"resources\": [\n        \"https://cdnjs.cloudflare.com/ajax/libs/react/15.6.1/react.min.js\",\n        \"https://cdnjs.cloudflare.com/ajax/libs/react/15.6.1/react-dom.min.js\",\n        \"https://unpkg.com/prop-types/prop-types.min.js\",\n        \"https://cdnjs.cloudflare.com/ajax/libs/immutable/3.8.1/immutable.js\",\n        \"https://cdnjs.cloudflare.com/ajax/libs/draft-js/0.10.1/Draft.js\",\n        \"https://cdnjs.cloudflare.com/ajax/libs/redux/3.6.0/redux.js\",\n        \"https://cdnjs.cloudflare.com/ajax/libs/react-redux/5.0.3/react-redux.js\"\n      ]\n    }\n  },\n  \"sourceFiles\": [\n    \"https://codepen.io/username/pen/penId.html\",\n    \"https://codepen.io/username/pen/penId.css\",\n    \"https://codepen.io/username/pen/penId.scss\",\n    \"https://codepen.io/username/pen/penId.js\",\n    \"https://codepen.io/username/pen/penId.babel\"\n  ]\n}\n```\n\n## Possible todo\n\n- [ ] Collect the images for each pen as well\n- [ ] Make a simple online service that would do all the work\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsamuherek%2Fbackup-codepen-cli","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsamuherek%2Fbackup-codepen-cli","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsamuherek%2Fbackup-codepen-cli/lists"}