{"id":13433916,"url":"https://github.com/kiliman/tailwindui-crawler","last_synced_at":"2025-05-15T04:00:15.827Z","repository":{"id":37871154,"uuid":"245535037","full_name":"kiliman/tailwindui-crawler","owner":"kiliman","description":"tailwindui-crawler downloads the component HTML files locally","archived":false,"fork":false,"pushed_at":"2025-03-28T17:41:34.000Z","size":1172,"stargazers_count":795,"open_issues_count":11,"forks_count":108,"subscribers_count":15,"default_branch":"master","last_synced_at":"2025-05-10T03:45:18.263Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kiliman.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-03-06T23:27:53.000Z","updated_at":"2025-04-30T09:48:05.000Z","dependencies_parsed_at":"2024-12-13T17:08:07.327Z","dependency_job_id":"102be582-09d0-44fe-b7fe-c3847f992538","html_url":"https://github.com/kiliman/tailwindui-crawler","commit_stats":null,"previous_names":[],"tags_count":27,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kiliman%2Ftailwindui-crawler","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kiliman%2Ftailwindui-crawler/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kiliman%2Ftailwindui-crawler/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kiliman%2Ftailwindui-crawler/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kiliman","download_url":"https://codeload.github.com/kiliman/tailwindui-crawler/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254270640,"owners_count":22042858,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T02:01:40.300Z","updated_at":"2025-05-15T04:00:15.775Z","avatar_url":"https://github.com/kiliman.png","language":"JavaScript","funding_links":[],"categories":["JavaScript","🤖 AI \u0026 Machine Learning"],"sub_categories":[],"readme":"# tailwindui-crawler\n\n\u003c!-- ALL-CONTRIBUTORS-BADGE:START - Do not remove or modify this section --\u003e\n\n[![All Contributors](https://img.shields.io/badge/all_contributors-10-orange.svg?style=flat-square)](#contributors-)\n\n\u003c!-- ALL-CONTRIBUTORS-BADGE:END --\u003e\n\n\u003cimg src=\"./images/tailwindui-crawler.png\"\u003e\n\nThis script will crawl the [tailwindcss.com/plus](https://tailwindcss.com/plus) website and download all the\ncomponents to the `./output` folder.\n\n## 🛠 How to use\n\nTo install, clone this repo and run `yarn` or `npm install` to pull down the dependencies.\n\nThen create a `.env` file with your email, password, and optional output folder.\n\n```ini\nEMAIL=youremail\nPASSWORD=yourpassword\n# OUTPUT optional, defaults to ./output\nOUTPUT=/path/to/output\n# LANGUAGES defaults to html\nLANGUAGES=html,react,vue,alpine\n# COMPONENTS commma-delimited list (defaults to all)\nCOMPONENTS=(all|marketing,application-ui,ecommerce)\n# BUILDINDEX generate index file to view components offline\nBUILDINDEX=(0 | 1)\n# TEMPLATES download template files\nTEMPLATES=(0 | 1)\n# FORCE_UPDATE force update existing components\nFORCE_UPDATE=(0 | 1)\n# DEBUG enable debug logging\nDEBUG=(0 | 1)\n```\n\n\u003e NOTE: The tool uses [dotenv-expand](https://github.com/motdotla/dotenv-expand)\n\u003e to support variable expansion like `$HOME/path/to/output` so if your password\n\u003e or any other value includes a `$`, make sure you add a `\\` (backslash) to\n\u003e escape the `$`. For example, `PASSWORD=p@\\$\\$w0rd`\n\u003e\n\u003e Also, `dotenv` does **not** support inline comments, so do not do something\n\u003e like `LANGUAGES=html,react,vue # some comment` as it will not get the correct\n\u003e values.\n\nThen finally, run `yarn start` or `npm start`\n\nThe script will login to [tailwindcss.com/plus](https://tailwindcss.com/plus) with your credentials, and download all the\ncomponents as individual files in the `./output` folder.\n\n## 🤔 What's it for?\n\nThe benefit of pulling down all the components is that you can commit them to a local or\nprivate repo, and by running this periodically, you can see exactly which files were added\nor changed. Hopefully, some time in the future, they will open up a private repo for those\nthat have purchased the library.\n\n## 🚀 New v5.0\n\nThe crawler has been updated to support the new Tailwind Plus site structure (tailwindcss.com/plus) as of 2024. It includes support for downloading all marketing UI blocks and properly handles the new component structure.\n\n### New Features:\n\n- Support for the new Tailwind Plus URL structure\n- Enhanced component naming for React components (generates descriptive names based on the component path)\n- Added `FORCE_UPDATE` option to overwrite existing components\n- Improved debugging with the `DEBUG` environment variable\n- Component skipping to avoid re-downloading existing components\n- Detailed summary statistics after download completes\n\n### 🗂 Preview page\n\nYou can set the `.env` key `BUILDINDEX=1` to have the crawler generate an index file similar to the components\npage on [tailwindcss.com/plus](https://tailwindcss.com/plus). Install and run the [serve](https://www.npmjs.com/package/serve) package\nto view the index.\n\n\u003e NOTE: The HTML Preview does not apply transformers. It's a copy of the\n\u003e component site on [tailwindcss.com/plus](https://tailwindcss.com/plus).\n\n```bash\nyarn global add serve\ncd $OUTPUT/preview # change to your preview folder\nserve\n```\n\n### ⚙️ Example `.env` file\n\n```ini\nEMAIL=******\nPASSWORD=******\nOUTPUT=$HOME/Projects/tailwindui\nLANGUAGES=html,react,vue,alpine\nCOMPONENTS=marketing,application-ui\nBUILDINDEX=1\nTEMPLATES=1\nFORCE_UPDATE=0\nDEBUG=0\n```\n\n## 🤖 Automatically keep a **private** GitHub Repository up-to-date\n\n**NOTE**: GitHub action has been updated in v4.0.0. Please make sure your _default.yml_ file is updated with the latest actions.\n\nYou can automatically keep a **private** GitHub repository up-to-date with component changes from TailwindUI by using this tool with GitHub Actions.\n\n1. [Create a **private** GitHub repository](https://github.com/new/).\n1. [Add `TAILWINDUI_EMAIL` and `TAILWINDUI_PASSWORD` secrets to the GitHub repository](https://help.github.com/en/actions/configuring-and-managing-workflows/creating-and-storing-encrypted-secrets#creating-encrypted-secrets).\n1. [Optionally create a `.env` file with additional settings for the crawler](#%EF%B8%8F-example-env-file).\n1. Create a new file `.github/workflows/default.yml`:\n\n   ```yml\n   name: Update\n   on:\n     schedule:\n       - cron: '0 0 * * *' # Every day at midnight\n\n   jobs:\n     update:\n       name: Update\n       runs-on: ubuntu-latest\n       steps:\n         - name: Checkout\n           uses: actions/checkout@v2\n         - name: Run crawler\n           uses: kiliman/tailwindui-crawler-action@v1.3.0\n           with:\n             email: ${{ secrets.TAILWINDUI_EMAIL }}\n             password: ${{ secrets.TAILWINDUI_PASSWORD }}\n   ```\n\n   \u003e NOTE: Make sure to update to the latest action `v1.1.0` to support the crawler v3+\n\n   Read more about the schedule cron syntax in [the official GitHub Actions documentation](https://help.github.com/en/actions/reference/events-that-trigger-workflows#scheduled-events-schedule).\n\n   \u003e NOTE: if you're creating a new repository or have updated your default branch from `master`, you will have to specify with `branch` (and/or `current_branch`) like so:\n\n   ```diff\n   # ...\n         - name: Run crawler\n           uses: kiliman/tailwindui-crawler-action@v1.3.0\n           with:\n             email: ${{ secrets.TAILWINDUI_EMAIL }}\n             password: ${{ secrets.TAILWINDUI_PASSWORD }}\n   +         branch: main\n   +         current_branch: main\n   ```\n\n### Email Notifications\n\nTo be emailed whenever there is a change to a component, simply setup [GitHub Notifications](https://help.github.com/en/github/administering-a-repository/about-email-notifications-for-pushes-to-your-repository#enabling-email-notifications-for-pushes-to-your-repository) on your repository.\n\n## 🚦 Upgrading to v5.\n\nThis is a major change that supports the new Tailwind Plus site structure. The crawler now downloads components from [tailwindcss.com/plus](https://tailwindcss.com/plus) instead of the old tailwindui.com site.\n\nNew features:\n\n- Better React component naming based on component paths\n- Skipping existing components for faster updates\n- Option to force update with FORCE_UPDATE=1\n- Enhanced debug output with DEBUG=1\n- Support for marketing components and all UI blocks\n\nNOTE: Since this script is essentially screen scraping, there's the potential\nof it breaking if the HTML structure changes. I will do my best to keep it in sync with\nthe website.\n\n## 😍 Thank you\n\nThanks to Adam and Steve for making an amazing library. This has definitely made creating\na UI for my applications a pleasant experience.\n\nEnjoy and let me know if you have any questions.\n\nKiliman\n\n## Contributors ✨\n\nThanks goes to these wonderful people ([emoji key](https://allcontributors.org/docs/en/emoji-key)):\n\n\u003c!-- ALL-CONTRIBUTORS-LIST:START - Do not remove or modify this section --\u003e\n\u003c!-- prettier-ignore-start --\u003e\n\u003c!-- markdownlint-disable --\u003e\n\u003ctable\u003e\n  \u003ctbody\u003e\n    \u003ctr\u003e\n      \u003ctd align=\"center\" valign=\"top\" width=\"14.28%\"\u003e\u003ca href=\"https://github.com/kiliman\"\u003e\u003cimg src=\"https://avatars3.githubusercontent.com/u/47168?v=4?s=100\" width=\"100px;\" alt=\"Kiliman\"/\u003e\u003cbr /\u003e\u003csub\u003e\u003cb\u003eKiliman\u003c/b\u003e\u003c/sub\u003e\u003c/a\u003e\u003cbr /\u003e\u003ca href=\"https://github.com/kiliman/tailwindui-crawler/commits?author=kiliman\" title=\"Code\"\u003e💻\u003c/a\u003e\u003c/td\u003e\n      \u003ctd align=\"center\" valign=\"top\" width=\"14.28%\"\u003e\u003ca href=\"http://www.cemfi.de\"\u003e\u003cimg src=\"https://avatars0.githubusercontent.com/u/8217108?v=4?s=100\" width=\"100px;\" alt=\"Simon Waloschek\"/\u003e\u003cbr /\u003e\u003csub\u003e\u003cb\u003eSimon Waloschek\u003c/b\u003e\u003c/sub\u003e\u003c/a\u003e\u003cbr /\u003e\u003ca href=\"https://github.com/kiliman/tailwindui-crawler/commits?author=sonovice\" title=\"Code\"\u003e💻\u003c/a\u003e\u003c/td\u003e\n      \u003ctd align=\"center\" valign=\"top\" width=\"14.28%\"\u003e\u003ca href=\"https://github.com/nawok\"\u003e\u003cimg src=\"https://avatars3.githubusercontent.com/u/159773?v=4?s=100\" width=\"100px;\" alt=\"Pavel Fomchenkov\"/\u003e\u003cbr /\u003e\u003csub\u003e\u003cb\u003ePavel Fomchenkov\u003c/b\u003e\u003c/sub\u003e\u003c/a\u003e\u003cbr /\u003e\u003ca href=\"https://github.com/kiliman/tailwindui-crawler/commits?author=nawok\" title=\"Code\"\u003e💻\u003c/a\u003e\u003c/td\u003e\n      \u003ctd align=\"center\" valign=\"top\" width=\"14.28%\"\u003e\u003ca href=\"https://robinmalfait.com\"\u003e\u003cimg src=\"https://avatars2.githubusercontent.com/u/1834413?v=4?s=100\" width=\"100px;\" alt=\"Robin Malfait\"/\u003e\u003cbr /\u003e\u003csub\u003e\u003cb\u003eRobin Malfait\u003c/b\u003e\u003c/sub\u003e\u003c/a\u003e\u003cbr /\u003e\u003ca href=\"https://github.com/kiliman/tailwindui-crawler/commits?author=RobinMalfait\" title=\"Code\"\u003e💻\u003c/a\u003e\u003c/td\u003e\n      \u003ctd align=\"center\" valign=\"top\" width=\"14.28%\"\u003e\u003ca href=\"https://miguelpiedrafita.com\"\u003e\u003cimg src=\"https://avatars0.githubusercontent.com/u/23558090?v=4?s=100\" width=\"100px;\" alt=\"Miguel Piedrafita\"/\u003e\u003cbr /\u003e\u003csub\u003e\u003cb\u003eMiguel Piedrafita\u003c/b\u003e\u003c/sub\u003e\u003c/a\u003e\u003cbr /\u003e\u003ca href=\"https://github.com/kiliman/tailwindui-crawler/commits?author=m1guelpf\" title=\"Code\"\u003e💻\u003c/a\u003e \u003ca href=\"https://github.com/kiliman/tailwindui-crawler/commits?author=m1guelpf\" title=\"Documentation\"\u003e📖\u003c/a\u003e \u003ca href=\"#ideas-m1guelpf\" title=\"Ideas, Planning, \u0026 Feedback\"\u003e🤔\u003c/a\u003e\u003c/td\u003e\n      \u003ctd align=\"center\" valign=\"top\" width=\"14.28%\"\u003e\u003ca href=\"https://github.com/vladdu\"\u003e\u003cimg src=\"https://avatars0.githubusercontent.com/u/9707?v=4?s=100\" width=\"100px;\" alt=\"Vlad Dumitrescu\"/\u003e\u003cbr /\u003e\u003csub\u003e\u003cb\u003eVlad Dumitrescu\u003c/b\u003e\u003c/sub\u003e\u003c/a\u003e\u003cbr /\u003e\u003ca href=\"https://github.com/kiliman/tailwindui-crawler/commits?author=vladdu\" title=\"Documentation\"\u003e📖\u003c/a\u003e\u003c/td\u003e\n      \u003ctd align=\"center\" valign=\"top\" width=\"14.28%\"\u003e\u003ca href=\"https://github.com/vesper8\"\u003e\u003cimg src=\"https://avatars1.githubusercontent.com/u/816028?v=4?s=100\" width=\"100px;\" alt=\"C-Bass\"/\u003e\u003cbr /\u003e\u003csub\u003e\u003cb\u003eC-Bass\u003c/b\u003e\u003c/sub\u003e\u003c/a\u003e\u003cbr /\u003e\u003ca href=\"https://github.com/kiliman/tailwindui-crawler/commits?author=vesper8\" title=\"Code\"\u003e💻\u003c/a\u003e\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003ctd align=\"center\" valign=\"top\" width=\"14.28%\"\u003e\u003ca href=\"https://gregbrimble.com/\"\u003e\u003cimg src=\"https://avatars.githubusercontent.com/u/8484333?v=4?s=100\" width=\"100px;\" alt=\"Greg Brimble\"/\u003e\u003cbr /\u003e\u003csub\u003e\u003cb\u003eGreg Brimble\u003c/b\u003e\u003c/sub\u003e\u003c/a\u003e\u003cbr /\u003e\u003ca href=\"https://github.com/kiliman/tailwindui-crawler/commits?author=GregBrimble\" title=\"Documentation\"\u003e📖\u003c/a\u003e \u003ca href=\"#tool-GregBrimble\" title=\"Tools\"\u003e🔧\u003c/a\u003e\u003c/td\u003e\n      \u003ctd align=\"center\" valign=\"top\" width=\"14.28%\"\u003e\u003ca href=\"https://github.com/yagnik\"\u003e\u003cimg src=\"https://avatars.githubusercontent.com/u/777296?v=4?s=100\" width=\"100px;\" alt=\"Yagnik\"/\u003e\u003cbr /\u003e\u003csub\u003e\u003cb\u003eYagnik\u003c/b\u003e\u003c/sub\u003e\u003c/a\u003e\u003cbr /\u003e\u003ca href=\"https://github.com/kiliman/tailwindui-crawler/commits?author=yagnik\" title=\"Code\"\u003e💻\u003c/a\u003e\u003c/td\u003e\n      \u003ctd align=\"center\" valign=\"top\" width=\"14.28%\"\u003e\u003ca href=\"https://debeijer.io/\"\u003e\u003cimg src=\"https://avatars.githubusercontent.com/u/71566757?v=4?s=100\" width=\"100px;\" alt=\"idebeijer\"/\u003e\u003cbr /\u003e\u003csub\u003e\u003cb\u003eidebeijer\u003c/b\u003e\u003c/sub\u003e\u003c/a\u003e\u003cbr /\u003e\u003ca href=\"https://github.com/kiliman/tailwindui-crawler/commits?author=idebeijer\" title=\"Code\"\u003e💻\u003c/a\u003e\u003c/td\u003e\n    \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\n\u003c!-- markdownlint-restore --\u003e\n\u003c!-- prettier-ignore-end --\u003e\n\n\u003c!-- ALL-CONTRIBUTORS-LIST:END --\u003e\n\nThis project follows the [all-contributors](https://github.com/all-contributors/all-contributors) specification. Contributions of any kind welcome!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkiliman%2Ftailwindui-crawler","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkiliman%2Ftailwindui-crawler","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkiliman%2Ftailwindui-crawler/lists"}