{"id":15641578,"url":"https://github.com/cicirello/generate-sitemap","last_synced_at":"2025-04-16T02:22:13.222Z","repository":{"id":38082070,"uuid":"284034073","full_name":"cicirello/generate-sitemap","owner":"cicirello","description":"Generate an XML sitemap for a GitHub Pages site using GitHub Actions","archived":false,"fork":false,"pushed_at":"2024-09-27T19:19:10.000Z","size":499,"stargazers_count":85,"open_issues_count":0,"forks_count":16,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-29T04:09:39.662Z","etag":null,"topics":["actions","github-actions","github-pages","html","lastmod","noindex","robots","robots-exclusion-protocol","site","sitemap","sitemap-generator","webpage","website","workflows","xml","xml-sitemap"],"latest_commit_sha":null,"homepage":"https://actions.cicirello.org/generate-sitemap/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cicirello.png","metadata":{"funding":{"github":"cicirello","patreon":null,"open_collective":null,"ko_fi":"cicirello","tidelift":null,"community_bridge":null,"liberapay":"cicirello","issuehunt":null,"otechie":null,"custom":null},"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-07-31T12:41:02.000Z","updated_at":"2025-03-21T13:43:25.000Z","dependencies_parsed_at":"2023-12-12T00:54:07.405Z","dependency_job_id":"ebf1282d-b091-4728-890c-6dc22518e376","html_url":"https://github.com/cicirello/generate-sitemap","commit_stats":{"total_commits":349,"total_committers":4,"mean_commits":87.25,"dds":0.08022922636103147,"last_synced_commit":"0166125c2a3b8e7c12460a19f9043ed3f01553e8"},"previous_names":[],"tags_count":25,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cicirello%2Fgenerate-sitemap","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cicirello%2Fgenerate-sitemap/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cicirello%2Fgenerate-sitemap/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cicirello%2Fgenerate-sitemap/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cicirello","download_url":"https://codeload.github.com/cicirello/generate-sitemap/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249183817,"owners_count":21226264,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["actions","github-actions","github-pages","html","lastmod","noindex","robots","robots-exclusion-protocol","site","sitemap","sitemap-generator","webpage","website","workflows","xml","xml-sitemap"],"created_at":"2024-10-03T11:43:05.136Z","updated_at":"2025-04-16T02:22:13.197Z","avatar_url":"https://github.com/cicirello.png","language":"Python","funding_links":["https://github.com/sponsors/cicirello","https://ko-fi.com/cicirello","https://liberapay.com/cicirello"],"categories":["GitHub"],"sub_categories":[],"readme":"# generate-sitemap\n\n[![cicirello/generate-sitemap - Generate XML sitemaps for static websites in GitHub Actions](https://actions.cicirello.org/images/generate-sitemap640.png)](#generate-sitemap)\n\nCheck out all of our GitHub Actions: https://actions.cicirello.org/\n\n## About\n\n| __GitHub Actions__ | [![GitHub release (latest by date)](https://img.shields.io/github/v/release/cicirello/generate-sitemap?label=Marketplace\u0026logo=GitHub)](https://github.com/marketplace/actions/generate-sitemap) [![Count of Action Users](https://badgen.net/github/dependents-repo/cicirello/generate-sitemap?icon=github\u0026label=used%20by)](https://github.com/cicirello/generate-sitemap/network/dependents) |\n| :--- | :--- |\n| __Build Status__ | [![build](https://github.com/cicirello/generate-sitemap/actions/workflows/build.yml/badge.svg)](https://github.com/cicirello/generate-sitemap/actions/workflows/build.yml) [![CodeQL](https://github.com/cicirello/generate-sitemap/actions/workflows/codeql-analysis.yml/badge.svg)](https://github.com/cicirello/generate-sitemap/actions/workflows/codeql-analysis.yml) |\n| __Source Info__ | [![GitHub](https://img.shields.io/github/license/cicirello/generate-sitemap)](https://github.com/cicirello/generate-sitemap/blob/master/LICENSE) [![GitHub top language](https://img.shields.io/github/languages/top/cicirello/generate-sitemap)](https://github.com/cicirello/generate-sitemap) |\n| __Support__ | [![GitHub Sponsors](https://img.shields.io/badge/sponsor-30363D?logo=GitHub-Sponsors\u0026logoColor=#EA4AAA)](https://github.com/sponsors/cicirello) [![Liberapay](https://img.shields.io/badge/Liberapay-F6C915?logo=liberapay\u0026logoColor=black)](https://liberapay.com/cicirello) [![Ko-Fi](https://img.shields.io/badge/Ko--fi-F16061?logo=ko-fi\u0026logoColor=white)](https://ko-fi.com/cicirello) |\n\nThe generate-sitemap GitHub action generates a sitemap for a website hosted on GitHub\nPages, and has the following features:\n* Support for both xml and txt sitemaps (you choose using one of the action's inputs). \n* When generating an xml sitemap, it uses the last commit date of \n  each file to generate the `\u003clastmod\u003e` tag in the sitemap entry. If the file\n  was created during that workflow run, but not yet committed, then it instead uses\n  the current date (however, we recommend if possible committing newly created files first).\n* Supports URLs for html and pdf files in the sitemap, and has inputs \n  to control the included file types (defaults include both html and pdf files in the sitemap).\n* Now also supports including URLs for a user specified list of \n  additional file extensions in the sitemap. \n* Checks content of html files for `\u003cmeta name=\"robots\" content=\"noindex\"\u003e` \n  directives, excluding any that do from the sitemap. \n* Parses a robots.txt, if present at the root of the website, excluding \n  any URLs from the sitemap that match `Disallow:` rules for `User-agent: *`.\n* Enables specifying a list of directories and/or specific files to exclude from\n  the sitemap.\n* Sorts the sitemap entries in a consistent order, such that the URLs are \n  first sorted by depth in the directory structure (i.e., pages at the website \n  root appear first, etc), and then pages at the same depth are sorted alphabetically. \n* It assumes that for files with the name `index.html` that the preferred URL for the page\n  ends with the enclosing directory, leaving out the `index.html`. For example,\n  instead of `https://WEBSITE/PATH/index.html`, the sitemap will contain\n  `https://WEBSITE/PATH/` in such a case. \n* Provides option to exclude `.html` extension from URLs listed in sitemap.   \n\nThe generate-sitemap GitHub action is designed to be used \nin combination with other GitHub Actions. For example, it \ndoes not commit and push the generated sitemap. See \nthe [Examples](#examples) for examples of combining\nwith other actions in your workflow.\n\nThe generate-sitemap action is for GitHub Pages sites,\nsuch that the repository contains the html, etc of the \nsite itself, regardless of whether or not the html was \ngenerated by a static site generator or written by \nhand. For example, I use it for multiple Java project \ndocumentation sites, where most of the site is generated \nby javadoc. I also use it with my personal website, which \nis generated with a custom static site generator. As long as\nthe repository for the GitHub Pages site contains the\nsite as served (e.g., html files, pdf files, etc), the \ngenerate-sitemap action is applicable.\n\nThe generate-sitemap action is not for GitHub Pages \nJekyll sites (unless you generate the site locally and \npush the html output instead of the markdown, but why would \nyou do that?). In the case of a GitHub Pages Jekyll site, \nthe repository contains markdown, and not the html that \nis generated from the markdown. The generate-sitemap action \ndoes not support that use-case. If you are looking to generate \na sitemap for a Jekyll website, there is \na [Jekyll plugin](https://github.com/jekyll/jekyll-sitemap) for that. \n\n## Table of Contents\n\nThe remainder of the documentation is organized into the following\nsections:\n* [Requirements](#requirements): Explanation of a pre-requisite step of any workflow\n  using the action.\n* [Inputs](#inputs): Documentation of all of the actions's inputs.\n* [Outputs](#outputs): Documentation of all of the actions's outputs.\n* [Examples](#examples): Several example workflows illustrating various features.\n* [Real Examples From Projects Using the Action](#real-examples-from-projects-using-the-action)\n* [Built With](#built-with): A list of languages, tools, etc used to develop this action.\n* [Blog Posts](#blog-posts): A selection of blog posts about the GitHub Action.\n* [Support the Project](#support-the-project): Information on various ways that you can support\n  the project.\n\n## Requirements\n\nThis action relies on `actions/checkout@v2` with `fetch-depth: 0`.\nSetting the `fetch-depth` to 0 for the checkout action ensures\nthat the `generate-sitemap` action will have access to the commit\nhistory, which is used for generating the `\u003clastmod\u003e` tags in the\n`sitemap.xml` file.  If you instead use the default when applying the\ncheckout action, the `\u003clastmod\u003e` tags will be incorrect.  So be\nsure to include the following as a step in your workflow:\n\n```yml\n    steps:\n    - name: Checkout the repo\n      uses: actions/checkout@v4\n      with:\n        fetch-depth: 0 \n```\n\n## Inputs\n\n### `path-to-root`\n\nThe path to the root of the website relative to the\nroot of the repository. Default `.` is appropriate in most cases,\nsuch as whenever the root of your Pages site is the root of the\nrepository itself. If you are using this for a GitHub Pages site\nin the `docs` directory, such as for a documentation website, then\njust pass `docs` for this input.\n\n### `base-url-path`\n\nThis is the url to your website. You must specify this\nfor your sitemap to be meaningful.  It defaults\nto `https://web.address.of.your.nifty.website/` for demonstration\npurposes.\n\n### `include-html`\n\nThis flag determines whether html files are included in\nyour sitemap (files with an extension of either `.html` \nor `.htm`). Default: `true`.\n\n### `include-pdf`\n\nThis flag determines whether pdf files are included in\nyour sitemap. Default: `true`.\n\n### `additional-extensions`\n\nIf you want to include URLs to other document types, you can use\nthe `additional-extensions` input to specify a list (separated by\nspaces) of file extensions. For example, Google (and other search\nengines) index a variety of other file types, including `docx`, `doc`,\nsource code for various common programming languages, etc. Here\nis an example:\n\n```yml\n    - name: Generate the sitemap\n      uses: cicirello/generate-sitemap@v1\n      with:\n        additional-extensions: doc docx ppt pptx\n```\n\n### `exclude-paths`\n\nThe action will automatically exclude any files or directories\nbased on a robots.txt file, if present. But if you have additional\ndirectories or individual files that you wish to exclude from the\nsitemap that are not otherwise blocked, you can use the `exclude-paths`\ninput to specify a list of them, separated by any whitespace characters. \nFor example, if you wish to exclude the directory `/exclude-these` as \nwell as the individual file `/nositemap.html`, you can use the following:\n\n```yml\n    - name: Generate the sitemap\n      uses: cicirello/generate-sitemap@v1\n      with:\n        exclude-paths: /exclude-these /nositemap.html\n```\n\nIf you have many such cases to exclude, your workflow may be easier to\nread if you use a YAML multi-line string, with the following:\n\n```yml\n    - name: Generate the sitemap\n      uses: cicirello/generate-sitemap@v1\n      with:\n        exclude-paths: \u003e\n          /exclude-these \n          /nositemap.html\n```\n\n### `sitemap-format`\n\nUse this to specify the sitemap format. Default: `xml`.\nThe `sitemap.xml` generated by the default will contain lastmod dates\nthat are generated using the last commit dates of each file. Setting \nthis input to anything other than `xml` will generate a plain text \n`sitemap.txt` simply listing the urls.\n\n### `drop-html-extension`\n\nThe `drop-html-extension` input provides the option to exclude `.html` extension \nfrom URLs listed in the sitemap. The default is `drop-html-extension: false`.  If\nyou want to use this option, just pass `drop-html-extension: true` to the action in\nyour workflow. GitHub Pages automatically serves the \ncorresponding html file if URL has no file extension. For example, if a user\nof your site browses to the URL, `https://WEBSITE/PATH/filename` (with no extension),\nGitHub Pages automatically serves `https://WEBSITE/PATH/filename.html` if it exists.\nThe default behavior of the `generate-sitemap` action includes the `.html` extension\nfor pages where the filename has the `.html` extension. If you prefer to exclude the\n`.html` extension from the URLs in your sitemap, then \npass `drop-html-extension: true` to the action in your workflow. \nNote that you should also ensure that any canonical links that you list within\nthe html files corresponds to your choice here.\n\n### `date-only`\n\nThe `date-only` input controls whether XML sitemaps include the full date and time in lastmod,\nor only the date. The default is `date-only: false`, which includes the full date and time\nin the lastmod fields. If you only want the date in the lastmod, then use `date-only: true`.\n\n## Outputs\n\n### `sitemap-path`\n\nThe generated sitemap is placed in the root of the website. This \noutput is the path to the generated sitemap file relative to the\nroot of the repository. If you didn't use the `path-to-root` input, then\nthis output should simply be the name of the sitemap file (`sitemap.xml`\nor `sitemap.txt`).\n\n### `url-count`\n\nThis output provides the number of URLs in the sitemap.\n\n### `excluded-count`\n\nThis output provides the number of URLs excluded from the sitemap due\nto either `\u003cmeta name=\"robots\" content=\"noindex\"\u003e` within html files,\nor due to exclusion from directives in a `robots.txt` file.\n\n## Examples\n\n### Basic Action Syntax\n\nYou can run the action with a step in your workflow like this:\n\n```yml\n    - name: Generate the sitemap\n      uses: cicirello/generate-sitemap@v1\n      with:\n        base-url-path: https://THE.URL.TO.YOUR.PAGE/\n```\n\nIn the above example, the major release version was used, which ensures that you'll\nbe using the latest patch level release, including any bug fixes, etc. If you prefer,\nyou can also use a specific version such as with:\n\n```yml\n    - name: Generate the sitemap\n      uses: cicirello/generate-sitemap@v1.10.1\n      with:\n        base-url-path: https://THE.URL.TO.YOUR.PAGE/\n```\n\n### Example 1: Minimal Example\n\nIn this example workflow, we use all of the default inputs except for\nthe `base-url-path` input. The result will be a `sitemap.xml`\nfile in the root of the repository. After completion, it then\nsimply echos the outputs.\n\n```yml\nname: Generate xml sitemap\n\non:\n  push:\n    branches: [ main ]\n\njobs:\n  sitemap_job:\n    runs-on: ubuntu-latest\n    name: Generate a sitemap\n\n    steps:\n    - name: Checkout the repo\n      uses: actions/checkout@v4\n      with:\n        fetch-depth: 0 \n\n    - name: Generate the sitemap\n      id: sitemap\n      uses: cicirello/generate-sitemap@v1\n      with:\n        base-url-path: https://THE.URL.TO.YOUR.PAGE/\n\n    - name: Output stats\n      run: |\n        echo \"sitemap-path = ${{ steps.sitemap.outputs.sitemap-path }}\"\n        echo \"url-count = ${{ steps.sitemap.outputs.url-count }}\"\n        echo \"excluded-count = ${{ steps.sitemap.outputs.excluded-count }}\"\n```\n\n### Example 2: Webpage for API Docs\n\nThis example workflow illustrates how you might use this to generate\na sitemap for a Pages site in the `docs` directory of the\nrepository. It also demonstrates excluding `pdf` files, and\nconfiguring a plain text sitemap.\n\n```yml\nname: Generate API sitemap\n\non:\n  push:\n    branches: [ main ]\n\njobs:\n  sitemap_job:\n    runs-on: ubuntu-latest\n    name: Generate a sitemap\n\n    steps:\n    - name: Checkout the repo\n      uses: actions/checkout@v4\n      with:\n        fetch-depth: 0 \n\n    - name: Generate the sitemap\n      id: sitemap\n      uses: cicirello/generate-sitemap@v1\n      with:\n        base-url-path: https://THE.URL.TO.YOUR.PAGE/\n        path-to-root: docs\n        include-pdf: false\n        sitemap-format: txt\n\n    - name: Output stats\n      run: |\n        echo \"sitemap-path = ${{ steps.sitemap.outputs.sitemap-path }}\"\n        echo \"url-count = ${{ steps.sitemap.outputs.url-count }}\"\n        echo \"excluded-count = ${{ steps.sitemap.outputs.excluded-count }}\"\n``` \n\n### Example 3: Including Additional Indexable File Types\n\nIn this example workflow, we add various additional types to the\nsitemap using the `additional-extensions` input. Note that this\nalso include html files and pdf files since the workflow is using the\ndefault values for `include-html` and `include-pdf`, which both default to\n`true`.\n\n```yml\nname: Generate xml sitemap\n\non:\n  push:\n    branches: [ main ]\n\njobs:\n  sitemap_job:\n    runs-on: ubuntu-latest\n    name: Generate a sitemap\n\n    steps:\n    - name: Checkout the repo\n      uses: actions/checkout@v4\n      with:\n        fetch-depth: 0 \n\n    - name: Generate the sitemap\n      id: sitemap\n      uses: cicirello/generate-sitemap@v1\n      with:\n        base-url-path: https://THE.URL.TO.YOUR.PAGE/\n        additional-extensions: doc docx ppt pptx xls xlsx\n\n    - name: Output stats\n      run: |\n        echo \"sitemap-path = ${{ steps.sitemap.outputs.sitemap-path }}\"\n        echo \"url-count = ${{ steps.sitemap.outputs.url-count }}\"\n        echo \"excluded-count = ${{ steps.sitemap.outputs.excluded-count }}\"\n```\n\n### Example 4: Combining With Other Actions\n\nPresumably you want to do something with your sitemap once it is \ngenerated. In this example workflow, we combine it with the action\n[peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request).\nFirst, the `cicirello/generate-sitemap` action generates the sitemap. And\nthen the `peter-evans/create-pull-request` monitors for changes, and\nif the sitemap changed will create a pull request.\n\n```yml\nname: Generate xml sitemap\n\non:\n  push:\n    branches: [ main ]\n\njobs:\n  sitemap_job:\n    runs-on: ubuntu-latest\n    name: Generate a sitemap\n\n    steps:\n    - name: Checkout the repo\n      uses: actions/checkout@v4\n      with:\n        fetch-depth: 0 \n\n    - name: Generate the sitemap\n      id: sitemap\n      uses: cicirello/generate-sitemap@v1\n      with:\n        base-url-path: https://THE.URL.TO.YOUR.PAGE/\n\n    - name: Create Pull Request\n      uses: peter-evans/create-pull-request@v3\n      with:\n        title: \"Automated sitemap update\"\n        body: \u003e \n          Sitemap updated by the [generate-sitemap](https://github.com/cicirello/generate-sitemap) \n          GitHub action. Automated pull-request generated by the \n          [create-pull-request](https://github.com/peter-evans/create-pull-request) GitHub action.\n```\n\n## Real Examples From Projects Using the Action\n\n### Personal Website\n\nThis first real example is from the [personal website](https://www.cicirello.org/) \nof the developer. One of the workflows, \n[sitemap-generation.yml](https://github.com/cicirello/cicirello.github.io/blob/staging/.github/workflows/sitemap-generation.yml), \nis strictly for generating the sitemap. It runs on pushes of either `*.html` or `*.pdf` \nfiles to the staging branch of this repository. After generating the sitemap, it uses \n[peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request)\nto generate a pull request. You can also replace that step with a commit and push instead.\nYou can find the resulting sitemap here: [sitemap.xml](https://www.cicirello.org/sitemap.xml).\n\n### Documentation Website for a Java Library\n\nThis next example is for the documentation website of \nthe [Chips-n-Salsa](https://chips-n-salsa.cicirello.org/) library. The\n[docs.yml](https://github.com/cicirello/Chips-n-Salsa/blob/master/.github/workflows/docs.yml)\nworkflow runs on push and pull-requests of either `*.java` files. It uses Maven\nto run javadoc (e.g., with `mvn javadoc:javadoc`). It then copies the generated javadoc\ndocumentation to the `docs` directory, from which the API website is served. This is followed\nby another GitHub Action, \n[cicirello/javadoc-cleanup](https://github.com/cicirello/javadoc-cleanup),\nwhich makes a few edits to the javadoc generated website to improve mobile browsing. \n\nNext, it commits any changes (without pushing yet) produced by javadoc and/or \njavadoc-cleanup. After performing those commits, it now runs the generate-sitemap\naction to generate the sitemap. It does this after committing the site changes so that\nthe lastmod dates will be accurate. Finally, it uses \n[peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request)\nto generate a pull request. You can also replace that step with a commit and push instead.\n\nYou can find the resulting sitemap here: [sitemap.xml](https://chips-n-salsa.cicirello.org/sitemap.xml).\n\n## Built With\n\nThe `generate-sitemap` action uses the following:\n* Python (implemented almost entirely in Python);\n* The [cicirello/pyaction](https://actions.cicirello.org/pyaction/) Docker container, which \n  is designed to support GitHub Actions development in the Python language (see\n  [pyaction's GitHub repository](https://github.com/cicirello/pyaction));\n* git to use last commit dates for last modified dates in the sitemaps; and\n* We started with our template repository for GitHub Actions implemented in \n  Python: [cicirello/python-github-action-template](https://github.com/cicirello/python-github-action-template).\n\n## Blog Posts\n\nHere is a selection of blog posts about generate-sitemap on DEV.to:\n* [Deploy a Documentation Website for a Java Library Using GitHub Actions](https://dev.to/cicirello/deploy-a-documentation-website-for-a-java-library-using-github-actions-197n), posted on DEV on November 30, 2022.\n* [Generate an XML Sitemap for a Static Website in GitHub Actions](https://dev.to/cicirello/generate-an-xml-sitemap-for-a-static-website-in-github-actions-20do), posted on DEV on November 23, 2022.\n\n## Support the Project\n\nYou can support the project in a number of ways:\n* __Starring__: If you find the `generate-sitemap` action \n  useful, consider starring the repository.\n* __Sharing with Others__: Consider sharing it with others who\n  you feel might find it useful.\n* __Reporting Issues__: If you find a bug or have a suggestion for\n  a new feature, please report it via \n  the [Issue tracker](https://github.com/cicirello/generate-sitemap/issues).\n* __Contributing Code__: If there is an open issue that you think\n  you can help with, submit a pull request.\n* __Sponsoring__: You can also consider \n  [becoming a sponsor](https://github.com/sponsors/cicirello).\n  \n## License\n\nThe scripts and documentation for this GitHub action is released under\nthe [MIT License](https://github.com/cicirello/generate-sitemap/blob/master/LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcicirello%2Fgenerate-sitemap","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcicirello%2Fgenerate-sitemap","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcicirello%2Fgenerate-sitemap/lists"}