{"id":20331178,"url":"https://github.com/comcast/sitemapper-for-js","last_synced_at":"2025-04-11T21:07:34.059Z","repository":{"id":46295604,"uuid":"189618151","full_name":"Comcast/sitemapper-for-js","owner":"Comcast","description":"Generate XML sitemaps for JS Websites (Supports Angular, React)","archived":false,"fork":false,"pushed_at":"2023-05-01T06:19:30.000Z","size":43,"stargazers_count":36,"open_issues_count":5,"forks_count":16,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-11T21:07:24.645Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Comcast.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-05-31T15:33:25.000Z","updated_at":"2025-03-05T14:53:13.000Z","dependencies_parsed_at":"2022-09-22T19:23:29.281Z","dependency_job_id":null,"html_url":"https://github.com/Comcast/sitemapper-for-js","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Comcast%2Fsitemapper-for-js","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Comcast%2Fsitemapper-for-js/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Comcast%2Fsitemapper-for-js/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Comcast%2Fsitemapper-for-js/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Comcast","download_url":"https://codeload.github.com/Comcast/sitemapper-for-js/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248480434,"owners_count":21110937,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-14T20:19:09.279Z","updated_at":"2025-04-11T21:07:34.012Z","avatar_url":"https://github.com/Comcast.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# SITEMAP GENERATOR FOR SPA (SINGLE PAGE APPLICATION)\n\n### About\n\nSitemaps are simple XML documents consisting of links of all pages in a website. This provides additional information about the page to search engine crawlers to categories the type of content and serve it to the users based on their search keywords. \n\nMost commonly used sitemap generators works well with websites with multiple pages that are built with PHP, ASP.NET or any old-school technologies since the browser reloads everytime the user navigates through different pages. Wherease, websites that rely heavily on Javascript (Like Angular / React / Vuejs ) will not reload during page navigation and only the `view` changes. This makes the existing generators difficult to capture different routes and create a sitemap out of it. \n\nThis Sitemap generator, built with `puppeteer (Google Chrome's Headless Chrome Node API) ` works well with Javascript based websites in creating Sitemaps\n\n\n### Setup \u0026 Configuration\n\n`npm install`\n\nTo start generating\n\n`npm start`\n\n**config.js**\n\n```js\nmodule.exports  = {\n    base: 'https://www.xfinity.com', // website url\n    urls: [ // list of pages you want to crawl\n        'https://www.xfinity.com/mobile', \n        'https://www.xfinity.com/mobile/plan',\n        'https://www.xfinity.com/mobile/byod',\n        'https://www.xfinity.com/mobile/support',\n        'https://www.xfinity.com/mobile/shop?category=device',\n        'https://www.xfinity.com/mobile/shop?category=accessories'\n    ],\n    strictPresence: 'www.xfinity.com/mobile/', // url will be added to xml only if this exists\n    ignoreStrings: [ // ignore any url that has these texts\n        'img.xfinity',\n        'styles.',\n        'm.me'\n    ],\n    autoCrawl: false, // Recursive crawling functionality\n    crawlLevel: 0, // Recursive calling for pages upto 'x' levels\n    pageLoad: { // page load configuration\n        waitUntil: 'networkidle0',\n        timeout: 3000000\n    },\n    disableHashRoutes: false, // disable routes with Hash in it\n    sortBy: 'asc' // 'asc' | 'dsc' | 'none'\n}\n```\n\n### Configurations\n\n**base**\n\n`base: 'https://www.xfinity.com'`\n\nWebsite that you want to create sitemap for\n\n**urls (only for manual Crawling)**\n\nArray of urls that you wanted to crawl. Links present in the mentional html pages will **not** be recursively called in this\n\n**strictPresence**\n\nAdd the url to XML only if this string presents\n\n**ignoreStrings**\n\nList of urls/strings you wanted to ignore in the links you are adding to Sitemap\n\n**autoCrawl**\n\nEnable/Disable Auto Crawling feature. Auto-crawling takes more time than manual crawl. Largely depends on the complexity of website\n\n\n**crawlLevel(only for auto-crawl)**\n\nMention number of child routes you would like to crawl in case of auto-crawling\n\nE.g.: \nLets assume the base url `https://abc.com/`, \n\n`crawlLevel=1` would crawl in pattern  `https://abc.com/\u003cany-path\u003e`\n`crawlLevel=2` would crawl in pattern  `https://abc.com/\u003cany-path\u003e/\u003cany-path\u003e`\n\n\n**pageLoad**\n\nPage load settings inherited from puppeteer configuration\n\nhttps://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#pagegotourl-options\n\n\n**disableHashRoutes**\n\nIgnore any routes with `#` in it\n\nE.g.:\n\nAvoids these urls\n\n`https://abc.com/#section2`\n`https://abc.com/#/section2`\n`https://abc.com/about#section4`\n\n\n## License\n\nThis repo is licensed under [Apache License 2.0](LICENSE).\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcomcast%2Fsitemapper-for-js","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcomcast%2Fsitemapper-for-js","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcomcast%2Fsitemapper-for-js/lists"}