https://github.com/andygrunwald/sico
A Sitemap Comparison that helps you to not fuck up your website migration.
https://github.com/andygrunwald/sico
hacktoberfest sitemap sitemap-comparison website-migration
Last synced: 9 months ago
JSON representation
A Sitemap Comparison that helps you to not fuck up your website migration.
- Host: GitHub
- URL: https://github.com/andygrunwald/sico
- Owner: andygrunwald
- License: mit
- Created: 2022-10-21T18:21:58.000Z (almost 4 years ago)
- Default Branch: main
- Last Pushed: 2022-10-21T20:27:50.000Z (almost 4 years ago)
- Last Synced: 2025-02-01T20:51:12.156Z (over 1 year ago)
- Topics: hacktoberfest, sitemap, sitemap-comparison, website-migration
- Language: Go
- Homepage:
- Size: 6.84 KB
- Stars: 4
- Watchers: 3
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# sico - A Sitemap comparison tool
A Sitemap Comparison that helps you to not fuck up your website migration.
## Usecase
### Website migration
Imagine you want to migrate a website.
Lets say, your personal website https://andygrunwald.com/ (incl. a blog) from [Hugo](https://gohugo.io/) to [Astro](https://astro.build/).
You have a few blog posts that rank pretty well in Google - You want to keep all previous URLs.
This tool checks if all URLs from the old website's sitemap are present in the new one.
It is not doing a 1:1 check!
The new site can contain more links in the sitemap.
## Usage
```
Usage of ./sico:
-exclude value
Regex to match against URLs in {source} sitemap that don't need to be in {new} sitemap. It can be defined multiple times.
-new string
New Sitemap URL - Sitemap entries you want to check for presence (default "https://example-new.com/sitemap.xml")
-newBaseURL new
Base URL that will be used if new contains a SitemapIndex to replace the SitemapIndex entries
-source string
Source Sitemap URL - Sitemap you want to check against (default "https://example.com/sitemap.xml")
```
### An example
The call ...
```sh
./sico -source "https://andygrunwald.com/sitemap.xml" \
-new "https://deploy-preview-7--spiffy-shortbread-df2800.netlify.app/sitemap-index.xml" \
-newBaseURL "https://deploy-preview-7--spiffy-shortbread-df2800.netlify.app/" \
-exclude "andygrunwald\\.com/tags/"
```
... means:
1. We read the `-source` sitemap from `https://andygrunwald.com/sitemap.xml` and collect the URLs:
```xml
https://andygrunwald.com/
2021-09-21T20:30:00+02:00
[...]
```
2. We read the `-new` sitemap from `https://deploy-preview-7--spiffy-shortbread-df2800.netlify.app/sitemap-index.xml` and collect the URLs:
```xml
https://andygrunwald.com/sitemap-0.xml
1. If this URL is a SitemapIndex ([one sitemap split into multiple sitemaps](https://developers.google.com/search/docs/crawling-indexing/sitemaps/large-sitemaps))
2. AND `-newBaseURL` is set, replace the Base URL (Scheme + Host) of the Sub-Sitemap with `-newBaseURL`
3. Means `https://andygrunwald.com/sitemap-0.xml` will be changed to `https://deploy-preview-7--spiffy-shortbread-df2800.netlify.app/sitemap-0.xml`
3. Loop through all URLs from `-source` (`https://andygrunwald.com/sitemap.xml`), check if the URL matches a defined `-exclude` and needs to be skipped; if not check if it is part of the `-new` sitemap. If yes, all good; if not, raise this as output (see below)
4. Result:
```
Result
=============
Source Sitemap: https://andygrunwald.com/sitemap.xml
URLs checked (from source sitemap): 60
New Sitemap: https://deploy-preview-7--spiffy-shortbread-df2800.netlify.app/sitemap-index.xml
Excludes configured: 1
URLs skipped because they matched an exclude: 24
URLs missing from source sitemap in new sitemap: 14
Missing URLs in the new sitemap:
https://andygrunwald.com/categories/
https://andygrunwald.com/categories/hardware/
[...]
```
## Production ready?
No, not really.
But it does the job.
This tool was created "on the get-go".
It has no focus on reliability, proper error handling, or things that fit into the *production ready* category.
However, this is (partially) not needed.
This tool only reads data from the web and compares it.
No write functionality or anything else.
Hence, no damage.
Means: You can see it as a (kind of) production ready.