Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/tmcw/notfoundbot
fix & archive outgoing links on your website
https://github.com/tmcw/notfoundbot
actions hugo jekyll
Last synced: 13 days ago
JSON representation
fix & archive outgoing links on your website
- Host: GitHub
- URL: https://github.com/tmcw/notfoundbot
- Owner: tmcw
- Created: 2020-05-02T17:59:32.000Z (over 4 years ago)
- Default Branch: main
- Last Pushed: 2024-05-11T16:09:07.000Z (6 months ago)
- Last Synced: 2024-10-26T11:35:08.522Z (19 days ago)
- Topics: actions, hugo, jekyll
- Language: TypeScript
- Homepage:
- Size: 2.9 MB
- Stars: 114
- Watchers: 4
- Forks: 7
- Open Issues: 10
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
Awesome Lists containing this project
README
# notfoundbot
[](https://codeclimate.com/github/tmcw/notfoundbot/maintainability) [](https://codeclimate.com/github/tmcw/notfoundbot/test_coverage)
notfoundbot is a GitHub Action that helps you automatically maintain the correctness of your
website's outgoing links. It finds links that need fixing and opens pull requests
that fix them.
This action is intended for websites and blogs powered by static site generators.
notfoundbot does the following fixes:
- Upgrades outgoing HTTP links to HTTPS
- Replaces broken outgoing links with links to the [Wayback Machine](https://web.archive.org/)
By using post dates derived from filenames, notfoundbot searches for Wayback Machine archives
of linked resources that are contemporary to the post itself: broken links in a 2011 blog post
will be linked to archives from around that era.
## Example YAML
```yaml
name: notfoundbot
on:
schedule:
- cron: "0 5 * * *"
jobs:
check:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Fix links
uses: tmcw/[email protected]
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
```
Some websites will return different information for notfoundbot than for a typical user. When false positives become a repetitive issue an exceptions list can be used. The exception list is a space separated list of hosts that will always return an ok status.
```yaml
name: notfoundbot
on:
schedule:
- cron: "0 5 * * *"
jobs:
check:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Fix links
uses: tmcw/[email protected]
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
EXCEPTIONS: www.host.com thisisok.org
```
By default notfoundbot will check `.md` files in the `_posts` directory. You can change this directory:
```yaml
- name: Fix links
uses: tmcw/[email protected]
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
content-folder: custom-content
```
Notes:
- I might forget to update the version on `notfoundbot` here - make sure that it's
the latest!
- Check out [crontab.guru](https://crontab.guru/#5_*_*_*_*) to customize the
schedule line, which can run the task more or less often if you want.
## Features
- Post date detection: supports filename-based dates, YAML & TOML frontmatter
- notfoundbot uses [magic-string](https://github.com/rich-harris/magic-string) to
selectively update links without affecting surrounding markup
## Workflow
- If there is an existing PR tagged `notfoundbot`, exit
- Gather post files and parse them, and then for each unique outlink URL
- If the URL is not http or https, ignore it
- If the URL is relative, ignore it
- If the URL has been checked recently and is in the cache, ignore it
- If the URL is HTTP, check its HTTPS equivalent.
- If the HTTPS equivalent exists, upgrade the link to HTTPS
- Otherwise, check the HTTP link
- If the HTTP link resolves, ignore it
- If the HTTP link fails, mark it as an error.
- If the URL is HTTPS, check to see if it resolves
- If the link resolves, ignore it
- If the link fails, mark it as an error
Then, for each link marked as an error:
- Check the Internet Archive to find contemporary archives of each failed URL
- If an archive exists, replace the link
- Otherwise, ignore it.