{"id":15647873,"url":"https://github.com/healeycodes/broken-link-crawler","last_synced_at":"2025-04-30T13:40:04.147Z","repository":{"id":54995528,"uuid":"178659483","full_name":"healeycodes/Broken-Link-Crawler","owner":"healeycodes","description":":robot: Python bot that crawls your website looking for dead stuff","archived":false,"fork":false,"pushed_at":"2022-09-29T19:55:48.000Z","size":152,"stargazers_count":43,"open_issues_count":5,"forks_count":12,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-11-29T20:02:33.485Z","etag":null,"topics":["bot","crawler","python"],"latest_commit_sha":null,"homepage":"https://healeycodes.com/python/beginners/tutorial/webdev/2019/04/02/dead-link-bot.html","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/healeycodes.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-03-31T08:03:53.000Z","updated_at":"2024-11-02T10:02:11.000Z","dependencies_parsed_at":"2022-08-14T08:31:14.796Z","dependency_job_id":null,"html_url":"https://github.com/healeycodes/Broken-Link-Crawler","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/healeycodes%2FBroken-Link-Crawler","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/healeycodes%2FBroken-Link-Crawler/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/healeycodes%2FBroken-Link-Crawler/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/healeycodes%2FBroken-Link-Crawler/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/healeycodes","download_url":"https://codeload.github.com/healeycodes/Broken-Link-Crawler/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":229088761,"owners_count":18018382,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bot","crawler","python"],"created_at":"2024-10-03T12:21:47.611Z","updated_at":"2024-12-10T15:35:10.453Z","avatar_url":"https://github.com/healeycodes.png","language":"Python","readme":"This was for my **[tutorial](https://healeycodes.com/python/beginners/tutorial/webdev/2019/04/02/dead-link-bot.html)** on building a dead link checker so its scope has been kept quite small.\n\n# Broken Link Crawler\n\n![Desktop](https://github.com/healeycodes/Broken-Link-Crawler/blob/master/bot-in-action.gif)\n\nLet's say I have a website and I want to find any dead links and images on this website.\n\n```bash\n$ python deadseeker.py 'https://healeycodes.com/'\n\u003e 404 - https://docs.python.org/3/library/missing.html\n\u003e 404 - https://github.com/microsoft/solitare2\n```\n\nThe website is crawled, and all `href` and `src` attributes are sent a request. Errors are reported. This bot doesn't observe `robots.txt` but _you should_.\n\n\n\n### It is not a clever bot. But it is a good bot.\n\n\u003cbr\u003e\n\nAccepting (small) PRs and issues!\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhealeycodes%2Fbroken-link-crawler","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhealeycodes%2Fbroken-link-crawler","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhealeycodes%2Fbroken-link-crawler/lists"}