{"id":13762741,"url":"https://github.com/hahwul/deadfinder","last_synced_at":"2025-05-16T07:05:56.414Z","repository":{"id":60179111,"uuid":"540459720","full_name":"hahwul/deadfinder","owner":"hahwul","description":"🏴‍☠️ Find dead-links (broken links)","archived":false,"fork":false,"pushed_at":"2025-05-09T01:46:38.000Z","size":464,"stargazers_count":145,"open_issues_count":3,"forks_count":11,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-05-09T02:41:40.480Z","etag":null,"topics":["broken-links","dead-links","hacktoberfest","security","seo","seo-optimization"],"latest_commit_sha":null,"homepage":"https://rubygems.org/gems/deadfinder","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hahwul.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null},"funding":{"github":"hahwul"}},"created_at":"2022-09-23T13:43:44.000Z","updated_at":"2025-05-09T01:46:35.000Z","dependencies_parsed_at":"2023-11-26T15:04:31.711Z","dependency_job_id":"546cb8e0-1abb-4a30-ad5c-5e6bdd24e1cd","html_url":"https://github.com/hahwul/deadfinder","commit_stats":{"total_commits":126,"total_committers":3,"mean_commits":42.0,"dds":"0.015873015873015928","last_synced_commit":"88f48b07a8247268df56126894c676e2a5e754d1"},"previous_names":[],"tags_count":27,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hahwul%2Fdeadfinder","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hahwul%2Fdeadfinder/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hahwul%2Fdeadfinder/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hahwul%2Fdeadfinder/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hahwul","download_url":"https://codeload.github.com/hahwul/deadfinder/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254485062,"owners_count":22078767,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["broken-links","dead-links","hacktoberfest","security","seo","seo-optimization"],"created_at":"2024-08-03T14:00:56.191Z","updated_at":"2025-05-16T07:05:51.404Z","avatar_url":"https://github.com/hahwul.png","language":"Ruby","funding_links":["https://github.com/sponsors/hahwul"],"categories":["Weapons","Ruby"],"sub_categories":["Tools"],"readme":"\u003cdiv align=\"center\"\u003e\n  \u003cpicture\u003e\n    \u003cimg alt=\"DeadFinder Logo\" src=\"https://github.com/user-attachments/assets/1523d0be-31dd-4031-ac97-5feda474a6e9\" width=\"500px;\"\u003e\n  \u003c/picture\u003e\n  \u003cp\u003eFind dead-links (broken links)\u003c/p\u003e\n\u003c/div\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"#contributions-welcome\"\u003e\u003cimg src=\"https://img.shields.io/badge/CONTRIBUTIONS-WELCOME-000000?style=for-the-badge\u0026labelColor=000000\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://app.codecov.io/gh/hahwul/deadfinder/\"\u003e\u003cimg src=\"https://img.shields.io/codecov/c/gh/hahwul/deadfinder?style=for-the-badge\u0026color=000000\u0026logo=codecov\u0026labelColor=000000\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://rubygems.org/gems/deadfinder\"\u003e\u003cimg src=\"https://img.shields.io/gem/v/deadfinder?style=for-the-badge\u0026color=000000\u0026logo=ruby\u0026labelColor=000000\u0026logoColor=red\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://formulae.brew.sh/formula/deadfinder\"\u003e\u003cimg src=\"https://img.shields.io/homebrew/v/deadfinder?style=for-the-badge\u0026color=000000\u0026logo=homebrew\u0026labelColor=000000\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\nDead link (broken link) means a link within a web page that cannot be connected. These links can have a negative impact to SEO and Security. This tool makes it easy to identify and modify.\n\n![](https://github.com/user-attachments/assets/92129de9-90c6-41e0-a424-883fe30858f6)\n\n## Installation\n### Install with Gem\n#### CLI\n```bash\ngem install deadfinder\n# https://rubygems.org/gems/deadfinder\n```\n\n#### Gemfile\n\n```ruby\ngem 'deadfinder'\n# and `bundle install`\n```\n\n### Install with Homebrew\n```bash\nbrew install deadfinder\n# https://formulae.brew.sh/formula/deadfinder\n```\n\n### Docker Image\n```shell\ndocker pull ghcr.io/hahwul/deadfinder:latest\n```\n\n## Using In\n### CLI\n```shell\ndeadfinder sitemap https://www.hahwul.com/sitemap.xml\n```\n\n### Github Action\n```yml\nsteps:\n- name: Run DeadFinder\n  uses: hahwul/deadfinder@1.7.1\n  # or uses: hahwul/deadfinder@latest\n  id: broken-link\n  with:\n    command: sitemap # url / file / sitemap\n    target: https://www.hahwul.com/sitemap.xml\n    # timeout: 10\n    # concurrency: 50\n    # silent: false\n    # headers: \"X-API-Key: 123444\"\n    # worker_headers: \"User-Agent: Deadfinder Bot\"\n    # include30x: false\n    # user_agent: \"Apple\"\n    # proxy: \"http://localhost:8070\"\n    # proxy_auth: \"id:pw\"\n    # match: \n    # ignore: \n\n- name: Output Handling\n  run: echo '${{ steps.broken-link.outputs.output }}'\n```\n\nIf you have found a Dead Link and want to automatically add it as an issue, please refer to the \"[Automating Dead Link Detection](https://www.hahwul.com/2024/10/20/automating-dead-link-detection/)\" article.\n\n### Ruby Code\n```ruby\nrequire 'deadfinder'\n\nrunner = DeadFinder::Runner.new\noptions = runner.default_options\noptions['concurrency'] = 30\n\nDeadFinder.run_url('https://www.hahwul.com/cullinan/csrf/', options)\nputs DeadFinder.output\n\n# {\"https://www.hahwul.com/cullinan/csrf/\" =\u003e [\"https://www.hahwul.com/tag/cullinan/\"]}\n```\n\nFor various examples and detailed usage, including sitemap, file, and other modes, please refer to the [rubydoc](https://rubydoc.info/gems/deadfinder/DeadFinder) and [examples](./examples) directory in the repository.\n\n## Usage\n```\nCommands:\n  deadfinder completion \u003cSHELL\u003e     # Generate completion script for shell.\n  deadfinder file \u003cFILE\u003e            # Scan the URLs from File. (e.g., deadfinder file urls.txt)\n  deadfinder help [COMMAND]         # Describe available commands or one specific command\n  deadfinder pipe                   # Scan the URLs from STDIN. (e.g., cat urls.txt | deadfinder pipe)\n  deadfinder sitemap \u003cSITEMAP-URL\u003e  # Scan the URLs from sitemap.\n  deadfinder url \u003cURL\u003e              # Scan the Single URL.\n  deadfinder version                # Show version.\n\nOptions:\n  r, [--include30x], [--no-include30x]  # Include 30x redirections\n  c, [--concurrency=N]                  # Number of concurrency\n                                        # Default: 50\n  t, [--timeout=N]                      # Timeout in seconds\n                                        # Default: 10\n  o, [--output=OUTPUT]                  # File to write result (e.g., json, yaml, csv)\n  f, [--output-format=OUTPUT_FORMAT]    # Output format\n                                        # Default: json\n  H, [--headers=one two three]          # Custom HTTP headers to send with initial request\n     [--worker-headers=one two three]   # Custom HTTP headers to send with worker requests\n     [--user-agent=USER_AGENT]          # User-Agent string to use for requests\n                                        # Default: Mozilla/5.0 (compatible; DeadFinder/1.7.1;)\n  p, [--proxy=PROXY]                    # Proxy server to use for requests\n     [--proxy-auth=PROXY_AUTH]          # Proxy server authentication credentials\n  m, [--match=MATCH]                    # Match the URL with the given pattern\n  i, [--ignore=IGNORE]                  # Ignore the URL with the given pattern\n  s, [--silent], [--no-silent]          # Silent mode\n  v, [--verbose], [--no-verbose]        # Verbose mode\n     [--debug], [--no-debug]            # Debug mode\n```\n\n## Modes\n```shell\n# Scan the URLs from STDIN (multiple URLs)\ncat urls.txt | deadfinder pipe\n\n# Scan the URLs from File. (multiple URLs)\ndeadfinder file urls.txt\n\n# Scan the Single URL.\ndeadfinder url https://www.hahwul.com\n\n# Scan the URLs from sitemap. (multiple URLs)\ndeadfinder sitemap https://www.hahwul.com/sitemap.xml\n```\n\n## JSON Handling\n```shell\ndeadfinder sitemap https://www.hahwul.com/sitemap.xml \\\n  -o output.json\n  \ncat output.json | jq\n```\n\n```json\n{\n  \"Origin URL\": [\n    \"DeadLink URL\",\n    \"DeadLink URL\",\n    \"DeadLink URL\"\n  ]\n}\n```\n\n## Contributions Welcome!\n\nWe welcome contributions from everyone! If you have an idea for an improvement or want to report a bug:\n\n- **Fork the repository.**\n- **Create a new branch** for your feature or bug fix (e.g., `feature/awesome-feature` or `bugfix/annoying-bug`).\n- **Make your changes.**\n- **Commit your changes** with a clear commit message.\n- **Push** to the branch.\n- **Submit a Pull Request (PR)** to our `main` branch.\n\nWe'll review your PR as soon as possible. Thank you for contributing to our project!\n\n### Contributors\n\n![](CONTRIBUTORS.svg)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhahwul%2Fdeadfinder","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhahwul%2Fdeadfinder","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhahwul%2Fdeadfinder/lists"}