{"id":13548964,"url":"https://github.com/spatie/http-status-check","last_synced_at":"2025-05-15T12:04:40.022Z","repository":{"id":41380772,"uuid":"44727732","full_name":"spatie/http-status-check","owner":"spatie","description":"CLI tool to crawl a website and check HTTP status codes","archived":false,"fork":false,"pushed_at":"2023-12-18T14:24:14.000Z","size":120,"stargazers_count":595,"open_issues_count":0,"forks_count":89,"subscribers_count":20,"default_branch":"main","last_synced_at":"2025-04-14T19:59:37.407Z","etag":null,"topics":["curl","health","http","seo","statuscode"],"latest_commit_sha":null,"homepage":"https://freek.dev/308-building-a-crawler-in-php","language":"PHP","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/spatie.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2015-10-22T06:57:28.000Z","updated_at":"2025-03-06T05:19:14.000Z","dependencies_parsed_at":"2023-12-18T15:47:26.524Z","dependency_job_id":"c5e92351-e1e6-4e08-a330-11f4f826b59c","html_url":"https://github.com/spatie/http-status-check","commit_stats":{"total_commits":123,"total_committers":27,"mean_commits":4.555555555555555,"dds":0.3739837398373984,"last_synced_commit":"89416a8aa680ec9abd7ca920982edcf52deaec60"},"previous_names":[],"tags_count":22,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spatie%2Fhttp-status-check","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spatie%2Fhttp-status-check/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spatie%2Fhttp-status-check/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spatie%2Fhttp-status-check/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/spatie","download_url":"https://codeload.github.com/spatie/http-status-check/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254337612,"owners_count":22054253,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["curl","health","http","seo","statuscode"],"created_at":"2024-08-01T12:01:16.570Z","updated_at":"2025-05-15T12:04:35.002Z","avatar_url":"https://github.com/spatie.png","language":"PHP","funding_links":[],"categories":["PHP"],"sub_categories":[],"readme":"# Check the HTTP status code of all links on a website\n\n[![Latest Version on Packagist](https://img.shields.io/packagist/v/spatie/http-status-check.svg?style=flat-square)](https://packagist.org/packages/spatie/http-status-check)\n[![Software License](https://img.shields.io/badge/license-MIT-brightgreen.svg?style=flat-square)](LICENSE.md)\n![Tests](https://github.com/spatie/http-status-check/workflows/Tests/badge.svg)\n[![Total Downloads](https://img.shields.io/packagist/dt/spatie/http-status-check.svg?style=flat-square)](https://packagist.org/packages/spatie/http-status-check)\n\nThis repository provides a tool to check the HTTP status code of every link on a given website.\n\n## Support us\n\n[\u003cimg src=\"https://github-ads.s3.eu-central-1.amazonaws.com/http-status-check.jpg?t=1\" width=\"419px\" /\u003e](https://spatie.be/github-ad-click/http-status-check)\n\nWe invest a lot of resources into creating [best in class open source packages](https://spatie.be/open-source). You can support us by [buying one of our paid products](https://spatie.be/open-source/support-us).\n\nWe highly appreciate you sending us a postcard from your hometown, mentioning which of our package(s) you are using. You'll find our address on [our contact page](https://spatie.be/about-us). We publish all received postcards on [our virtual postcard wall](https://spatie.be/open-source/postcards).\n\n## Installation\n\nThis package can be installed via Composer:\n\n```bash\ncomposer global require spatie/http-status-check\n```\n\n## Usage\n\nThis tool will scan all links on a given website:\n\n```bash\nhttp-status-check scan https://example.com\n```\n\nIt outputs a line per link found. Here's an example on Laracast website scan:\n\n![screenshot](https://freek.dev/uploads/2015/11/screenshot.png)\n\nWhen the crawling process is finished a summary will be shown.\n\nBy default the crawler uses 10 concurrent connections to speed up the crawling process. You can change that number by passing a different value to the `--concurrency` option:\n\n```bash\nhttp-status-check scan https://example.com --concurrency=20\n```\n\nYou can also write all urls that gave a non-2xx or non-3xx response to a file:\n\n```bash\nhttp-status-check scan https://example.com --output=log.txt\n```\n\nWhen the crawler finds a link to an external website it will by default crawl that link as well. If you don't want the crawler to crawl such external urls use the `--dont-crawl-external-links` option:\n\n```bash\nhttp-status-check scan https://example.com --dont-crawl-external-links\n```\n\nBy default, requests timeout after 10 seconds. You can change this by passing the number of seconds to the `--timeout` option:\n\n```bash\nhttp-status-check scan https://example.com --timeout=30\n```\n\nBy default, the crawler will respect robots data. You can ignore them though with the `--ignore-robots` option:\n\n```bash\nhttp-status-check scan https://example.com --ignore-robots\n```\n\nIf your site requires a basic authentification, you can use the `--auth` option:\n\n```bash\nhttp-status-check scan https://example.com --auth=username:password\n```\n\n## Testing\n\nTo run the tests, first make sure you have [Node.js](https://nodejs.org/) installed. Then start the included node based server in a separate terminal window:\n\n```bash\ncd tests/server\nnpm install\nnode server.js\n```\n\nWith the server running, you can start testing:\n\n```bash\nvendor/bin/phpunit\n```\n\n## Changelog\n\nPlease see [CHANGELOG](CHANGELOG.md) for more information on what has changed recently.\n\n## Contributing\n\nPlease see [CONTRIBUTING](https://github.com/spatie/.github/blob/main/CONTRIBUTING.md) for details.\n\n## Security\n\nIf you've found a bug regarding security please mail [security@spatie.be](mailto:security@spatie.be) instead of using the issue tracker.\n\n## Credits\n\n-   [Freek Van der Herten](https://github.com/freekmurze)\n-   [Sebastian De Deyne](https://github.com/sebastiandedeyne)\n-   [All Contributors](../../contributors)\n\n## License\n\nThe MIT License (MIT). Please see [License File](LICENSE.md) for more information.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fspatie%2Fhttp-status-check","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fspatie%2Fhttp-status-check","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fspatie%2Fhttp-status-check/lists"}