{"id":16641581,"url":"https://github.com/scrwdrv/siege-crawler","last_synced_at":"2025-04-05T01:25:46.017Z","repository":{"id":51640219,"uuid":"233047528","full_name":"scrwdrv/siege-crawler","owner":"scrwdrv","description":"This CLI tool will find same domain urls in a web page and requesting them to find even more urls until server crash (or at the end of benchmark). It is used to test maximun capacity of server or finding for glitches that users might encounter.","archived":false,"fork":false,"pushed_at":"2022-12-10T14:28:34.000Z","size":137,"stargazers_count":1,"open_issues_count":3,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-13T03:48:43.536Z","etag":null,"topics":["benchmark","cli","crawler","ddos","debug","siege","tool"],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/scrwdrv.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-01-10T12:59:19.000Z","updated_at":"2023-02-17T01:05:50.000Z","dependencies_parsed_at":"2023-01-26T07:45:59.117Z","dependency_job_id":null,"html_url":"https://github.com/scrwdrv/siege-crawler","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scrwdrv%2Fsiege-crawler","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scrwdrv%2Fsiege-crawler/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scrwdrv%2Fsiege-crawler/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scrwdrv%2Fsiege-crawler/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/scrwdrv","download_url":"https://codeload.github.com/scrwdrv/siege-crawler/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247273993,"owners_count":20912029,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["benchmark","cli","crawler","ddos","debug","siege","tool"],"created_at":"2024-10-12T07:47:07.127Z","updated_at":"2025-04-05T01:25:45.997Z","avatar_url":"https://github.com/scrwdrv.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# siege-crawler\n This CLI tool will find same domain urls in a web page and requesting them to find even more urls until server crash (or at the end of benchmark). It is used to test maximun capacity of server or finding for glitches that users might encounter.\n\n\n## Installation\n```sh\nnpm i -g siege-crawler\n```\n\n## Usage\n```sh\nsiege-crawler https://google.com \n# requesting `https://google.com` with default rate 50/sec until ^c is fired\n```\n```sh\nsiege-crawler --duration 5 --rate 10 https://google.com \n# or          -d 5 -r 10\n# requesting `https://google.com` with custom rate 10/sec for 5 secs or ^c is fired\n```\n\n### Skip Specific URLs\nRule out urls by regular expression. Notice, regex will match **full url** not just path. \n```sh\nsiege-crawler --ruleout /google\\.com\\/search/ https://google.com \n```\n```sh\nsiege-crawler --ruleout /^https://google\\.com\\/search\\?.+$/i https://google.com \n```\n### Ouput\n```sh\n[google.com] rate: 5/sec, duration: 5 secs, ruleout: /google\\.com\\/search/i\n\nSent: 26, Received: 26, Lost: 0 (0.00% loss)\nData received: 5,614,377 bytes\nResponse Time(ms): Max = 544, Min = 219, Avg = 311.00\n```\n\n### Request Log\nsiege-crawler will generate a complete log in current directory listed all the requested urls, status and response time in real order.\n```log\n Status | Time(ms) | URL\n     OK |      284 | https://google.com\n     OK |      339 | https://google.com\n     OK |      234 | https://google.com\n     OK |      304 | https://google.com/\n     OK |      246 | https://google.com/advanced_search?hl=en-US\u0026fg=1\n     OK |      396 | https://google.com/history/privacyadvisor/search/unauth?utm_source=googlemenu\u0026fg=1\n     OK |      319 | https://google.com/history/optout?hl=en-US\u0026fg=1\n     OK |      219 | https://google.com\n     OK |      283 | https://google.com/\n     OK |      246 | https://google.com/advanced_search?hl=en-US\u0026fg=1\n     OK |      485 | https://google.com/history/privacyadvisor/search/unauth?utm_source=googlemenu\u0026fg=1\n     OK |      265 | https://google.com/webhp?tab=ww\n     OK |      318 | https://google.com/preferences?hl=en-US\n     OK |      309 | https://google.com/history/optout?hl=en-US\u0026fg=1\n     OK |      265 | https://google.com/history/optout?utm_source=search-privacy-advisor\n     OK |      230 | https://google.com\n     OK |      265 | https://google.com/\n     OK |      235 | https://google.com/advanced_search?hl=en-US\u0026fg=1\n     OK |      416 | https://google.com/history/privacyadvisor/search/unauth?utm_source=googlemenu\u0026fg=1\n     OK |      298 | https://google.com/webhp?tab=ww\n     OK |      305 | https://google.com/preferences?hl=en-US\n     OK |      301 | https://google.com/history/optout?hl=en-US\u0026fg=1\n     OK |      241 | https://google.com/webhp\n     OK |      544 | https://google.com/support/websearch?p=ws_cookies_notif\u0026hl=en-US\n     OK |      265 | https://google.com/history/optout?hl=en-US\n     OK |      474 | https://google.com/intl/en-US/help.html\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscrwdrv%2Fsiege-crawler","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fscrwdrv%2Fsiege-crawler","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscrwdrv%2Fsiege-crawler/lists"}