{"id":19975645,"url":"https://github.com/k0nxt3d/web-scrapers","last_synced_at":"2025-12-31T01:06:05.937Z","repository":{"id":213927094,"uuid":"435372577","full_name":"K0NXT3D/Web-Scrapers","owner":"K0NXT3D","description":"Web Scraping Scripts in PhP and Bash","archived":false,"fork":false,"pushed_at":"2021-12-31T07:04:24.000Z","size":168,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-01T18:46:58.714Z","etag":null,"topics":["bash","bot","clone","cloning","crawler","curl","curlphp","download","mirroring","scraping","scraping-websites","seo","seo-optimization","shell-script","spider","wget"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/K0NXT3D.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2021-12-06T05:34:13.000Z","updated_at":"2021-12-31T07:04:27.000Z","dependencies_parsed_at":"2023-12-24T11:27:30.555Z","dependency_job_id":"2c565e4e-6977-4c6f-868b-0cf6ed4abc50","html_url":"https://github.com/K0NXT3D/Web-Scrapers","commit_stats":null,"previous_names":["k0nxt3d/web-scrapers"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/K0NXT3D/Web-Scrapers","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/K0NXT3D%2FWeb-Scrapers","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/K0NXT3D%2FWeb-Scrapers/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/K0NXT3D%2FWeb-Scrapers/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/K0NXT3D%2FWeb-Scrapers/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/K0NXT3D","download_url":"https://codeload.github.com/K0NXT3D/Web-Scrapers/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/K0NXT3D%2FWeb-Scrapers/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286079811,"owners_count":27282121,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-11-27T02:00:05.795Z","response_time":58,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bash","bot","clone","cloning","crawler","curl","curlphp","download","mirroring","scraping","scraping-websites","seo","seo-optimization","shell-script","spider","wget"],"created_at":"2024-11-13T03:19:37.123Z","updated_at":"2025-11-27T02:01:06.009Z","avatar_url":"https://github.com/K0NXT3D.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# web-scrapers\nWeb Scraping Scripts in PhP and Bash\n\n\u003ch4\u003eTerrorBot - Paparazzi - BashKat\u003c/h4\u003e\n\nThese are all nice and simple, easy on the processor and server loads.\nThese worked great with proxychains, so if you're still using that..\n\nThe code for each is really the same.\nGeneral variations to the \"bot\" itself and how I use them on my end.\nUsing Cron is the best way if you're able to proxychains your IP address.\nI only suggest that because these can get you blacklisted.\nOther options include sleep 6h6m5s etc, BUT... it helps to strart a completely\nnew instance of the bot and refresh your IP/Browser and I'll be working on the domain spoofing.\nIt's probably right in front of me.\n\nNormally I don't even run the filenames and I have them in their own folders as \"bot\"\nchmod +x and run ./bot.\nI have the multi-bot launcher of course, I'm sure it's around here somewhere?\n\n\u003cp\u003eDir BashKat/\u003c/p\u003e\n     \u003cp\u003e./bot\u003c/p\u003e\n     \n\u003cem\u003ePretty Simple, Have fun making it your own.\u003c/em\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fk0nxt3d%2Fweb-scrapers","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fk0nxt3d%2Fweb-scrapers","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fk0nxt3d%2Fweb-scrapers/lists"}