{"id":34075563,"url":"https://github.com/supratikchatterjee16/serp_bot","last_synced_at":"2025-12-14T09:51:15.769Z","repository":{"id":57465829,"uuid":"378208241","full_name":"supratikchatterjee16/serp_bot","owner":"supratikchatterjee16","description":"A generic SERP bot, that can be used with just about any search engine.","archived":false,"fork":false,"pushed_at":"2023-12-12T18:59:18.000Z","size":104,"stargazers_count":1,"open_issues_count":0,"forks_count":2,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-11-28T01:46:00.395Z","etag":null,"topics":["bot","crawler","python","requests","scraping","search","serp","user-agent-spoofer"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/supratikchatterjee16.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-06-18T16:28:55.000Z","updated_at":"2023-12-06T17:07:00.000Z","dependencies_parsed_at":"2022-09-17T18:01:26.450Z","dependency_job_id":null,"html_url":"https://github.com/supratikchatterjee16/serp_bot","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/supratikchatterjee16/serp_bot","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/supratikchatterjee16%2Fserp_bot","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/supratikchatterjee16%2Fserp_bot/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/supratikchatterjee16%2Fserp_bot/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/supratikchatterjee16%2Fserp_bot/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/supratikchatterjee16","download_url":"https://codeload.github.com/supratikchatterjee16/serp_bot/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/supratikchatterjee16%2Fserp_bot/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":27725393,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-12-14T02:00:11.348Z","response_time":56,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bot","crawler","python","requests","scraping","search","serp","user-agent-spoofer"],"created_at":"2025-12-14T09:51:11.761Z","updated_at":"2025-12-14T09:51:15.761Z","avatar_url":"https://github.com/supratikchatterjee16.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# SERP Bot\n\nThis is a utility that has gives the basic use of scraping web engines. This is simple in the way it works, however, just using some common sense would allow you to make some rather advanced uses of this.\n\nCaution : SERP bots are a legal gray zone.\n\nThis can be installed through pip.\n```shell\npip install serp-bot\n```\n\nAlternatively, you could download this repository and install it with pip.\n```shell\npip3 install .\n```\n\n## Using\n\nThis has 3 components that can be made use of. SERPBot, GenericSearchEngine and RequestDispatcher.\n\nSample usage :\n```python\nfrom serp_bot import SERPBot, RequestDispatcher\n\nbot = SERPBot()\ndispatcher = RequestDispatcher()\nengine = bot.get_random_search_engine()\nprint(engine)\nengine.build_base_query('hello world')\nprint(engine.get_current_url())\ndispatcher.get(engine.get_current_url()[0])\nprint(dispatcher.last_response.text)\nwith open('sample.html', 'w+') as html_file:\n\thtml_file.write(dispatcher.last_response.text)\n```\n\nAlternatively, using the RequestDispatcher can help avoid some Web Scraping detection units.\n\n```python\nfrom serp_bot import RequestDispatcher\n\nrequest_dispatcher = RequestDispatcher()\nresponse = request_dispatcher.get('https://google.com/search?q=crapper+zapper')\nprint(response.content)\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsupratikchatterjee16%2Fserp_bot","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsupratikchatterjee16%2Fserp_bot","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsupratikchatterjee16%2Fserp_bot/lists"}