{"id":16410050,"url":"https://github.com/nirala96/web_scraping_scripts","last_synced_at":"2025-10-26T17:32:45.441Z","repository":{"id":50774859,"uuid":"303561267","full_name":"nirala96/Web_Scraping_scripts","owner":"nirala96","description":"Some Web Scraping Scripts using scrapy written in python that scrapes the data from a website","archived":false,"fork":false,"pushed_at":"2021-05-30T07:21:45.000Z","size":652,"stargazers_count":5,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-31T22:11:23.835Z","etag":null,"topics":["python","scrapy","scrapy-crawler","webscraping"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nirala96.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-10-13T02:02:19.000Z","updated_at":"2022-02-07T17:55:56.000Z","dependencies_parsed_at":"2022-09-11T15:41:30.630Z","dependency_job_id":null,"html_url":"https://github.com/nirala96/Web_Scraping_scripts","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nirala96%2FWeb_Scraping_scripts","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nirala96%2FWeb_Scraping_scripts/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nirala96%2FWeb_Scraping_scripts/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nirala96%2FWeb_Scraping_scripts/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nirala96","download_url":"https://codeload.github.com/nirala96/Web_Scraping_scripts/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":238380598,"owners_count":19462402,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["python","scrapy","scrapy-crawler","webscraping"],"created_at":"2024-10-11T06:22:50.688Z","updated_at":"2025-10-26T17:32:45.073Z","avatar_url":"https://github.com/nirala96.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Web Scraping Scripts\n\u003cp align = \"center\"\u003e\n    \u003ca href = \"\"\u003e\n\u003cimg src=\"https://github.com/nirala69/Web_Scraping_scripts/blob/main/Scrapy-Logo-big.png?raw=true\" width=\"400\" align='center'\u003e\n        \u003c/a\u003e\n\nOverview\n========\n\nScrapy is a fast high-level web crawling and web scraping framework, used to\ncrawl websites and extract structured data from their pages. It can be used for\na wide range of purposes, from data mining to monitoring and automated testing.\n\nCheck the Scrapy homepage at https://scrapy.org for more information,\nincluding a list of features.\n\n### My Script contains \u003cimg src=\"https://github.com/nirala69/Web_Scraping_scripts/blob/main/spider-clipart-animation-3.gif?raw=true\" width=\"40\" align='center'\u003e\n- [IMDB in multiple domains](https://github.com/nirala69/Web_Scraping_scripts/tree/main/IMDB_detailed_scrape/IMDB)  \u003cimg src=\"https://github.com/nirala69/Web_Scraping_scripts/blob/main/Imdb.jpg?raw=true\" width=\"50\" align=''\u003e\n    1. Most popular movies.\n    2. Lowest rated movies.\n    3. Most popular TV shows.\n    4. Top rated TV shows.\n    5. Top rated movies.\n- [All restaurants in gandhinagar (Gujarat) from zomato.](https://github.com/nirala69/Web_Scraping_scripts/tree/main/zomato_scrape) \u003cimg src=\"https://github.com/nirala69/Web_Scraping_scripts/blob/main/zomato.png?raw=true\" width=\"100\" align=''\u003e\n- Basic scraping of quotes.scrape.com\n\n### Technologies used\n\u003cimg src=\"https://github.com/nirala69/Web_Scraping_scripts/blob/main/mop.gif?raw=true\" width=\"400\" align='right'\u003e\n- Scrapy\n\n### Languages used\n- Python \n- SQL\n\n\n\nRequirements\n============\n\n* Python 3.6+\n* Works on Linux, Windows, macOS, BSD\n\nInstall\n=======\n\nThe quick way::\n\n    pip install scrapy\n\nSee the install section in the documentation at\nhttps://docs.scrapy.org/en/latest/intro/install.html for more details.\n\nDocumentation\n=============\n\nDocumentation is available online at https://docs.scrapy.org/ and in the ``docs``\ndirectory.\n\nReleases\n========\n\nYou can check https://docs.scrapy.org/en/latest/news.html for the release notes.\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnirala96%2Fweb_scraping_scripts","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnirala96%2Fweb_scraping_scripts","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnirala96%2Fweb_scraping_scripts/lists"}