https://github.com/nirala96/web_scraping_scripts
Some Web Scraping Scripts using scrapy written in python that scrapes the data from a website
https://github.com/nirala96/web_scraping_scripts
python scrapy scrapy-crawler webscraping
Last synced: 8 months ago
JSON representation
Some Web Scraping Scripts using scrapy written in python that scrapes the data from a website
- Host: GitHub
- URL: https://github.com/nirala96/web_scraping_scripts
- Owner: nirala96
- Created: 2020-10-13T02:02:19.000Z (over 5 years ago)
- Default Branch: main
- Last Pushed: 2021-05-30T07:21:45.000Z (about 5 years ago)
- Last Synced: 2025-01-31T22:11:23.835Z (over 1 year ago)
- Topics: python, scrapy, scrapy-crawler, webscraping
- Language: Python
- Homepage:
- Size: 637 KB
- Stars: 5
- Watchers: 1
- Forks: 0
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Web Scraping Scripts
Overview
========
Scrapy is a fast high-level web crawling and web scraping framework, used to
crawl websites and extract structured data from their pages. It can be used for
a wide range of purposes, from data mining to monitoring and automated testing.
Check the Scrapy homepage at https://scrapy.org for more information,
including a list of features.
### My Script contains 
- [IMDB in multiple domains](https://github.com/nirala69/Web_Scraping_scripts/tree/main/IMDB_detailed_scrape/IMDB) 
1. Most popular movies.
2. Lowest rated movies.
3. Most popular TV shows.
4. Top rated TV shows.
5. Top rated movies.
- [All restaurants in gandhinagar (Gujarat) from zomato.](https://github.com/nirala69/Web_Scraping_scripts/tree/main/zomato_scrape) 
- Basic scraping of quotes.scrape.com
### Technologies used
- Scrapy
### Languages used
- Python
- SQL
Requirements
============
* Python 3.6+
* Works on Linux, Windows, macOS, BSD
Install
=======
The quick way::
pip install scrapy
See the install section in the documentation at
https://docs.scrapy.org/en/latest/intro/install.html for more details.
Documentation
=============
Documentation is available online at https://docs.scrapy.org/ and in the ``docs``
directory.
Releases
========
You can check https://docs.scrapy.org/en/latest/news.html for the release notes.