Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/mirusu400/pinterest-infinite-crawler
An infinite Pinterest crawler/scraper. Crawl image with inifnite-scroll!
https://github.com/mirusu400/pinterest-infinite-crawler
crawler hacktoberfest pinterest pinterest-downloader python scraper scraping selenium
Last synced: about 2 months ago
JSON representation
An infinite Pinterest crawler/scraper. Crawl image with inifnite-scroll!
- Host: GitHub
- URL: https://github.com/mirusu400/pinterest-infinite-crawler
- Owner: mirusu400
- License: mit
- Created: 2021-08-20T15:25:49.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2024-02-03T22:07:41.000Z (11 months ago)
- Last Synced: 2024-04-18T13:05:28.134Z (8 months ago)
- Topics: crawler, hacktoberfest, pinterest, pinterest-downloader, python, scraper, scraping, selenium
- Language: Python
- Homepage:
- Size: 2.87 MB
- Stars: 39
- Watchers: 3
- Forks: 7
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Pinterest-infinite-crawler
An **infinite** pinterest crawler, crawl image by page.
# Requirements
* Python 3.7+
* Selenium, requests, beautifulsoup4, pyyaml
* Chrome + Chromedriver
# Installation
1. Download requirements
```
git clone https://github.com/mirusu400/Pinterest-infinite-crawler.git
cd Pinterest-infinite-crawler
pip install -r requirements.txt
```
2. Download chromedriver
You **MUST** download [ChromeDriver](https://chromedriver.chromium.org/downloads) as the same version of [Chrome](chrome://settings/help).
And replace it the same directory with `main.py`.
3. (Optional) Set `config.yaml`
Copy `.config.yaml` to `config.yaml` and fill your Pinterest's email, password and directorys to save images
```
email: [your email here]
password: [your password here]
directory: ./download
```
# Usage
```
python main.py
```
# Using argument
You can also run crawler by passing argument, here are full document:
```
usage: main.py [-h] [-e EMAIL] [-p PASSWORD] [-d DIRECTORY] [-l LINK] [-g PAGE]
optional arguments:
-h, --help show this help message and exit
-e EMAIL, --email EMAIL Your Pinterest account email
-p PASSWORD, --password PASSWORD Your Pinterest account password
-d DIRECTORY, --directory DIRECTORY Directory you want to download
-l LINK, --link LINK Link of Pinterest which you want to scrape
-g PAGE, --page PAGE Number of pages which you want to scrape
-b BATCH, --batch BATCH Enable batch mode (Please read README.md!!)
```
**Example:**
> main.py -e [email protected] -p [your_password] -d download_image -l https://pinterest.com/ -g 10
# Batch mode
You can download multiple Pinterest links in a one, using batch mode
1. Copy and paste `.batch.json` to `batch.json` and modify json array files.
```
[
{
"index": "1",
"link": "https://www.pinterest.co.kr/pin/362750944993136496/",
"dir": "./download1"
},
{
"index": "2",
"link": "https://www.pinterest.co.kr/",
"dir": "./download2"
},
...
]
```
2. Use Batch mode in command line
> main.py -b
# Q & A
### What is `Link to scrape` mean?
You can select **any** pages what you want to scrape in Pinterest, not only main page. Such as:
* [Releative-pins of one pin](https://www.pinterest.co.kr/pin/643240759283703965/)
* [Someone's board](https://www.pinterest.co.kr/eaobrienae/croquies/)
* [A search result](https://www.pinterest.co.kr/search/pins/?q=Github)
* Or anything!
### Does it can download video?
No, you can only download jpg images from this tool. Video is not support for now.
# Contribute
If you find an issue or wants to contribute, please issue or pull request.