Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/huaying/instagram-crawler
Get Instagram posts/profile/hashtag data without using Instagram API
https://github.com/huaying/instagram-crawler
auto autoliker instagram instagram-bot instagram-crawler instagram-liker instagram-scraper likers python scraper webdriver
Last synced: 7 days ago
JSON representation
Get Instagram posts/profile/hashtag data without using Instagram API
- Host: GitHub
- URL: https://github.com/huaying/instagram-crawler
- Owner: huaying
- License: mit
- Created: 2017-09-15T09:41:15.000Z (over 7 years ago)
- Default Branch: master
- Last Pushed: 2024-05-03T19:40:11.000Z (9 months ago)
- Last Synced: 2025-01-19T05:35:10.738Z (14 days ago)
- Topics: auto, autoliker, instagram, instagram-bot, instagram-crawler, instagram-liker, instagram-scraper, likers, python, scraper, webdriver
- Language: Python
- Homepage:
- Size: 26.1 MB
- Stars: 1,191
- Watchers: 40
- Forks: 287
- Open Issues: 63
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Instagram Crawler [](https://travis-ci.org/huaying/instagram-crawler)
Below is what you can do with this program:
- Get Instagram posts/profile/hashtag data without using Instagram API. `crawler.py`
- Like posts automatically. `liker.py`
This crawler could fail due to updates on instagram’s website. If you encounter any problems, please contact me.
## Install
1. Make sure you have Chrome browser installed.
2. Download [chromedriver](https://sites.google.com/a/chromium.org/chromedriver/) and put it into bin folder: `./inscrawler/bin/chromedriver`
3. Install Selenium: `pip3 install -r requirements.txt`
4. `cp inscrawler/secret.py.dist inscrawler/secret.py`
## User Auth
1. Open `inscrawler/secret.py` file.
2. Change the `username` and `password` variables' value to the ones corresponding to your Instagram account.
````
username = 'my_ig_username'
password = '***********'
````
## Crawler
### Usage
```
positional arguments:
mode
options: [posts, posts_full, profile, hashtag]
optional arguments:
-n NUMBER, --number NUMBER
number of returned posts
-u USERNAME, --username USERNAME
instagram's username
-t TAG, --tag TAG instagram's tag name
-o OUTPUT, --output OUTPUT
output file name(json format)
--debug see how the program automates the browser
--fetch_comments fetch comments
# Turning on the flag might take forever to fetch data if there are too many commnets.
--fetch_likes_plays fetch like/play number
--fetch_likers fetch all likers
# Instagram might have rate limit for fetching likers. Turning on the flag might take forever to fetch data if there are too many likes.
--fetch_mentions fetch users who are mentioned in the caption/comments (startwith @)
--fetch_hashtags fetch hashtags in the caption/comments (startwith #)
--fetch_details fetch username and photo caption
# only available for "hashtag" search
```
### Example
```
python crawler.py posts_full -u cal_foodie -n 100 -o ./output
python crawler.py posts_full -u cal_foodie -n 10 --fetch_likers --fetch_likes_plays
python crawler.py posts_full -u cal_foodie -n 10 --fetch_comments
python crawler.py profile -u cal_foodie -o ./output
python crawler.py hashtag -t taiwan -o ./output
python crawler.py hashtag -t taiwan -o ./output --fetch_details
python crawler.py posts -u cal_foodie -n 100 -o ./output # deprecated
```
1. Choose mode `posts`, you will get url, caption, first photo for each post; choose mode `posts_full`, you will get url, caption, all photos, time, comments, number of likes and views for each posts. Mode `posts_full` will take way longer than mode `posts`. **[`posts` is deprecated. For the recent posts, there is no quick way to get the post caption]**
2. Return default 100 hashtag posts(mode: hashtag) and all user's posts(mode: posts) if not specifying the number of post `-n`, `--number`.
3. Print the result to the console if not specifying the output path of post `-o`, `--output`.
4. It takes much longer to get data if the post number is over about 1000 since Instagram has set up the rate limit for data request.
5. Don't use this repo crawler Instagram if the user has more than 10000 posts.
The data format of `posts`:
The data format of `posts_full`:
## Liker
Set up your username/password in `secret.py` or set them as environment variables.
### Usage
```
positional arguments:
tag
optional arguments:
-n NUMBER, --number NUMBER (default 1000)
number of posts to like
```
### Example
```
python liker.py foodie
```