Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

https://github.com/2hands10fingers/Reddit-Image-Scraper-1.0

Scrapes/downloads a selected subreddit's posted images by a specified date range on http://reddit.com
https://github.com/2hands10fingers/Reddit-Image-Scraper-1.0

Last synced: 3 months ago
JSON representation

Scrapes/downloads a selected subreddit's posted images by a specified date range on http://reddit.com

Lists

README 

        
# W A R N I N G : CRUCIAL FEATURE DEPRECATED. NO LONGER WORKS.

 Unfortunately, Reddit has removed the ability to scan for submissions by date range. This eliminates the usefulness of this program and I'm deeply sorry to anyone who enjoyed using it for their use. Please message Reddit's technical team so we can have it back! Click here for the official announcement: https://www.reddit.com/r/changelog/comments/7tus5f/update_to_search_api/



 Thanks -2hands10fingers



# Reddit Image Scraper 1.0 brought to you by pyStudyGroup

Credits: /u/2hands10fingers, /u/Joe_Anonimist, /u/iakovosbelonias, tjcim



Scrapes Reddit images from any public subreddit by a user-specified date range and saves them to your computer.





____



# Introduction



Many tools scrape Reddit but don't give you the power to grab data on your terms. This changes things.



# How it Works



Use Python 3 to scrape any .jpg or .png images in the available subreddit (you can always add or remove file extensions within the code). Simply run the program in terminal using $ python3 RedditImageScraper.py under the correct directory where RedditImageScraper.py is downloaded, enter your start and ending dates, then you will select any public subreddit of your choice, and *PRESTO*! you will begin downloading images.



# Installation



1. Download



You can download this repository directly from GitHub and place the files where you want to run the program from. Remember, where the program is located, that is where your pictures will download to. You can change `file_path` in config.py to direct all downloads to that directory.



2. Install Dependencies



We make sure things are easy to install and test by using the requirements.txt file. In your terminal, type  $ pip install -r requirements.txt and you should have everything you need to get going. 



3. Obtaining Reddit Access



Oauth2 is required. Reddit doesn't like anonymous people taking their stuff, so you will need to copy the config.py.sample to config.py then open up the config.py file to add your own username, password, client_id, and client_secret to the file. Please follow Reddit's Oauth2 simple to follow guidelines to grab the client_id and client_secret here.



# Testing



RIS now comes with testing abilities and error logging. We recommend doing this within python3's virtual environement.



### Setupping up the virtual testing environment:




$ python3 -m venv env



$ source env/bin/activate



$ pip install -r requirements.txt



$ pytest






# Notes



Download speeds may vary depending on the length of your selected date range, the Reddit server's current performance, or your current internet speed. You will only be able to select the years 2005 (when Reddit was launched) through the current year. If you want to download all of the images for the current day, simply enter the date twice (e.g., 3-14-2017 -- 3-14-2017).



Thank you and enjoy,



2hands10fingers