https://github.com/pitops/reddit-bot-scraper

Scrape subreddits based on search criteria or get the X latest from 'hot' or 'new' categories
https://github.com/pitops/reddit-bot-scraper

puppeteer reddit-bot

Last synced: 1 day ago
JSON representation

Scrape subreddits based on search criteria or get the X latest from 'hot' or 'new' categories

• Host: GitHub
• URL: https://github.com/pitops/reddit-bot-scraper
• Owner: pitops
• Created: 2019-06-03T13:47:42.000Z (over 5 years ago)
• Default Branch: master
• Last Pushed: 2021-03-01T03:50:56.000Z (over 3 years ago)
• Last Synced: 2024-08-02T16:15:36.720Z (3 months ago)
• Topics: puppeteer, reddit-bot
• Language: JavaScript
• Size: 17.6 KB
• Stars: 25
• Watchers: 1
• Forks: 4
• Open Issues: 2
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Reddit Bot Scraper

## Description

The motivation behind this bot is to showcase how you can create a reddit scraper bot. The application for this specific bot is to get the `hot` or `new` comments(topics) in a specific category.

Moreover, it can be configured to search for specific keywords and bring only relevant results.

## Setup

Open up `index.js` and configure the subreddit to scrape from. See examples below for various ways to use it.

Example:

```javascript

// index.js

// initialize(subreddit: String, options: Object)

// if no options are passed it defaults to { headless: false, devtools: true}

await reddit.initialize("smallbusiness", {

  headless: false,

  devtools: false

});

```

### Usage Examples

```javascript

// Get latest 25 topics in specific subreddit

const results = await reddit.get();

// Search the 'new' tab of subreddit and bring first 10 results

const results = await reddit.get({

  type: "new",

  number: 10

});

// Get up to 5 results

const results = await reddit.get({

  limit: 5

});

// limit to 5 results matching keyword criteria

const results = await reddit.searchFor(["reminder", "appointment"]).get(5);

// Same as above in a more verbose way

const results = await reddit

  .searchFor(["reminder", "appointment"])

  .get({ limit: 5 });

// Get 100 topics and THEN filter by keywords

const results = await reddit.getLatest({

  type: "hot",

  number: 100,

  keywords: ["appointment", "reminder"]

});

// Alias of .get()

const results = await reddit.getLatestHot();

// Alias of .get() configured for the 'new' tab in Reddit

const results = await reddit.getLatestNew();

```

## Contributions welcome