https://github.com/drewish/fullfeeds

Follows links in RSS feeds, and merges the linked content back into a new feed.
https://github.com/drewish/fullfeeds

Last synced: 16 days ago
JSON representation

Follows links in RSS feeds, and merges the linked content back into a new feed.

• Host: GitHub
• URL: https://github.com/drewish/fullfeeds
• Owner: drewish
• Created: 2012-01-27T04:49:01.000Z (almost 13 years ago)
• Default Branch: master
• Last Pushed: 2013-02-05T15:39:31.000Z (almost 12 years ago)
• Last Synced: 2024-11-01T13:11:42.315Z (2 months ago)
• Language: JavaScript
• Homepage:
• Size: 344 KB
• Stars: 4
• Watchers: 3
• Forks: 1
• Open Issues: 5
• Metadata Files:
  • Readme: Readme.md

Awesome Lists containing this project

README

        
# Full Feeds

  Enhances RSS feeds by following their links, extracting additional content

  from those pages and incorporating it into a new feed.

  You're also able to modify the fetched URLs e.g. load a print friendly

  version or skip over ad posts.

## Requirements

  Dependencies:

  * async

  * feedparser

  * jsdom

  * redis

  * rss

  Full Feeds uses redis to cache the linked aticles so you'll need to have a

  server. Currently, the configuration for that is hardcoded to the defaults.

## Simple Example

    var fullfeeds = require('fullfeeds'),

        fs = require('fs');

    var config = [

      {

        name: 'paulgraham',

        url: 'http://www.aaronsw.com/2002/feeds/pgessays.rss',

        // Optionally, you can specify a function to pull modify an article's URL.

        urlExtractor: function(article) {

          return article.guid;

        },

        // Use a Sizzle selector to specify the content on the page.

        selector: 'table table tr td'

      }

    ];

    // Invoke it with the configuration and a callback. The callback will be

    // passed an error and then results. Results will be an array with information

    fullfeeds(config, function(err, results) {

      results.forEach(function(result) {

        var path = __dirname + '/output/' + result.config.name + '.xml';

        console.log("%s: Saving to %s", result.config.name, path);

        fs.writeFile(path, result.build_feed.xml(), function(err) {

          if (err) {

            console.error(err);

          }

        });

      })

    });