{"id":13858660,"url":"https://github.com/acavalin/rrss","last_synced_at":"2025-02-21T18:26:30.736Z","repository":{"id":179879020,"uuid":"15148298","full_name":"acavalin/rrss","owner":"acavalin","description":"RSS feed reader with web interface and SQLite data storage","archived":false,"fork":false,"pushed_at":"2014-10-23T19:26:42.000Z","size":432,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-02-17T19:47:49.616Z","etag":null,"topics":["atom-aggregator","atom-feed","atom-reader","rss","rss-aggregator","rss-reader","ruby","scraping","sinatra","webapp"],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/acavalin.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2013-12-12T21:46:44.000Z","updated_at":"2024-09-10T08:22:33.000Z","dependencies_parsed_at":null,"dependency_job_id":"5e7df5d3-ea97-4ef6-aae6-75232cc255e3","html_url":"https://github.com/acavalin/rrss","commit_stats":null,"previous_names":["acavalin/rrss"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/acavalin%2Frrss","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/acavalin%2Frrss/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/acavalin%2Frrss/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/acavalin%2Frrss/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/acavalin","download_url":"https://codeload.github.com/acavalin/rrss/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240066320,"owners_count":19742656,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["atom-aggregator","atom-feed","atom-reader","rss","rss-aggregator","rss-reader","ruby","scraping","sinatra","webapp"],"created_at":"2024-08-05T03:02:16.544Z","updated_at":"2025-02-21T18:26:30.708Z","avatar_url":"https://github.com/acavalin.png","language":"Ruby","readme":"# RRSS - Ruby RSS feed reader ##################################################\nRRSS is a Rss/Rdf/Atom feed reader written in [Ruby](http://www.ruby-lang.org) and using [Sinatra](http://www.sinatrarb.com) and [SQLite](http://www.sqlite.org).\n\nIt reads simple configuration files in [YML](http://yaml.org/spec/1.1) format, downloads and stores items in various SQLite databases and sports a nice web GUI to read and manage them (modify, comment, mark, etc...).\n\nRRSS is also able to:\n\n  - download gzipped files\n  - use scripts for scraping a web page (placed in ./scrapes)\n  - use scripts for manipulating the downloaded file (placed in ./scripts)\n  - use regular expressions for manipulating the downloaded file (:regexp: option)\n  - export stored items in JSON or ATOM format (with mark status too)\n  - run in batch mode without starting the GUI\n  - GUI: set custom feed favicon\n  - GUI: use a custom skin (CSS)\n\n\n# 0. Installation ##############################################################\n\n  1. check the ruby version you have with `ruby -v` and make sure it is **\u003e= 1.9** \n  2. obtain a copy of the project from github, you can either:\n    \n    - download the ZIP: `wget https://github.com/acavalin/rrss/archive/master.zip \u0026\u0026 cd rrss-master`\n    - clone the repository: `git clone https://github.com/acavalin/rrss.git \u0026\u0026 cd rrss`\n    \n  3. install the **bundler** gem: `gem install bundler`\n  4. now you can install all required gems with `bundle install`\n  5. edit **config.yml** as you prefer\n  5. create a **feeds.yml** config file (see *feeds.yml.example*)\n  6. run the application with `./rrss.rb`\n  7. point your browser to http://localhost:3333\n  8. ????\n  9. profit! ;^)\n\n# 1. Configuration #############################################################\n\n## 1.1 config.yml ##\n\n### 1.1.1 rss_dler options ###\nConfiguration for the feed downloader and parser:\n\u003ctable border=\"1\"\u003e\n  \u003ctr\u003e\u003cth\u003eKey\u003c/th\u003e\u003cth\u003eDescr\u003c/th\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003ehash_keys    \u003c/td\u003e\u003ctd\u003eItem properties to be hashed for generating its unique id.\u003cbr/\u003e\n                                Available keys are: :id, :link, :title, :descr, :date       \u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003emax_item_days\u003c/td\u003e\u003ctd\u003eNumber of required days for an item to be marked as old     \u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003emax_old_items\u003c/td\u003e\u003ctd\u003eMaximum number of old items to keep                         \u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003eparse_timeout\u003c/td\u003e\u003ctd\u003eRss parsing timeout (in seconds)                            \u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003eperiod       \u003c/td\u003e\u003ctd\u003eFeed check default interval time (in minutes)               \u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003etimeout      \u003c/td\u003e\u003ctd\u003eDownload/scrape timeout (in seconds)                        \u003c/td\u003e\u003c/tr\u003e\n\u003c/table\u003e\n\n### 1.1.2 rss_mngr options ###\nConfiguration for the feed manager (the web GUI):\n\u003ctable border=\"1\"\u003e\n  \u003ctr\u003e\u003cth\u003eKey\u003c/th\u003e\u003cth\u003eDescr\u003c/th\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003echeck_interval \u003c/td\u003e\u003ctd\u003eperiodic feed check interval time (in minutes)\u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003eexit_grace_time\u003c/td\u003e\u003ctd\u003efeed download grace time on exit (in minutes) \u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003elayout         \u003c/td\u003e\u003ctd\u003elayout css file name                          \u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003eport           \u003c/td\u003e\u003ctd\u003ewebserver (GUI) listening port                \u003c/td\u003e\u003c/tr\u003e\n\u003c/table\u003e\n\n## 1.2 feeds.yml ##\nThe file represents the feeds tree as an array of key-value options,\nhere is an example (see also *feeds.yml.example*):\n\n      ---\n      # this item is on the root folder\n      - :name:    example1\n        :link:    http://www.foo.com\n        :period:  10\n        :enabled: true\n        :regexp:  [['HELLO', 'Hello'], ['hi', 'HI']]\n      \n      - :name:    example2\n        :url:     http://bar.org/rss.xml\n        :period:  60\n        :enabled: true\n      \n      # an open folder (w/o ':' at the beginning)\n      - folder:\n        # a collapsed subfolder (w/ ':' at the beginning)\n        - :subfolder:\n          # these two items are children of \"subfolder\"\n          - :name:    example3A\n            :url:     http://www.foo2.com/rss.php\n            :period:  720\n            :enabled: true\n          \n          - :name:    example3B\n            :url:     http://www.bar2.com/atom.xml\n            :period:  720\n            :enabled: true\n        # these two items are children of \"folder\"\n        - :name:    example4A\n          :url:     http://www.fb.org/en/feeds/news.rss\n          :period:  1440\n          :enabled: true\n      \n        - :name:    example4B\n          :url:     http://www.xyz.net/news.xml\n          :period:  1440\n          :enabled: true\n\nEvery feed has a set of options you can use to customize it:\n\n\u003ctable border=\"1\"\u003e\n  \u003ctr\u003e\u003cth\u003eKey\u003c/th\u003e\u003cth\u003eDescr\u003c/th\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003e:name:         \u003c/td\u003e\u003ctd\u003efeed feedname ([a-z_])                                            \u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003e:enabled:      \u003c/td\u003e\u003ctd\u003eenable the periodic download for this feed (default false)        \u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003e:hash_keys:    \u003c/td\u003e\u003ctd\u003earray of item properties to be hashed for generating the unique id\u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003e:limit:        \u003c/td\u003e\u003ctd\u003eonly consider this quantity of most recent downloaded items       \u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003e:link:         \u003c/td\u003e\u003ctd\u003eclickable link on the feeds tree                                  \u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003e:parse_timeout:\u003c/td\u003e\u003ctd\u003eoverrider default parse timeout (in seconds)                      \u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003e:period:       \u003c/td\u003e\u003ctd\u003eperiodic download interval (in minutes)                           \u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003e:regexp:       \u003c/td\u003e\u003ctd\u003earray of pairs [regexp, replace_string] to manipulate items       \u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003e:summary:      \u003c/td\u003e\u003ctd\u003esave and show the summary of the item (default true)              \u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003e:timeout:      \u003c/td\u003e\u003ctd\u003eoverrider default downaload timeout (in seconds)                  \u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003e:url:          \u003c/td\u003e\u003ctd\u003eurl of the xml file to download                                   \u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003e:validation:   \u003c/td\u003e\u003ctd\u003eapply feed validation during parsing (default true)               \u003c/td\u003e\u003c/tr\u003e\n\u003c/table\u003e\n\nAs you can see in the previous example, a folder comes in two flavors:\n\n  - \u003c\u003c name **:**       \u003e\u003e renders an expanded folder on the feeds tree\n  - \u003c\u003c **:** name **:** \u003e\u003e renders a collapsed folder on the feeds tree\n\n### 1.2.1 OPML import/conversion ###############################################\nHere is a useful command line combo to perform an easy OPML (indented XML) to YML\nconversion:\n\n      cat feedlist.opml | \\\n        sed 's/\u003coutline title=\"\\(.*\\)\" text=\".*\"\u003e/- :\\L\\1:/' | \\\n        sed 's/\\( \\+\\)\u003coutline text=\"\\([^\"]*\\)\".*htmlUrl=\"\\([^\"]*\\)\" xmlUrl=\"\\([^\"]*\\)\".*\\/\u003e/\\1- :name:    \\L\\2\\E\\n\\1  :url:     \\4\\n\\1  :link:    \\3\\n\\1  :enabled: true\\n/' | \\\n        grep -v \"\u003c.outline\u003e\" \u003e feeds.yml\n\n## 1.3 Feed processing ##\nWhen adding a new feed, keep in mind the retrival/manipulation steps the\napplication will perform on the downloaded file:\n\n  1. if **./scrapes/feedname** exists and is executable then run it and capture its\n     output\n  2. otherwise download the file specified in **:url:**\n  3. if **./scripts/feedname** exists and is executable then use it to convert the\n     previous output (it must read the input from stdin and print output to stdout)\n  4. sequentially apply every eventual regexp specified in **:regexp:**\n  5. convert contents to UTF-8, parse and store them to **./db/feedname.db**\n  6. autopurge old items (only read and unkept ones)\n\n# 2. Feed GUI/Webserver ########################################################\n\n## 2.1 Keyboard shortcuts ######################################################\n\n\u003ctable border=\"1\"\u003e\n  \u003ctr\u003e\u003cth\u003eKey \u003c/th\u003e\u003cth\u003eFunction                           \u003c/th\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003eh   \u003c/td\u003e\u003ctd\u003eshow help                          \u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003en   \u003c/td\u003e\u003ctd\u003eselect next unread item            \u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003edown\u003c/td\u003e\u003ctd\u003eselect next item                   \u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003em/up\u003c/td\u003e\u003ctd\u003eselect previous item               \u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003ehome\u003c/td\u003e\u003ctd\u003eselect first item                  \u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003eend \u003c/td\u003e\u003ctd\u003eselect last item                   \u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003eu   \u003c/td\u003e\u003ctd\u003etoggle unread on selected item     \u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003ek   \u003c/td\u003e\u003ctd\u003etoggle kept on selected item       \u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003eesc \u003c/td\u003e\u003ctd\u003eclose/reset view                   \u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003ev   \u003c/td\u003e\u003ctd\u003echange view filter                 \u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003el   \u003c/td\u003e\u003ctd\u003eshow linear view in list mode      \u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003eL   \u003c/td\u003e\u003ctd\u003eshow linear view in thumbs mode    \u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003er   \u003c/td\u003e\u003ctd\u003erefresh feeds tree                 \u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003eR   \u003c/td\u003e\u003ctd\u003emark all feed items as read        \u003c/td\u003e\u003c/tr\u003e\n  \u003ctr\u003e\u003ctd\u003es   \u003c/td\u003e\u003ctd\u003esearch items in current feed/folder\u003c/td\u003e\u003c/tr\u003e\n\u003c/table\u003e\n\n## 2.2 Feed items export #######################################################\nYou can download all desired feed items by using the following urls:\n\n  - http://ip_address:port/dump/feed_name.xml    *(atom feed)*\n  - http://ip_address:port/dump/feed_name.json   *(json object)*\n  \n**Note**: Feed/item preferences are included in the XML/Atom file within *dc_type*\ntags.\n\n## 2.3 Change feed favicon #####################################################\nTo set a custom favicon for a specific feed use the script *set_favicon.rb*:\n\n    set_favicon.rb feed_name favicon_uri\n\nwhere **feed_name** is the name specified in **:name:** and the **favicon_uri**\ncan be either an URL or a local file PATH.\n\n# 3. Feed downloader ###########################################################\n\n## 3.1 Batch mode/dump #########################################################\nYou can run the download process of your feeds in batch mode using the script\n**check_feeds.rb**:\n\n    check_feeds.rb [dump_dir [format]]\n\nif you supply a dump directory then the processed feed will be dumped on that\nplace.\nThe format can be either *xml* or *json*.\n\n# 4. Referrer/External resources ###############################################\nWebservers tend to block a *localhost* referrer for feeds that rely on external\nresources like images :'(\n\nIf you use Firefox, you can bypass this problem by installing the\n[Referrer Control](https://addons.mozilla.org/firefox/addon/referrer-control/)\nextension; you can find the full documentation on its\n[wiki page](https://github.com/muzuiget/referrer_control/wiki)\n \nYou just need to add a custom rule:\n\n    *localhost*, \u003cany\u003e, \u003cremove\u003e\n\nand remember to set the default rule to **Skip** if you wish to preserve the\nbrowser default behaviour.\n\n# A. Reference documentation ###################################################\nHere is a list of the specs, libraries and tools used to develop RRSS:\n\n  * RSS\n    - http://ruby-doc.com/stdlib-1.9.2/libdoc/rss/rdoc/RSS.html\n    - http://www.cozmixng.org/~rwiki/?cmd=view;name=RSS+Parser%3A%3ATutorial.en\n    - http://en.wikipedia.org/wiki/RSS\n  * Ruby libs\n    - http://ruby-doc.org/stdlib-1.9.3/libdoc/timeout/rdoc/Timeout.html\n    - http://ruby-doc.org/stdlib-1.9.3/libdoc/open-uri/rdoc/OpenURI.html\n    - http://ruby-doc.org/stdlib-1.9.3/libdoc/digest/rdoc/Digest.html\n    - http://ruby-doc.org/stdlib-1.9.3/libdoc/fileutils/rdoc/FileUtils.html\n    - http://ruby-doc.org/stdlib-1.9.3/libdoc/open3/rdoc/Open3.html\n    - http://ruby-doc.com/stdlib-1.9.2/libdoc/zlib/rdoc/Zlib.html\n    - http://ruby-doc.org/stdlib-1.9.3/libdoc/base64/rdoc/Base64.html\n    - http://ruby-doc.org/stdlib-1.9.3/libdoc/stringio/rdoc/StringIO.html\n    - http://ruby-doc.org/stdlib-1.9.3/libdoc/logger/rdoc/Logger.html\n    - http://www.ruby-doc.org/stdlib-1.9.3/libdoc/json/rdoc/JSON.html\n  * Sinatra\n    - http://www.sinatrarb.com/configuration.html\n    - http://www.sinatrarb.com/extensions.html\n    - http://www.sinatrarb.com/contrib/reloader\n  * SQLite\n    - http://sqlite-ruby.rubyforge.org/sqlite3/faq.html\n    - http://www.sqlite.org/lang.html\n    - http://www.sqlite.org/pragma.html#syntax\n    - http://www.sqlite.org/datatype3.html\n  * YAML\n    - http://yaml.org/spec/1.1/\n  * Firefox referrer control\n    - https://github.com/muzuiget/referrer_control/wiki\n","funding_links":[],"categories":["Ruby"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Facavalin%2Frrss","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Facavalin%2Frrss","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Facavalin%2Frrss/lists"}