{"id":15517328,"url":"https://github.com/skx/rss2email","last_synced_at":"2025-04-04T15:09:22.677Z","repository":{"id":38383174,"uuid":"139706545","full_name":"skx/rss2email","owner":"skx","description":"Convert RSS feeds to emails","archived":false,"fork":false,"pushed_at":"2025-03-08T19:19:36.000Z","size":409,"stargazers_count":116,"open_issues_count":3,"forks_count":19,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-03-28T14:07:59.452Z","etag":null,"topics":["email","golang","golang-application","rss","rss2email"],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/skx.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"skx","custom":"https://steve.fi/donate/"}},"created_at":"2018-07-04T10:24:02.000Z","updated_at":"2025-03-25T08:12:21.000Z","dependencies_parsed_at":"2022-07-18T10:39:04.157Z","dependency_job_id":"d06f3930-8553-415b-9af6-9a22dde4ba98","html_url":"https://github.com/skx/rss2email","commit_stats":{"total_commits":285,"total_committers":13,"mean_commits":"21.923076923076923","dds":0.1578947368421053,"last_synced_commit":"224eec7368595386cfb51b3d52299060202d5f97"},"previous_names":[],"tags_count":32,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/skx%2Frss2email","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/skx%2Frss2email/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/skx%2Frss2email/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/skx%2Frss2email/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/skx","download_url":"https://codeload.github.com/skx/rss2email/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247198461,"owners_count":20900080,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["email","golang","golang-application","rss","rss2email"],"created_at":"2024-10-02T10:12:32.711Z","updated_at":"2025-04-04T15:09:22.657Z","avatar_url":"https://github.com/skx.png","language":"Go","funding_links":["https://github.com/sponsors/skx","https://steve.fi/donate/"],"categories":[],"sub_categories":[],"readme":"[![Go Report Card](https://goreportcard.com/badge/github.com/skx/rss2email)](https://goreportcard.com/report/github.com/skx/rss2email)\n[![license](https://img.shields.io/github/license/skx/rss2email.svg)](https://github.com/skx/rss2email/blob/master/LICENSE)\n[![Release](https://img.shields.io/github/release/skx/rss2email.svg)](https://github.com/skx/rss2email/releases/latest)\n\nTable of Contents\n=================\n\n* [RSS2Email](#rss2email)\n* [Installation](#installation)\n  * [bash completion](#bash-completion)\n* [Feed Configuration](#feed-configuration)\n* [Usage](#usage)\n* [Daemon Mode](#daemon-mode)\n* [Initial Run](#initial-run)\n* [Assumptions](#assumptions)\n* [Email Customization](#email-customization)\n  * [Changing default From address](#changing-default-from-address)\n* [Implementation Overview](#implementation-overview)\n* [Logging Notes](#logging-notes)\n* [Github Setup](#github-setup)\n\n\n\n\n# RSS2Email\n\nThis project began life as a naive port of the python-based [r2e](https://github.com/wking/rss2email) utility to golang.\n\nOver time we've now gained a few more features:\n\n* The ability to customize the email-templates which are generated and sent.\n  * See [email customization](#email-customization) for details.\n* The ability to send email via STMP, or via `/usr/sbin/sendmail`.\n  * See [SMTP-setup](#smtp-setup) for details.\n* The ability to include/exclude feed items from the emails.\n  * For example receive emails only of feed items that contain the pattern \"playstation\".\n* A well-behaved HTTP-polling behaviour, using the appropriate cache-related HTTP-headers.\n\n\n\n# Installation\n\nIf you have golang installed you can fetch, build, and install the latest binary by running:\n\n```sh\ngo install github.com/skx/rss2email@latest\n```\n\nIf you prefer you can also fetch our latest binary release from [our release page](https://github.com/skx/rss2email/releases).\n\nTo install from source simply clone the repository and build in the usual manner:\n\n```sh\ngit clone https://github.com/skx/rss2email\ncd rss2email\ngo install .\n```\n\nFinally you can find automatically generated docker images, these are built on a nightly basis, and when releases are made:\n\n* https://github.com/skx/rss2email/pkgs/container/rss2email\n\n\n\n**Version NOTES**:\n\n* You'll need go version **1.21** or higher to build.\n  * We use `go embed` to embed the (default) email-template within the binary, this was introduced with golang **v1.17**.\n  * We use the [slog logging package](https://go.dev/blog/slog) introduced with golang **v1.21**.\n  * We use the fuzzing support which was introduced with golang **v1.18** to test our configuration-file loading/parsing.\n    * See [configfile/FUZZING.md](configfile/FUZZING.md) for details of using that.\n\n\n\n## bash completion\n\nThe binary has integrated support for TAB-completion, for bash.  To enable this update your [dotfiles](https://github.com/skx/dotfiles/) to include the following:\n\n```\nsource \u003c(rss2email bash-completion)\n```\n\n\n\n\n# Feed Configuration\n\nOnce you have installed the application you'll need to configure the feeds to monitor, this could be done by editing the configuration file:\n\n* `~/.rss2email/feeds.txt`\n\nThere are several built-in sub-commands for manipulating the feed-list, for example you can add a new feed to monitor via the `add` sub-command:\n\n     $ rss2email add https://example.com/blog.rss\n\nOPML files can be imported via the `import` sub-command:\n\n     $ rss2email import feeds.opml\n\nThe list of feeds can be displayed via the `list` subcommand (note that adding the `-verbose` flag will fetch each of the feeds and that will be slow):\n\n     $ rss2email list [-verbose]\n\nFinally you can remove an entry from the feed-list via the `delete` sub-command:\n\n     $ rss2email delete https://example.com/foo.rss\n\nThe configuration file in its simplest form is nothing more than a list of URLs, one per line.  However there is also support for adding per-feed options:\n\n       https://foo.example.com/\n        - key:value\n       https://foo.example.com/\n        - key2:value2\n\nThis is documented and explained in the integrated help:\n\n    $ rss2email help config\n\nAdding per-feed items allows excluding feed-entries by regular expression, for example this does what you'd expect:\n\n       https://www.filfre.net/feed/rss/\n        - exclude-title: The Analog Antiquarian\n\n\n\n\n# Usage\n\nOnce you've populated your feed list, via a series of `rss2email add ..` commands, or by editing the configuration file directly, you are now ready to actually launch the application.\n\nTo run the application, announcing all new feed-items by email to `user@host.com` you'd run this:\n\n    $ rss2email cron user@host.com\n\nOnce the feed-list has been fetched, and items processed, the application will terminate.  It is expected that you'll add an entry to your `crontab` file to ensure this runs regularly.  For example you might wish to run the check \u0026 email process once every 15 minutes, so you could add this:\n\n     # Announce feed-changes via email four times an hour\n     */15 * * * * $HOME/go/bin/rss2email cron recipient@example.com\n\nWhen new items appear in the feeds they will then be sent to you via email.\nEach email will be multi-part, containing both `text/plain` and `text/html`\nversions of the new post(s).  There is a default template which should contain\nthe things you care about:\n\n* A link to the item posted.\n* The subject/title of the new feed item.\n* The HTML and Text content of the new feed item.\n\nIf you wish you may customize the template which is used to generate the notification email, see [email-customization](#email-customization) for details.  It is also possible to run in a [daemon mode](#daemon-mode) which will leave the process running forever, rather than terminating after walking the feeds once.\n\nThe state of feed-entries is recorded beneath `~/.rss2email/state.db`, which is a [boltdb database](https://pkg.go.dev/go.etcd.io/bbolt).\n\n\n\n\n# Daemon Mode\n\nTypically you'd invoke `rss2email` with the `cron` sub-command as we documented above.  This works in the naive way you'd expect:\n\n* Read the contents of each URL in the feed-list.\n* For each feed-item which is new generate and send an email.\n* Terminate.\n\nThe `daemon` process does a similar thing, however it does __not__ terminate.  Instead the process becomes:\n\n* Read the contents of each URL in the feed-list.\n* For each feed-item which is new generate and send an email.\n* Sleep for 5 minutes.\n* Begin the process once more.\n\nWith this behaviour every feed will be fetched and processed every five minutes, which is almost certainly too frequently.  To change this we have a notion of \"frequency\" - a feed will never be fetched more frequently than the given frequency value.\n\n* Set the `SLEEP` environmental variable if you wish to change globally.\n  * e.g. \"`export SLEEP=15`\" will cause our main loop to fetch the feeds only once every fifteen minutes.\n* Set the per-feed `frequency` option to a different value.\n  * That would mean all feeds would get fetched every fifteen minutes.\n  * Except for the specific one that has a different value.\n\n\u003e NOTE: Frequency values of less than five minutes will be ignored.\n\nIn short the process runs forever, in the foreground.  This is expected to be driven by `docker` or a systemd-service.  Creating the appropriate configuration is left as an exercise, but you might examine the following two files for inspiration:\n\n* [Dockerfile](Dockerfile)\n* [docker-compose.yml](docker-compose.yml)\n\n\n\n\n# Initial Run\n\nWhen you add a new feed all the items contained within that feed will initially be unseen/new, and this means you'll receive a flood of emails if you were to run:\n\n     $ rss2email add https://blog.steve.fi/index.rss\n     $ rss2email cron user@domain.com\n\nTo avoid this you can use the `-send=false` flag, which will merely\nrecord each item as having been seen, rather than sending you emails:\n\n     $ rss2email add https://blog.steve.fi/index.rss\n     $ rss2email cron -send=false user@domain.com\n\n\n\n\n# Assumptions\n\nBecause this application is so minimal there are a number of assumptions baked in:\n\n* We assume that `/usr/sbin/sendmail` exists and will send email successfully.\n  * You can cause emails to be sent via SMTP, see [SMTP-setup](#smtp-setup) for details.\n* We assume the recipient and sender email addresses can be the same.\n  * i.e. If you mail output to `bob@example.com` that will be used as the sender address.\n  * You can change the default sender via the [email-customization](#email-customization) process described next if you prefer though.\n\n\n\n\n# SMTP Setup\n\nBy default the outgoing emails we generate are piped to `/usr/sbin/sendmail` to be delivered.  If that is unavailable, or unsuitable, you can instead configure things such that SMTP is used directly.\n\nTo configure SMTP you need to setup the following environmental-variables (environmental variables were selected as they're natural to use within Docker and systemd-service files).\n\n\n| Name              | Example Value     |\n|-------------------|-------------------|\n| **SMTP_HOST**     | `smtp.gmail.com`  |\n| **SMTP_PORT**     | `587`             |\n| **SMTP_USERNAME** | `bob@example.com` |\n| **SMTP_PASSWORD** | `secret!value`    |\n\nIf those values are present then SMTP will be used, otherwise the email will be sent via the local MTA.\n\n\n\n\n# Email Customization\n\nBy default the emails are sent using a template file which is embedded in the application.  You can override the template by creating the file `~/.rss2email/email.tmpl`, if that is present then it will be used instead of the default.\n\nYou can view the default template via the following command:\n\n    $ rss2email list-default-template\n\nYou can copy the default-template to the right location by running the following, before proceeding to edit it as you wish:\n\n    $ rss2email list-default-template \u003e ~/.rss2email/email.tmpl\n\nThe default template contains a brief header documenting the available fields, and functions, which you can use.  As the template uses the standard Golang [text/template](https://golang.org/pkg/text/template/) facilities you can be pretty creative with it!\n\nIf you're a developer who wishes to submit changes to the embedded version you should carry out the following two-step process to make your change.\n\n* Edit `template/template.txt`, which is the source of the template.\n* Rebuild the application to update the embedded copy.\n\n**NOTE**: If you read the earlier section on configuration you'll see that it is possible to add per-feed configuration values to the config file.  One of the supported options is to setup a feed-specific template-file.\n\n\n\n## Changing default From address\n\nAs noted earlier when sending the notification emails the recipient address is used as the sender-address too.   There are no flags for changing the From: address used to send the emails, however using the section above you can [use a customized email-template](#email-customization), and simply update the template to read something like this:\n\n```\nFrom: my.sender@example.com\nTo: {{.To}}\nSubject: [rss2email] {{.Subject}}\nX-RSS-Link: {{.Link}}\nX-RSS-Feed: {{.Feed}}\n```\n\n* i.e. Change the `{{.From}}` to your preferred sender-address.\n\n\n\n\n# Implementation Overview\n\nThe two main commands are `cron` and `daemon` and they work in roughly the same way:\n\n* They instantiate [processor/processor.go](processor/processor.go) to run the logic\n  * That walks over the list of feeds from [configfile/configfile.go](configfile/configfile.go).\n  * For each feed [httpfetch/httpfetch.go](httpfetch/httpfetch.go) is used to fetch the contents.\n  * The result is a collection of `*gofeed.Feed` items, one for each entry in the remote feed.\n    * These are wrapped via [withstate/feeditem.go](withstate/feeditem.go) so we can test if they're new.\n    * [processor/emailer/emailer.go](processor/emailer/emailer.go) is used to send the email if necessary.\n    * Either by SMTP or by executing `/usr/sbin/sendmail`\n\nThe other subcommands mostly just interact with the feed-list, via the use of [configfile/configfile.go](configfile/configfile.go) to add/delete/list the contents of the feed-list.\n\n\n\n\n# Logging Notes\n\nThe application is configured to use a common logger, which will output all messages to STDERR.  The codebase will log messages at three levels:\n\n* DEBUG\n  * This will be used when new features are added, and contain implementation-related notices.\n  * The messages here will be helpful for debugging, or extending the application.\n* WARN\n  * This level is shown by default.\n  * This level is used for messages which are not fatal errors, but which a user might wish to be aware of.\n    * For example failure to fetch a remote feed, or a count of retried HTTP-fetches.\n* ERROR\n  * This level is shown by default.\n  * This level is used for fatal-errors.\n  * This should only be used for messages which are immediately followed by an application exit.\n\nThere are several environmental variables which can be used to modify the logging output:\n\n* `LOG_LEVEL`\n  * This may be set to one of several values:\n    * `ERROR`: Show only errors.\n    * `WARN`: Show errors, and warnings.\n    * `DEBUG`: Show errors, warnings, and internal debugging messages (very verbose).\n* `LOG_FILE_PATH`\n  * The name of the file to duplicate logging message to, defaults to being `rss2email.log`.\n* `LOG_FILE_DISABLE`\n  * Set this to any non-empty value to disable the logfile entirely.\n* `LOG_JSON`\n  * If this is set to a non-empty string the logging messages will be output in JSON format.\n  * This is useful if you're collecting (container) messages in datadog, loki, sumologic, or something similar.\n\nBot the `cron` and `daemon` sub-commands will switch to showing DEBUG messages if you supply the `-verbose` flag to them, which avoids the need for setting environmental variables.\n\n\n\n\n# Github Setup\n\nThis repository is configured to run tests upon every commit, and when\npull-requests are created/updated.  The testing is carried out via\n[.github/run-tests.sh](.github/run-tests.sh) which is used by the\n[github-action-tester](https://github.com/skx/github-action-tester) action.\n\nReleases are automated in a similar fashion via [.github/build](.github/build),\nand the [github-action-publish-binaries](https://github.com/skx/github-action-publish-binaries) action.\n\nSteve\n--\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fskx%2Frss2email","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fskx%2Frss2email","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fskx%2Frss2email/lists"}