Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/slurdge/goeland

An alternative to rss2email written in golang with many filters
https://github.com/slurdge/goeland

email-sender go golang hacktoberfest hacktoberfest2022 pipes rss rss-feed rss-feed-scraper rss2email

Last synced: 2 months ago
JSON representation

An alternative to rss2email written in golang with many filters

Awesome Lists containing this project

README 

        
# goeland



![goeland](cmd/asset/goeland_small.png)



![GitHub release (latest by date)](https://img.shields.io/github/v/release/slurdge/goeland)

![version](https://img.shields.io/github/go-mod/go-version/slurdge/goeland)

![GitHub](https://img.shields.io/github/license/slurdge/goeland)

![Image license](https://img.shields.io/badge/Images-CC%20BY--SA%204.0-blueviolet)



[![Build Status](https://github.com/slurdge/goeland/actions/workflows/build.yml/badge.svg)](https://github.com/slurdge/goeland/actions/workflows/build.yml)[![Docker images](https://github.com/slurdge/goeland/actions/workflows/docker.yml/badge.svg)](https://github.com/slurdge/goeland/actions/workflows/docker.yml)

![CodeQL](https://github.com/slurdge/goeland/actions/workflows/codeql-analysis.yml/badge.svg)



An RSS to email, à la rss2email, written in Go.



Support this project by giving it a ⭐️ and sharing it.



## About



Goeland excels at creating beautiful emails from RSS feeds, tailored for daily or weekly digest.



It includes a number of filters (see below) that can transform the RSS content along the way. It can also consume other sources, such as Imgur tags.



Goeland transforms this...



```xml



Phoronix

https://www.phoronix.com/



Linux Hardware Reviews, Benchmarks & Open-Source News



en-us



Google Announces KataOS As Security-Focused OS, Leveraging Rust & seL4 Microkernel



https://www.phoronix.com/news/Google-KataOS

https://www.phoronix.com/news/Google-KataOS



Google this week has announced the release of KataOS as their newest operating system effort focused on embedded devices running ambient machine learning workloads. KataOS is security-minded, exclusively uses the Rust programming language, and is built atop the seL4 microkernel as its foundation...



Sun, 16 Oct 2022 06:10:25 -0400



```



into this



![email](documentation/screenshots/phoronix_mixed.png)



Goeland has a size-fits-all default template that works well with mobile, tablet, desktop and webmail clients.



Goeland can extract full text from most article sources, enabling a ready to consume email.



## Status



Goeland is used in production with many email clients, and has sent over thousands of emails. It is considered stable.



## Installation



Grab the latest binary release from the [release page](https://github.com/slurdge/goeland/releases/latest/).

Binaries are available for the following platforms:



* linux/386

* linux/amd64

* linux/arm

* linux/arm64

* darwin/amd64

* windows/amd64

* windows/386



Just put it in a folder where you have write permissions and run it first with :



```console

goeland run

```



If you are interested for another platform to be supported, please open a PR or submit a feature request.



## Usage



On first run, if it doesn't exist yet, goeland will create a `config.toml` with the default values. You need to adjust the `[email]` section with your SMTP server details.

The config values can also be set with environment variables (e.g. `GOELAND_EMAIL_PASSWORD_FILE=/path/to/pass`).



### Sources



Afterwards, fill the `[sources]` and `[pipes]` sections.

Source are identified by their name after the `[source.]` field:



```toml

[sources.hackernews]

type = "feed"

url = "https://hnrss.org/newest"

filters = ["all", "today"]

```



You can then use `'hackernews'` in the following pipes.



The different source types are:



* `"feed"`: RSS, Atom or JSON feed (all supported formats can be found [here](https://github.com/mmcdole/gofeed#supported-feed-types)). Fill in the `url` field.

* `"imgur"`: Return most recent results for a tag. Fill in the the `tag` field.

* `"merge"`: Will merge two or more sources together. Fill in the `sources` field with a list of sources: `sources = ["source1", "source2"]`. Especially useful to merge different sources on the same topic. Don't forget to `digest` or `combine` it later.



### Filtering



One powerful aspect of goeland is filtering. Instead of sending the content of the RSS directly to the email system, it can transform it in a number of ways in order to make it easier to read, process, etc.



Any number of filters can be defined, **the order is important**. For example, the following:



```toml

filters = ["unseen", "retrieve", "digest"]

```



Will first keep only previously `unseen` entries, then make it nicer with the `retrieve` filter, and, at last, will put them all together with `digest`. This will create only one email with a SourceTitle as the title of the RSS feed.



Filters can have options. For example, to get the 3 newest post, you would do:

```toml

filters = ["first(3)"]

```



The available filters are as follows:



* none: Removes all entries

* all: Default, include all entries

* first: Keep only the first (usually newest) entries (default 1)

* last: Keep only the last (usually oldest) entries (default 1)

* reverse: Reverse the order of the entries

* random: Keep 1 or more random entries (default 5)

* unseen: Keep only unseen entry. Entries that have been seen will be put in a `goeland.db` file. Use the `purge` command to remove seen entries

* today: Keep only the entries of the day

* lasthours: Keep only the entries that are from the X last hours (default 24)

* digest: Make a digest of all entries (optional heading level, default 2)

* combine: Combine all the entries into one source and use the first entry title as source title. Useful for merged sources

* links: Rewrite relative links src="// and href="// to have an https:// prefix

* embedimage: Embed a picture if the entry has an attachment with a type of picture (optional position: top|bottom|left|right, default top)

* replace: Replace a string with another. Use with an argument like this: replace(myreplace) and define



```toml

[replace.myreplace]

        from="A string"

        to="Another string"

```



in your config file.



* includelink: Include the link of entries in the digest form

* includesourcetitle: Include source titles of entries in the digest form

* retrieve: Retrieves the full content from a goquery. E.g. you can use `retrieve(div.content)` to get the full excerpts of Next INpact's [LeBrief](https://www.nextinpact.com/lebrief/)

* language: Keep only the specified languages (best effort detection), use like this: `language(en,de)`

* untrack: Removes feedburner pixel tracking

* reddit: Better formatting for reddit rss

* sanitize: Sanitize the content of entries (to be used if --unsafe-no-sanitize-filter was passed)

* toc: Create a special table of content entry containing the titles of all entries. Use `toc(title)` to use the Title as a link

* limitwords: Limit the number of words in the entry, use like this:  `limitwords(32)`



### Pipes



After defining some sources, you can send them to a pipe. One source can be sent to multiple pipes, but a pipe can only have one source. If you need to combine sources together, use the above special `merge`.



A pipe has the following structure:



```toml

[pipes.hackernews]

source = "hackernews"

destination = "email"

email_to = "[email protected]"

email_from = "HackerNews "

email_title = "{{.EntryTitle}}"

template = "/path/to/template.html" # optional

```



You can use EntryTitle, SourceTitle and SourceName in the email template. SourceTitle is the title of the RSS feed.



For debugging purposes, or in order to pipe to other systems, you can set the destination to `terminal`.



### Email



In the email section you need to specify your outgoing mail server. You can specify both `encryption` and `allow-insecure` to connect to self-hosted servers. You can also specify `authentication` to select the appropriate option for your server (the options available are `"none"`, `"plain"`, `"login"` and `"crammd5"`; if unspecified it defaults to `"plain"`; see [`go-simple-mail`](https://pkg.go.dev/github.com/xhit/go-simple-mail/v2#AuthType)'s documentation for details).



```toml

[email]

host = "smtp.example.com"

port = 25

username = "default"

password = "p4ssw0rd"

# password_file = /run/password/goeland_smtp_pass

encryption = "tls"

allow-insecure = false

authentication = "plain"

#Email customization

include-header = true

include-footer = true

#footer = Your custom footer

#logo = internal:goeland.png

#template = /path/to/template.html

```



You can create your own template, see [relevant documentation](documentation/templates.md).

The pipe template takes precedence over the main template defined in the `[email]` section.



## Examples



This will bring you 6 puppies to your inbox.



```toml

loglevel = "info"

dry-run = false



[email]

host = "smtp.sendgrid.net"

port = 587

username = "apikey"

password = ""



[sources]



[sources.insta]

url = "https://rssbridge.example.com/?action=display&bridge=Instagram&context=Hashtag&h=puppy&media_type=picture&direct_links=on&format=MRss"

type = "feed"

filters = ["random(3)"]



[sources.imgur]

type = "imgur"

tag = "puppy"

filters = ["random(3)"]



[sources.puppies]

type = "merge"

sources = ["insta", "imgur"]

filters = ["combine"]



[pipes]



[pipes.puppies]

source = "puppies"

destination = "email"

email_to = ["[email protected]"]

email_from = "DailyPuppy "

```



This will give you the latest article on a specific subreddit:



```toml

loglevel = "none"

dry-run = false

database = "goeland.db"



[email]

host = "example.com"

port = 25

username = "username"

password = "password"



[sources]



[sources.reddit]

url = "https://www.reddit.com/r/selfhosted/top.rss"

type = "feed"

filters = ["unseen", "includelink", "digest"]



[pipes.reddit]

source = "reddit"

destination = "email"

email_to = ["[email protected]"]

email_from = "Reddit "

```



It is possible to send an email to multiple addresses, just put them in a list:



```toml

[pipes.reddit]

source = "reddit"

destination = "email"

email_to = ["[email protected]", "[email protected]", "[email protected]"]

email_from = "Reddit "

```



See also the `examples/` folder.



## Contributing



Feel free to open issues or PR for bugs and suggestions for more filters and source types.



If you encounter a problematic feed, please open an issue with the content of the feed attached.



## Future



Here is a list of things that could be nice



* image inliner

* embedded scripting language for filters&manipulation

* remove tags for instagram

* footer text

* use enclosure of the feed as header image

*