{"id":17022537,"url":"https://github.com/rknightuk/echo","last_synced_at":"2025-03-17T09:31:25.583Z","repository":{"id":67574691,"uuid":"587303068","full_name":"rknightuk/echo","owner":"rknightuk","description":"Post RSS feeds to Micro.blog and Mastodon","archived":false,"fork":false,"pushed_at":"2024-04-12T21:29:09.000Z","size":1152,"stargazers_count":74,"open_issues_count":0,"forks_count":5,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-05-23T04:04:37.482Z","etag":null,"topics":["feed","indieweb","mastodon","microblog","now","rss"],"latest_commit_sha":null,"homepage":"https://echo.rknight.me","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rknightuk.png","metadata":{"files":{"readme":"readme.md","changelog":"changelog.md","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2023-01-10T12:48:00.000Z","updated_at":"2024-05-11T09:17:46.000Z","dependencies_parsed_at":"2023-11-18T03:02:38.206Z","dependency_job_id":"685eb5ee-09fc-4507-a46e-08ab24eaf40c","html_url":"https://github.com/rknightuk/echo","commit_stats":{"total_commits":47,"total_committers":3,"mean_commits":"15.666666666666666","dds":0.06382978723404253,"last_synced_commit":"8a86e4b0af8b72fa76077851b5c0ed2e44f94e16"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rknightuk%2Fecho","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rknightuk%2Fecho/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rknightuk%2Fecho/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rknightuk%2Fecho/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rknightuk","download_url":"https://codeload.github.com/rknightuk/echo/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243731045,"owners_count":20338761,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["feed","indieweb","mastodon","microblog","now","rss"],"created_at":"2024-10-14T07:10:36.704Z","updated_at":"2025-03-17T09:31:25.166Z","avatar_url":"https://github.com/rknightuk.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Echo - RSS Cross Poster\n\n![Echo screenshot](screenshot.png)\n\n## What is it?\n\nEcho is a node script to post new items from an RSS feed to various services including Micro.blog and Mastodon.\n\n### Why \"Echo\"?\n\nIt does RSS feeds, so Feeder. Feeder are a band with an album called Echo Park. Echo is a good name because the album link AND the meaning of the word echo. So there.\n\n## Requirements\n\n- Node 19 (it might work with earlier versions but that's what I used)\n- A server/computer/potato to run it on\n\n## Installation\n\n1. Clone this repository\n2. Run `npm install` to install the node module\n3. Run `cp config.example.js config.js` to create a new config file\n4. Setup your RSS feeds and services (see [configuration](#configuration) for options)\n5. Run `node index.js init` to setup - this will store the latest ID so only new posts going forward will be posted. If you want to post _some_ items from a feed, add the ID of the latest item you don't want to post in `data/nameofsite.txt`.\n6. Setup a cron to run `node index.js` regularly\n\n🚨 **Warning**: If you don't run `node index.js init` first, the script will post **all** the posts in the RSS feeds. You _probably_ don't want this.\n\nYou can also run `node index.js dry`. This will log which posts will be created, but _will not_ post anything.\n\nEcho keeps track of the last item posted so on subsequent runs it will only post new posts.\n\nYou can also run Echo with GitHub actions. [See Lewis' blog post for more info](https://lewisdale.dev/post/using-gitea-github-actions-for-triggering-echo/)\n\n## Configuration\n\nThere are two parts to configure: `sites` and `services`. `sites` is the RSS feeds you want to cross-post and `services` is the services you want to cross-post to.\n\nGo to [the Echo website](https://echo.rknight.me) to use the config generator and paste the generated config into `config.js` or see below for setting it up manually.\n\n### Sites\n\n`config.sites` is an array of RSS feeds you wish to cross-post. A site has five attribute:\n\n- `name` (required): this can be anything (this is used in a filename so probably don't use special characters).\n- `feed` (required): the feed URL you want to post to Micro.blog (e.g. \u003chttps://mycoolsite.com/feed\u003e).\n- `categories` (optional - only for Micro.blog): An array of categories to assign to you posts for the site (e.g. `[\"Cat One\", \"Cat Two\"]`).\n- `services`: The services you want to cross-post to. See [services](#services).\n- `transform`: this is an object with two functions (see below for preset transforms):\n  - `getId`: This tells Echo which attribute to use for the ID of each feed item. Most feeds use `id` or `guid` but if it's something different you can set that here.\n  - `format`: This is how you format the title, body, and date of the post. This returns an object with content, date, and an optional title.\n  - `filter` (optional): Use this if you need to filter out specific items in a feed. For example, Letterboxd includes items for lists being updated which I don't want to be posted.\n\n#### Example Site Configuration\n\n```js\n{\n    name: \"example.com\",\n    feed: \"http://example.com/feed\",\n    categories: [\"my category\"],\n    services: [SERVICES.MICROBLOG, SERVICES.WEBHOOK]\n    transform: {\n        getId: (data) =\u003e {\n            return data.id\n        },\n        format: (data) =\u003e {\n            return {\n                content: data.content,\n                date: data.isoDate,\n                title: data.title, // optional\n            }\n        },\n        filter: (items) =\u003e {\n            return items.filter(item =\u003e {\n                return !item.link.includes('/list/')\n            })\n        }\n    }\n}\n```\n\n### Preset Transforms\n\nEcho has a few presets you can use instead of having to write the `getId` and `format` functions for every site. These can be seen in [`presets.js`](lib/presets.js). For example, to use the Letterboxd or status.lol preset you can do the following:\n\n```js\n{\n    name: \"letterboxd.com\",\n    feed: \"http://letterboxd.com/exampleuser/rss\",\n    categories: [\"movies\"],\n    transform: presets.letterboxd,\n},\n{\n    name: \"status.lol\",\n    feed: \"http://exampleuser.status.lol/feed\",\n    categories: [\"status\"],\n    transform: presets.statuslol,\n}\n```\n\nYou can define the body of your post in `format` to make your posts look exactly how you want. For ease, Echo includes `helpers.js` with some helper libraries.\n\n- To convert HTML to markdown, use `helpers.toMarkdown(text)`\n- To use [Cheerio](https://cheerio.js.org/) use `helpers.cheerioLoad(text)`\n- To generate a UUID use `helpers.generateUuid()`\n- Get the length of the post based on how Mastodon calculates this (links are always 23 characters, for example) `helpers.getMastodonLength(string)`\n- Get all links as an array `helpers.getLinks(string)`\n- Encode and decode HTML entities with `helpers.decode(string)` and `helpers.encode(string)`\n\n```js\nformat: (data) =\u003e {\n    const formatted = presets.default.format(data)\n\n    // get the first link with Cheerio\n    // and append it to the content\n    const $ = helpers.cheerioLoad(formatted.content)\n    const firstLink = $('a:first').attr('href')\n    formatted.content += ` ${firstLink}`\n\n    // format to markdown\n    formatted.content = helpers.toMarkdown(formatted.content)\n    return formatted\n}\n```\n\n### Services\n\nEach service requires a different set of values depending on how the API works.\n\n#### Micro.blog\n\n|Key|Value|Notes|\n|---|---|---|\n|`siteUrl`|The Micro.blog site you're posting to|e.g. `https://coolsite.micro.blog`|\n|`apiKey`|A Micro.blog API key|Get an API key from [https://micro.blog/account/apps](https://micro.blog/account/apps)|\n\n#### Mastodon\n\n|Key|Value|Notes|\n|---|---|---|\n|`instance`|Your Mastodon instance|e.g. `https://social.lol`|\n|`accessToken`|Go to `Preferences \u003e Development \u003e New Application` on your Mastodon instance and grab the access token|\n|`visibility` (optional, default `public`)|`public` `unlisted` `private` `direct`|\n|`sensitive` (optional, default `false`)|`false` `true`|\n\n#### Webhooks\n\nThe webhook service will send a `POST` request with the result of `transform.format` (as set in your site config) to a given url.\n\n|Key|Value|Notes|\n|---|---|---|\n|`url`|The URL to post to|\n\n#### Omnivore\n\nThe Omnivore service will save a URL to your Omnivore account.\n\n|Key|Value|Notes|\n|---|---|---|\n|`apiKey`|Your Omnivore API key|\n\n\n#### GitHub\n\nCreate a new file on a GitHub repository.\n\n| Key      | Value                 | Notes |\n| -------- | --------------------- | ----- |\n| `token` | Your GitHub token |\n| `repo` | The repository to commit to | e.g. `rknightuk/echo`\n| `branch` | The branch to commit to |\n| `committer` | An object with `name` and `email` values | e.g. `{ name: 'Robb', email: 'robb@example.com }`\n\nFor posting to Github your `format` function must return `content` and `filePath` where `filePath` is the path to where the file will be in the Github repository, for example `src/posts/movies/2024-02-09.md`. It can optionally return a `commit` message, which will fallback to `New post` if none is set. Example `format` function for GitHub:\n\n```js\nformat: (data) =\u003e {\n    return {\n        content: data.title,\n        date: new Date(data.isoDate).toISOString(),\n        filePath: `src/posts/movies/${new Date().getFullYear()}/${new Date().toISOString().split('T').md`,\n        commit: `Add ${data.title}`,\n    }\n}\n```\n\n#### LinkAce\n\n|Key|Value|Notes|\n|---|---|---|\n|`domain`|The LinkAce domain where you have it installed|e.g.`https://links.example.com`|\n|`apiKey`|A LinkAce API key||\n\n`content` returns from `transform.format` should be a link. Categories on the site will be converted to tags. Any tags included from `format` will be merged with `categories`.\n\n```js\n{\n    name: 'mycoollinkfeed',\n    feed: 'https://example.com/linkfeed.xml',\n    categories: ['ATag'],\n    transform: {\n        getId: presets.default.getId,\n        format: (data) =\u003e {\n            return {\n                content: data.external_url,\n                date: data.date_published,\n                tags: data._custom.tags,\n            }\n        }\n    },\n    services: [SERVICES.LINKACE],\n},\n```\n\n#### Webmentions\n\n| Key      | Value                                          | Notes                           |\n| -------- | ---------------------------------------------- | ------------------------------- |\n| no config is required for webmentions |  |  |\n\n`content` returns from `transform.format` should be a link.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frknightuk%2Fecho","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frknightuk%2Fecho","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frknightuk%2Fecho/lists"}