{"id":22941706,"url":"https://github.com/plaidweb/pushl","last_synced_at":"2025-05-11T18:15:01.121Z","repository":{"id":47417161,"uuid":"150941671","full_name":"PlaidWeb/Pushl","owner":"PlaidWeb","description":"Push notification adapter for feeds","archived":false,"fork":false,"pushed_at":"2025-03-20T17:49:33.000Z","size":706,"stargazers_count":31,"open_issues_count":11,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-31T21:42:53.797Z","etag":null,"topics":["atom","feeds","rss","syndication","webmention","websub"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/PlaidWeb.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-09-30T07:15:22.000Z","updated_at":"2025-03-20T17:49:38.000Z","dependencies_parsed_at":"2024-01-30T01:45:34.847Z","dependency_job_id":"197fc5f4-e1d5-40a0-882f-2b7e068a7814","html_url":"https://github.com/PlaidWeb/Pushl","commit_stats":{"total_commits":179,"total_committers":3,"mean_commits":"59.666666666666664","dds":"0.027932960893854775","last_synced_commit":"80bbcadb0dfa92e6c6c87d80f3a49d39a3252cd0"},"previous_names":[],"tags_count":31,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PlaidWeb%2FPushl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PlaidWeb%2FPushl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PlaidWeb%2FPushl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PlaidWeb%2FPushl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/PlaidWeb","download_url":"https://codeload.github.com/PlaidWeb/Pushl/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253453365,"owners_count":21911083,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["atom","feeds","rss","syndication","webmention","websub"],"created_at":"2024-12-14T13:44:46.564Z","updated_at":"2025-05-10T17:31:42.634Z","avatar_url":"https://github.com/PlaidWeb.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Pushl\n\nA simple tool that parses content feeds and sends out appropriate push notifications (WebSub, webmention, etc.) when they change.\n\nSee http://publ.beesbuzz.biz/blog/113-Some-thoughts-on-WebMention for the motivation.\n\n## Features\n\n* Supports any feed supported by [feedparser](https://github.com/kurtmckee/feedparser)\n    and [mf2py](https://github.com/microformats/mf2py) (RSS, Atom, HTML pages containing\n    `h-entry`, etc.)\n* Will send WebSub notifications for feeds which declare a WebSub hub\n* Will send WebMention notifications for entries discovered on those feeds or specified directly\n* Can perform autodiscovery of additional feeds on entry pages\n* Can do a full backfill on Atom feeds configured with [RFC 5005](https://tools.ietf.org/html/rfc5005)\n* When configured to use a cache directory, can detect entry deletions and updates to implement the webmention update and delete protocols (as well as saving some time and bandwidth)\n\n\n## Site setup\n\nIf you want to support WebSub, have your feed implement [the WebSub protocol](https://indieweb.org/WebSub). The short version is that you should have a `\u003clink rel=\"hub\" href=\"http://path/to/hub\" /\u003e` in your feed's top-level element.\n\nThere are a number of WebSub hubs available; I use [Superfeedr](http://pubsubhubbub.superfeedr.com).\n\nFor [WebMentions](https://indieweb.org/Webmention), configure your site templates with the various microformats; by default, Pushl will use the following tags as the top-level entry container, in descending order of priority:\n\n* Anything with a `class` of `h-entry`\n* An `\u003carticle\u003e` tag\n* Anything with a `class` of `entry`\n\nFor more information on how to configure your site templates, see the [microformats h-entry specification](http://microformats.org/wiki/h-entry).\n\n### mf2 feed notes\n\nIf you're using an mf2 feed (i.e. an HTML-formatted page with `h-entry` declarations), only entries with a `u-url` property will be used for sending webmentions; further, Pushl will retrieve the page from that URL to ensure it has the full content. (This is to work around certain setups where the `h-feed` only shows summary text.)\n\nAlso, there is technically no requirement for an HTML page to declare an `h-feed`; all entities marked up with `h-entry` will be consumed.\n\n## Installation\n\nThe easiest way to install pushl is via [pipx](https://pipx.pypa.io/stable/installation/), with e.g.\n\n```bash\npipx install pushl\n```\n\nYou can also install pushl as a dependency in whatever other Python virtual environment you're using (via [poetry](https://python-poetry.org/) or the like), and this will make the pushl wrapper script available in its path.\n\n## Usage\n\n### Basic\n\n```bash\npushl -c $HOME/var/pushl-cache http://example.com/feed.xml\n```\n\nWhile you can run it without the `-c` argument, its use is highly recommended so that subsequent runs are both less spammy and so that it can detect changes and deletions.\n\n### Sending pings from individual entries\n\nIf you just want to send webmentions from an entry page without processing an entire feed, the `-e/--entry` flag indicates that the following URLs are pages or entries, rather than feeds; e.g.\n\n```bash\npushl -e http://example.com/some/page\n```\n\nwill simply send the webmentions for that page.\n\n### Additional feed discovery\n\nThe `-r/--recurse` flag will discover any additional feeds that are declared on entries and process them as well. This is useful if you have per-category feeds that you would also like to send WebSub notifications on. For example, [my site](http://beesbuzz.biz) has per-category feeds which are discoverable from individual entries, so `pushl -r http://beesbuzz.biz/feed` will send WebSub notifications for all of the categories which have recent changes.\n\nNote that `-r` and `-e` in conjunction will also cause the feed declared on the entry page to be processed further. While it is tempting to use this in a feed autodiscovery context e.g.\n\n```bash\npushl -re http://example.com/blog/\n```\n\nthis will also send webmentions from the blog page itself which is probably *not* what you want to have happen.\n\n### Backfilling old content\n\nIf your feed implements [RFC 5005](https://tools.ietf.org/html/rfc5005), the `-a` flag will scan past entries for WebMention as well. It is recommended to only use this flag when doing an initial backfill, as it can end up taking a long time on larger sites (and possibly make endpoint operators very grumpy at you). To send updates of much older entries it's better to just use `-e` to do it on a case-by-case basis.\n\n### Dual-protocol/multi-domain websites\n\nIf you have a website which has multiple URLs that can access it (for example, http+https, or multiple domain names), you generally only want WebMentions to be sent from the canonical URL. The best solution is to use `\u003clink rel=\"canonical\"\u003e` to declare which one is the real one, and Pushl will use that in sending the mentions; so, for example:\n\n\n```bash\npushl -r https://example.com/feed http://example.com/feed http://alt-domain.example.com/feed\n```\n\nAs long as both `http://example.com` and `http://alt-domain.example.com` declare the `https://example.com` version as canonical, only the webmentions from `https://example.com` will be sent.\n\nIf, for some reason, you can't use `rel=\"canonical\"` you can use the `-s/--websub-only` flag on Pushl to have it only send WebSub notifications for that feed; for example:\n\n```bash\npushl -r https://example.com/feed -s https://other.example.com/feed\n```\n\nwill send both Webmention and WebSub for `https://example.com` but only WebSub for `https://other.example.com`.\n\n## Automated updates\n\n`pushl` can be run from a cron job, although it's a good idea to use `flock -n` to prevent multiple instances from stomping on each other. An example cron job for updating a site might look like:\n\n```crontab\n*/5 * * * * flock -n $HOME/.pushl-lock pushl -rc $HOME/.pushl-cache http://example.com/feed\n```\n\n### My setup\n\nIn my setup, I have `pushl` installed in my website's pipenv:\n\n```bash\ncd $HOME/beesbuzz.biz\npipenv install pushl\n```\n\nand created this script as `$HOME/beesbuzz.biz/pushl.sh`:\n\n```bash\n#!/bin/bash\n\ncd $(dirname \"$0\")\nLOG=logs/pushl-$(date +%Y%m%d.log)\n\n# redirect log output\nif [ \"$1\" == \"quiet\" ] ; then\n    exec \u003e\u003e $LOG 2\u003e\u00261\nelse\n    exec 2\u003e\u00261 | tee -a $LOG\nfi\n\n# add timestamp\ndate\n\n# run pushl\nflock -n $HOME/var/pushl/run.lock $HOME/.local/bin/pipenv run pushl -rvvkc $HOME/var/pushl \\\n    https://beesbuzz.biz/feed\\?push=1 \\\n    http://publ.beesbuzz.biz/feed\\?push=1 \\\n    https://tumblr.beesbuzz.biz/rss \\\n    https://novembeat.com/feed\\?push=1 \\\n    http://beesbuzz.biz/feed\\?push=1 \\\n    -s http://beesbuzz.biz/feed-summary https://beesbuzz.biz/feed-summary\n\n# while we're at it, clean out the log and pushl cache directory\nfind logs $HOME/var/pushl -type f -mtime +30 -print -delete\n```\n\nThen I have a cron job:\n\n```crontab\n*/15 * * * * $HOME/beesbuzz.biz/pushl.sh quiet\n```\n\nwhich runs it every 15 minutes.\n\nI also have a [git deployment hook](http://publ.beesbuzz.biz/441) for my website, and its final step (after restarting `gunicorn`) is to run `pushl.sh`, in case a maximum latency of 15 minutes just isn't fast enough.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fplaidweb%2Fpushl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fplaidweb%2Fpushl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fplaidweb%2Fpushl/lists"}