{"id":13857127,"url":"https://github.com/PyFeeds/PyFeeds","last_synced_at":"2025-07-13T20:30:48.454Z","repository":{"id":7978447,"uuid":"55342385","full_name":"PyFeeds/PyFeeds","owner":"PyFeeds","description":"DIY Atom feeds in times of social media and paywalls","archived":false,"fork":false,"pushed_at":"2024-05-29T11:51:43.000Z","size":798,"stargazers_count":81,"open_issues_count":8,"forks_count":11,"subscribers_count":7,"default_branch":"master","last_synced_at":"2024-11-05T11:58:36.480Z","etag":null,"topics":["atom","feeds","python","rss","scrapy"],"latest_commit_sha":null,"homepage":"https://pyfeeds.readthedocs.io","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/PyFeeds.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-04-03T11:05:07.000Z","updated_at":"2024-10-31T05:36:33.000Z","dependencies_parsed_at":"2024-02-09T01:40:33.987Z","dependency_job_id":"2957c973-f8e3-4cc6-b2ec-c9965d73c791","html_url":"https://github.com/PyFeeds/PyFeeds","commit_stats":{"total_commits":651,"total_committers":6,"mean_commits":108.5,"dds":"0.24577572964669736","last_synced_commit":"d628c5e52a23ff00c96d52c5f8c8ba8213c2db0d"},"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PyFeeds%2FPyFeeds","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PyFeeds%2FPyFeeds/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PyFeeds%2FPyFeeds/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PyFeeds%2FPyFeeds/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/PyFeeds","download_url":"https://codeload.github.com/PyFeeds/PyFeeds/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225912599,"owners_count":17544210,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["atom","feeds","python","rss","scrapy"],"created_at":"2024-08-05T03:01:26.785Z","updated_at":"2024-11-22T14:32:06.234Z","avatar_url":"https://github.com/PyFeeds.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"Feeds\n=====\n\n|pypi| |support| |licence|\n\n|readthedocs|\n\n|pyfeedsci|\n\nOnce upon a time every website offered an RSS feed to keep readers updated\nabout new articles/blog posts via the users' feed readers. These times are\nlong gone. The once iconic orange RSS icon has been replaced by \"social share\"\nbuttons.\n\nFeeds aims to bring back the good old reading times. It creates Atom feeds for\nwebsites that don't offer them (anymore). It allows you to read new articles of\nyour favorite websites in your feed reader (e.g. TinyTinyRSS_) even if this is\nnot officially supported by the website.\n\nFurthermore it can also enhance existing feeds by inlining the actual content\ninto the feed entry so it can be read without leaving the feed reader.\n\nFeeds is based on Scrapy_, a framework for extracting data from websites, and\nit's easy to add support for new websites. Just take a look at the existing\nspiders_ and feel free to open a `pull request`_!\n\nDocumentation\n-------------\nFeeds comes with extensive documentation. It is available at\n`https://pyfeeds.readthedocs.io \u003chttps://pyfeeds.readthedocs.io/en/latest/\u003e`_.\n\nSupported Websites\n------------------\n\nFeeds is currently able to create full text Atom feeds for various sites. The\ncomplete list of `supported websites is available in the documentation\n\u003chttps://pyfeeds.readthedocs.io/en/latest/spiders.html\u003e`_.\n\nContent behind paywalls\n~~~~~~~~~~~~~~~~~~~~~~~\n\nSome sites (Falter_, Konsument_, LWN_, `Oberösterreichische Nachrichten`_,\nÜbermedien_) offer articles only behind a paywall. If you have a paid\nsubscription, you can configure your username and password in ``feeds.cfg`` and\nalso read paywalled articles from within your feed reader. For the less\nfortunate who don't have a subscription, paywalled articles are tagged with\n``paywalled`` so they can be filtered, if desired.\n\nAll feeds contain the articles in full text so you never have to leave your\nfeed reader while reading.\n\nInstallation\n------------\n\nFeeds is meant to be installed on your server and run periodically in a cron\njob or similar job scheduler. We recommend to install Feeds inside a virtual\nenvironment.\n\nFeeds can be installed from PyPI using ``pip``:\n\n.. code-block:: bash\n\n   $ pip install PyFeeds\n\nYou may also install the current development version. The master branch is\nconsidered stable enough for daily use:\n\n.. code-block:: bash\n\n   $ pip install https://github.com/pyfeeds/pyfeeds/archive/master.tar.gz\n\nAfter installation ``feeds`` is available in your virtual environment.\n\nFeeds supports Python 3.8+.\n\nQuickstart\n----------\n\n* List all available spiders::\n\n  $ feeds list\n\n* Feeds allows to crawl one or more spiders without configuration, e.g.::\n\n  $ feeds crawl tvthek.orf.at\n\n* A configuration file is supported too. Simply copy the template configuration\n  and adjust it. Enable the spiders you are interested in and adjust the output\n  path where Feeds stores the scraped Atom feeds::\n\n  $ cp feeds.cfg.dist feeds.cfg\n  $ $EDITOR feeds.cfg\n  $ feeds --config feeds.cfg crawl\n\n* Point your feed reader to the generated Atom feeds and start reading. Feeds\n  works best when run periodically in a cron job.\n* Run ``feeds --help`` or ``feeds \u003csubcommand\u003e --help`` for help and usage\n  details.\n\nCaching\n-------\n\nFeeds caches HTTP responses by default to save bandwidth. Entries are cached\nfor 90 days by default (this can be overwritten in the config file). Outdated\nentries are purged from cache automatically after a crawl. It's also possible\nto explicitly purge the cache from outdated entries::\n\n  $ feeds --config feeds.cfg cleanup\n\nRelated work\n------------\n\n* `morss \u003chttps://github.com/pictuga/morss\u003e`_ creates feeds, similar to Feeds\n  but in \"real-time\", i.e. on (HTTP) request.\n* `Full-Text RSS \u003chttps://bitbucket.org/fivefilters/full-text-rss\u003e`_ converts\n  feeds to contain the full article and not only a teaser based on heuristics\n  and rules. Feeds are converted in \"real-time\", i.e. on request basis.\n* `f43.me \u003chttps://github.com/j0k3r/f43.me\u003e`_ converts feeds to contain the\n  full article and also improves articles by adding links to the comment\n  sections of Hacker News and Reddit. Feeds are converted periodically.\n\nHow to contribute\n-----------------\n\nIssues\n~~~~~~\n\n* Search the existing issues in the `issue tracker`_.\n* File a `new issue`_ in case the issue is undocumented.\n\nPull requests\n~~~~~~~~~~~~~\n\n* Fork the project to your private repository.\n* Create a topic branch and make your desired changes.\n* Open a pull request. Make sure the GitHub CI checks are passing.\n\nAuthors\n-------\nFeeds is written and maintained by `Florian Preinstorfer \u003chttps://nblock.org\u003e`_\nand `Lukas Anzinger \u003chttps://www.notinventedhere.org\u003e`_.\n\nLicense\n-------\n\nAGPL3, see https://pyfeeds.readthedocs.io/en/latest/license.html for details.\n\n.. _issue tracker: https://github.com/pyfeeds/pyfeeds/issues\n.. _new issue: https://github.com/pyfeeds/pyfeeds/issues/new\n.. _Scrapy: https://www.scrapy.org\n.. _TinyTinyRSS: https://tt-rss.org\n.. _pull request: https://pyfeeds.readthedocs.io/en/latest/contribute.html\n.. _spiders: https://github.com/PyFeeds/PyFeeds/tree/master/feeds/spiders\n.. _Falter: https://pyfeeds.readthedocs.io/en/latest/spiders/falter.at.html\n.. _Konsument: https://pyfeeds.readthedocs.io/en/latest/spiders/konsument.at.html\n.. _LWN: https://pyfeeds.readthedocs.io/en/latest/spiders/lwn.net.html\n.. _Oberösterreichische Nachrichten: https://pyfeeds.readthedocs.io/en/latest/spiders/nachrichten.at.html\n.. _Übermedien: https://pyfeeds.readthedocs.io/en/latest/spiders/uebermedien.de.html\n\n.. |pypi| image:: https://img.shields.io/pypi/v/pyfeeds.svg?style=flat-square\n    :target: https://pypi.org/project/pyfeeds/\n    :alt: pypi version\n\n.. |support| image:: https://img.shields.io/pypi/pyversions/pyfeeds.svg?style=flat-square\n    :target: https://pypi.org/project/pyfeeds/\n    :alt: supported Python version\n\n.. |licence| image:: https://img.shields.io/pypi/l/pyfeeds.svg?style=flat-square\n    :target: https://pypi.org/project/pyfeeds/\n    :alt: licence\n\n.. |readthedocs| image:: https://img.shields.io/readthedocs/pyfeeds/latest.svg?style=flat-square\u0026label=Read%20the%20Docs\n   :alt: Read the documentation at https://pyfeeds.readthedocs.io/en/latest/\n   :target: https://pyfeeds.readthedocs.io/en/latest/\n\n.. |pyfeedsci| image:: https://github.com/PyFeeds/PyFeeds/workflows/PyFeeds%20CI/badge.svg\n    :target: https://github.com/PyFeeds/PyFeeds/actions?query=workflow%3A%22PyFeeds+CI%22\n    :alt: GitHub PyFeeds CI\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FPyFeeds%2FPyFeeds","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FPyFeeds%2FPyFeeds","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FPyFeeds%2FPyFeeds/lists"}