{"id":13557726,"url":"https://github.com/feedhq/feedhq","last_synced_at":"2025-04-03T12:31:03.755Z","repository":{"id":2541098,"uuid":"3518605","full_name":"feedhq/feedhq","owner":"feedhq","description":"FeedHQ is a web-based feed reader","archived":false,"fork":false,"pushed_at":"2022-03-21T22:16:34.000Z","size":2521,"stargazers_count":572,"open_issues_count":36,"forks_count":50,"subscribers_count":20,"default_branch":"master","last_synced_at":"2024-11-04T08:39:10.505Z","etag":null,"topics":["atom-feed","atom-feed-parser","django","elasticsearch","feed-reader","feeds","postgresql","python","redis","rss","rss-aggregator","rss-feed","rss-feed-parser"],"latest_commit_sha":null,"homepage":"https://feedhq.org","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/feedhq.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2012-02-22T20:28:09.000Z","updated_at":"2024-10-18T19:14:05.000Z","dependencies_parsed_at":"2022-08-30T00:11:44.350Z","dependency_job_id":null,"html_url":"https://github.com/feedhq/feedhq","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/feedhq%2Ffeedhq","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/feedhq%2Ffeedhq/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/feedhq%2Ffeedhq/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/feedhq%2Ffeedhq/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/feedhq","download_url":"https://codeload.github.com/feedhq/feedhq/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247002203,"owners_count":20867425,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["atom-feed","atom-feed-parser","django","elasticsearch","feed-reader","feeds","postgresql","python","redis","rss","rss-aggregator","rss-feed","rss-feed-parser"],"created_at":"2024-08-01T12:04:30.756Z","updated_at":"2025-04-03T12:31:02.171Z","avatar_url":"https://github.com/feedhq.png","language":"Python","readme":"FeedHQ\n======\n\n.. image:: https://travis-ci.org/feedhq/feedhq.svg?branch=master\n   :alt: Build Status\n   :target: https://travis-ci.org/feedhq/feedhq\n\n.. image:: https://img.shields.io/coveralls/feedhq/feedhq/master.svg\n   :alt: Coverage Status\n   :target: https://coveralls.io/r/feedhq/feedhq?branch=master\n\nFeedHQ is a simple, lightweight web-based feed reader. Main features:\n\nUser-facing features\n--------------------\n\n* RSS and ATOM support\n\n* Grouping by categories\n\n* Awesome pagination and intelligent browsing\n\n* Great readability on all screen sizes (smatphones, tablets and desktops)\n\n* Mobile-first, retina-ready\n\n* Reading list management with Instapaper, Readability or Read It Later\n  support\n\n* Filter out already read entries\n\n* Hides images/media by default (and therefore filters ads and tracking stuff)\n\n* Multiple user support\n\n* `OPML import`_\n\n* Syntax highlighting, awesome for reading tech blogs\n\n* Keyboard navigation\n\n* `Subtome`_ support\n\nDeveloper- / Sysadmin-facing features\n-------------------------------------\n\n* Nice with web servers, uses ETag and Last-Modified HTTP headers\n\n* Handles HTTP status codes nicely (permanent redirects, gone, not-modified…)\n\n* Exponential backoff support\n\n* `PubSubHubbub`_ support\n\n.. _PubSubHubbub: http://code.google.com/p/pubsubhubbub/\n\n.. _OPML import: http://www.opml.org/\n\n.. _Subtome: https://www.subtome.com/\n\nInstallation\n------------\n\nRequirements:\n\n* Python 3.4 or greater\n* Redis (2.6+ recommended)\n* PostgreSQL (9.2+ recommended but anything \u003e= 8.4 should work)\n* Elasticsearch (see compatibility table below)\n\nGetting the code::\n\n    git clone https://github.com/feedhq/feedhq.git\n    cd feedhq\n    virtualenv -p python2 env\n    source env/bin/activate\n    add2virtualenv .\n    pip install -r requirements.txt\n\nElasticsearch version requirements:\n\n============ ============ ==========\nCommit hash  Date         ES version\n============ ============ ==========\nUp to 3aea18 Sep 18, 2014 1.1\nFrom 6d5cbc  Oct 28, 2014 1.3\n============ ============ ==========\n\nConfiguration\n-------------\n\nFeedHQ relies on environment variables for its configuration. The required\nenvironment variables are:\n\n* ``DJANGO_SETTINGS_MODULE``: set it to ``feedhq.settings``.\n* ``SECRET_KEY``: set to a long random string.\n* ``ALLOWED_HOSTS``: space-separated list of hosts which serve the web app.\n  E.g. ``www.feedhq.org feedhq.org``.\n* ``FROM_EMAIL``: the email address that sends automated emails (password\n  lost, etc.). E.g. ``FeedHQ \u003cfeedhq@example.com\u003e``.\n* ``REDIS_URL``: a URL for configuring redis. E.g.\n  ``redis://localhost:6354/1``.\n* ``DATABASE_URL``: a heroku-like database URL. E.g.\n  ``postgres://user:password@host:port/database``.\n\nOptionally you can customize:\n\n* ``DEBUG``: set it to a non-empty value to enable the Django debug mode.\n* ``MEDIA_ROOT``: the absolute location where media files (user-generated) are\n  stored. This must be a public directory on your webserver available under\n  ``MEDIA_URL``.\n* ``MEDIA_URL``: the URL that handles media files (user-generated) served from\n  ``MEDIA_ROOT``. By default, it is set to ``/media/``.\n* ``STATIC_ROOT``: the absolute location where static files (CSS/JS files) are\n  stored. This must be a public directory on your webserver available under\n  the ``/static/`` URL.\n* ``STATIC_URL``: the URL that serves static files (CSS/JS files) located in\n  ``STATIC_ROOT``. By default, it is set to ``/static/``.\n* ``SENTRY_DSN``: a DSN to enable `Sentry`_ error reporting.\n* ``SESSION_COOKIE_PATH``: the path set on the session cookie. E.g., ``/``.\n* ``HTTPS``: set-it to a non-empty value to configure FeedHQ for SSL access.\n* ``EMAIL_URL``: a URL for configuring email. E.g.\n  ``smtp://user:password@host:port/?backend=my.EmailBackend\u0026use_tls=true``.\n  The ``backend`` querystring parameter sets the Django ``EMAIL_BACKEND``\n  setting. By default emails only go to the development console.\n* ``ES_NODES``: space-separated list of elasticsearch nodes to use for cluster\n  discovery. Defaults to ``localhost:9200``.\n* ``ES_INDEX``: the name of the elasticsearch index. Defaults to ``feedhq``.\n* ``ES_ALIAS_TEMPLATE``: string template to format the per-user alias that is\n  created in elasticsearch. Defaults to ``feedhq-{0}``.\n* ``ES_SHARDS``: number of shards that you want in your elasticsearch index.\n  This cannot be changed once you have created the index. Defaults to 5.\n* ``ES_REPLICAS``: number of times you want your shards to be replicated on\n  the elasticsearch cluster. Defaults to 1. Set it to 0 if you only have one\n  node. This is only used at index creation but the index setting can be\n  altered dynamically.\n* ``HEALTH_SECRET``: a shared secret between your FeedHQ server(s) and your\n  monitoring. The ``/health/`` endpoint can be protected by requiring clients\n  to provide a shared secret in an ``X-Token`` header. If no secret is set,\n  the health endpoint is open to all.\n* ``LOG_SYSLOG``: set it to ``1`` to send logs to ``/dev/log`` instead of\n  stdout (the default). Logs are formatted in JSON.\n\n.. _Sentry: https://www.getsentry.com/\n\nFor integration with external services:\n\n* ``READITLATER_API_KEY``: your `Pocket`_ API key.\n* ``INSTAPAPER_CONSUMER_KEY``, ``INSTAPAPER_CONSUMER_SECRET``: your\n  `Instapaper`_ API keys.\n* ``READABILITY_CONSUMER_KEY``, ``READABILITY_CONSUMER_SECRET``: your\n  `Readability`_ API keys.\n\n.. _Pocket: http://getpocket.com/\n.. _Instapaper: http://www.instapaper.com/\n.. _Readability: https://www.readability.com/\n\nThen deploy the Django app using the recipe that fits your installation. More\ndocumentation on the `Django deployment guide`_. The WSGI application is\nlocated at ``feedhq.wsgi.application``.\n\n.. _Django deployment guide: http://docs.djangoproject.com/en/dev/howto/deployment/\n\nTo create the Elasticsearch index::\n\n    django-admin.py create_index\n\nThen you'll need to create the appropriate PostgreSQL database and run::\n\n    django-admin.py syncdb\n    django-admin.py migrate\n\nNote that additionally to the web server, you need to run one or more\nconsumers for the task queue. This is done with the ``rqworker`` management\ncommand::\n\n    django-admin.py rqworker store high default low favicons\n\nThe arguments are queue names.\n\nOnce your application is deployed (you've run ``django-admin.py syncdb`` to\ncreate the database tables, ``django-admin.py migrate`` to run the initial\nmigrations and ``django-admin.py collectstatic`` to collect your static\nfiles), you can add users to the application. On the admin interface, add\nas many users as you want. Then add some some categories and feeds to your\naccount using the regular interface.\n\nCrawl for updates::\n\n    django-admin.py sync_scheduler\n    django-admin.py updatefeeds\n\nSet up a cron job to update your feeds on a regular basis. This puts the\noldest-updated feeds in the update queue::\n\n    */5 * * * * /path/to/env/django-admin.py updatefeeds\n\nThe ``updatefeeds`` command puts 1/12th of the feeds in the update queue. Feeds\nwon't update if they've been updated in the past 60 minutes, so the 5-minute\nperiod for cron jobs distributes nicely the updates along the 1-hour\nperiod.\n\nA cron job should also be set up for picking and updating favicons (the\n``--all`` switch processes existing favicons in case they have changed, which\nyou should probably do every month or so)::\n\n    @monthly /path/to/env/bin/django-admin.py favicons --all\n\nHere is a full list of management commands that you should schedule:\n\n* ``add_missing`` creates the missing denormalized URLs for crawling. Since\n  URLs are denormalized it's recommended to run it every now and then to\n  ensure consistency.\n\n  Recommended frequency: hourly.\n\n  Resource consumption: negligible (2 database queries).\n\n* ``delete_unsubscribed`` is the delete counterpart of ``add_missing``.\n\n  Recommended frequency: hourly.\n\n  Resource consumption: negligible (2 database queries).\n\n* ``favicons --all`` forces fetching the favicons for all existing URLs. It's\n  useful for picking up new favicons when they're updated. Depending on your\n  volume of data, this can be resource-intensive.\n\n  Recommended frequency: monthly.\n\n  Resource consumption: the command itself only triggers async jobs but the\n  jobs perform network I/O, HTML parsing, disk I/O and database queries.\n\n* ``updatefeeds`` picks 1/12th of the URLs and fetches them.\n\n  Recommended frequency: every 5 minutes.\n\n  Resource consumption: the command itself only triggers async jobs but the\n  jobs perform network I/O, HTML parsing and -- when updates are found --\n  database queries.\n\n* ``sync_scheduler`` adds missing URLs to the scheduler. Also useful to run\n  every now and then.\n\n  Recommended frequency: every hour.\n\n  Resource consumption: one large database query per chunk of 10k feeds which\n  aren't in the scheduler, plus one redis ``HMSET`` per URL that's not in the\n  scheduler. As a routine task it's not resource-intensive.\n\n* ``sync_pubsubhubbub`` unsubscribes from unneeded PubSubHubbub\n  subscriptions.\n\n  Recommended frequency: once a day.\n\n  Resource consumption: low.\n\n* ``clean_rq`` removes stale RQ jobs.\n\n  Recommended frequency: once a day.\n\n  Resource consumption: low. Only makes requests to Redis.\n\n* ``delete_old`` removes expired entries as determined by each user's entry TTL.\n\n  Recommended frequency: once a day.\n\n  Resource consumption: medium, makes ``delete_by_query`` queries to ES (1 per\n  user).\n\n* ``delete_expired_tokens`` removes expired API tokens. Tokens are valid for 7\n  days, after which they are renewed by client apps.\n\n  Recommended frequency: once a day.\n\n  Resource consumption: low (one ``DELETE`` query).\n\nDevelopment\n-----------\n\nInstall the development requirements::\n\n    pip install -r requirements-dev.txt\n\nRun the tests::\n\n    make test\n\nOr if you want to run the tests with ``django-admin.py`` directly, make sure\nyou use ``feedhq.test_settings`` as the ``DJANGO_SETTINGS_MODULE`` environment\nvariable to avoid making network calls while running the tests.\n\nThe Django debug toolbar is enabled when the ``DEBUG`` environment variable is\ntrue and the ``django-debug-toolbar`` package is installed.\n\n`Foreman`_ is used in development to start a lightweight Django server and run\n`RQ`_ workers. Environment variables are managed using a `python port`_ of\nDaemontools' ``envdir`` utility. A running `Redis`_ server is required for\nthis workflow::\n\n    make run\n\n.. _Foreman: http://ddollar.github.com/foreman/\n.. _RQ: http://python-rq.org/\n.. _Redis: http://redis.io/\n.. _Daemontools: http://cr.yp.to/daemontools.html\n.. _python port: https://pypi.python.org/pypi/envdir\n\nWhen running ``django-admin.py updatefeeds`` on your development machine,\nmake sure you have the ``DEBUG`` environment variable present to avoid making\nPubSubHubbub subscription requests without any valid callback URL.\n\nEnvironment variables for development are set in the ``envdir`` directory. For\ntests, they are located in the ``tests/envdir`` directory.\n\nWhen working on frontend assets (SCSS or js files), `watchman`_ can be used to\nautomatically run compass and uglify on file changes. Install `watchman`_,\n`Compass`_ (``gem install bundle \u0026\u0026 bundle install``) and `npm`_ (part of\nnodejs) to get started. Then run::\n\n    make watch\n\n.. _watchman: https://github.com/facebook/watchman\n.. _Compass: http://compass-style.org/\n.. _npm: https://www.npmjs.org/\n\nOnce you're done working with assets, simply kill watchman::\n\n    pkill watchman\n","funding_links":[],"categories":["Python","django"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffeedhq%2Ffeedhq","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffeedhq%2Ffeedhq","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffeedhq%2Ffeedhq/lists"}