Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/LukeB42/Emissary
A microservice for archiving the news.
https://github.com/LukeB42/Emissary
Last synced: about 2 months ago
JSON representation
A microservice for archiving the news.
- Host: GitHub
- URL: https://github.com/LukeB42/Emissary
- Owner: LukeB42
- License: mit
- Created: 2014-07-21T19:51:18.000Z (about 10 years ago)
- Default Branch: master
- Last Pushed: 2016-05-22T18:16:48.000Z (over 8 years ago)
- Last Synced: 2024-07-27T06:41:02.229Z (about 2 months ago)
- Language: Python
- Homepage:
- Size: 6.36 MB
- Stars: 164
- Watchers: 14
- Forks: 17
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
Emissary
========
An intelligence utility / test for researchers, programmers and generally carnivorous primates who want personally curated news archives.
Emissary is a web content extractor that has a RESTful API and the ability to run pre-store scripts.
Emissary stores the full text of linked articles from RSS feeds or URLs containing links.
Documentation lives [here](http://docs.psybernetics.org/).
--------
Installation requires the python interpreter headers, libevent, libxml2 and libxslt headers.
Optional article compression requires libsnappy.
All of these can be obtained on debian-based systems with:
sudo apt-get install -y zlib1g-dev libxml2-dev libxslt1-dev python-dev libevent-dev libsnappy-dev
You're then ready to install the package for all users:
sudo python setup.py install
Usage: python -m emissary.run
-h, --help show this help message and exit
-c, --crontab Crontab to parse
--config (defaults to emissary.config)
-a, --address (defaults to 0.0.0.0)
-p, --port (defaults to 6362)
--export Write the existing database as a crontab
--key SSL key file
--cert SSL certificate
--pidfile (defaults to ./emissary.pid)
--logfile (defaults to ./emissary.log)
--stop
--debug Log to stdout
-d Run in the background
--run-as (defaults to the invoking user)
--scripts-dir (defaults to ./scripts/)
Some initial setup has to be done before the system will start.
Communication with Emissary is mainly done over HTTPS connections
and for that you're going to need an SSL certificate and a key:
user@host $ openssl genrsa 4096 > key
user@host $ openssl req -new -x509 -nodes -sha256 -days 365 -key key > cert
To prevent your API keys ever getting put into version control for all
the world to see, you need to put a database URI into the environment:
export EMISSARY_DATABASE="sqlite://///home/you/.emissary.db"
Protip: Put that last line in your shells' rc file.
Start an instance in the foreground to obtain your first API key:
user@host $ python -m emissary.run --cert cert --key key
14/06/2015 16:31:30 - Emissary - INFO - Starting Emissary 2.0.0.
e5a59e0a-b457-45c6-9d30-d983419c43e1
^That UUID is your Primary API key. Add it to this example crontab:
user@host $ cat feeds.txt
apikey: your-api-key-here
# url name group minute hour day month weekday
http://news.ycombinator.com/rss "HN" "HN" */15 * * * *
http://phys.org/rss-feed/ "Phys.org" "Phys.org" 1 12 * * *
http://feeds.nature.com/news/rss/most_recent "Nature" "Nature" 30 13 * * *
user@host $ python -m emissary.run -c feeds.txt
Using API key "Primary".
Primary: Creating feed group HN.
Primary: HN: Creating feed "HN"
Emissary supports multiple apikey directives in one crontab.
Subsequent feed definitions are associated with the previous key.
Start an instance in the background and connect to it:
user@host $ python -m emissary.run -d --cert cert --key key
user@host $ python -m emissary.repl
Emissary 2.0.0
Psybernetics 2015
(3,204) > help
If the prospect of creating an NSA profile of your reading habits is
something that rightfully bothers you then my advice is to subscribe
to many things and then use Emissary to read the things that really
interest you.
