https://github.com/nprapps/clerk
A cron job --> Slack webhook that posts new action from the House floor
https://github.com/nprapps/clerk
cron federal-government slack
Last synced: about 1 year ago
JSON representation
A cron job --> Slack webhook that posts new action from the House floor
- Host: GitHub
- URL: https://github.com/nprapps/clerk
- Owner: nprapps
- Created: 2017-02-03T18:25:00.000Z (over 9 years ago)
- Default Branch: master
- Last Pushed: 2017-02-03T18:52:24.000Z (over 9 years ago)
- Last Synced: 2025-03-25T09:22:10.444Z (about 1 year ago)
- Topics: cron, federal-government, slack
- Language: Python
- Homepage:
- Size: 15.6 KB
- Stars: 5
- Watchers: 6
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# clerk
## What is this?
**Clerk is a cron job that posts to a Slack webhook whenever new action happens on the House floor. It pulls from the [official House clerk](http://clerk.house.gov/floorsummary/floor.aspx) XML feed**
## Assumptions
The following things are assumed to be true in this documentation.
* You are running OSX.
* You are using Python 3.
* You have [virtualenv](https://pypi.python.org/pypi/virtualenv) and [virtualenvwrapper](https://pypi.python.org/pypi/virtualenvwrapper) installed and working.
For more details our stack, see our [development environment blog post](http://blog.apps.npr.org/2013/06/06/how-to-setup-a-developers-environment.html).
## What's in here?
The project contains the following folders and important files:
* ``fabfile`` -- [Fabric](http://docs.fabfile.org/en/latest/) commands for automating setup, deployment, data processing, etc.
* ``app_config.py`` -- Global project configuration for scripts, deployment, etc.
* ``crontab`` -- Cron jobs to be installed as part of the project.
* ``requirements.txt`` -- Python requirements.
* ``run_on_server.sh`` -- Shell script to run at the beginning of cron jobs to source environment variables and virtual environments.
## Bootstrap the project
```
cd clerk
mkvirtualenv clerk
pip install -r requirements.txt
```
## Hide project secrets
Project secrets should **never** be stored in ``app_config.py`` or anywhere else in the repository. They will be leaked to the client if you do. Instead, always store passwords, keys, etc. in environment variables and document that they are needed here in the README.
Any environment variable that starts with ``$PROJECT_SLUG_`` will be automatically loaded when ``app_config.get_secrets()`` is called.
## Connecting to Slack
To connect Clerk to your Slack, you will need to create an [incoming webhook](https://api.slack.com/incoming-webhooks) and copy the webhook endpoint to an environment variable called `clerk_WEBHOOK`. The default server setup expects the environment variable to be placed in `/etc/environment`.
## Setting up servers
Use the server configuration in [app_config.py](https://github.com/nprapps/clerk/blob/master/app_config.py#L36-L46) to set up your servers correctly. You can set the user, python version, and the folders where various pieces install.
## Deploy to EC2
1. Run ``fab staging master servers.setup`` to configure the server.
2. Run ``fab staging master deploy`` to deploy the project to the server. Note that this will also install your crontab.
## Managing cron jobs
Cron jobs are defined in the file `crontab`. Each task should use the `run_on_server.sh` shim to ensure the project's virtualenv is properly activated prior to execution. For example:
```
* * * * * ubuntu bash /home/ubuntu/apps/clerk/repository/cron.sh fab $DEPLOYMENT_TARGET cron_jobs.test >> /var/log/clerk/crontab.log 2>&1
```
The cron jobs themselves should be defined in `fabfile/cron_jobs.py` whenever possible.
To install the cronjob, run either `fab staging master deploy`, which will also synchronize the repo with the latest on Github, or `fab staging master servers.install_crontab` to simply copy the crontab on the server to `etc/cron.d`.
To uninstall the cronjob (a recommended practice for disabling the job), run `fab staging master servers.uninstall_cronjob`.