Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/onyxfish/median
An experiment in zero-infrastructure charting.
https://github.com/onyxfish/median
Last synced: about 1 month ago
JSON representation
An experiment in zero-infrastructure charting.
- Host: GitHub
- URL: https://github.com/onyxfish/median
- Owner: onyxfish
- Created: 2015-07-20T00:15:57.000Z (over 9 years ago)
- Default Branch: master
- Last Pushed: 2015-07-20T00:54:04.000Z (over 9 years ago)
- Last Synced: 2023-04-12T09:29:24.685Z (over 1 year ago)
- Language: JavaScript
- Homepage:
- Size: 727 KB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
median
========================
* [What is this?](#what-is-this)
* [Assumptions](#assumptions)
* [What's in here?](#whats-in-here)
* [Bootstrap the project](#bootstrap-the-project)
* [Hide project secrets](#hide-project-secrets)
* [Add a page to the site](#add-a-page-to-the-site)
* [Run the project](#run-the-project)
* [Compile static assets](#compile-static-assets)
* [Test the rendered app](#test-the-rendered-app)
* [Deploy to S3](#deploy-to-s3)
* [Deploy to EC2](#deploy-to-ec2)
* [Install cron jobs](#install-cron-jobs)
* [Install web services](#install-web-services)
* [Run a remote fab command](#run-a-remote-fab-command)
What is this?
-------------
An experiment in zero-infrastructure charting.
Assumptions
-----------
The following things are assumed to be true in this documentation.
* You are running OSX.
* You are using Python 2.7. (Probably the version that came OSX.)
* You have [virtualenv](https://pypi.python.org/pypi/virtualenv) and [virtualenvwrapper](https://pypi.python.org/pypi/virtualenvwrapper) installed and working.
* You have NPR's AWS credentials stored as environment variables locally.
For more details on the technology stack used with the app-template, see our [development environment blog post](http://blog.apps.npr.org/2013/06/06/how-to-setup-a-developers-environment.html).
What's in here?
---------------
The project contains the following folders and important files:
* ``confs`` -- Server configuration files for nginx and uwsgi. Edit the templates then ``fab servers.render_confs``, don't edit anything in ``confs/rendered`` directly.
* ``fabfile`` -- [Fabric](http://docs.fabfile.org/en/latest/) commands for automating setup, deployment, data processing, etc.
* ``jst`` -- Javascript ([Underscore.js](http://documentcloud.github.com/underscore/#template)) templates.
* ``less`` -- [LESS](http://lesscss.org/) files, will be compiled to CSS and concatenated for deployment.
* ``templates`` -- HTML ([Jinja2](http://jinja.pocoo.org/docs/)) templates, to be compiled locally.
* ``www`` -- Static and compiled assets to be deployed. (a.k.a. "the output")
* ``app.py`` -- A [Flask](http://flask.pocoo.org/) app for rendering the project locally.
* ``app_config.py`` -- Global project configuration for scripts, deployment, etc.
* ``crontab`` -- Cron jobs to be installed as part of the project.
* ``public_app.py`` -- A [Flask](http://flask.pocoo.org/) app for running server-side code.
* ``render_utils.py`` -- Code supporting template rendering.
* ``requirements.txt`` -- Python requirements.
* ``static.py`` -- Static Flask views used in both ``app.py`` and ``public_app.py``.
Bootstrap the project
---------------------
Node.js is required for the static asset pipeline. If you don't already have it, get it like this:
```
brew install node
curl https://npmjs.org/install.sh | sh
```
Then bootstrap the project:
```
cd median
mkvirtualenv median
pip install -r requirements.txt
npm install
fab update
```
**Problems installing requirements?** You may need to run the pip command as ``ARCHFLAGS=-Wno-error=unused-command-line-argument-hard-error-in-future pip install -r requirements.txt`` to work around an issue with OSX.
Hide project secrets
--------------------
Project secrets should **never** be stored in ``app_config.py`` or anywhere else in the repository. They will be leaked to the client if you do. Instead, always store passwords, keys, etc. in environment variables and document that they are needed here in the README.
Any environment variable that starts with ``$PROJECT_SLUG_`` will be automatically loaded when ``app_config.get_secrets()`` is called.
Adding a page to the site
-------------------------
A site can have any number of rendered pages, each with a corresponding template and view. To create a new one:
* Add a template to the ``templates`` directory. Ensure it extends ``_base.html``.
* Add a corresponding view function to ``app.py``. Decorate it with a route to the page name, i.e. ``@app.route('/filename.html')``
* By convention only views that end with ``.html`` and do not start with ``_`` will automatically be rendered when you call ``fab render``.
Run the project
---------------
A flask app is used to run the project locally. It will automatically recompile templates and assets on demand.
```
workon $PROJECT_SLUG
fab app
```
Visit [localhost:8000](http://localhost:8000) in your browser.
Compile static assets
---------------------
Compile LESS to CSS, compile javascript templates to Javascript and minify all assets:
```
workon median
fab render
```
(This is done automatically whenever you deploy to S3.)
Test the rendered app
---------------------
If you want to test the app once you've rendered it out, just use the Python webserver:
```
cd www
python -m SimpleHTTPServer
```
Deploy to S3
------------
```
fab staging master deploy
```
Deploy to EC2
-------------
You can deploy to EC2 for a variety of reasons. We cover two cases: Running a dynamic web application (`public_app.py`) and executing cron jobs (`crontab`).
Servers capable of running the app can be setup using our [servers](https://github.com/nprapps/servers) project.
For running a Web application:
* In ``app_config.py`` set ``DEPLOY_TO_SERVERS`` to ``True``.
* Also in ``app_config.py`` set ``DEPLOY_WEB_SERVICES`` to ``True``.
* Run ``fab staging master servers.setup`` to configure the server.
* Run ``fab staging master deploy`` to deploy the app.
For running cron jobs:
* In ``app_config.py`` set ``DEPLOY_TO_SERVERS`` to ``True``.
* Also in ``app_config.py``, set ``INSTALL_CRONTAB`` to ``True``
* Run ``fab staging master servers.setup`` to configure the server.
* Run ``fab staging master deploy`` to deploy the app.
You can configure your EC2 instance to both run Web services and execute cron jobs; just set both environment variables in the fabfile.
Install cron jobs
-----------------
Cron jobs are defined in the file `crontab`. Each task should use the `cron.sh` shim to ensure the project's virtualenv is properly activated prior to execution. For example:
```
* * * * * ubuntu bash /home/ubuntu/apps/median/repository/cron.sh fab $DEPLOYMENT_TARGET cron_jobs.test
```
To install your crontab set `INSTALL_CRONTAB` to `True` in `app_config.py`. Cron jobs will be automatically installed each time you deploy to EC2.
The cron jobs themselves should be defined in `fabfile/cron_jobs.py` whenever possible.
Install web services
---------------------
Web services are configured in the `confs/` folder.
Running ``fab servers.setup`` will deploy your confs if you have set ``DEPLOY_TO_SERVERS`` and ``DEPLOY_WEB_SERVICES`` both to ``True`` at the top of ``app_config.py``.
To check that these files are being properly rendered, you can render them locally and see the results in the `confs/rendered/` directory.
```
fab servers.render_confs
```
You can also deploy only configuration files by running (normally this is invoked by `deploy`):
```
fab servers.deploy_confs
```
Run a remote fab command
-------------------------
Sometimes it makes sense to run a fabric command on the server, for instance, when you need to render using a production database. You can do this with the `fabcast` fabric command. For example:
```
fab staging master servers.fabcast:deploy
```
If any of the commands you run themselves require executing on the server, the server will SSH into itself to run them.