https://github.com/webengineer/django-fabfile

Fabric tasks for Django and Amazon Web Services
https://github.com/webengineer/django-fabfile

Last synced: 11 days ago
JSON representation

Fabric tasks for Django and Amazon Web Services

• Host: GitHub
• URL: https://github.com/webengineer/django-fabfile
• Owner: webengineer
• Created: 2011-08-26T18:56:31.000Z (about 13 years ago)
• Default Branch: master
• Last Pushed: 2013-08-29T11:02:58.000Z (about 11 years ago)
• Last Synced: 2024-08-02T14:12:22.984Z (4 months ago)
• Language: Python
• Homepage:
• Size: 527 KB
• Stars: 21
• Watchers: 0
• Forks: 3
• Open Issues: 0
• Metadata Files:
  • Readme: README.rst
  • Security: docs/security_groups.rst

Awesome Lists containing this project

README

        
Fabric tasks for Django and Amazon Web Services

***********************************************

Required arrangements

---------------------

.. note:: `django_fabfile` package should be listed in your path. It's already

   in place if your are installed it using `pip install django-fabfile`. But if

   you are working within repository, then::

    export PYTHONPATH=

   in your shell.

Pre-run configuration

---------------------

Django settings will be checked first if environment variable

`DJANGO_SETTINGS_MODULE` configured properly. If option was not configured

within Django settings, then its value will be taken from ``./fabfile.cfg``

file - copy-paste there rows that should be overriden from defaults

:download:`django_fabfile/fabfile.cfg.def <../django_fabfile/fabfile.cfg.def>`.

Django settings should be configured in following form (with preserving

sections case)::

    FABFILE = {

        'DEFAULT': {

            'LOGGING_FOLDER': '.',

        },

        'Credentials': {

            'AWS_ACCESS_KEY_ID': 'AKIAABCDEFGHIJKLMNOP',

            'AWS_SECRET_ACCESS_KEY': 'ABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLM',

        },

    }

.. note::

   Boto config files http://code.google.com/p/boto/wiki/BotoConfig will be

   examined at least - if ``[Credentials]`` or some other options was filled

   neither in Django settings nor in ``./fabfile.cfg`` or :download:`django_fabfile/fabfile.cfg.def <../django_fabfile/fabfile.cfg.def>`.

Example of required AWS IAM Policy

----------------------------------

For using Fabric tasks you need AWS account with following permissions:

* create-snapshot::

    {

      "Statement": [

        {

          "Sid": "Stmt1304003463574",

          "Action": [

            "ec2:CreateSnapshot",

            "ec2:CreateTags",

            "ec2:DescribeInstances",

            "ec2:DescribeRegions",

            "ec2:DescribeSnapshots",

            "ec2:DescribeTags",

            "ec2:DescribeVolumes"

          ],

          "Effect": "Allow",

          "Resource": "*"

        }

      ]

    }

* delete-snapshot::

    {

      "Statement": [

        {

          "Sid": "Stmt1306410750989",

          "Action": [

            "ec2:DescribeRegions",

            "ec2:DescribeSnapshots",

            "ec2:DeleteSnapshot"

          ],

          "Effect": "Allow",

          "Resource": "*"

        }

      ]

    }

* reboot-instance::

    {

      "Statement": [

        {

          "Sid": "Stmt1312204628195",

          "Action": [

            "ec2:RebootInstances"

          ],

          "Effect": "Allow",

          "Resource": "*"

        },

        {

          "Sid": "Stmt1312276311113",

          "Action": [

            "ec2:DetachVolume"

          ],

          "Effect": "Allow",

          "Resource": "*"

        }

      ]

    }

* rsync-snapshot::

    {

      "Statement": [

        {

          "Effect": "Allow",

          "Action": [

            "EC2:AttachVolume",

            "EC2:CreateKeyPair",

            "EC2:CreateVolume",

            "EC2:DeleteKeyPair",

            "EC2:DeleteVolume",

            "EC2:DescribeAvailabilityZones",

            "EC2:DescribeImages",

            "EC2:DescribeKeyPairs",

            "EC2:RunInstances",

            "EC2:TerminateInstances"

          ],

          "Resource": "*"

        }

      ]

    }

Backup script

-------------

.. note:: Instances and snaphots are filtered by tag "Earmarking" value

   "production". This can be configured in settings - see "Pre-run

   configuration" section above.

Following backup commands may be configured with crontab to run daily::

    #!/bin/sh

    VE=/home/backup-agent/.virtualenvs/backup

    . $VE/bin/activate && \

    fab --keepalive=60 -f $VE/lib/python2.6/site-packages/django_fabfile/backup.py \

            backup_instances_by_tag \

            trim_snapshots \

            rsync_region:src_region_name=us-east-1,dst_region_name=eu-west-1 \

            rsync_region:src_region_name=us-west-1,dst_region_name=eu-west-1 \

            rsync_region:src_region_name=eu-west-1,dst_region_name=us-east-1

With 30 production instances it tooks around 6 hours to replicate

snapshots updates. First replication tooks much more time obviously.

Running tests

-------------

The package has test cases to make sure, that all methods, being tested,

keep on the entire program logic. The test cases are using packages ``fudge``

and ``unittest``. The test files (``'test_*.py'``) are located in folder

``'/django_fabfile/tests/'``. To run tests from command line, implement::

    python -m unittest discover django_fabfile

Recent changes

--------------

See :doc:`changelog` for recent updates.

Licensing

---------

The product is licensed by oDesk http://www.odesk.com/ under

:download:`GNU GENERAL PUBLIC LICENSE Version 3` except

portions with attached inline license information like

:func:`django_fabfile.backup._trim_snapshots`.