{"id":15284007,"url":"https://github.com/propertyshelf/mls.fabfile","last_synced_at":"2026-04-28T23:02:13.552Z","repository":{"id":11859323,"uuid":"14418237","full_name":"propertyshelf/mls.fabfile","owner":"propertyshelf","description":"Deploy and manage Propertyshelf MLS applications using Fabric.","archived":false,"fork":false,"pushed_at":"2016-02-26T07:59:41.000Z","size":46,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-03-01T07:36:19.226Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/propertyshelf.png","metadata":{"files":{"readme":"README.rst","changelog":"CHANGES.rst","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2013-11-15T07:52:22.000Z","updated_at":"2016-02-26T07:59:44.000Z","dependencies_parsed_at":"2022-09-26T18:11:47.243Z","dependency_job_id":null,"html_url":"https://github.com/propertyshelf/mls.fabfile","commit_stats":null,"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/propertyshelf%2Fmls.fabfile","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/propertyshelf%2Fmls.fabfile/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/propertyshelf%2Fmls.fabfile/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/propertyshelf%2Fmls.fabfile/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/propertyshelf","download_url":"https://codeload.github.com/propertyshelf/mls.fabfile/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245138124,"owners_count":20566872,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-09-30T14:48:38.397Z","updated_at":"2026-04-28T23:02:13.500Z","avatar_url":"https://github.com/propertyshelf.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"mls.fabfile\n===========\n\nThis project contains a bunch of fabric commands we use at `Propertyshelf`_\nto deploy and maintain our MLS systems.\n\n\nRequirements\n------------\n\n`mls.fabfile` currently uses `knife`_ to communicate with Rackspace servers.\nPlease make sure `knife` is installed and configured successfully on your\nsystem.\n\n\nInstall\n-------\n\nYou can install `mls.fabfile` with PIP::\n\n    pip install mls.fabfile\n\nAll required dependencies will be installed automatically.\n\n\nUsage\n-----\n\nFirst, we need a working `knife.rb` file to interact with our Chef server and\nthe Rackspace cloud eco system. Below is an example `knife.rb` file that gets\nall its required info from environment variables. This way you can add this\n`knife.rb` file inside a `.chef` directory to your project and savely put it\nunder version control::\n\n    # Logging.\n    log_level                         :info\n    log_location                      STDOUT\n\n    # Chef server configuration.\n    chef_server_url                   \"#{ENV['KNIFE_CHEF_SERVER']}\"\n    client_key                        \"#{ENV['KNIFE_CLIENT_KEY']}\"\n    node_name                         \"#{ENV['KNIFE_NODE_NAME']}\"\n    validation_client_name            \"#{ENV['KNIFE_VALIDATION_CLIENT_NAME']}\"\n    validation_key                    \"#{ENV['KNIFE_VALIDATION_CLIENT_KEY']}\"\n    encrypted_data_bag_secret         \"#{ENV['ENCRYPTED_DATA_BAG_SECRET_FILE']}\"\n\n    # Rackspace API configuration.\n    knife[:rackspace_api_key]       = \"#{ENV['RACKSPACE_API_KEY']}\"\n    knife[:rackspace_api_username]  = \"#{ENV['RACKSPACE_USERNAME']}\"\n    knife[:rackspace_endpoint]      = \"#{ENV['RACKSPACE_ENDPOINT']}\"\n    knife[:rackspace_version]       = \"#{ENV['RACKSPACE_VERSION']}\"\n\n\nNext, we need a `fabfile.py`. All we need to do is to import `mls.fabfile`\nto make the fabric commands available and the available environments we can\nwork with from `propertyshelf.fabfile.common`.::\n\n    # -*- coding: utf-8 -*-\n    \"\"\"Sample MLS deployment script.\"\"\"\n\n    from fabric import api\n\n    from mls.fabfile import *\n    from propertyshelf.fabfile.common.environments import *\n\n\n    # Definition of role names to be used.\n    api.env.role_database = 'mls_db'\n    api.env.role_frontend = 'mls_fe'\n    api.env.role_staging = 'mls_staging'\n    api.env.role_worker = 'mls_app'\n\n    # Definition of used Rackspace flavors (server sized) for our servers.\n    api.env.flavor_database = '5'\n    api.env.flavor_frontend = '2'\n    api.env.flavor_staging = '2'\n    api.env.flavor_worker = '3'\n\n    # Definition of node names to be used.\n    api.env.nodename_database = 'mls-db'\n    api.env.nodename_frontend = 'mls-fe'\n    api.env.nodename_staging = 'mls-staging'\n    api.env.nodename_worker = 'mls-app'\n\n    # The Rackspace server image we use. This is a Debian 6.0.6.\n    api.env.os_image = '92a28e50-181d-4fc7-a071-567d26fc95f6'\n\n    # MLS specific configuration.\n    api.env.domain = 'mls-example.com'\n    api.env.mls_customizations = ['mlsext.realtorcom', ]\n    api.env.mls_policy_enabled = True\n    api.env.mls_policy_package = 'mlspolicy.example'\n    api.env.mls_policy_package_url = 'git https://github.com/yourname/mlspolicy.example'\n\nYou can now use fabric to manage your MLS::\n\n    $ fab -l\n    Sample MLS deployment script.\n\n    Available commands:\n\n        development                 Work locally with vagrant.\n        production                  Work with the production environment.\n        staging                     Work with the staging environment.\n        bootstrap.bundle_db_fe_app  Bootstrap a new MLS bundle: Database, Frontend, App Worker.\n        bootstrap.database          Bootstrap a new standalone database server.\n        bootstrap.frontend          Bootstrap a new standalone frontend server.\n        bootstrap.staging           Bootstrap a staging system.\n        bootstrap.worker            Bootstrap a new standalone application worker.\n        client.remove               Remove an existing MLS application client.\n        client.restart              Restart the application client component.\n        client.update               Update the client packages.\n        database.backup             Perform a backup of Zope's data on the server.\n        database.download_blobs     Download blob part of Zope's data from the server.\n        database.download_data      Download the database files from the server.\n        database.download_zodb      Download ZODB part of Zope's data from the server.\n        database.restart            Restart the database component.\n        database.restore            Restore an existing backup of Zope's data on the server.\n        database.upload_blob        Upload blob part of Zope's data to the server.\n        database.upload_data        Upload the database files to the server.\n        database.upload_zodb        Upload ZODB part of Zope's data to the server.\n        frontend.restart            Restart the frontend components.\n        frontend.restart_haproxy    Restart the HA-Proxy load balancer component.\n        frontend.restart_nginx      Restart the NginX web server component.\n        frontend.restart_varnish    Restart the Varnish caching proxy component.\n        roles.check                 Check if the required roles are available.\n        roles.create_missing        Create missing roles on the chef server.\n\nBefore we can start it is a good idea to check if all roles we defined are\navailable on the chef server::\n\n    $ fab roles.check\n    Role mls_fe NOT available.\n    Role mls_db NOT available.\n    Role mls_staging NOT available.\n    Role mls_app NOT available.\n\n    Done.\n\nTo create the missing roles based on our configuration, we simply have to do::\n\n    $ fab roles.create_missing\n    Created role mls_db\n    Created role mls_fe\n    Created role mls_app\n    Created role mls_staging\n\n    Done.\n\nNow we can create our staging system::\n\n    $ fab bootstrap.staging\n    [localhost] local: knife rackspace server create -S mls-staging -N mls-staging -f 5 -I 92a28e50-181d-4fc7-a071-567d26fc95f6 -r role[rackspace],role[mls_staging] -E staging\n\n    ...\n\n    Done.\n\nNote that there can only be one staging system. If you try to add another one\nwith the same name, you'll get an error message::\n\n    $ fab bootstrap.staging\n\n    Fatal error: Server \"mls-staging\" already exists in environment \"staging\".\n\n    Aborting.\n\nIf you need a second one, you can adjust the node name manually::\n\n    $ fab bootstrap.staging:nodename=mls-staging2\n    [localhost] local: knife rackspace server create -S mls-staging2 -N mls-staging2 -f 5 -I 92a28e50-181d-4fc7-a071-567d26fc95f6 -r role[rackspace],role[mls_ni_staging] -E staging\n\n    ...\n\n    Done.\n\nYou can now manage the single components::\n\n    $ fab staging frontend.restart\n    [x.x.x.x] Executing task 'frontend.restart'\n    [x.x.x.x] sudo: /etc/init.d/haproxy restart\n    [x.x.x.x] out: sudo password:\n\n    [x.x.x.x] out: Restarting haproxy: haproxy.\n    [x.x.x.x] out:\n\n    [x.x.x.x] sudo: /etc/init.d/varnish restart\n    [x.x.x.x] out: sudo password:\n    [x.x.x.x] out: Stopping HTTP accelerator: varnishd.\n    [x.x.x.x] out: Starting HTTP accelerator: varnishd.\n    [x.x.x.x] out:\n\n    [x.x.x.x] sudo: /etc/init.d/nginx restart\n    [x.x.x.x] out: sudo password:\n    [x.x.x.x] out: Restarting nginx: nginx.\n    [x.x.x.x] out:\n\n\n    Done.\n    Disconnecting from x.x.x.x... done.\n\nWe also support download of the database files for local testing::\n\n    $ fab production database.download_data\n    [x.x.x.x] Executing task 'database.download_data'\n    This will overwrite your local Data.fs. Are you sure you want to continue? [Y/n]\n    [localhost] local: mkdir -p var/filestorage\n    [localhost] local: mv var/filestorage/Data.fs var/filestorage/Data.fs.bak\n    [x.x.x.x] out: sudo password:\n    [x.x.x.x] sudo: rsync -a var/filestorage/Data.fs /tmp/Data.fs\n    [x.x.x.x] out: sudo password:\n    [x.x.x.x] out:\n    [x.x.x.x] download: /Volumes/Work/Propertyshelf/MLS/Provisioning/var/filestorage/Data.fs \u003c- /tmp/Data.fs\n    This will overwrite your local blob files. Are you sure you want to continue? [Y/n]\n    [localhost] local: rm -rf var/blobstorage_bak\n    [localhost] local: mv var/blobstorage var/blobstorage_bak\n    [x.x.x.x] sudo: rsync -a ./var/blobstorage /tmp/\n    [x.x.x.x] out: sudo password:\n    [x.x.x.x] out:\n    [x.x.x.x] sudo: tar czf blobstorage.tgz blobstorage\n    [x.x.x.x] out: sudo password:\n    [x.x.x.x] out:\n    [x.x.x.x] download: /Volumes/Work/Propertyshelf/MLS/Provisioning/var/blobstorage.tgz \u003c- /tmp/blobstorage.tgz\n\n    Warning: Local file /Volumes/Work/Propertyshelf/MLS/Provisioning/var/blobstorage.tgz already exists and is being overwritten.\n\n    [localhost] local: tar xzf blobstorage.tgz\n\n    Done.\n    Disconnecting from x.x.x.x... done.\n\nOnce we have local data files, we can upload them to our development environment\n(a vagrant VM)::\n\n    $ fab development database.upload_data client.restart\n    [localhost] local: vagrant ssh-config | grep IdentityFile\n    [127.0.0.1:2222] Executing task 'database.upload_data'\n    This will overwrite your remote Data.fs. Are you sure you want to continue? [y/N] y\n    [127.0.0.1:2222] sudo: mkdir -p /tmp/upload\n    [127.0.0.1:2222] put: var/filestorage/Data.fs -\u003e /tmp/upload/Data.fs\n    [127.0.0.1:2222] sudo: chown mls /tmp/upload/Data.fs\n    [127.0.0.1:2222] sudo: supervisorctl stop zeoserver\n    [127.0.0.1:2222] out: zeoserver: stopped\n    [127.0.0.1:2222] out:\n\n    [127.0.0.1:2222] sudo: mv var/filestorage/Data.fs var/filestorage/Data.fs.bak\n    [127.0.0.1:2222] sudo: mv /tmp/upload/Data.fs var/filestorage/Data.fs\n    This will overwrite your remote blob files. Are you sure you want to continue? [y/N] y\n    [127.0.0.1:2222] sudo: mkdir -p /tmp/upload\n    [localhost] local: tar czf blobstorage_upload.tgz blobstorage\n    [127.0.0.1:2222] put: var/blobstorage_upload.tgz -\u003e /tmp/upload/blobstorage.tgz\n    [127.0.0.1:2222] sudo: chown mls /tmp/upload/blobstorage.tgz\n    [127.0.0.1:2222] sudo: tar xzf blobstorage.tgz\n    [127.0.0.1:2222] sudo: supervisorctl stop zeoserver\n    [127.0.0.1:2222] out: zeoserver: ERROR (not running)\n    [127.0.0.1:2222] out:\n\n    [127.0.0.1:2222] sudo: mv var/blobstorage var/blobstorage_bak\n    [127.0.0.1:2222] sudo: mv /tmp/upload/blobstorage var\n    [127.0.0.1:2222] sudo: supervisorctl start zeoserver\n    [127.0.0.1:2222] out: zeoserver: started\n    [127.0.0.1:2222] out:\n\n    [127.0.0.1:2222] Executing task 'client.restart'\n    [127.0.0.1:2222] sudo: supervisorctl restart application\n    [127.0.0.1:2222] out: application: stopped\n    [127.0.0.1:2222] out: application: started\n    [127.0.0.1:2222] out:\n\n\n    Done.\n    Disconnecting from 127.0.0.1:2222... done.\n\nWe can also get a list of nodes for already defined roles::\n\n    $ fab roles.list_nodes\n    Role: mls_fe\n    - mls-fe: x.x.x.x\n\n    Role: mls_db\n    - mls-db: x.x.x.x\n\n    Role: mls_staging\n    - mls-staging: x.x.x.x\n    - vagrant-mls-staging: 10.0.2.15\n\n    Role: mls_app\n    - mls-app-01: x.x.x.x\n\n\n    Done.\n\nThis can be useful if we want to execute a task only for a given node::\n\n    $ fab frontend.restart_nginx:hosts=x.x.x.x\n    [x.x.x.x] Executing task 'frontend.restart_nginx'\n    [x.x.x.x] sudo: /etc/init.d/nginx restart\n    [x.x.x.x] out: sudo password:\n    [x.x.x.x] out: Restarting nginx: nginx.\n    [x.x.x.x] out:\n\n\n    Done.\n    Disconnecting from x.x.x.x... done.\n\n\n.. _`Propertyshelf`: http://propertyshelf.com\n.. _`knife`: http://docs.opscode.com/knife.html\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpropertyshelf%2Fmls.fabfile","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpropertyshelf%2Fmls.fabfile","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpropertyshelf%2Fmls.fabfile/lists"}