{"id":16563086,"url":"https://github.com/ab77/cloud-harness","last_synced_at":"2025-12-04T00:02:14.904Z","repository":{"id":92447838,"uuid":"38174676","full_name":"ab77/cloud-harness","owner":"ab77","description":"Python wrapper for cloud service provider APIs/SDKs","archived":false,"fork":false,"pushed_at":"2016-03-25T14:47:29.000Z","size":160,"stargazers_count":3,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-01-15T17:18:01.671Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ab77.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2015-06-27T20:43:16.000Z","updated_at":"2019-05-18T09:08:59.000Z","dependencies_parsed_at":"2023-06-01T03:00:10.462Z","dependency_job_id":null,"html_url":"https://github.com/ab77/cloud-harness","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ab77%2Fcloud-harness","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ab77%2Fcloud-harness/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ab77%2Fcloud-harness/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ab77%2Fcloud-harness/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ab77","download_url":"https://codeload.github.com/ab77/cloud-harness/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241975135,"owners_count":20051428,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-11T20:38:41.038Z","updated_at":"2025-12-04T00:02:14.852Z","avatar_url":"https://github.com/ab77.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"## Cloud Harness\nPython wrapper for cloud service provider APIs/SDKs.\n\n### Supported Cloud Providers\n* Microsoft Azure `./cloud-harness.py azure`, incorporating:\n\t- Azure Service Management APIs via [Microsoft Azure Python SDK/API](https://github.com/Azure/azure-sdk-for-python)\n\t- [Azure Resource Management APIs](https://msdn.microsoft.com/en-us/library/azure/dn948464.aspx)\n\n### Installation and Configuration\n* from a working Python environment, run `pip install -r requirements.txt \u0026\u0026 git clone https://github.com/ab77/cloud-harness.git /opt/cloud-harness \u0026\u0026 pushd /opt/cloud-harness`\n* copy `cloud-harness.sample.conf` to `cloud-harness.conf`\n* [download](https://manage.windowsazure.com/publishsettings) and save your Azure PublishSettings file with `.publishsettings` extension to the same directory\n* run `./cloud-harness.py azure` for the first time to extract your management certificate and update the config file automatically (or [manually](http://stuartpreston.net/2015/02/retrieving-microsoft-azure-management-certificates-for-use-in-cross-platform-automationprovisioning-tools/) if you wish)\n* set default `location_name` in `cloud-harness.conf` config file (e.g. East US), run `python ./cloud-harness.py azure` for the default action `list_locations`\n* set other configuration properties as required in `cloud-harness.conf` config file (e.g. `storage_account` and `service_certificate`)\n* to use the new Azure Resource Management (ARM) APIs, you'll need to follow [these](http://blog.davidebbo.com/2014/12/azure-service-principal.html) steps. \n\n### General Usage\n* run `python ./cloud-harness.py azure --help` to see all available command line options\n* to get get a list of required parameters for a particular action (e.g. `add_role`), run `python ./cloud-harness.py azure --action add_role`\n* specify `--verbose` flag to see various run-time properties\n* specify `--readonly` flag to limit operations, which would otherwise perform changes (usually together with `--verbose`)\n* specify `--async` flag to return from calls immediately, without waiting for operation completion\n\n### Supported Resource Extensions\nThe following resource extensions are supported (use `--extension \u003cextension\u003e \u003cextension\u003e ... \u003cext\u003e`):\n* ChefClient (Windows|Linux)\n* CustomScript (Windows|Linux)\n* VMAccessAgent (Windows|Linux)\n* OSPatching (Linux)\n* DockerExtension (Linux)\n* DSC (Windows)\n* PuppetEnterpriseAgent (Windows|Linux)\n* BGInfo (Windows|Linux)\n* OctopusDeploy (Windows)\n\n### Examples\nSome useful examples to deploy virtual machines and various resource extensions.\n\n##### Create storage account (name must be unique as it forms part of the storage URL, check with `--action check_storage_account_name_availability`):\n\n    ./cloud-harness.py azure --action create_storage_account \\\n    --account myuniquestorageaccountname01 \\\n    --verbose\n\nRemember to update `storage_account` in `cloud-harness.py` with the name of the storage account you've just created.\n\n##### Create a new hosted service (name must be unique within `cloudapp.net` domain, check with `--action check_storage_account_name_availability`):\n\n    ./cloud-harness.py azure --action create_hosted_service \\\n    --service my-hosted-service \\\n    --label 'my hosted service' \\\n    --verbose\n\n##### Add x.509 certificate containing RSA public key for SSH authentication to the hosted service:\n\n    ./cloud-harness.py azure --action add_service_certificate \\\n    --service my-hosted-service \\\n    --certificate service_certificate.cer \\\n    --verbose\n\n##### Create a reserved IP address for the hosted service:\n\n    ./cloud-harness.py azure --action create_reserved_ip_address \\\n    --ipaddr my-reserved-ip-address \\\n    --verbose\n\n##### Create Virtual Network:\n\n    ./cloud-harness.py azure --action create_virtual_network_site \\\n\t--network VNet1 \\\n\t--subnet Subnet-1 \\\n\t--subnetaddr 10.0.0.0/11 \\\n\t--vnetaddr 10.0.0.0/8 \\\n\t--verbose\n\n##### List OS Images:\n\n    ./cloud-harness.py azure --action list_os_images\n\t\n##### Create a new Linux virtual machine deployment and role with reserved IP, SSH authentication and `CustomScript` resource extension[n3]:\n\n    ./cloud-harness.py azure --action create_virtual_machine_deployment \\\n    --service my-hosted-service \\\n    --deployment my-virtual-machine-deployment \\\n    --name my-ubuntu-virtual-machine \\\n    --label 'my deployment' \\\n    --account my-storage-account \\\n    --blob b39f27a8b8c64d52b05eac6a62ebad85__Ubuntu_DAILY_BUILD-wily-15_10-amd64-server-20150722-en-us-30GB \\\n    --os Linux \\\n    --network my-virtual-network-name \\\n    --subnet my-subnet-name \\\n    --ipaddr my-reserved-ip-address \\\n    --size Medium \\\n    --extension CustomScript \\\n    --ssh_auth \\\n    --disable_pwd_auth \\\n    --verbose\n\n##### Add `Docker` extension to the Linux virtual machine[n6]:\n\n    ./cloud-harness.py azure --action add_resource_extension \\\n    --service my-hosted-service \\\n    --deployment my-virtual-machine-deployment \\\n    --name my-ubuntu-virtual-machine \\\n    --extension DockerExtension \\\n    --docker_compose compose.yaml \\\n    --verbose\t\n\nNote, the Docker extension relies on certain pre-requisites, incl. certificates. See [this](http://docs.docker.com/engine/articles/https/) article on how to generate these and don't forget to update the `[DockerExtension]` section in `cloud-harness.conf` afterwards.\n\n##### Create a Linux virtual machine (role) with a random alpha-numeric password[n2], add `CustomScript` and `ChefClient` extensions:\n\n    ./cloud-harness.py azure --action add_role \\\n    --service my-hosted-service \\\n    --deployment my-virtual-machine-deployment \\\n    --name my-second-ubuntu-virtual-machine \\\n    --label 'my Linux (Ubuntu) virtual machine' \\\n    --account my-storage-account \\\n    --blob b39f27a8b8c64d52b05eac6a62ebad85__Ubuntu_DAILY_BUILD-wily-15_10-amd64-server-20150722-en-us-30GB \\\n    --os Linux \\\n    --network my-virtual-network-name \\\n    --subnet my-subnet-name \\\n    --size Medium \\\n\t--extension CustomScript ChefClient \\\n    --verbose\n\n##### Add data disk to virtual machine:\n\n    ./cloud-harness.py azure --action add_data_disk \\\n    --service my-hosted-service \\\n    --deployment my-virtual-machine-deployment \\\n    --name my-second-ubuntu-virtual-machine \\\n    --account my-storage-account \\\n    --verbose\n\n##### Create a Windows virtual machine (role) with random alpha-numeric password, and `CustomScript` extension[n4]:\n\n    ./cloud-harness.py azure --action add_role \\\n    --service my-hosted-service \\\n    --deployment my-virtual-machine-deployment \\\n    --name my-windows-virtual-machine \\\n    --label 'my Windows 2K8R2 virtual machine' \\\n    --account my-storage-account \\\n    --os Windows \\\n    --blob a699494373c04fc0bc8f2bb1389d6106__Win2K8R2SP1-Datacenter-201505.01-en.us-127GB.vhd \\\n    --network my-virtual-network-name \\\n    --subnet my-subnet-name \\\n    --size Medium \\\n\t--extension CustomScript \\\n    --verbose\n\n##### Add `ChefClient` and `DSC` (Desired State Configuration) extensions to the Windows virtual machine[n7]:\n\n    ./cloud-harness.py azure --action add_resource_extension \\\n    --service my-hosted-service \\\n    --deployment my-virtual-machine-deployment \\\n    --name my-windows-virtual-machine \\\n    --extension ChefClient DSC \\\n\t--dsc_module IISInstall.ps1.zip \\\n    --verbose\t\n\t\n##### Reset the Administrator password on the Windows VM using `VMAccess` extension:\n\n    ./cloud-harness.py azure --action add_resource_extension \\\n    --service my-hosted-service \\\n    --deployment my-virtual-machine-deployment \\\n    --name my-second-ubuntu-virtual-machine \\\n    --extension VMAccessAgent \\\n    --password new-s3cure-passw0rd \\\n    --verbose\n\n##### Update Linux virtual machine (role) using `OSPatching` extension:\n\n    ./cloud-harness.py azure --action add_resource_extension \\\n    --service my-hosted-service \\\n    --deployment my-virtual-machine-deployment \\\n    --name my-second-ubuntu-virtual-machine \\\n    --extension OSPatching \\\n    --patching_oneoff \\\n    --verbose\n\n##### Secure the virtual machine, by adding ACLs to the public facing port(s)[n5]:\n\n    ./cloud-harness.py azure --action set_epacls \\\n    --service my-hosted-service \\\n    --deployment my-virtual-machine-deployment \\\n    --name my-ubuntu-virtual-machine \\\n    --subnet my-subnet-name \\\n    --verbose\n\n##### **DESTROY** service, deployment, virtual machines (roles), disks and associated VHDs:\n\n    ./cloud-harness.py azure --action delete_hosted_service \\\n    --service my-hosted-service \\\n    --delete_disks \\\n    --delete_vhds \\\n    --verbose\n\n##### **DELETE** reserved IP address:\n\n    ./cloud-harness.py azure --action delete_reserved_ip_address \\\n    --ipaddr my-reserved-ip-address\n\n#### Fiddler Proxy\nTo use `Fiddler2` to capture HTTPS traffic to the API\n* export your Azure Management Certificate as base64 encoded  x.509 as ClientCertificate.cer\n* place it into your `Fiddler2` directory (e.g. `C:\\Users\\\u003cuser\u003e\\Documents\\Fiddler2`)\n* set `proxy = True` in `cloud-harness.conf` and re-launch `Fiddler2` [n1]\n\n### Further Work\nLots, including:\n* implement new Azure Resource Management APIs\n* implement (at least) one additional cloud service provider (e.g. [DigitalOcean](https://www.digitalocean.com/?refcode=937b01397c94))\n* add a suitable unittest framework\n* implement additional VM extensions (OctopusDeploy, etc.)\n* where it makes sense, move user-configurable defaults to config file\n* automate Chef Server deployment from Azure [image](http://azure.microsoft.com/blog/2015/04/01/chef-server-in-marketplace-chef-azure-provisioning-and-more/)\n\n-- [ab1](https://plus.google.com/+AntonBelodedenko?rel=author)\n\n#### Notes\n[n1] For more information, see [Using Fiddler to decipher Windows Azure PowerShell or REST API HTTPS traffic](http://blogs.msdn.com/b/avkashchauhan/archive/2013/01/30/using-fiddler-to-decipher-windows-azure-powershell-or-rest-api-https-traffic.aspx).\n\n[n2] SSH authentication is not compatible with `ChefClient` extension due to the way it currently handles certificates [PR45](https://github.com/chef-partners/azure-chef-extension/pull/45).\n\n[n3] `CustomScript` extension on Linux by default, will run `bootstrap.sh` to upgrade `WAAgent` as well as un-pack/execute `linux_custom_data.dat` where you can put additional bootstrap commands.\n\n[n4] `CustomScript` extension on Windows by default, will run `bootstrap.ps1` to un-pack/execute `windows_custom_data.dat` where you can put additional bootstrap commands.\n\n[n5] `update_role()` currently resets ACLs, use `--action set_epacls` to set them again if you get a warning. Also, this operation will cause a reboot and currently generates new public facing port numbers.\n\n[n6] `Docker` is secured by default with SSL, using a server certificate signed by a private CA.\n\n[n7] `DSC` configuration archive can be compiled using PowerShell, run `Publish-AzureVMDscConfiguration .\\MyConfiguration.ps1 -ConfigurationArchivePath .\\MyConfiguration.ps1.zip`\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fab77%2Fcloud-harness","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fab77%2Fcloud-harness","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fab77%2Fcloud-harness/lists"}