{"id":19402759,"url":"https://github.com/cscfi/etherpad-deployment-demo","last_synced_at":"2025-04-24T07:32:26.885Z","repository":{"id":71818869,"uuid":"73453033","full_name":"CSCfi/etherpad-deployment-demo","owner":"CSCfi","description":"Deploy Etherpad on OpenStack","archived":false,"fork":false,"pushed_at":"2023-08-01T08:24:25.000Z","size":89,"stargazers_count":3,"open_issues_count":2,"forks_count":1,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-01-28T21:11:45.685Z","etag":null,"topics":["ansible","etherpad","heat","openstack"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/CSCfi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2016-11-11T06:46:50.000Z","updated_at":"2023-08-01T08:24:31.000Z","dependencies_parsed_at":"2023-03-11T11:27:11.655Z","dependency_job_id":null,"html_url":"https://github.com/CSCfi/etherpad-deployment-demo","commit_stats":{"total_commits":13,"total_committers":2,"mean_commits":6.5,"dds":0.07692307692307687,"last_synced_commit":"31f82980a67e84deb474033a138c9f5dc3e4514e"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CSCfi%2Fetherpad-deployment-demo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CSCfi%2Fetherpad-deployment-demo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CSCfi%2Fetherpad-deployment-demo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CSCfi%2Fetherpad-deployment-demo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/CSCfi","download_url":"https://codeload.github.com/CSCfi/etherpad-deployment-demo/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223945413,"owners_count":17229623,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ansible","etherpad","heat","openstack"],"created_at":"2024-11-10T11:25:38.877Z","updated_at":"2024-11-10T11:25:39.331Z","avatar_url":"https://github.com/CSCfi.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Etherpad deployment demo\n\n#### Table of Contents\n\n1. [Overview](#overview)\n2. [Requirements](#requirements)\n    * [Software](#software)\n    * [Cloud environment](#cloud-environment)\n    * [Virtual machine images](#virtual-machine-images)\n3. [Usage](#usage)\n4. [Technical details](#technical-details)\n    * [Tricks used](#tricks-used)\n    * [Caveats](#caveats)\n5. [Contributors](#contributors)\n\n## Overview\n\nThis repository contains Ansible playbooks for deploying Etherpad with a Galera\nbackend using Heat and Docker. This is mainly meant to be an example on how to\nuse these tools together, but you could also potentially use this to deploy an\nEtherpad application on OpenStack.\n\n**If you are going to use the code here as an example, please read the \"Caveats\"\nchapter below.**\n\n## Requirements\n\n### Software\n\n* Ansible \u003e= 2.2\n* shade \u003e= 1.8.0\n\nFor instructions on how to install Ansible, see [the official\ndocumentation](https://docs.ansible.com/).\n\nShade is a client library used by Ansible to communicate with OpenStack clouds.\nThe easiest way to install it is using pip:\n\n```bash\n$ pip install shade\n```\n\n### Cloud environment\n\nYou will also need access to an OpenStack cloud. The project you use for\ndeployment needs to have a network and a router in place. If floating IPs are\navailable for your project then this should already be the case.\n\nThe OpenStack installation needs to have at least Heat and Neutron in addition\nto Nova. The version of the Heat template used here is for Liberty. If you want\nto run against an older version of OpenStack, you will need to modify the\ntemplate version. It is not guaranteed that the template will work against\nolder versions of OpenStack. Contact your OpenStack operator's support if you\nare unsure whether the cloud fulfills these requirements.\n\nMake sure the project you use for the stack has sufficient quota. You will need\nsufficient quota for these additional resources on top of whatever you already\nhave running in the project:\n\n  * Four virtual machines (the exact quota requirement depends on the flavors)\n  * One floating IP\n  * Two security groups and the handful of rules in those security groups\n\n### Virtual machine images\n\nThe database backend deployment has been tested to work with CentOS 7 and\nUbuntu 16.04 images.\n\nThe Etherpad frontend has been tested to work with Ubuntu 16.04. It should also\nwork on CentOS 7, but due to a bug in the node.js role it does not work with\nthat operating system at the time of this writing.\n\n## Usage\n\nFirst you'll need to clone this repository to a directory on your machine. We\nwill refer to the root of the cloned repository with `etherpad-deployment-demo`\nlater in this text.\n\nInstall the third party Ansible roles used:\n\n```bash\n$ ansible-galaxy install -r requirements.yml\n```\n\nYou will need to get an openrc file from OpenStack so that Ansible can interact\nwith it. The easiest way to get it is to login to the web interface and go to\nCompute -\u003e Access \u0026 Security -\u003e API Access -\u003e Download OpenStack RC File. Once\nyou have the file, you will need to source it (the name of the file may be\ndifferent):\n\n```bash\n$ source openrc.sh\n```\n\nAfter that you can fill in the parameters for the Heat stack. First copy the\nexample Heat parameter file to your current working directory:\n\n```bash\n$ cd etherpad-deployment-demo\n$ cp files/example-heat-params.yml playbooks/my-heat-params.yml\n```\n\nEdit the file with your favorite editor and fill in all the variables. You can\nfind documentation about the variables in the Heat template under\n`files/etherpad-heat-stack.yml`.\n\nOnce you have completed the steps above, you are ready to spin up the stack in\nOpenStack. You will need to specify the name of the network you filled in in\nyour Heat parameters:\n\n```bash\n$ ansible-playbook site.yml \\\n  -e \"etherpad_network_name=\u003cthe openstack network shared by instances\u003e\"\n```\n\nThe default user account name used to log in to virtual machines is\n\"cloud-user\". If the images you are using have a different default user account\nname, then you will need to also set the `vm_user_account` variable.\n\nYou can find out the public IP address of the application after the playbook\nrun has finished by looking at the automatically generated `etherpad_inventory`\nfile. The public IP is the value of `ansible_ssh_host` for `etherpad_node`:\n\n```bash\n$ cat etherpad_inventory\n```\n\nOnce the playbook run finishes, you can access the deployed application by\npointing your browser to its public IP address.\n\n## Technical details\n\nThe playbooks here will spin up the following stack:\n\n![Etherpad architecture](images/etherpad-demo-architecture.png)\n\nThe setup of Etherpad is split into stages that are implemented as separate\nplaybooks. You can find these playbooks under the `playbooks/` directory. These\nare all gathered together in the correct order in `site.yml`. The stages are as\nfollows:\n\n1. Start Heat stack\n2. Configure database cluster\n3. Configure HAproxy and Etherpad\n\nYou can follow the flow of execution by starting from `site.yml` and reading the\nincluded playbooks in the order listed.\n\n### Tricks used\n\n**Start a Heat stack and create an Ansible inventory dynamically out of its\noutputs.**\n\nIn the `outputs` section of the Heat template, you'll find an example on how to\nget data out from Heat after its done with its deployment. This output is\nplaced into an Ansible variable using the `register` keyword. The variable is\nthen used to dynamically add the freshly created hosts to Ansible's inventory.\nThe `add_host` module is used for this. An inventory file is also generated\n(`etherpad_inventory`), though this is not used during the Ansible run. It can\nbe used once the stack is running for troubleshooting purposes.\n\n**Connection to hosts with no public IP through a bastion host.**\n\nThe virtual machine used to host Etherpad is also used as a bastion host to\nconnect to the database backend. This is achieved by using the ProxyCommand\nfeature of SSH. The ProxyCommand option is filled in using\n`ansible_ssh_common_args` set in the context of the database cluster nodes (see\n`group_vars/`).\n\n**Automatic generation of passwords.**\n\nThe database passwords required are generated automatically using Ansible's\n`lookup` function. This function generates a password and puts it in a file\nunder `playbooks/credentials`. If the file is already in place, the password in\nit will be used instead. You can see this in the database configuration\nplaybook.\n\n### Caveats\n\nSome shortcuts were taken to finish these playbooks in time for a live demo, so\nif you want to use this repo as an example you should be aware of the following\ncaveats. First of all a caveat about this list: it is almost certainly not\ncomplete, and there are other things that could be done better as there always\nare. As a general principle, you should not copy implementation details from an\nexample without understanding the implications.\n\n**Bad Docker image and container management.**\n\nThere is an actual set of modules for managing Docker images and containers.\nThese are not used here, but they should be. What is done here instead is\nrunning Docker commands directly using `shell` or `command`. For documentation\non the proper way to do manage Docker from Ansible, see\n[Ansible's\ndocumentation](https://docs.ansible.com/ansible/list_of_cloud_modules.html#docker).\n\n**Broken load balancing to the database cluster.**\n\nThe configuration for the load balancing to the database cluster in HAproxy\nwill be something like this:\n\n```\nlisten galera_cluster\n  bind 127.0.0.1:3306\n  mode tcp\n  balance source\n    server galera_node0 192.168.1.176:3306 check\n    server galera_node1 192.168.1.174:3306 check\n    server galera_node2 192.168.1.177:3306 check\n```\n\nDue to the way Galera works, **this will not work properly if one of the\nbackend nodes fails**. What is being done here is dead simple load balancing in\nTCP mode.  However, there is no guarantee that a cluster member that has port\n3306 open is able to serve requests. If you want to use HAproxy for load\nbalancing to a Galera cluster, you will need to configure a script in a\ndifferent port on the cluster nodes that will respond with the status of the\ndatabase node with an HTTP status. Something like this:\n\n```\nlisten galera_cluster\n  bind 127.0.0.1:3306\n  mode tcp\n  balance roundrobin\n  option httpchk\n  server galera_node0 192.168.1.176:3306 check port 9200 inter 2000 rise 2 fall 5\n  server galera_node1 192.168.1.174:3306 backup check port 9200 inter 2000 rise 2 fall 5\n  server galera_node2 192.168.1.177:3306 backup check port 9200 inter 2000 rise 2 fall 5\n```\n\nHere there is a service in port 9200 that reports on the status of the cluster\nnode. Because the check uses HTTP, `option httpchk` is set. With these\nplaybooks, the service in port 9200 doesn't exist, so this configuration cannot\nbe copied without setting that up first. You should be able to find\ndocumentation on how to do that by searching for keywords like e.g. \"haproxy\ngalera cluster load balancing\".\n\nAnother way to configure the load balancing is documented in [the Galera\ncluster documentation on\nHAproxy](http://galeracluster.com/documentation-webpages/haproxy.html).\n\n**No affinity setting for the Galera cluster nodes.**\n\nIf you want to create a production database cluster in OpenStack, you need to\nensure that all virtual machines end up on different physical servers. This is\npossible in OpenStack by using server groups and setting the affinity policy\nfor the server group to anti-affinity. This can be done in Heat by using the\n`OS::Nova::ServerGroup` resource. This is not currently done by this deployment\nexample.\n\n## Contributors\n\n  * Risto Laurikainen - https://github.com/rlaurika\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcscfi%2Fetherpad-deployment-demo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcscfi%2Fetherpad-deployment-demo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcscfi%2Fetherpad-deployment-demo/lists"}