{"id":19581108,"url":"https://github.com/beyondtheclouds/openstackoid","last_synced_at":"2025-09-12T20:39:07.740Z","repository":{"id":68615155,"uuid":"160845272","full_name":"BeyondTheClouds/openstackoid","owner":"BeyondTheClouds","description":"Make your OpenStacks Collaborative","archived":false,"fork":false,"pushed_at":"2019-12-06T15:02:43.000Z","size":159,"stargazers_count":3,"open_issues_count":0,"forks_count":2,"subscribers_count":8,"default_branch":"master","last_synced_at":"2025-04-27T09:35:26.284Z","etag":null,"topics":["collaborative","openstack"],"latest_commit_sha":null,"homepage":"","language":"Lua","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/BeyondTheClouds.png","metadata":{"files":{"readme":"README.org","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-12-07T16:01:04.000Z","updated_at":"2019-02-18T23:03:57.000Z","dependencies_parsed_at":"2023-02-21T11:30:21.880Z","dependency_job_id":null,"html_url":"https://github.com/BeyondTheClouds/openstackoid","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/BeyondTheClouds/openstackoid","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BeyondTheClouds%2Fopenstackoid","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BeyondTheClouds%2Fopenstackoid/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BeyondTheClouds%2Fopenstackoid/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BeyondTheClouds%2Fopenstackoid/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/BeyondTheClouds","download_url":"https://codeload.github.com/BeyondTheClouds/openstackoid/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BeyondTheClouds%2Fopenstackoid/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":274873613,"owners_count":25365824,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-12T02:00:09.324Z","response_time":60,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["collaborative","openstack"],"created_at":"2024-11-11T07:32:42.524Z","updated_at":"2025-09-12T20:39:07.734Z","avatar_url":"https://github.com/BeyondTheClouds.png","language":"Lua","funding_links":[],"categories":[],"sub_categories":[],"readme":"#+TITLE: OpenStackoïd\n\n/Make your OpenStacks Collaborative/\n\nThis PoC aims at making several independent OpenStack clouds\ncollaborative. The main idea consists in extending the OpenStack CLI\nto define the collaboration process. A dedicated option, called\n~--os-scope~, specifies which services (e.g., compute, image,\nidentity, ...) of which OpenStack cloud (e.g., CloudOne, CloudTwo,\n...) an OpenStack is made of to perform the CLI request. For instance,\nthe provisioning of a VM in CloudOne with an image in CloudTwo looks\nlike as follows:\n\n: openstack server create my-vm --flavor m1.tiny --image cirros \\\n:   --os-scope '{\"compute\": \"CloudOne\", \"image\": \"CloudTwo\"}'\n\nThis approach enables the segregation of the infrastructure into\ndistinct areas. It removes the relying on a single control plane and\nthus, resolves network partitioning and scalability challenges in most\ncases.\n\nTo get more insight, read the [[#how-it-works][how it works]] section or [[#try-it][try it]] by\nyourself.\n\n* Table of Contents                                                  :TOC@2@gh:\n- [[#try-it][Try it]]\n- [[#limitation][Limitation]]\n- [[#how-it-works][How it works]]\n  - [[#haproxy-rules-the-flow][HAProxy rules the flow]]\n  - [[#scope-should-follow-the-workflow][Scope should follow the workflow]]\n  - [[#rest-client-instance-variable-in-keystonemiddleware][Rest client instance variable in Keystonemiddleware]]\n- [[#setup][Setup]]\n  - [[#provisioning-openstack-with-devstack][Provisioning OpenStack with Devstack]]\n  - [[#provisioning-the-scope-interpretation][Provisioning the scope interpretation]]\n- [[#project-structure][Project structure]]\n- [[#acknowledgment][Acknowledgment]]\n\n* Try it\n  :PROPERTIES:\n  :CUSTOM_ID: try-it\n  :END:\nGet the code with a git clone.\n: git clone --recurse-submodules git@github.com:BeyondTheClouds/openstackoid.git -b stable/rocky --depth 1\n\nThen starts the two OpenStack clouds (require tmux v2 and Vagrant v2.2\n-- if you don't want to use tmux, refer to the [[#setup][setup]] section).\n: cd openstackoid; ./setup-env.sh\n\nThe previous command starts two tmux windows and launches two\nOpenStack clouds, each in a virtual machine, thanks to Vagrant. After\n15 to 20 minutes, the time for Devstack to deploy the two OpenStack\nclouds, the output may look like the following. The top window\nconnects to OpenStack ~CloudOne~ and the bottom one to ~CloudTwo~.\nThese two clouds are completely independent and shared no services.\n\n#+begin_example\nstack@CloudOne:~$\n─────────────────────────────────────────────────────────────────────────────────────────────────────────────\nstack@CloudTwo:~$\n#+end_example\n\nFrom there you can issue a standard ~openstack~ command, such as\n~openstack image list~. Note the difference of ~ID~.\n\n#+begin_example\nstack@CloudOne:~$ openstack image list\n+--------------------------------------+--------------------------+--------+\n| ID                                   | Name                     | Status |\n+--------------------------------------+--------------------------+--------+\n| 440263d5-20a7-432b-b0db-693787bd2579 | cirros-0.3.5-x86_64-disk | active |\n+--------------------------------------+--------------------------+--------+\n─────────────────────────────────────────────────────────────────────────────────────────────────────────────\nstack@CloudTwo:~$ openstack image list\n+--------------------------------------+--------------------------+--------+\n| ID                                   | Name                     | Status |\n+--------------------------------------+--------------------------+--------+\n| 45da7b62-163c-4c78-aac7-363cbf4627a4 | cirros-0.3.5-x86_64-disk | active |\n+--------------------------------------+--------------------------+--------+\n#+end_example\n\nMoreover, you can use the ~--os-scope~ option that tells OpenStack of\na specific cloud to do something by using a service from another\ncloud. For example, you can tell to the ~CloudOne~ to start a VM by\nusing the ~image~ service from ~CloudTwo~. In the scope, a service is\nimplicitly bound to the local cloud, which prevents to explicitly\nspecify all services. Note the ~ID~ of ~image~ at the end that comes\nfrom ~CloudTwo~.\n\n#+begin_example\nstack@CloudOne:~$ openstack server create my-vm \\\n  --os-scope '{\"image\": \"CloudTwo\"}' \\\n  --image  cirros-0.3.5-x86_64-disk \\\n  --flavor m1.tiny \\\n  --wait\n+-------------------------------------+-----------------------------------------------------------------+\n| Field                               | Value                                                           |\n+-------------------------------------+-----------------------------------------------------------------+\n| OS-DCF:diskConfig                   | MANUAL                                                          |\n| ...                                 | ...                                                             |\n| image                               | cirros-0.3.5-x86_64-disk (45da7b62-163c-4c78-aac7-363cbf4627a4) |\n| name                                | my-vm                                                           |\n| ...                                 | ...                                                             |\n| status                              | ACTIVE                                                          |\n| user_id                             | 2d9440f8a4d546c88d1f5b661dc6e69b                                |\n+-------------------------------------+-----------------------------------------------------------------+\n─────────────────────────────────────────────────────────────────────────────────────────────────────────\nstack@CloudTwo:~$ openstack image list\n+--------------------------------------+--------------------------+--------+\n| ID                                   | Name                     | Status |\n+--------------------------------------+--------------------------+--------+\n| 45da7b62-163c-4c78-aac7-363cbf4627a4 | cirros-0.3.5-x86_64-disk | active |\n+--------------------------------------+--------------------------+--------+\n#+end_example\n\n🎉\n\nSee [[file:misc/examples.sh][misc/examples.sh]] for other examples.\n\n* TODO Limitation\n  :PROPERTIES:\n  :CUSTOM_ID: limitation\n  :END:\n- Same project, domain id for non-public resources\n- Same keystone credential\n- Resource of another cloud should be accessible from the first one\n  (e.g., image is OK, flat network is NOK unless the two clouds share\n  the same infra).\n\n* How it works\n  :PROPERTIES:\n  :CUSTOM_ID: how-it-works\n  :END:\n** HAProxy rules the flow\nIn brief, every OpenStack cloud comes with a proxy (here HAProxy)\nin front of it. In such deployment, a service (e.g., Glance API of\n~CloudOne~) is available via two addresses:\n- The /Backend/ address (i.e., ~10.0.2.15/image~) that directly\n  targets Glance API.\n- The /Frontend/ address (i.e., ~192.168.141.245:8888/image~)\n  that targets HAProxy. HAProxy then evaluates the request and, in\n  most cases, forwards it to the Backend.\n\nHere, we add HAProxy the capability [[file:playbooks/haproxy/lua/interpret_scope.lua][to interprets]] the ~--os-scope~.\nInstead of forwarding the request to the local Backend, HAProxy\ndetermines the cloud of the targeted service from the scope and\nURL. It then forwards the request to the local Backend only if the\ncurrent cloud is equivalent to the determined one. Otherwise, it\nforwards the request to the Frontend of the determined cloud.\n\nAs an example, here is a sample of the HAProxy configuration on\n~CloudOne~ for the ~image~ service.\n\n#+begin_src conf-space -n\nlisten http-proxy\n  bind 192.168.141.245:8888           # (ref:local-front)\n  http-request del-header X-Forwarded-Proto if { ssl_fc }\n  use_backend %[lua.interpret_scope]  # (ref:lua-scope)\n\n# Target concrete backend\nbackend CloudOne_image_public\n  server CloudOne 10.0.2.15:80 check inter 2000 rise 2 fall 5 # (ref:local-back)\n\n# Target HA of OS cloud named CloudTwo\nbackend CloudTwo_image_public\n  http-request set-header Host 192.168.141.245:8888\n  server CloudTwo 192.168.142.245:8888 check inter 2000 rise 2 fall 5 # (ref:remote-front)\n\n# Do the same for compute, identity, ...\n#+end_src\n\nThe ~lua.interpret_scope~ line [[(lua-scope)]] is a [[file:playbooks/haproxy/lua/interpret_scope.lua][Lua script]] that\ndetermines the name of the backend based on the ~--os-scope '{\"image\":\n\"CloudTwo\"}~ and URL of the targeted service. From there, it\nforwards the request whether to the local Backend ~10.0.2.15~ (l.\n[[(local-back)]]) or Frontend of the remote cloud ~192.168.142.245~ (l.\n[[(remote-front)]]).\n\n*** Generating the HAProxy configuration file\n    :PROPERTIES:\n    :CUSTOM_ID: generating-the-haproxy-configuration-file\n    :END:\nBased on a short description list of all services (see lst.\n[[lst:services-desc]]), it is easy to [[file:playbooks/haproxy/haproxy.cfg.j2][generate the HAProxy configuration\nfile]] automatically. The description list, on the other hand, partially\ncomes with the next OpenStack command. The addresses of the Frontend\nand Backend for all services still have to be added.\n\n: openstack endpoint list --format json \\\n:   -c \"Service Type\" -c \"Interface\" -c \"URL\" -c \"Region\"\n\n#+NAME: lst:services-desc\n#+CAPTION: Services description list\n#+begin_src json\n{ \"services\" :\n  [\n    {\n      \"Service Type\": \"image\",\n      \"Interface\": \"public\",\n      \"URL\": \"192.168.141.245:8888/image\",\n      \"Region\": \"CloudOne\",\n      \"Frontend\": \"192.168.141.245:8888\",\n      \"Backend\": \"10.0.2.15:80\"\n    },\n    ...\n    {\n      \"Service Type\": \"image\",\n      \"Interface\": \"public\",\n      \"URL\": \"192.168.142.245:8888/image\",\n      \"Region\": \"CloudTwo\",\n      \"Frontend\": \"192.168.142.245:8888\",\n      \"Backend\": \"10.0.2.15:80\"\n    },\n    ...\n  ]\n}\n#+end_src\n\n** Scope should follow the workflow\nHAProxy determines from the ~--os-scope~ the address of the targeted\nservice. Which means, the scope has to be defined for every request\nand subsequent requests. For instance, when Alice does an ~openstack\nserver create --os-scope ...~, the value of the ~--os-scope~ should\nnot only be attached to the initial ~POST /servers~ request made by\nthe CLI. But also, to all subsequent requests of the workflow,\nincluding Nova request to Keystone to check Alice credentials, Nova\nrequest to Glance to check/get the image. Glance request to Keystone\nto check Alice credentials ... and so on.\n\nA first solution is to modify the OpenStack code of all services to\nensure that, e.g., when Alice contacts Nova with a specific\n~--os-scope~, then Nova propagates that ~--os-scope~ in the subsequent\nrequests. However, in OpenStackoïd, we want to avoid as much as\npossible modifications to the vanilla code.\n\nAnother naive implementation would try to implement the scope\npropagation at HAProxy level -- and keep OpenStack code as it is.\nUnfortunately, this doesn't work since HAProxy is unlikely to figure\nout that, e.g., the current request from Nova to Glance comes from a\nprevious request from Alice to Nova with a specific ~--os-scope~.\n\nLuckily, every OpenStack service already propagates information from\none service to another during the entire workflow of a command: the\nKeystone ~X-Auth-Token~ that contains Alice credentials. Here we reuse\nthat information to piggyback the ~--os-scope~. Then, HAProxy seeks\nfor the ~X-Auth-Token~, extracts the scope and finally interprets it\nto forwards the request to the good cloud.\n\n** TODO Rest client instance variable in Keystonemiddleware\n   :PROPERTIES:\n   :CUSTOM_ID: rest-client-instance-variable-in-keystonemiddleware\n   :END:\nTODO\n\n* Setup\n  :PROPERTIES:\n  :CUSTOM_ID: setup\n  :END:\nThe setup is made of, but not limited to, two distinct VirtualBox VMs\nwith an All-in-One OpenStack inside each. The [[file:setup-env.sh][setup-env.sh]] script\nstarts two tmux windows and runs vagrant inside each window. Vagrant\nis in charge of deploying the All-in-One OpenStack and then\nconfiguring OpenStack to interpret the ~--os-scope~.\n\nThe [[file:Vagrantfile][Vagrantfile]] contains the description of the two All-in-One\nOpenStack at its top (see ~os_clouds~). The ~:name~ refers to the name\nof the cloud, ~:ip~ to the Frontend address (has to be accessible by\nother clouds), and ~:ssh~ to the port used by Vagrant for SSH\nconnections. Doing a ~vagrant up~ reads that configuration and starts\ntwo Ubuntu/16.04 VMs with these characteristics. Adding a third entry\nin ~os_clouds~ and running ~vagrant up~ again will start a third\nAll-in-One OpenStack.\n\n#+CAPTION: Configuration of OpenStack clouds\n#+begin_src ruby\nos_clouds = [\n  {\n    :name =\u003e \"CloudOne\",\n    :ip =\u003e \"192.168.141.245\",\n    :ssh =\u003e 2141\n  },\n  {\n    :name =\u003e \"CloudTwo\",\n    :ip =\u003e \"192.168.142.245\",\n    :ssh =\u003e 2142\n  }\n]\n#+end_src\n\nIt is also possible to start only one OpenStack cloud by giving its\nname after the ~vagrant up~. For instance, the following command only\nstarts and configures the ~CloudOne~.\n\n: vagrant up CloudOne\n\n** Provisioning OpenStack with Devstack\nA ~vagrant up \u003cCloudName\u003e~, on its first run, automatically deploys\nOpenStack with Devstack and then configures it for the ~--os-scope~.\nBut, it is possible to only run the deployment of Devstack with the\nfollowing commands.\n\n: vagrant up \u003cCloudName\u003e --no-provision\n: vagrant provision \u003cCloudName\u003e --provision-with devstack\n\nThe ~--provision-with devstack~ refers to the Ansible\n[[file:playbooks/devstack.yml][playbooks/devstack.yml]] playbook. In brief, this playbook:\n1. Adds a stack user.\n2. Clones Devstack stable/rocky.\n3. Generates a local.conf.\n4. Runs Devstack deployment.\n\nIf something goes wrong during the execution of this playbook,\neverything is OK. Simply rerun the ~vagrant provision \u003cCloudName\u003e\n--provision-with devstack~, since Ansible playbooks are idempotent.\n\n** Provisioning the scope interpretation\nIn the same manner of the previous section, it is also possible to\nonly run the configurations of one OpenStack cloud to interpret the\n~--os-scope~ with the next command.\n\n: vagrant provision \u003cCloudName\u003e --provision-with os-scope\n\nThe ~--provision-with os-scope~ refers to the Ansible\n[[file:playbooks/os-scope.yml][playbooks/os-scope.yml]] playbook. In brief, this playbook:\n1. Computes the list of services as explained in the \"How it works\"\n   (see, [[#generating-the-haproxy-configuration-file][Generating the HAProxy configuration file]]).\n2. Uses that list to generate the HAProxy configuration file, and then\n   deploys HAProxy.\n3. Installs a new plugin for python-openstackclient that adds the\n   ~--os-scope~ in the CLI.\n4. Workaround the rest client instance variable in Keystonemiddleware\n   (see, [[#rest-client-instance-variable-in-keystonemiddleware][Rest client instance variable in Keystonemiddleware]]).\n5. Ensures that HTTP requests of OpenStack services go through the\n   proxy (on that particular point, read the next section).\n\nIf something goes wrong during the execution of this playbook,\neverything is OK. Simply rerun the ~vagrant provision \u003cCloudName\u003e\n--provision-with os-scope~, since Ansible playbooks are idempotent.\n\n*** ~[HACK]~ tag in the code\nDevstack doesn't provide HAProxy deployment by default and we want to\navoid the modification of Devstack -- or any other OpenStack services\n-- as much as possible. Thus, we deployed HAProxy after Devstack and\nthen ensure each request to OpenStack goes through the proxy thanks to\nthe ~HTTP_PROXY~ environment variable. This is referenced in the\ncurrent code with the ~[HACK]~ tag. In a real-world deployment (à la\nKolla), services are already hidden behind HAProxy and code marked\nwith the ~[HACK]~ tag should be removed.\n\n* Project structure\n#+begin_example\n.\n├── keystonemiddleware@...        Fork of k-middleware\n│   └── ...\n├── misc                          Miscellaneous\n│   ├── examples.sh               - OS CLI examples with the --os-scope\n│   └── ...\n├── playbooks                     List of provisioning playbooks\n│   ├── devstack.yml              - Devstack provisioning\n│   ├── os-scope.yml              - OpenStackoïd provisioning\n│   └── haproxy                   - HAProxy conf files for OpenStackoïd\n├── python-openstackoidclient     OpenStackoïd CLI plugin\n│   └── ...\n├── setup-env.sh                  Tmux setup script\n└── Vagrantfile                   Vagrant conf that setups the 2 OS\n#+end_example\n\n* Acknowledgment\nWe would like to thanks members of the OpenStack community, and\nespecially members of the [[https://twitter.com/tcarrez/status/1061665184530481152][OpenStack Berlin Hackathon]] (team 5) which\nhave laid some of the initial foundation for this work:\n- [[https://www.linkedin.com/in/lebre/][Adrien Lebre]]\n- [[https://www.linkedin.com/in/ashkan-kamyab-a97b0495/][Ashkan Kamyab]]\n- [[https://www.linkedin.com/in/curtis-collicutt-99037295/][Curtis Collicutt]]\n- [[https://www.linkedin.com/in/elvissn/][Elvis Noudjeu]]\n- [[https://www.linkedin.com/in/ilya-alekseyev-7a29b310/][Ilya Alekseyev]]\n- [[https://www.linkedin.com/in/jrbalderrama/][Javier Rojas Balderrama]]\n- [[https://rcherrueau.github.io/][Ronan-Alexandre Cherrueau]]\n- [[https://www.linkedin.com/in/magyarizsolt/][Zsolt Magyari]]\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbeyondtheclouds%2Fopenstackoid","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbeyondtheclouds%2Fopenstackoid","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbeyondtheclouds%2Fopenstackoid/lists"}