{"id":31074113,"url":"https://github.com/vbattista78/microstack-autoscaling-project","last_synced_at":"2026-05-16T18:07:56.702Z","repository":{"id":313635756,"uuid":"1052107483","full_name":"vbattista78/microstack-autoscaling-project","owner":"vbattista78","description":"Demo of CPU-driven autoscaling with load balancing on MicroStack/OpenStack: create clone on high CPU, delete base on low.","archived":false,"fork":false,"pushed_at":"2025-09-14T17:02:52.000Z","size":88,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-09-14T18:38:04.005Z","etag":null,"topics":["bash","cloud-computing","cloud-init","cpu-metrics","load-balancing","microstack","openstack","python","vm-cloning"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vbattista78.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-09-07T12:22:36.000Z","updated_at":"2025-09-14T17:02:56.000Z","dependencies_parsed_at":"2025-09-07T14:33:03.392Z","dependency_job_id":"35c4875e-48d6-4560-adb2-17de3ffdcaa6","html_url":"https://github.com/vbattista78/microstack-autoscaling-project","commit_stats":null,"previous_names":["vbattista78/microstack-autoscaling-project"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/vbattista78/microstack-autoscaling-project","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vbattista78%2Fmicrostack-autoscaling-project","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vbattista78%2Fmicrostack-autoscaling-project/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vbattista78%2Fmicrostack-autoscaling-project/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vbattista78%2Fmicrostack-autoscaling-project/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vbattista78","download_url":"https://codeload.github.com/vbattista78/microstack-autoscaling-project/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vbattista78%2Fmicrostack-autoscaling-project/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":275348511,"owners_count":25448626,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-16T02:00:10.229Z","response_time":65,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bash","cloud-computing","cloud-init","cpu-metrics","load-balancing","microstack","openstack","python","vm-cloning"],"created_at":"2025-09-16T02:02:48.940Z","updated_at":"2025-09-16T02:03:23.937Z","avatar_url":"https://github.com/vbattista78.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# MicroStack Autoscaling with Load Balancing: CPU Metrics to Scale Actions (Create on High, Delete on Low)\n\nThis project implements a practical **autoscaling with load balancing** pattern on a single-node **MicroStack (OpenStack)** lab. A controller monitors a baseline VM’s CPU/memory; on sustained high load it **creates a clone**, and on sustained low load it **deletes the baseline** and keeps the **clone as the new primary**. The goal is a small, reproducible lab that demonstrates scale-out and graceful handover.\n\n**Scripts in this repo**\n- `deploy_secure_vm.py` — Idempotent provisioning: network/router/security group/keypair, baseline **CirrOS** VM, **Floating IP**, optional **snapshot retention**, and **cleanup**. It also sets minimal security-group rules so SSH and ICMP work out of the box.\n- `autoscale_watch.py` — Autoscaling controller: polls **CPU and MEM** via SSH, triggers **clone creation** and **baseline removal**; expects to call the deploy tool as `~/deploy_secure_vm.py`.\n- `split_after_scale.sh` — Load generator: produces a deterministic **100% CPU spike** on the baseline to force scale-out and later a **~50/50 split** across base+clone; includes a `--stop` to cleanly stop all loads.\n\n---\n\n## Quick Start (3 steps)\n\n```bash\n# 0) Load OpenStack CLI credentials (MicroStack)\nsource /var/snap/microstack/common/etc/microstack.rc\n\n# 1) Create a baseline VM (CirrOS + Floating IP) with minimal SG and (optionally) no snapshot\npython3 ./deploy_secure_vm.py --name VM-test --no-snapshot \\\n  --keypair lab-key --pubkey-file ~/.ssh/lab-key.pub\n\n# Note the actual name (auto-numbered, e.g., VM-test_1):\nmicrostack.openstack server list\n\n# 2) Start the autoscaling controller (monitor CPU/MEM on the baseline)\npython3 ./autoscale_watch.py \\\n  --server VM-test_1 \\\n  --clone legacy \\\n  --high 80 --low 20 \\\n  --min-up 4 --min-down 4 \\\n  --metric max\n\n# 3) In another terminal, trigger load and watch the scale-out + handover\n./split_after_scale.sh\n```\n\n**What you’ll see**\n- On sustained **HIGH** (≥ `--high` for `--min-up` samples), the controller **creates a clone** named from `\u003cbase\u003e_clone` (automatically numbered by the deploy tool).\n- On sustained **LOW** (≤ `--low` for `--min-down` samples), the controller **deletes the baseline** and continues monitoring the **clone** (handover).\n\n**Why each command matters**\n- `source …microstack.rc` → loads OpenStack auth variables into your shell (auth URL, project, token). Without this, OpenStack CLI calls will fail.\n- `deploy_secure_vm.py …` → creates everything needed (network, SG, keypair) and a **CirrOS** VM with a Floating IP. `--no-snapshot` skips the automatic snapshot (faster for tests).\n- `server list` → shows the **actual VM name** created (e.g., `VM-test_1`), which you’ll pass to `--server`.\n- `autoscale_watch.py …` → starts the **controller** that measures CPU/MEM via SSH and applies **scale-out / handover** logic.\n- `split_after_scale.sh` → generates load (first 100% to trigger scale-out, then ~50/50 on base+clone to demonstrate load balancing).\n\n---\n\n## One-time setup (keys, SDK, path)\n\n1) **SSH keys** (keep both; CirrOS often prefers RSA):\n```bash\nssh-keygen -t ed25519 -f ~/.ssh/lab-key -N \"\"\nssh-keygen -t rsa -b 2048 -f ~/.ssh/lab-key-rsa -N \"\"\n```\n*Explanation:* ed25519 is modern/compact; RSA ensures compatibility with CirrOS. Keys enable passwordless SSH.\n\n2) **OpenStack SDK** \u0026 **clouds.yaml** (the scripts use `cloud='microstack'`):\n```bash\npython3 -m pip install --user openstacksdk\nmkdir -p ~/.config/openstack\ncat \u003e ~/.config/openstack/clouds.yaml \u003c\u003c'YAML'\nclouds:\n  microstack:\n    auth:\n      auth_url: https://\u003cMICROSTACK_IP\u003e:5000/v3\n      username: admin\n      password: \u003cKEYSTONE_PASSWORD\u003e\n      project_name: admin\n      user_domain_name: default\n      project_domain_name: default\n    region_name: microstack\n    interface: public\n    identity_api_version: 3\n    verify: false\nYAML\n```\n*Explanation:* centralizes credentials for SDK/CLI; `verify: false` avoids self-signed certificate issues in the lab. Replace `\u003cMICROSTACK_IP\u003e` and `\u003cKEYSTONE_PASSWORD\u003e`.\n\n3) **Make the deployer reachable from `$HOME`**  \nThe controller invokes it as `~/deploy_secure_vm.py`:\n```bash\ncp ./deploy_secure_vm.py ~/deploy_secure_vm.py\nchmod +x ~/deploy_secure_vm.py\n```\n*Security-group note:* by default, the deployer opens **ICMP** to `0.0.0.0/0` and **SSH 22** only from `10.20.20.1/32`. Change that CIDR to your **host IP/32** (or temporarily open to `0.0.0.0/0` while testing) so SSH works immediately.\n\n---\n\n## How it works — detailed command explanations\n\n### 1) Baseline provisioning — `deploy_secure_vm.py`\n\n**What it does \u0026 why**\n1. **Networking (idempotent):** creates/validates `lab-net`, `lab-net-subnet (192.168.100.0/24)`, and `lab-router` with **external** gateway, then attaches the subnet to the router → instances can reach the Internet.\n2. **Security Group `sg-secure`:** enables **ICMP** (diagnostics) and **SSH 22** from your IP → safe admin access without opening to the entire Internet.\n3. **Nova keypair:** if missing, imports your public key (`--pubkey-file`) under name `--keypair` → **passwordless SSH**.\n4. **CirrOS `m1.tiny` VM:** creates a baseline named **`\u003cBASE\u003e_N`** (e.g., `VM-test_1`) to avoid name collisions across repeated runs.\n5. **Floating IP:** allocates and associates a FIP with the VM → reachable from your host/LAN.\n6. **Optional snapshots:** unless you pass `--no-snapshot`, creates a snapshot and **retains only the last `--retain`** (production-like hygiene in a lab).\n7. **Cleanup:** with `--cleanup \u003cBASE\u003e` removes all VMs starting with `\u003cBASE\u003e`, orphan Floating IPs, and (optionally) related snapshots.\n\n**Examples**\n```bash\n# Create/update a baseline VM (no snapshot)\npython3 ./deploy_secure_vm.py --name VM-test --no-snapshot \\\n  --keypair lab-key --pubkey-file ~/.ssh/lab-key.pub\n\n# Full cleanup for a base prefix (VMs/FIPs; add snapshots with --wipe-snaps)\npython3 ./deploy_secure_vm.py --cleanup VM-test --wipe-snaps --yes\n```\n\n\u003e After deploy, the script prints ready-to-use SSH commands (ed25519 and RSA) and reminds the default CirrOS console password (`cubswin:)`).\n\n---\n\n### 2) Autoscaling controller — `autoscale_watch.py`\n\n**What it measures \u0026 how it decides**\n- **Metrics:** reads **CPU** (`/proc/stat`) and **MEM** (`/proc/meminfo`) via SSH; choose `--metric cpu`, `--metric mem`, or `--metric max` (default, most conservative).\n- **SSH user/key:** user **`cirros`**, **RSA** key for compatibility; point to it with `--ssh-key-path` if needed.\n- **Scale-out:** when the metric stays ≥ `--high` for `--min-up` consecutive samples → calls the deployer to create a clone using `\u003cbase\u003e_clone` as base (the deployer auto-numbers, e.g., `_clone_1`).\n- **Handover:** when the metric stays ≤ `--low` for `--min-down` samples **and a clone exists** → deletes the **baseline** (via the deployer) and switches monitoring to the **clone** (new primary).\n\n**Key parameters (and why)**\n```bash\npython3 ./autoscale_watch.py \\\n  --server VM-test_1 \\        # VM to monitor (if omitted: interactive selection in a TTY)\n  --clone legacy \\            # kept for compatibility; the deployer decides the actual name\n  --high 80 --low 20 \\        # thresholds: high to create, low to hand over\n  --min-up 4 --min-down 4 \\   # consecutive samples required (anti-flap)\n  --interval 5 \\              # polling interval in seconds\n  --metric max \\              # cpu | mem | max (use \"max\" to be conservative)\n  --ssh-key-path ~/.ssh/lab-key-rsa \\\n  --deploy-keypair lab-key \\\n  --deploy-pubkey-file ~/.ssh/lab-key.pub\n```\n\n**What you’ll see in logs**\n- metric lines: `[metrics] cpu=… mem=… -\u003e max=…`\n- **SCALE UP**: log lines showing scale up and the deployer being invoked\n- **SCALE DOWN**: log lines like “deleting baseline … and proceeding with clone …”\n\n\u003e Why `~/deploy_secure_vm.py`? The controller reuses the deployer’s idempotent logic for both **clone creation** and **baseline deletion**; keeping it in `$HOME` makes the call stable regardless of the current working directory.\n\n---\n\n### 3) Load generator — `split_after_scale.sh`\n\n**What it does \u0026 why**\n- If **no clone exists**, it starts **100% CPU** on the baseline (one `yes \u003e /dev/null` per vCPU) until the controller scales; then it waits for the clone to have a FIP/SSH, **stops 100%**, and starts a **~50/50 duty cycle** on **both** baseline and clone to showcase load balancing.\n- If a clone **already exists**, it skips the 100% spike and starts **50/50** directly on base+clone.\n- `--stop`: safely stops the load on base and clone (automatically detects active pairs).\n\n**Usage**\n```bash\n# Start: interactively choose an ACTIVE VM as the baseline\n./split_after_scale.sh\n\n# Stop: end the load on all detected base/clone pairs\n./split_after_scale.sh --stop\n```\n\n---\n\n## What to expect (end-to-end)\n\n1. `deploy_secure_vm.py --name VM-test` → **`VM-test_1`** ACTIVE with a Floating IP and correct SG in place.  \n2. `autoscale_watch.py --server VM-test_1` → monitoring starts (CPU/MEM).  \n3. `split_after_scale.sh` → sustained load ≥ 80% → controller **creates `\u003cbase\u003e_clone_*`**; when load falls ≤ 20% sustained → controller **deletes the baseline** and continues on the **clone**.\n\n---\n\n## Repository layout\n\n- `deploy_secure_vm.py` — provisioning (network/router/SG/keypair/VM), Floating IP, snapshots, cleanup  \n- `autoscale_watch.py` — autoscaler (CPU/MEM polling, scale-out via deployer, handover)  \n- `split_after_scale.sh` — deterministic load generator (`100%` spike + `~50/50` balancer with `--stop`)\n\n## Limitations\n\n- **Single-node MicroStack lab:** no high availability and limited performance; intended for reproducible demos rather than production.\n- **Guest image:** the baseline uses **CirrOS**, which has a minimal userspace and limited cloud-init features (no package manager).\n- **Telemetry path:** metrics are sampled over **SSH** from inside the guest (no Ceilometer/Gnocchi/Monasca).\n- **Load balancing scope:** there is no L4/L7 load balancer; the “balancing” is demonstrated by splitting CPU load across the two instances during the overlap window.\n\n### Note on the (deferred) Horizon dashboard\n\nInitially, the project aimed to provide a small **Horizon panel** under the “Project” section to orchestrate the same workflow from the GUI. On **MicroStack**, however, Horizon is delivered as a **snap**; the snap mount is **read-only**, and enabling a custom panel requires changing Horizon’s Python/Django modules and settings (e.g., `INSTALLED_APPS`, panel registrations, `local_settings.py`). Because these files live inside the read-only snap at runtime, the panel cannot be dropped in or enabled without **rebuilding the Horizon snap** or running a **separate Horizon deployment** (e.g., a source/DevStack build or a containerized Horizon) where those changes are allowed.\n\nFor this reason, the files needed to explore a dashboard implementation were **authored but not executed** on MicroStack. If you plan to pursue a GUI path on a different OpenStack build, refer to the **official Horizon Dashboard Developer Guide** for panel/plugin structure and enablement steps.\n\n### Reference documentation used for implementation and configuration:\n- [MicroStack Documentation](https://microstack.run/docs/)\n- [MicroStack Start Reference](https://discourse.ubuntu.com/t/get-started-with-microstack/13998)\n- [Official Ubuntu Cloud images](https://cloud-images.ubuntu.com/)\n- [OpenStack CLI Reference](https://docs.openstack.org/python-openstackclient/latest/)\n\n### Disclaimers\n* This README is both a user guide and a summary report of the project for the period of August 2025.\n* Names and IPs have been anonymized for the creation of the repository!\n* The laboratory was created without the use of certificates to avoid conflicts and problems during development. The use of certificates is recommended in the event of operational use.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvbattista78%2Fmicrostack-autoscaling-project","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvbattista78%2Fmicrostack-autoscaling-project","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvbattista78%2Fmicrostack-autoscaling-project/lists"}