{"id":50766539,"url":"https://github.com/kuldeep-poonia/loadequilibrium","last_synced_at":"2026-06-11T14:01:36.659Z","repository":{"id":345940648,"uuid":"1187937042","full_name":"kuldeep-poonia/loadequilibrium","owner":"kuldeep-poonia","description":"Predictive infrastructure congestion control engine that models distributed systems as coupled queues to forecast saturation risk and generate proactive scaling or load-shedding signals.","archived":false,"fork":false,"pushed_at":"2026-06-06T04:16:21.000Z","size":1181,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-06-06T06:11:46.240Z","etag":null,"topics":["autoscaling","control-theory","distributed-systems","infrastructure","observability","queueing-theory"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kuldeep-poonia.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-03-21T11:51:25.000Z","updated_at":"2026-06-06T04:16:25.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/kuldeep-poonia/loadequilibrium","commit_stats":null,"previous_names":["kuldeep-poonia/loadequilibrium"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/kuldeep-poonia/loadequilibrium","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kuldeep-poonia%2Floadequilibrium","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kuldeep-poonia%2Floadequilibrium/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kuldeep-poonia%2Floadequilibrium/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kuldeep-poonia%2Floadequilibrium/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kuldeep-poonia","download_url":"https://codeload.github.com/kuldeep-poonia/loadequilibrium/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kuldeep-poonia%2Floadequilibrium/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34201842,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-11T02:00:06.485Z","response_time":57,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["autoscaling","control-theory","distributed-systems","infrastructure","observability","queueing-theory"],"created_at":"2026-06-11T14:01:35.724Z","updated_at":"2026-06-11T14:01:36.646Z","avatar_url":"https://github.com/kuldeep-poonia.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# LoadEquilibrium\r\n\r\n![Go](https://img.shields.io/badge/Go-1.20+-blue?style=flat-square)\r\n![Docker](https://img.shields.io/badge/Docker-Ready-brightgreen?style=flat-square)\r\n![Kubernetes](https://img.shields.io/badge/Kubernetes-Compatible-326ce5?style=flat-square)\r\n![License](https://img.shields.io/badge/License-MIT-green?style=flat-square)\r\n\r\n\u003e Predictive auto-scaling for Docker \u0026 Kubernetes. Watches your services, predicts failures 60 seconds ahead, and scales automatically using control theory (MPC + RL).\r\n\r\n---\r\n\r\n## What It Does\r\n\r\nYou add one label to each service you want monitored. LoadEquilibrium does the rest:\r\n\r\n- Watches your services every 2 seconds\r\n- Builds a live mathematical model of each service's queue, latency, and load\r\n- Predicts when a service is about to fail — before it actually does\r\n- Issues precise scaling decisions automatically\r\n- Shows you everything in a live dashboard\r\n\r\nIt is not a threshold alarm. It does not wait until latency spikes to react. It uses the same class of control system used in aircraft autopilots — applied to your software.\r\n\r\n---\r\n\r\n## Getting Started in 3 Steps\r\n\r\n### Step 1 — Add a label to each service you want monitored\r\n\r\n```yaml\r\n# your existing docker-compose.yml\r\nservices:\r\n  my-api:\r\n    image: your-app:latest\r\n    labels:\r\n      le.enable: \"true\"    # ← add this one line\r\n```\r\n\r\nThat is the only change you make to your existing service. No agents to install. No config files to write.\r\n\r\n### Step 2 — Add LoadEquilibrium to your compose file\r\n\r\n```yaml\r\n  loadequilibrium:\r\n    image: ghcr.io/your-org/loadequilibrium:latest\r\n    ports:\r\n      - \"8080:8080\"\r\n    volumes:\r\n      - /var/run/docker.sock:/var/run/docker.sock:ro\r\n```\r\n\r\n### Step 3 — Start it\r\n\r\n```bash\r\ndocker compose up -d\r\n```\r\n\r\nOpen your browser at `http://localhost:8080`. You will see your services appear automatically within about 5 seconds.\r\n\r\n---\r\n\r\n## What Your Services Need to Expose\r\n\r\nYour services must expose a `/metrics` endpoint in Prometheus format. This is standard for any service built with:\r\n\r\n- Go (`prometheus/client_golang`)\r\n- Python (`prometheus_client`)\r\n- Node.js (`prom-client`)\r\n- Java (`micrometer`)\r\n- Any language with a Prometheus client library\r\n\r\nThe collector auto-detects the port. If your service exposes metrics on a non-standard port, add one more label:\r\n\r\n```yaml\r\nlabels:\r\n  le.enable: \"true\"\r\n  le.port: \"9100\"    # only needed for non-standard ports\r\n```\r\n\r\n---\r\n\r\n## What You See in the Dashboard\r\n\r\n**Monitor page** — read-only view of what is happening:\r\n\r\n- Live metrics per service: requests per second, queue depth, wait time, capacity used\r\n- Failure risk score — how likely each service is to stop responding in the next 60 seconds\r\n- Incident timeline — every problem detected, what the engine predicted, what action it took\r\n- Engine reasoning feed — plain-English explanation of what the autopilot is thinking right now\r\n- Live event stream — everything happening across all services in chronological order\r\n\r\n**Control page** — actions you can take:\r\n\r\n- Enable or freeze the autopilot (freeze = it keeps watching but stops issuing commands)\r\n- Switch operating policy: Safe Mode / Normal / Performance\r\n- Run a stress test on any service to see how the autopilot responds\r\n- Force the engine to step manually or retrain its model\r\n\r\n---\r\n\r\n## Architecture: One Image, One Port\r\n\r\nThe entire system runs in a single Docker container exposing port `8080`:\r\n\r\n```\r\nYour Services (with le.enable=true label)\r\n        │\r\n        │  Auto-discovered via Docker socket\r\n        │  Scraped every 2 seconds\r\n        ▼\r\n┌─────────────────────────────────────────┐\r\n│         LoadEquilibrium :8080           │\r\n│                                         │\r\n│  Collector ──► Telemetry Store          │\r\n│                      │                  │\r\n│               Tick Engine (2s)          │\r\n│                 │         │             │\r\n│          Autopilot    Reasoning         │\r\n│          (MPC+RL)     (Events)          │\r\n│                 │         │             │\r\n│          WebSocket Broadcast            │\r\n│                 │                       │\r\n│            UI (React)                  │\r\n└─────────────────────────────────────────┘\r\n        │\r\n        ▼\r\n  http://localhost:8080\r\n```\r\n\r\n- Port `8080` — dashboard UI, WebSocket live feed, REST API, Prometheus metrics\r\n- No separate collector container\r\n- No separate nginx\r\n- No Prometheus required (it is optional, for Grafana users)\r\n- No database required (optional, for persistent history)\r\n\r\n---\r\n\r\n## Optional: Grafana + Prometheus\r\n\r\nIf you already use Grafana, LoadEquilibrium exposes a `/metrics` endpoint that Prometheus can scrape. A pre-built dashboard is included.\r\n\r\nAdd to your compose file:\r\n\r\n```yaml\r\n  prometheus:\r\n    image: prom/prometheus:v2.52.0\r\n    volumes:\r\n      - ./monitoring/prometheus.yml:/etc/prometheus/prometheus.yml:ro\r\n    ports:\r\n      - \"9090:9090\"\r\n\r\n  grafana:\r\n    image: grafana/grafana:10.4.0\r\n    ports:\r\n      - \"3000:3000\"\r\n    volumes:\r\n      - ./monitoring/grafana/datasource.yml:/etc/grafana/provisioning/datasources/datasource.yml:ro\r\n      - ./monitoring/grafana/provider.yml:/etc/grafana/provisioning/dashboards/provider.yml:ro\r\n      - ./monitoring/grafana/loadequilibrium-dashboard.json:/var/lib/grafana/dashboards/loadequilibrium-dashboard.json:ro\r\n```\r\n\r\nGrafana dashboard shows: traffic, latency, queue depth, autopilot decisions, signal quality, and Go runtime metrics.\r\n\r\n---\r\n\r\n## Optional: Persistent History (PostgreSQL)\r\n\r\nWithout a database, the engine runs entirely in memory. Data is lost on restart but the system works perfectly for real-time monitoring.\r\n\r\nTo keep a history of engine snapshots:\r\n\r\n```yaml\r\n  loadequilibrium:\r\n    environment:\r\n      DATABASE_URL: \"postgres://le:yourpassword@postgres:5432/le?sslmode=disable\"\r\n\r\n  postgres:\r\n    image: postgres:16-alpine\r\n    environment:\r\n      POSTGRES_USER:     le\r\n      POSTGRES_PASSWORD: yourpassword\r\n      POSTGRES_DB:       le\r\n```\r\n\r\nThe schema is created automatically on first start. No migrations to run.\r\n\r\n---\r\n\r\n## Environment Variables\r\n\r\nAll settings have safe defaults. You only need to set `INGEST_TOKEN` in production.\r\n\r\n| Variable | Default | What it does |\r\n|---|---|---|\r\n| `INGEST_TOKEN` | *(empty)* | Auth token for the ingest API. Set this in production. |\r\n| `DATABASE_URL` | *(empty)* | Postgres DSN. Leave empty to run in-memory. |\r\n| `LISTEN_ADDR` | `:8080` | Port to listen on. |\r\n| `TICK_INTERVAL` | `2s` | How often the engine runs. Do not change unless you have a reason. |\r\n| `UTILISATION_SETPOINT` | `0.70` | Target capacity utilisation (70%). Leave 30% headroom. |\r\n| `MAX_SERVICES` | `200` | Maximum number of services to track. |\r\n| `LE_PORT` | `8080` | Host port (compose only). Change if 8080 is taken on your machine. |\r\n\r\nFull list of all variables is in the [Configuration Reference](#configuration-reference) section below.\r\n\r\n---\r\n\r\n## Kubernetes Deployment\r\n\r\nFor production on Kubernetes, use the manifests in `k8s/`. The image is the same — just one Deployment.\r\n\r\n```bash\r\n# 1. Create secrets (never put real values in the YAML files)\r\nkubectl create secret generic loadequilibrium-secrets \\\r\n  --from-literal=database-url='postgres://le:YOURPASS@postgres-svc:5432/le?sslmode=require' \\\r\n  --from-literal=ingest-token='your-real-token' \\\r\n  -n loadequilibrium\r\n\r\n# 2. Apply manifests\r\nkubectl apply -f k8s/00-namespace.yml\r\nkubectl apply -f k8s/01-secrets.yml\r\nkubectl apply -f k8s/02-configmap.yml\r\nkubectl apply -f k8s/03-postgres.yml\r\nkubectl apply -f k8s/04-deployment.yml\r\nkubectl apply -f k8s/05-prometheus.yml\r\nkubectl apply -f k8s/06-grafana.yml\r\nkubectl apply -f k8s/07-ingress.yml   # edit hostnames first\r\n\r\n# 3. Watch it start\r\nkubectl rollout status deployment/loadequilibrium -n loadequilibrium\r\n\r\n# 4. Access before ingress is ready\r\nkubectl port-forward svc/loadequilibrium-svc 8080:80 -n loadequilibrium\r\n```\r\n\r\n**Important**: Run exactly 1 replica. The engine keeps in-memory state that is not distributed across pods. If you need high availability, run active-passive with a shared PostgreSQL backend (bot[...]\r\n\r\n---\r\n\r\n## How to Send Metrics Manually (Without Docker)\r\n\r\nIf you are not using Docker (e.g. running services as systemd units, on bare metal, or in a different container runtime), use `collector.py` to push metrics from an existing Prometheus server:\r\n\r\n```bash\r\npip install requests\r\nPROMETHEUS_URL=http://your-prometheus:9090 \\\r\nINGEST_URL=http://loadequilibrium:8080/api/v1/ingest \\\r\npython3 collector.py\r\n```\r\n\r\nOr push directly to the ingest API from your application code:\r\n\r\n```bash\r\ncurl -X POST http://localhost:8080/api/v1/ingest \\\r\n  -H \"Content-Type: application/json\" \\\r\n  -d '[{\r\n    \"service_id\":    \"my-api\",\r\n    \"request_rate\":  142.5,\r\n    \"error_rate\":    0.002,\r\n    \"latency\": { \"p50\": 12.1, \"p95\": 48.3, \"p99\": 91.2, \"mean\": 18.4 },\r\n    \"queue_depth\":   23,\r\n    \"active_conns\":  87\r\n  }]'\r\n```\r\n\r\n---\r\n\r\n## API Reference\r\n\r\n| Method | Path | What it does |\r\n|---|---|---|\r\n| `GET` | `/` | Dashboard UI |\r\n| `GET` | `/ws` | WebSocket — live tick stream |\r\n| `GET` | `/api/v1/snapshot` | Last tick state as JSON (no WebSocket needed) |\r\n| `GET` | `/health` | Liveness check — returns `{\"status\":\"ok\"}` |\r\n| `GET` | `/metrics` | Prometheus metrics |\r\n| `POST` | `/api/v1/ingest` | Push telemetry points |\r\n| `POST` | `/api/v1/control/toggle` | Enable / freeze autopilot |\r\n| `POST` | `/api/v1/policy/update` | Change policy preset |\r\n| `POST` | `/api/v1/control/chaos-run` | Inject a load spike for testing |\r\n| `POST` | `/api/v1/control/replay-burst` | Replay a traffic burst |\r\n| `POST` | `/api/v1/runtime/step` | Force one engine tick manually |\r\n| `POST` | `/api/v1/alerts/ack` | Acknowledge a reasoning event |\r\n\r\n---\r\n\r\n## How the Engine Works (For Those Who Want to Know)\r\n\r\nEvery 2 seconds, this sequence runs:\r\n\r\n1. **Collect** — scrape `/metrics` from all labelled services\r\n2. **Window** — compute EWMA fast/slow, variance, confidence score, signal quality per service\r\n3. **Topology** — build a dependency graph from upstream call data (if services report it)\r\n4. **Model** — queue physics per service: utilisation ρ, mean wait time, queue depth, burst amplification\r\n5. **Reason** — rule engine fires events: `collapse_risk`, `cascade_risk`, `saturation_predicted`, `keystone_degraded`\r\n6. **Simulate** — Monte Carlo forward projection: what happens in the next 60 seconds under current trend?\r\n7. **Autopilot** — MPC (Model Predictive Control) + RL (policy gradient) computes target capacity\r\n8. **Decide** — Control authority converts float target to integer replica count, enforces cooldowns\r\n9. **Actuate** — Send scaling directive to your orchestrator (Kubernetes, Nomad, etc.)\r\n10. **Broadcast** — Push full tick state to all WebSocket clients (your dashboard)\r\n\r\nThe sandbox runs every 10 ticks: it takes the current service state, generates a synthetic load spike from real statistics, runs two competing control strategies against it, and uses the result t[...]\r\n\r\n---\r\n\r\n## Verifying Everything Works\r\n\r\nRun these three commands. All three must pass before deploying to production.\r\n\r\n```bash\r\n# Compile check — catches any broken wiring\r\ngo build ./...\r\n\r\n# Unit + integration tests with race detector\r\ngo test -race -count=1 -timeout=300s ./internal/...\r\n\r\n# Full autopilot system test — 10 scenarios, must all pass\r\ngo build -o system_test_runner ./cmd/system_test_runner/\r\n./system_test_runner 2\u003e/dev/null\r\n# Expected output line: \"STABLE_PRODUCTION_GRADE — 10/10 — 0 SLA breaches\"\r\n```\r\n\r\n---\r\n\r\n## Files You Can Delete\r\n\r\nThese files are safe to remove. They are developer tools that are never included in the Docker image:\r\n\r\n| File | What it is | Safe to delete? |\r\n|---|---|---|\r\n| `Dockerfile.collector` | Old sidecar build — replaced by embedded goroutine | ✅ Yes |\r\n| `ui/Dockerfile` | Old standalone nginx UI build — replaced by Go binary | ✅ Yes |\r\n| `run_physics_validation.sh` | Developer script to validate physics model output | ✅ Yes (keep if you develop the engine) |\r\n| `collector.py` | Alternative Python collector for non-Docker environments | ❌ Keep — useful for Prometheus users |\r\n| `Makefile` | Build convenience commands | ❌ Keep — useful for developers |\r\n| `cmd/system_test_runner/` | Autopilot test suite | ❌ Keep — CI depends on it |\r\n\r\n---\r\n\r\n## Configuration Reference\r\n\r\n### Core Engine\r\n\r\n| Variable | Default | Description |\r\n|---|---|---|\r\n| `LISTEN_ADDR` | `:8080` | HTTP server bind address |\r\n| `TICK_INTERVAL` | `2s` | Control tick frequency |\r\n| `TICK_DEADLINE` | `1800ms` | Max time per tick before adaptive stretch |\r\n| `MIN_TICK_INTERVAL` | `1s` | Minimum tick interval under stretch |\r\n| `MAX_TICK_INTERVAL` | `10s` | Maximum tick interval under stretch |\r\n| `WORKER_POOL_SIZE` | `8` | Parallel workers for window computation |\r\n\r\n### Telemetry\r\n\r\n| Variable | Default | Description |\r\n|---|---|---|\r\n| `RING_BUFFER_DEPTH` | `300` | Samples retained per service |\r\n| `MAX_SERVICES` | `200` | Maximum tracked services |\r\n| `STALE_SERVICE_AGE` | `5m` | Prune threshold for inactive services |\r\n| `INGEST_TOKEN` | `` | Auth token — set this in production |\r\n\r\n### Control Policy\r\n\r\n| Variable | Default | Description |\r\n|---|---|---|\r\n| `UTILISATION_SETPOINT` | `0.70` | Target utilisation — 70% leaves 30% headroom |\r\n| `COLLAPSE_THRESHOLD` | `0.90` | Utilisation above which collapse risk is flagged |\r\n| `EWMA_FAST_ALPHA` | `0.30` | Fast EWMA — responds in ~3 ticks |\r\n| `EWMA_SLOW_ALPHA` | `0.10` | Slow EWMA — trend signal, responds in ~10 ticks |\r\n| `PID_KP` | `-1.5` | PID proportional gain |\r\n| `PID_KI` | `-0.3` | PID integral gain |\r\n| `PID_KD` | `-0.1` | PID derivative gain |\r\n\r\n### Simulation\r\n\r\n| Variable | Default | Description |\r\n|---|---|---|\r\n| `SIM_BUDGET` | `45ms` | Wall-clock budget per tick for Monte Carlo sim |\r\n| `SIM_HORIZON_MS` | `60000` | Simulation lookahead — 60 seconds |\r\n| `SIM_SHOCK_FACTOR` | `2.0` | Worst-case load multiplier in simulation |\r\n\r\n### Persistence\r\n\r\n| Variable | Default | Description |\r\n|---|---|---|\r\n| `DATABASE_URL` | `` | Postgres DSN — if empty, runs in-memory |\r\n| `PERSIST_INTERVAL` | `30s` | How often snapshots flush to DB |\r\n\r\n---\r\n\r\n## License\r\n\r\nSee `LICENSE` for terms.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkuldeep-poonia%2Floadequilibrium","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkuldeep-poonia%2Floadequilibrium","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkuldeep-poonia%2Floadequilibrium/lists"}