{"id":50748748,"url":"https://github.com/vimalk78/decode-rapl","last_synced_at":"2026-06-10T23:04:00.226Z","repository":{"id":354142950,"uuid":"1222274761","full_name":"vimalk78/decode-rapl","owner":"vimalk78","description":"Predicting Intel RAPL package power from OS-visible signals — a research log of what worked, what didn't, and why","archived":false,"fork":false,"pushed_at":"2026-04-27T09:26:52.000Z","size":27390,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-04-27T11:18:54.244Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vimalk78.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-04-27T07:52:44.000Z","updated_at":"2026-04-27T09:26:57.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/vimalk78/decode-rapl","commit_stats":null,"previous_names":["vimalk78/decode-rapl"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/vimalk78/decode-rapl","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vimalk78%2Fdecode-rapl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vimalk78%2Fdecode-rapl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vimalk78%2Fdecode-rapl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vimalk78%2Fdecode-rapl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vimalk78","download_url":"https://codeload.github.com/vimalk78/decode-rapl/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vimalk78%2Fdecode-rapl/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34174148,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-10T02:00:07.152Z","response_time":89,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-06-10T23:03:59.361Z","updated_at":"2026-06-10T23:04:00.216Z","avatar_url":"https://github.com/vimalk78.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# DECODE-RAPL\n\n**Predicting Intel RAPL package power from OS-visible signals — a record of what worked, what didn't, and why.**\n\nThis repository is a research log of an attempt to build a deep-learning model that emulates the Intel RAPL (Running Average Power Limit) energy counter using only signals that are visible to a VM guest (`/proc/stat` user/system/iowait %, context switches, etc.). RAPL is implemented in silicon and is exposed only to bare-metal hosts via MSRs or `/sys/class/powercap/intel_rapl`; if a model can be trained on the bare-metal RAPL ground truth using *only* features that a VM guest would also be able to observe, that model could later run inside guest VMs that have no direct power telemetry — a direction relevant to projects like [Kepler](https://github.com/sustainable-computing-io/kepler).\n\n\u003e **Headline finding.** Across four model families and many hyperparameter sweeps, every variant achieves **R² ≈ 0.95–0.96, MAE ≈ 2 W on a shuffled stress-ng test set**, but degrades to **R² ≈ 0.30** with a **+3 to +15 W idle-baseline bias** on live, non-stress-ng workloads. The persistent gap suggests a *feature ceiling*: VM-visible OS metrics are blind to the microarchitectural state (instruction mix, cache pressure, functional-unit activation) that ultimately determines power. This is the canonical interim conclusion in [`DECORE-RAPL-REPORT.md`](DECORE-RAPL-REPORT.md) — that document is the most important thing to read.\n\n---\n\n## Where to look first\n\n| If you want… | Read |\n|---|---|\n| The full story and final analysis | [`DECORE-RAPL-REPORT.md`](DECORE-RAPL-REPORT.md) |\n| The 12-slide visual walkthrough (reveal.js) | [`presentation.html`](presentation.html) — open in a browser |\n| A developer-level quickstart for the v2/v3/v4 model code | [`QUICKSTART.md`](QUICKSTART.md) |\n| The current model architecture (1D-CNN encoder + power head) | [`decode_rapl_v4.md`](decode_rapl_v4.md) |\n| Per-version training reports | [`V2_TRAINING_REPORT.md`](V2_TRAINING_REPORT.md), [`results/`](results) |\n| Future direction notes (multi-machine generalization) | [`decode_rapl_v-later.md`](decode_rapl_v-later.md) |\n| The MS-TCN attempt (separate architecture, also bare-metal) | [`archive/ms-tcn/docs/MS_TCN_APPROACH_REPORT.md`](archive/ms-tcn/docs/MS_TCN_APPROACH_REPORT.md), [`archive/ms-tcn/results/RESULTS.md`](archive/ms-tcn/results/RESULTS.md) |\n| The very first DECODE-RAPL attempt (kept for context) | [`archive/decode-rapl-v1/README.md`](archive/decode-rapl-v1/README.md), [`archive/decode-rapl-v1/LSTM_ANALYSIS_SUMMARY.md`](archive/decode-rapl-v1/LSTM_ANALYSIS_SUMMARY.md) |\n\n## Repository layout\n\nThe current research lives at the repo root (the v2/v3/v4 model code, configs, reports, scripts, and the trained checkpoint). Three earlier attempts that were superseded are kept under [`archive/`](archive) so the progression of ideas remains visible.\n\n```\n.\n├── README.md, LICENSE\n├── DECORE-RAPL-REPORT.md      ← canonical interim report — read this first\n├── QUICKSTART.md               ← developer quickstart for the model code\n├── decode_rapl_v3.md, decode_rapl_v4.md, decode_rapl_v-later.md\n├── V2_TRAINING_REPORT.md, TRAINING.md, report-improvement.md\n├── src/         (model.py, train.py, inference.py, power_predictor.py, utils.py)\n├── scripts/     (run_workloads.sh, prepare_training_data.py, start_training_v4.sh, …)\n├── collector/   (my_data_collector.go — Go-based 16 ms RAPL+OS-metrics collector)\n├── config/      (v2/v3/v4 YAML configs for τ ∈ {1, 4, 8})\n├── docs/        (architecture.md, data_collection.md, training.md, inference.md, …)\n├── results/     (per-run training logs, summaries, plots)\n├── checkpoints/v4_tau1/best_model.pt  (1.6 MB — shipped for inference)\n└── archive/\n    ├── lstm/             ← earliest baseline (per-VM CPU% → power via small LSTM)\n    ├── ms-tcn/           ← MS-TCN with attention pooling; hit a data-distribution wall\n    └── decode-rapl-v1/   ← first DECODE-RAPL attempt (AE+LSTM); latent bottleneck identified\n```\n\n## Test environment\n\nAll data collection and live tests were run on a single bare-metal host:\n\n| Field | Value |\n|---|---|\n| Hardware | Dell PowerEdge R440 |\n| CPU | Intel Xeon Silver 4210R @ 2.40 GHz (Cascade Lake R) |\n| Sockets / cores / threads | 1 / 10 / 20 (HT on) |\n| NUMA nodes | 1 |\n| RAM | 96 GB DDR4 |\n| RAPL domains | package (`/sys/class/powercap/intel-rapl:0`) |\n| OS | Fedora 39 |\n\n## Data availability\n\nThe core training dataset is **5.8 M samples** collected at 16 ms intervals across 2,025+ stress-ng workload combinations on the bare-metal data-collection rig above. **The dataset itself was collected at the author's employer and its public release is currently under internal review** — it is not yet downloadable from this repo.\n\nUntil then, the pipeline is fully reproducible from scratch on your own bare-metal hardware:\n\n* Workload generator: [`scripts/run_workloads.sh`](scripts/run_workloads.sh)\n* Low-overhead Go collector: [`collector/my_data_collector.go`](collector/my_data_collector.go)\n* Preprocessing: [`scripts/prepare_training_data.py`](scripts/prepare_training_data.py)\n* Training (v4): [`scripts/start_training_v4.sh`](scripts/start_training_v4.sh) + [`config/v4_tau1.yaml`](config/v4_tau1.yaml)\n\n## Quick inference (no retraining needed)\n\nA trained v4_tau1 checkpoint is shipped at [`checkpoints/v4_tau1/best_model.pt`](checkpoints/v4_tau1/best_model.pt). Note that **inference uses RAPL only as a comparison reference** — for prediction itself the model only needs the four VM-visible features, but the live-prediction script reads RAPL to display the error. If you run it inside a VM, drop the `--compare-rapl` flag.\n\n```bash\nsudo python src/power_predictor.py \\\n    --model checkpoints/v4_tau1/best_model.pt \\\n    --config config/v4_tau1.yaml \\\n    --live\n```\n\nSee [`docs/inference.md`](docs/inference.md) for full inference options.\n\n## Status\n\nResearch-quality. There is no claim of production readiness, no stable API, and the open-source release is intended to share the *thinking and the negative result*, not to ship a product. Issues, PRs, and especially counter-examples (workloads on which the v4_tau1 checkpoint behaves differently than reported) are very welcome.\n\n## License\n\nApache License 2.0 — see [`LICENSE`](LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvimalk78%2Fdecode-rapl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvimalk78%2Fdecode-rapl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvimalk78%2Fdecode-rapl/lists"}