{"id":15776109,"url":"https://github.com/mahendrapaipuri/ceems","last_synced_at":"2026-01-12T02:53:35.966Z","repository":{"id":207397681,"uuid":"719047710","full_name":"mahendrapaipuri/ceems","owner":"mahendrapaipuri","description":"A Prometheus exporter and a REST API server to export metrics of compute units of resource managers like SLURM, Openstack, k8s, _etc_","archived":false,"fork":false,"pushed_at":"2024-10-29T13:32:58.000Z","size":8414,"stargazers_count":13,"open_issues_count":8,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-10-29T16:07:21.149Z","etag":null,"topics":["cloud","containers","dashboards","ebpf","emissions","energy-monitor","grafana","green-computing","hpc","json-api","kubernetes","metrics-server","metrics-visualization","monitoring","observability","openstack","performance-monitoring","prometheus","prometheus-exporter","slurm"],"latest_commit_sha":null,"homepage":"https://mahendrapaipuri.github.io/ceems/","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mahendrapaipuri.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-11-15T10:40:30.000Z","updated_at":"2024-10-29T13:32:56.000Z","dependencies_parsed_at":"2024-02-14T15:25:53.535Z","dependency_job_id":"bad0ca74-47b7-490b-8e1f-23863f360e33","html_url":"https://github.com/mahendrapaipuri/ceems","commit_stats":{"total_commits":533,"total_committers":4,"mean_commits":133.25,"dds":"0.21575984990619135","last_synced_commit":"3c3ce6dd00a6193fe8d256d6230cd6534d6755bb"},"previous_names":["mahendrapaipuri/batchjob-exporter","mahendrapaipuri/batchjob_monitoring","mahendrapaipuri/batchjob_metrics_monitor","mahendrapaipuri/batchjob_monitor","mahendrapaipuri/ceems"],"tags_count":16,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mahendrapaipuri%2Fceems","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mahendrapaipuri%2Fceems/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mahendrapaipuri%2Fceems/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mahendrapaipuri%2Fceems/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mahendrapaipuri","download_url":"https://codeload.github.com/mahendrapaipuri/ceems/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246390896,"owners_count":20769475,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cloud","containers","dashboards","ebpf","emissions","energy-monitor","grafana","green-computing","hpc","json-api","kubernetes","metrics-server","metrics-visualization","monitoring","observability","openstack","performance-monitoring","prometheus","prometheus-exporter","slurm"],"created_at":"2024-10-04T17:04:53.217Z","updated_at":"2025-03-30T22:31:37.505Z","avatar_url":"https://github.com/mahendrapaipuri.png","language":"C","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Compute Energy \u0026 Emissions Monitoring Stack (CEEMS)\n\u003c!-- markdown-link-check-disable --\u003e\n\n|         |                                                                                                                                                                                                                                                                                                                                                                                                                 |\n| ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| CI/CD   | [![ci](https://github.com/mahendrapaipuri/ceems/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/mahendrapaipuri/ceems/actions/workflows/ci.yml?query=branch%3Amain) [![CircleCI](https://dl.circleci.com/status-badge/img/circleci/8jSYT1wyKY8mKQRTqNLThX/TzM1Mr3AEAqmehnoCde19R/tree/main.svg?style=svg\u0026circle-token=28db7268f3492790127da28e62e76b0991d59c8b)](https://dl.circleci.com/status-badge/redirect/circleci/8jSYT1wyKY8mKQRTqNLThX/TzM1Mr3AEAqmehnoCde19R/tree/main)  [![Coverage](https://img.shields.io/badge/Coverage-73.7%25-brightgreen)](https://github.com/mahendrapaipuri/ceems/actions/workflows/ci.yml?query=branch%3Amain)                                                                                          |\n| Docs    | [![docs](https://img.shields.io/badge/docs-passing-green?style=flat\u0026link=https://mahendrapaipuri.github.io/ceems/docs/)](https://mahendrapaipuri.github.io/ceems/)                                                                                                                                                                                                                               |\n| Package | [![Release](https://img.shields.io/github/v/release/mahendrapaipuri/ceems.svg?include_prereleases)](https://github.com/mahendrapaipuri/ceems/releases/latest)                                                                                                                                                                     |\n| Meta    | [![GitHub License](https://img.shields.io/github/license/mahendrapaipuri/ceems)](https://github.com/mahendrapaipuri/ceems) [![Go Report Card](https://goreportcard.com/badge/github.com/mahendrapaipuri/ceems)](https://goreportcard.com/report/github.com/mahendrapaipuri/ceems) [![code style](https://img.shields.io/badge/code%20style-gofmt-blue.svg)](https://pkg.go.dev/cmd/gofmt) |\n\n\u003c!-- markdown-link-check-enable --\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://raw.githubusercontent.com/mahendrapaipuri/ceems/main/website/static/img/logo.png\" width=\"200\"\u003e\n\u003c/p\u003e\n\nCompute Energy \u0026 Emissions Monitoring Stack (CEEMS) (pronounced as *kiːms*) contains\na Prometheus exporter to export metrics of compute instance units and a REST API\nserver that serves the metadata and aggregated metrics of each\ncompute unit. Optionally, it includes a TSDB load balancer that supports basic access\ncontrol on TSDB so that one user cannot access metrics of another user.\n\n\"Compute Unit\" in the current context has a wider scope. It can be a batch job in HPC,\na VM in cloud, a pod in k8s, *etc*. The main objective of the repository is to quantify\nthe energy consumed and estimate emissions by each \"compute unit\". The repository itself\ndoes not provide any frontend apps to show dashboards and it is meant to use along\nwith Grafana and Prometheus to show statistics to users.\n\nAlthough CEEMS was born out of a need to monitor energy and carbon footprint of compute\nworkloads, it supports monitoring performance metrics as well. In addition, it leverages\n[eBPF](https://ebpf.io/what-is-ebpf/) framework to monitor IO and network metrics\nin a resource manager agnostic way.\n\n## Features\n\n- Monitor energy, performance, IO and network metrics for different types of resource\nmanagers (SLURM, Openstack, k8s)\n- Support NVIDIA (MIG and vGPU) and AMD GPUs\n- Provides targets using [HTTP Discovery Component](https://grafana.com/docs/alloy/latest/reference/components/discovery/discovery.http/)\nto [Grafana Alloy](https://grafana.com/docs/alloy/latest) to continuously profile compute units\n- Realtime access to metrics *via* Grafana dashboards\n- Access control to Prometheus datasource in Grafana\n- Stores aggregated metrics in a separate DB that can be retained for long time\n- CEEMS apps are [capability aware](https://tbhaxor.com/understanding-linux-capabilities/)\n\n## Install CEEMS\n\n\u003e [!WARNING]\n\u003e DO NOT USE pre-release versions as the API has changed quite a lot between the\npre-release and stable versions.\n\nInstallation instructions of CEEMS components can be found in\n[docs](https://mahendrapaipuri.github.io/ceems/docs/category/installation).\n\n## Visualizing metrics with Grafana\n\nCEEMS is meant to be used with Grafana for visualization and below are some of the\nscreenshots of dashboards.\n\n### Time series compute unit CPU metrics\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://raw.githubusercontent.com/mahendrapaipuri/ceems/main/website/static/img/dashboards/cpu_ts_stats.png\" width=\"1200\"\u003e\n\u003c/p\u003e\n\n### Time series compute unit GPU metrics\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://raw.githubusercontent.com/mahendrapaipuri/ceems/main/website/static/img/dashboards/gpu_ts_stats.png\" width=\"1200\"\u003e\n\u003c/p\u003e\n\n### List of compute units of user with aggregate metrics\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://raw.githubusercontent.com/mahendrapaipuri/ceems/main/website/static/img/dashboards/job_list_user.png\" width=\"1200\"\u003e\n\u003c/p\u003e\n\n### Aggregate usage metrics of a user\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://raw.githubusercontent.com/mahendrapaipuri/ceems/main/website/static/img/dashboards/agg.png\" width=\"1200\"\u003e\n\u003c/p\u003e\n\n## Talks and Demos\n\n- [An Introduction to CEEMS at ISC 2024](https://drive.google.com/file/d/1kUbD3GgDKwzgIuxjrTY95YJN5aSuIejQ/view?usp=drive_link)\n- [CEEMS Architecture and Usage](https://docs.google.com/presentation/d/1xNQTCsmPUz37KDb2BLrpWExuQWxk49NpVN9VDbxSe6Y/edit#slide=id.p)\n\n## Contributing\n\nWe welcome contributions to this project, we hope to see this project grow and become\na useful tool for people who are interested in the energy and carbon footprint of their\nworkloads.\n\nPlease feel free to open issues and/or discussions for any potential ideas of\nimprovement.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmahendrapaipuri%2Fceems","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmahendrapaipuri%2Fceems","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmahendrapaipuri%2Fceems/lists"}