{"id":19312138,"url":"https://github.com/bigcommerce/nomad-workload-cpu-actuals-report-generator","last_synced_at":"2025-04-22T15:31:59.023Z","repository":{"id":141666380,"uuid":"260006224","full_name":"bigcommerce/nomad-workload-cpu-actuals-report-generator","owner":"bigcommerce","description":null,"archived":false,"fork":false,"pushed_at":"2020-10-23T17:48:18.000Z","size":8,"stargazers_count":2,"open_issues_count":0,"forks_count":4,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-04-02T00:51:18.371Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Groovy","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bigcommerce.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-04-29T18:15:34.000Z","updated_at":"2024-02-25T11:07:48.000Z","dependencies_parsed_at":null,"dependency_job_id":"3a0b8d1d-407d-4668-aa0e-93cd642b2956","html_url":"https://github.com/bigcommerce/nomad-workload-cpu-actuals-report-generator","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bigcommerce%2Fnomad-workload-cpu-actuals-report-generator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bigcommerce%2Fnomad-workload-cpu-actuals-report-generator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bigcommerce%2Fnomad-workload-cpu-actuals-report-generator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bigcommerce%2Fnomad-workload-cpu-actuals-report-generator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bigcommerce","download_url":"https://codeload.github.com/bigcommerce/nomad-workload-cpu-actuals-report-generator/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250267446,"owners_count":21402482,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-10T00:33:05.214Z","updated_at":"2025-04-22T15:31:59.009Z","avatar_url":"https://github.com/bigcommerce.png","language":"Groovy","funding_links":[],"categories":[],"sub_categories":[],"readme":"# nomad-workload-cpu-actuals-report\n\n## Summary\n\nA tool used for balacning workloads -- finding over and under allocated jobs/tasks particularly when the Nomad Client cluster has non-uniform CPU classes / clockspeeds.\n\n## Usage\n\n```\n./nomadWorkloadCPUActualsReport --help\nerror: Missing required option: e\nusage: ./nomadWorkloadCPUActualsReport -e \u003ccomma delimited environments\u003e \u003cother args\u003e\nNomad Workload CPU Actuals Report\n -adstats,--advancedStats              Writes advanced stats including Std Dev, Kurtosis, and Skewness\n -avg,--avgKnownNomadClients           Use instead of --nomadFallbackClock to average the known Nomad clients for historical workloads\n -d1d,--disableOneDayQuery             Disable 1 day query\n -d1h,--disableOneHourQuery            Disable 1 hour query\n -d30d,--disableThirtyDayQuery         Disable 30 day query\n -d7d,--disableSevenDayQuery           Disable 7 day query\n -e,--environments \u003carg\u003e               Environments (comma delimited)\n -e60d,--enableSixytDayQuery           Enable 60 day query\n -e90d,--enableNinetyDayQuery          Enable 90 day query\n -fbc,--nomadFallbackClock \u003carg\u003e       The approximate clockspeed of out-of-service Nomad nodes that were used to run historical workloads tracked in Prometheus [defaults to 2300]\n -h,--help                             Usage Information\n -jobs,--targetJobs \u003carg\u003e              Target specific jobs (comma delimited)\n -jst,--taskSleepTime \u003carg\u003e            Task sleep time in seconds [defaults to 5]\n -miss,--includeMissRate               Write the miss rate\n -nc,--nomadTLSCertFilename \u003carg\u003e      Nomad TLS Key Filename [defaults to %env%.nomad.key]\n -nca,--nomadTLSCACertFilename \u003carg\u003e   Nomad TLS CA Certificate Filename [defaults to nomadca.crt]\n -nh,--nomadHost \u003carg\u003e                 Nomad host [defaults to https://nomad.service.%env%-dc1.consul:4646]\n -nk,--nomadTLSKeyFilename \u003carg\u003e       Nomad TLS Certificate Filename [defaults to %env%.nomad.crt]\n -ph,--prometheusHost \u003carg\u003e            Prometheus host [defaults to http://prometheus-read.service.%env%-dc1.consul:9090]\n -qst,--querySleepTime \u003carg\u003e           Query sleep time in seconds [defaults to 5]\nEnvironment queries are run in parallel to reduce report generation time. Use %env% to inject environment into --nomadTLSKeyFilename, --nomadTLSCertFilename, --nomadHost, --prometheusHost\n```\n\n## Prometheus Queries\n\nThere are two metrics pulled out of prometheus:\n\n1. `nomad_client_allocs_cpu_total_percent` - A guage of the percentage of the total CPU resources consumed by the task across all cores.\n2. `nomad_client_allocs_memory_usage` - A guage of the total amount of memory bytes used by the task. \n\nFor more information see [the Nomad metrics](https://www.nomadproject.io/docs/telemetry/metrics) page.\n\n## Queries\n\nDepending on what intervals are enabled per the aforementioned list of arguments, a number of ranges are used. By default the following are enabled:\n\n- 1 Hour\n- 1 Day\n- 7 Days\n- 30 Days\n\nThese flags can be disabled. Also available are 60 day and 90 day windows, which require explicitly enabling (are disabled by default). The queries all use `avg_over_time` over the two metrics for various range vector selectors (for time duration of the input metrics) and windows/range selectors for the `avg_over_time` function:\n\n| Period     | Range Selectors  |\n| ---------- | ---------------- |\n| 1 Hour     | `[5s])[1h:5s]`   |\n| 1 Day      | `[1m])[1d:1m]`   |\n| 7 Days     | `[1m])[7d:1m]`   |\n| 30 Days    | `[1h])[30d:6h]`  |\n| 60 Days    | `[1h])[60d:12h]` |\n| 90 Days    | `[1h])[90d:24h]` |\n\nQuery examples as issued to `/api/v1/query?query=`:\n\n```\navg_over_time(nomad_client_allocs_cpu_total_percent{task=\"container-job-x\"}[5s])[1h:5s]\navg_over_time(nomad_client_allocs_memory_usage{task=\"container-job-x\"}[5s])[1h:5s]\navg_over_time(nomad_client_allocs_cpu_total_percent{task=\"container-job-x\"}[1m])[1d:1m]\navg_over_time(nomad_client_allocs_memory_usage{task=\"container-job-x\"}[1m])[1d:1m]\navg_over_time(nomad_client_allocs_cpu_total_percent{task=\"container-job-x\"}[1m])[7d:1m]\navg_over_time(nomad_client_allocs_memory_usage{task=\"container-job-x\"}[1m])[7d:1m]\navg_over_time(nomad_client_allocs_cpu_total_percent{task=\"container-job-x\"}[1h])[30d:6h]\navg_over_time(nomad_client_allocs_memory_usage{task=\"container-job-x\"}[1h])[30d:6h]\navg_over_time(nomad_client_allocs_cpu_total_percent{task=\"container-job-x\"}[1h])[60d:12h]\navg_over_time(nomad_client_allocs_memory_usage{task=\"container-job-x\"}[1h])[60d:12h]\navg_over_time(nomad_client_allocs_cpu_total_percent{task=\"container-job-x\"}[1h])[90d:24h]\navg_over_time(nomad_client_allocs_memory_usage{task=\"container-job-x\"}[1h])[90d:24h]\n```\n\n## Understand the report\n\nReports are output with the timestamp from which the report generation process was started. Reports have a tab for each environment passed to the command line application. There are shaded or colored sections of the workbook that compare the following values to the `Nomad Job -\u003e Task Group -\u003e Tasks's` Allocated MHz:\n\n1. Mean\n2. 50th Percentile\n3. 95th Percentile\n4. Max\n\n... with the colors:\n\n- green: meaning a value is somewhere bewteen 0 and the Allocated MHz, richer meaning further (0 is darkest or richest green while closest to Allocated MHz is more green/white or white)\n- yellow: meaning a value is somewhere between the Allocated MHz and 2x the Allocated MHz, richer meaning further (2x Allocated MHz is darkest or richest yellow, while closest to Allocated MHz is more yellow/white or white)\n- red: meaning a value is or is greather than 2x the Allocated MHz\n\n### See Also\n\n- [Using Prometheus to Monitor Nomad Metrics](https://www.nomadproject.io/guides/operations/monitoring-and-alerting/prometheus-metrics.html)\n- [Shape of data](https://brownmath.com/stat/shape.htm)\n- [Understanding Descriptive Statistics](https://towardsdatascience.com/understanding-descriptive-statistics-c9c2b0641291)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbigcommerce%2Fnomad-workload-cpu-actuals-report-generator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbigcommerce%2Fnomad-workload-cpu-actuals-report-generator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbigcommerce%2Fnomad-workload-cpu-actuals-report-generator/lists"}