{"id":47669870,"url":"https://github.com/chrisbraddock/gpu-tune","last_synced_at":"2026-04-02T12:29:38.284Z","repository":{"id":249289040,"uuid":"831104756","full_name":"chrisbraddock/gpu-tune","owner":"chrisbraddock","description":"This project analyzes GPU performance metrics from AI training and inference tasks, highlighting the optimal power settings for efficient operation. It generates visualizations to help identify the best max power settings for training and inference based on energy consumption.","archived":false,"fork":false,"pushed_at":"2025-05-19T03:52:42.000Z","size":290,"stargazers_count":1,"open_issues_count":5,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-05-19T04:30:36.835Z","etag":null,"topics":["ai","artificial-intelligence","gpu-monitoring","nvidia","nvidia-cuda","nvidia-gpu"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/chrisbraddock.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-07-19T17:07:22.000Z","updated_at":"2025-05-19T03:51:07.000Z","dependencies_parsed_at":"2025-05-19T04:37:01.064Z","dependency_job_id":null,"html_url":"https://github.com/chrisbraddock/gpu-tune","commit_stats":null,"previous_names":["chrisbraddock/gpu-tune"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/chrisbraddock/gpu-tune","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chrisbraddock%2Fgpu-tune","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chrisbraddock%2Fgpu-tune/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chrisbraddock%2Fgpu-tune/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chrisbraddock%2Fgpu-tune/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/chrisbraddock","download_url":"https://codeload.github.com/chrisbraddock/gpu-tune/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chrisbraddock%2Fgpu-tune/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31306038,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-02T09:48:21.550Z","status":"ssl_error","status_checked_at":"2026-04-02T09:48:19.196Z","response_time":89,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","artificial-intelligence","gpu-monitoring","nvidia","nvidia-cuda","nvidia-gpu"],"created_at":"2026-04-02T12:29:37.703Z","updated_at":"2026-04-02T12:29:38.264Z","avatar_url":"https://github.com/chrisbraddock.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# GPU Performance Metrics and Recommendations\n\n\u003e [!CAUTION]\n\u003e **This was written by GPT-4o** with guidance. It's incomplete and needs human review. Use appropriately and PRs are welcome.\n\n\u003e This project analyzes GPU performance metrics from AI training and inference tasks, highlighting the optimal power settings for efficient operation. It generates visualizations to help identify the best max power settings for training and inference based on energy consumption.\n\nExample report:\n![Example report](./report.png)\n\n## Dependencies\n\n### Install System Dependencies\n\nEnsure the necessary NVIDIA libraries and tools are installed:\n\n- NVIDIA CUDA Toolkit\n- NVIDIA driver\n- NVIDIA System Management Interface (nvidia-smi)\n\nTo install the NVIDIA driver and tools, follow the instructions on the official NVIDIA documentation:\n\n- [NVIDIA CUDA Installation Guide for Linux](https://docs.nvidia.com/cuda/cuda-installation-guide-linux/)\n- `nvidia-smi` is typically included with the NVIDIA driver installation.\n\n### Install Python Dependencies\n\nInstall the dependencies using pip:\n\n```bash\npip install -r requirements.txt\n```\n\n## Usage\n\n### 1. Run Experiments\n\nThe `run_experiments.py` script orchestrates the running of experiments and data collection. This script will use `llm_inference.py` and `llm_training.py` to collect raw data.\n\n```bash\npython run_experiments.py\n```\n\n### 2. Process Experiment Data\n\nThe `process_experiment_data.py` script processes the raw experiment data into a more usable format. It will generate `training_stats.csv` and `inference_stats.csv`.\n\n```bash\npython process_experiment_data.py\n```\n\n### 3. Generate Plots and Recommendations\n\nThe `generate_report.py` script generates visualizations and highlights the recommended max power settings based on the provided data.\n\n```bash\npython generate_report.py\n```\n\n### 4. Viewing the Output\n\nThe output image `report.png` contains the following:\n\n- **Header**: Current date and time of generation.\n- **Charts**: Visualizations for both inference and training metrics side by side.\n- **Summary**: Recommended settings based on energy consumption.\n\n## Explanation of Recommendations\n\nThe recommended settings are based on the lowest energy consumption (Watt-min) for each scenario. Energy consumption is calculated as the product of power draw and total time taken.\n\n## File Descriptions\n\n- `training_stats.csv`: Contains columns such as `max_watt`, `tokens_per_sec`, `temperature`, `gpu_utilization`, `memory_utilization`, `loss`, and `timestamp`.\n- `inference_stats.csv`: Contains columns such as `max_watt`, `tokens_per_sec`, `temperature`, `gpu_utilization`, `memory_utilization`, and `timestamp`.\n- `generate_report.py`:\n  - Loads data from CSV files.\n  - Cleans data by removing outliers.\n  - Calculates average metrics for each max power setting.\n  - Generates smooth curve plots and highlights recommended settings.\n  - Adds a header and summary to the output image.\n- `recommend.py`:\n  - Loads data from CSV files.\n  - Calculates summary statistics for each max power setting.\n  - Recommends the optimal max power settings based on energy consumption.\n  - Can be run standalone or used as a module in other scripts.\n- `gpu_metrics_utils.py`: Utility functions for processing and analyzing GPU metrics.\n- `llm_inference.py`: Contains functions and scripts for running inference on large language models.\n- `llm_training.py`: Contains functions and scripts for training large language models.\n- `process_experiment_data.py`: Processes raw experiment data into a more usable format.\n- `run_experiments.py`: Orchestrates the running of experiments and data collection.\n- `clean.sh`: Script to clean up generated files such as logs, CSVs, and images.\n\n## License\n\nThis project is licensed under the MIT License.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchrisbraddock%2Fgpu-tune","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fchrisbraddock%2Fgpu-tune","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchrisbraddock%2Fgpu-tune/lists"}