Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/greenscale-ai/genai-carbon-footprint
GenAI Carbon Footprint is a tool to estimate energy use (kWh) and carbon emissions (gCO2eq) from LLM usage
https://github.com/greenscale-ai/genai-carbon-footprint
Last synced: 17 days ago
JSON representation
GenAI Carbon Footprint is a tool to estimate energy use (kWh) and carbon emissions (gCO2eq) from LLM usage
- Host: GitHub
- URL: https://github.com/greenscale-ai/genai-carbon-footprint
- Owner: greenscale-ai
- License: mit
- Created: 2024-02-18T13:51:04.000Z (5 months ago)
- Default Branch: main
- Last Pushed: 2024-02-29T18:35:32.000Z (4 months ago)
- Last Synced: 2024-03-21T09:28:47.888Z (3 months ago)
- Language: Go
- Size: 11.7 KB
- Stars: 1
- Watchers: 2
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Lists
- awesome-green-ai - GenAI Carbon Footprint - A tool to estimate energy use (kWh) and carbon emissions (gCO2eq) from LLM usage. (🛠Tools / Calculation Tools)
README
# GenAI Carbon Footprint calculator for LLM Inference
This repository contains a Go library (github.com/greenscale-ai/genai-carbon-footprint) designed to estimate the energy usage and carbon emissions associated with running Large Language Model (LLM) inferences, taking into account various factors such as the carbon intensity of electricity, device Thermal Design Power (TDP), memory consumption, and inference timing.
# Features
* Carbon Intensity Calculation: Calculate the carbon emissions based on user-provided or default values for electricity carbon intensity, device TDP, memory usage, and inference timing.
* Flexible Time Calculation: Supports specifying total inference time directly or calculating it based on first token latency and subsequent token latency times.
* CLI Tool: A command-line interface (CLI) tool for easy interaction with the library without needing to integrate it into other Go applications.
# Getting Started
## Prerequisites
Go version 1.15 or higher
## Installing
Clone the repository to your local machine:
```
git clone https://github.com/greenscale-ai/genai-carbon-footprint.git
cd genai-carbon-footprint
```
Build the CLI tool:
```
go build -o genai-carbon-footprint-cli cli/main.go
```
## Usage
Run the CLI tool with the necessary flags to calculate the carbon emissions. For example:
```
./genai-carbon-footprint-cli --tdp 350 --mem 8000 --total-inference-latency 16000 --carbon-intensity 450
```
## Flags
* `-tdp`: Model TDP in Watts (required).
* `--total-inference-latency`: Total time of one inference procedure in milliseconds.
* `--first-latency`: First token latency in milliseconds.
* `--next-latency`: Next token latency in milliseconds.
* `-mem`: Memory consumption in MB.
* `--token-size`: Output token number in one inference (default: 32).
* `--carbon-intensity`: Carbon intensity of electricity (default: 475 gCO2e/kWh - world average).
# Contributing
We welcome contributions to improve this project. Please follow these steps to contribute:
1. Fork the repository.
2. Create a new branch (git checkout -b feature/AmazingFeature).
3. Commit your changes (git commit -m 'Add some AmazingFeature').
4. Push to the branch (git push origin feature/AmazingFeature).
5. Open a Pull Request.
# License
This project is subject to the MIT License. A copy of this license can be found in [LICENSE](LICENSE) at the root of this repo.
# Acknowledgments
Intel Corporation for the initial Python version of the carbon calculator ([link](https://github.com/intel/intel-extension-for-transformers/blob/main/intel_extension_for_transformers/utils/llm_carbon_calc.py)).