https://github.com/teaglebuilt/llm_cost_exporter
https://github.com/teaglebuilt/llm_cost_exporter
Last synced: 6 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/teaglebuilt/llm_cost_exporter
- Owner: teaglebuilt
- Created: 2025-04-10T14:12:28.000Z (about 1 year ago)
- Default Branch: master
- Last Pushed: 2025-04-10T15:32:43.000Z (about 1 year ago)
- Last Synced: 2025-04-10T16:04:38.701Z (about 1 year ago)
- Language: Rust
- Size: 9.77 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# LLM Cost Exporter
Monitor cost metrics for llm providers that can be scraped and ingested by prometheus.
**Supported Providers**
* `openai`
* `anthropic`
* `bedrock`
## Instructions
### Prerequisites
Obvisouly, this is a prometheus exporter so make sure you have prometheus up and running.
#### Running on Kubernetes
To understand how to set this up on kubernetes beyond the basic helm installation, then refer to this example for [Cost Alerting on Kubernetes](./examples/k8s-cost-alerting/README.md)
**Installing helm chart**
* `OpenAI`
```
helm upgrade --install llm-cost-monitor ./chart \
--namespace ${NAMESPACE} \
--set providers.openai.secretName=openai-api-secret
```
* `Anthropic`
```
helm upgrade --install llm-cost-monitor ./chart \
--namespace ${NAMESPACE} \
--set providers.anthropic secretName=anthropic-api-secret
```
* `AWS Bedrock`
- with credentials
```
helm upgrade --install llm-cost-monitor ./llm-cost-monitor \
--namespace llm-monitoring \
--set providers.bedrock.credentials.enabled=true \
--set providers.bedrock.credentials.aws_access_key_id=$AWS_ACCESS_KEY_ID \
--set providers.bedrock.credentials.aws_secret_access_key=$AWS_SECRET_ACCESS_KEY
```
- with multiple accounts
```
helm upgrade --install llm-cost-monitor ./chart \
--namespace llm-monitoring \
--set providers.bedrock.enabled=true \
--set providers.bedrock.iam.roleArn=arn:aws:iam::123456789012:role/bedrock-access-role
```
#### Running with Docker
Feel free to use the [Compose Stack Example](./examples/compose-stack/README.md) for a live local example
Otherwise, here is the basic instructions.
add the host of the llm exporter to your `prometheus.yml`.
```yaml
scrape_configs:
- job_name: 'llm_cost_monitor'
static_configs:
- targets: ['localhost:8000']
```