https://github.com/cloudfoundry/firehose_exporter
Cloud Foundry Firehose Prometheus exporter
https://github.com/cloudfoundry/firehose_exporter
cloud-foundry firehose metrics prometheus
Last synced: 5 days ago
JSON representation
Cloud Foundry Firehose Prometheus exporter
- Host: GitHub
- URL: https://github.com/cloudfoundry/firehose_exporter
- Owner: cloudfoundry
- License: apache-2.0
- Created: 2016-09-19T19:27:28.000Z (about 9 years ago)
- Default Branch: master
- Last Pushed: 2025-04-14T03:42:45.000Z (6 months ago)
- Last Synced: 2025-04-14T04:33:29.150Z (6 months ago)
- Topics: cloud-foundry, firehose, metrics, prometheus
- Language: Go
- Size: 34.8 MB
- Stars: 26
- Watchers: 8
- Forks: 23
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
Awesome Lists containing this project
README
# Cloud Foundry Firehose Exporter [](https://travis-ci.org/cloudfoundry/firehose_exporter)
A [Prometheus][prometheus] exporter proxy for [Cloud Foundry Firehose][firehose] metrics. Please refer to the [FAQ][faq]
for general questions about this exporter.
## Architecture overview
## Installation
### Binaries
Download the already existing [binaries](/releases) for your platform:
```bash
$ ./firehose_exporter
```
### From source
Using the standard `go install` (you must have [Go][golang] already installed in your local machine):
```bash
$ go install github.com/cloudfoundry/firehose_exporter
$ firehose_exporter
```
### Docker
To run the firehose exporter as a Docker container, run:
```bash
$ docker run -p 9186:9186 boshprometheus/firehose-exporter
```
### Cloud Foundry
The exporter can be deployed to an already existing [Cloud Foundry][cloudfoundry] environment:
```bash
$ git clone https://github.com/cloudfoundry/firehose_exporter.git
$ cd firehose_exporter
```
Modify the included [application manifest file][manifest] to include your [Cloud Foundry Firehose][firehose] properties.
Then you can push the exporter to your Cloud Foundry environment:
```bash
$ cf push
```
### BOSH
This exporter can be deployed using the [Prometheus BOSH Release][prometheus-boshrelease].
## Usage
### Flags
| Flag / Environment Variable | Required | Default | Description |
| --------------------------- | -------- | ------- | ----------- |
| `retro_compat.disable`
`FIREHOSE_EXPORTER_RETRO_COMPAT_DISABLE` | No | `False` | Disable retro compatibility |
| `retro_compat.enable_delta`
`FIREHOSE_EXPORTER_RETRO_COMPAT_ENABLE_DELTA` | No | `False` | Enable retro compatibility delta in counter |
| `metrics.shard_id`
`FIREHOSE_EXPORTER_DOPPLER_SUBSCRIPTION_ID` | No | `prometheus` | Cloud Foundry Nozzle Subscription ID |
| `metrics.expiration`
`FIREHOSE_EXPORTER_DOPPLER_METRIC_EXPIRATION` | No | `10 minutes` | How long Cloud Foundry metrics received from the Firehose are valid |
| `metrics.batch_size`
`FIREHOSE_EXPORTER_METRICS_BATCH_SIZE` | No | `infinite buffer` | Batch size for nozzle envelop buffer |
| `metrics.node_index`
`FIREHOSE_EXPORTER_NODE_INDEX` | No | `0` | Node index to use |
| `metrics.timer_rollup_buffer_size`
`FIREHOSE_EXPORTER_TIMER_ROLLUP_BUFFER_SIZE` | No | `0` | The number of envelopes that will be allowed to be buffered while timer http metric aggregations are running |
| `filter.deployments`
`FIREHOSE_EXPORTER_FILTER_DEPLOYMENTS` | No | | Comma separated deployments to filter |
| `filter.events`
`FIREHOSE_EXPORTER_FILTER_EVENTS` | No | | Comma separated events to filter. If not set, all events will be enabled (`ContainerMetric`, `CounterEvent`, `HttpStartStop`, `ValueMetric`) |
| `logging.url`
`FIREHOSE_EXPORTER_LOGGING_URL` | Yes | | Cloud Foundry Log Stream URL |
| `logging.tls.ca`
`FIREHOSE_EXPORTER_LOGGING_TLS_CA` | No | | Path to ca cert to connect to rlp |
| `logging.tls.cert`
`FIREHOSE_EXPORTER_LOGGING_TLS_CERT` | Yes | | Path to cert to connect to rlp in mtls |
| `logging.tls.key`
`FIREHOSE_EXPORTER_LOGGING_TLS_KEY` | Yes | | Path to key to connect to rlp in mtls |
| `metrics.namespace`
`FIREHOSE_EXPORTER_METRICS_NAMESPACE` | No | `firehose` | Metrics Namespace |
| `metrics.environment`
`FIREHOSE_EXPORTER_METRICS_ENVIRONMENT` | Yes | | Environment label to be attached to metrics |
| `skip-ssl-verify`
`FIREHOSE_EXPORTER_SKIP_SSL_VERIFY` | No | `false` | Disable SSL Verify |
| `web.listen-address`
`FIREHOSE_EXPORTER_WEB_LISTEN_ADDRESS` | No | `:9186` | Address to listen on for web interface and telemetry |
| `web.telemetry-path`
`FIREHOSE_EXPORTER_WEB_TELEMETRY_PATH` | No | `/metrics` | Path under which to expose Prometheus metrics |
| `web.auth.username`
`FIREHOSE_EXPORTER_WEB_AUTH_USERNAME` | No | | Username for web interface basic auth |
| `web.auth.password`
`FIREHOSE_EXPORTER_WEB_AUTH_PASSWORD` | No | | Password for web interface basic auth |
| `web.tls.cert_file`
`FIREHOSE_EXPORTER_WEB_TLS_CERTFILE` | No | | Path to a file that contains the TLS certificate (PEM format). If the certificate is signed by a certificate authority, the file should be the concatenation of the server's certificate, any intermediates, and the CA's certificate |
| `web.tls.key_file`
`FIREHOSE_EXPORTER_WEB_TLS_KEYFILE` | No | | Path to a file that contains the TLS private key (PEM format) |
| `profiler.enable`
`FIREHOSE_EXPORTER_ENABLE_PROFILER` | No | `False` | Enable pprof profiling on app on /debug/pprof |
| `log.level`
`FIREHOSE_EXPORTER_LOG_LEVEL` | No | `info` | Only log messages with the given severity or above. Valid levels: [debug, info, warn, error, fatal] |
| `log.in_json`
`FIREHOSE_EXPORTER_LOG_IN_JSON` | No | `False` | Log in json |
### Metrics
For a list of [Cloud Foundry Firehose][firehose] metrics check the [Cloud Foundry Component Metrics][cfmetrics]
documentation.
The exporter returns additionally the following internal metrics:
| Metric | Description | Labels |
| ------ | ----------- | ------ |
| *metrics.namespace*_total_envelopes_received | Total number of envelopes received from Cloud Foundry Firehose | `environment` |
| *metrics.namespace*_last_envelope_received_timestamp | Number of seconds since 1970 since last envelope received from Cloud Foundry Firehose | `environment` |
| *metrics.namespace*_total_metrics_received | Total number of metrics received from Cloud Foundry Firehose | `environment` |
| *metrics.namespace*_last_metric_received_timestamp | Number of seconds since 1970 since last metric received from Cloud Foundry Firehose | `environment` |
| *metrics.namespace*_total_container_metrics_received | Total number of container metrics received from Cloud Foundry Firehose | `environment` |
| *metrics.namespace*_last_container_metric_received_timestamp | Number of seconds since 1970 since last container metric received from Cloud Foundry Firehose | `environment` |
| *metrics.namespace*_total_counter_events_received | Total number of counter events received from Cloud Foundry Firehose | `environment` |
| *metrics.namespace*_last_counter_event_received_timestamp | Number of seconds since 1970 since last counter event received from Cloud Foundry Firehose | `environment` |
| *metrics.namespace*_total_http_received | Total number of http start stop received from Cloud Foundry Firehose | `environment` |
| *metrics.namespace*_last_http_received_timestamp | Number of seconds since 1970 since last http start stop received from Cloud Foundry Firehose | `environment` |
| *metrics.namespace*_total_value_metrics_received | Total number of value metrics received from Cloud Foundry Firehose | `environment` |
| *metrics.namespace*_last_value_metric_received_timestamp | Number of seconds since 1970 since last value metric received from Cloud Foundry Firehose | `environment` |
## Contributing
Refer to [CONTRIBUTING.md](https://github.com/cloudfoundry/firehose_exporter/blob/master/CONTRIBUTING.md).
## License
Apache License 2.0, see [LICENSE](https://github.com/cloudfoundry/firehose_exporter/blob/master/LICENSE).
[cloudfoundry]: https://www.cloudfoundry.org/
[cfmetrics]: https://docs.cloudfoundry.org/loggregator/all_metrics.html
[faq]: https://github.com/cloudfoundry/firehose_exporter/blob/master/FAQ.md
[firehose]: https://docs.cloudfoundry.org/loggregator/architecture.html#firehose
[golang]: https://golang.org/
[manifest]: https://github.com/cloudfoundry/firehose_exporter/blob/master/manifest.yml
[prometheus]: https://prometheus.io/
[prometheus-boshrelease]: https://github.com/bosh-prometheus/prometheus-boshrelease