Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/michael-doubez/filestat_exporter

Prometheus exporter gathering metrics about file size, modification time and other stats
https://github.com/michael-doubez/filestat_exporter

file-hash file-modification-time file-monitoring file-size line-counter prometheus-exporter

Last synced: 9 days ago
JSON representation

Prometheus exporter gathering metrics about file size, modification time and other stats

Host: GitHub
URL: https://github.com/michael-doubez/filestat_exporter
Owner: michael-doubez
License: apache-2.0
Created: 2019-12-07T21:36:08.000Z (almost 5 years ago)
Default Branch: master
Last Pushed: 2024-07-04T22:11:26.000Z (4 months ago)
Last Synced: 2024-08-02T15:48:06.945Z (3 months ago)
Topics: file-hash, file-modification-time, file-monitoring, file-size, line-counter, prometheus-exporter
Language: Go
Homepage:
Size: 69.3 KB
Stars: 47
Watchers: 3
Forks: 13
Open Issues: 6
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE

Awesome Lists containing this project

README

        # File statistics exporter

[![CircleCI](https://circleci.com/gh/michael-doubez/filestat_exporter/tree/master.svg?style=shield)][circleci]

[![Docker Pulls](https://img.shields.io/docker/pulls/mdoubez/filestat_exporter.svg?maxAge=604800)][dockerhub]

[![GitHub All Releases](https://img.shields.io/github/downloads/michael-doubez/filestat_exporter/total)][releases]

[![Go Report Card](https://goreportcard.com/badge/github.com/michael-doubez/filestat_exporter)][goreportcard]

Prometheus exporter gathering metrics about file size, modification time and other statistics.

## Quickstart

Pre-built binaries are available on the [GitHub release page][releases].

### Usage

Configure target files on command line, passing glob patterns in parameters

    ./filestat_exporter '*'

Optional flags:

* __`-config.file `:__ The path to the configuration file (use "none" to disable).

* __`-log.level `:__ Logging level \[debug, info, warn, error\]. (default: `info`)

* __`-version`:__ Print the version of the exporter and exit.

* __`-web.listen-address `:__ Address to listen on for web interface and telemetry. (default: `9943`)

* __`-web.telemetry-path `:__ Path under which to expose metrics. (default: `/metrics`)

* __`-web.config `:__ Path to config yaml file that can enable TLS or authentication.

* __`-web.systemd-socket`:__ Flag to use systemd socket activation listeners instead of port listeners (Linux only).

* __`-path.cwd `:__ Change working directory of path pattern collection.

* __`-metric.crc32`:__ Generate CRC32 hash metric of files.

* __`-metric.nb_lines`:__ Generate line number metric of files.

The exporter can read a config file in yaml format (`filestat.yaml` by default).

```yaml

exporter:

  # Optional network parameters

  listen_address: ':9943'

  #metrics_path: /metrics

  

  # Optional working directory - overridden by parameter '-path.cwd'

  working_directory: "/path/to/my/project"

  # Default enable/disable of metrics - overridden if not set by parameter '-metric.*'

  enable_crc32_metric: true

  # enable_nb_line_metric: false

  # list of patterns to apply - metrics can be enable/disabled for each group

  files:

    - patterns: ["*.html","assets/*.css","scripts/*.js"]

    - patterns: ["data/*.csv"]

      enable_nb_line_metric: true

    - patterns: ["archives/*.tar.gz"]

      enable_crc32_metric: false

      enable_nb_line_metric: false

```

Note: if a file is matched by a pattern more than once, only the first match's config is used

### Pattern format

Pattern uses the glob implementation of [bmatcuk/doublestar](https://github.com/bmatcuk/doublestar#patterns) project:

* Doublestar (`**`) can be used to match directories recursively. It should appear surrounded by path separators such as `/**/`.

* Usual [Glob syntax](https://en.wikipedia.org/wiki/Glob_(programming)#Syntax) is still supported.

Pattern can also use golang template format with the following functions:

| Function  | Description                                   | Example                       |

| --------- | --------------------------------------------- | ----------------------------- |

| now       | Current [time](https://pkg.go.dev/time#Time)  | `{{ now.Locale.Year }}/*.tgz` |

### Exported Metrics

| Metric                       | Description                                  | Labels   |

| ---------------------------- | -------------------------------------------- | -------- |

| file_glob_match_number       | Number of files matching pattern             | pattern  |

| file_stat_size_bytes         | Size of file in bytes                        | path     |

| file_stat_modif_time_seconds | Last modification time of file in epoch time | path     |

| file_content_hash_crc32  (*) | CRC32 hash of file content                   | path     |

| file_content_line_number (*) | Number of lines in file                      | path     |

Note: metrics with `(*)` are only provided if configured

## Building and running

Prerequisites:

* [Go compiler](https://golang.org/dl/) - currently version v1.22

* Linux with make installed

* Essential build environment for dependencies

### Building

    go get github.com/michael-doubez/filestat_exporter

    cd ${GOPATH-$HOME/go}/src/github.com/michael-doubez/filestat_exporter

    make

    ./filestat_exporter  

To see all available configuration flags:

    ./filestat_exporter -h

The Makefile provides several targets:

* `make check`: Running checks and tests

* `make run`: Run exporter from go

* `make version`: Print current version

* `make build`: Build exporter without checks

### Cross compiled distribution

To build all distribustion packages

    make dist

To build a specific os/architecture package

    make dist--

    make dist-linux-amd64

    ...

## Using Docker

The `filestat_exporter` is designed to monitor files on the host system.

Try it out in minutes on [Katakoda docker playground][dockerplay]:

```bash

# create local file

docker container run --rm -d -v ~/my_files:/my_files --name my_files bash -c 'echo "Hello world" > /my_files/sample.txt'

# launch exporter watching the files

docker run -d -p 9943:9943 --name=filestats -v ~/my_files:/data mdoubez/filestat_exporter -path.cwd /data '*'

# see file metrics

curl -s docker:9943/metrics | grep file_

```

## TLS and basic authentication

Filestat Exporter supports TLS and basic authentication. This enables better

control of the various HTTP endpoints.

To use TLS and/or basic authentication, you need to pass a configuration file

using the `--web.config.file` parameter. The format of the file is described

[in the exporter-toolkit repository](https://github.com/prometheus/exporter-toolkit/blob/master/docs/web-configuration.md).

## License

Apache License 2.0, see [LICENSE](https://github.com/michael-doubez/filestat_exporter/blob/master/LICENSE).

[circleci]: https://circleci.com/gh/michael-doubez/filestat_exporter

[dockerhub]: https://hub.docker.com/r/mdoubez/filestat_exporter/

[goreportcard]: https://goreportcard.com/report/github.com/michael-doubez/filestat_exporter

[dockerplay]: https://www.katacoda.com/courses/docker/playground

[releases]: https://github.com/michael-doubez/filestat_exporter/releases