Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/dmaes/nomad-logger
Watch Nomad allocations and update logshipper config
https://github.com/dmaes/nomad-logger
Last synced: 3 months ago
JSON representation
Watch Nomad allocations and update logshipper config
- Host: GitHub
- URL: https://github.com/dmaes/nomad-logger
- Owner: dmaes
- License: apache-2.0
- Archived: true
- Created: 2021-12-23T13:02:19.000Z (almost 3 years ago)
- Default Branch: main
- Last Pushed: 2024-02-12T10:29:00.000Z (9 months ago)
- Last Synced: 2024-06-19T13:47:04.317Z (5 months ago)
- Language: Go
- Size: 74.2 KB
- Stars: 8
- Watchers: 1
- Forks: 2
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-nomad - dmaes/nomad-logger - Watch Nomad allocations and update logshipper config. (Utilities)
README
# Nomad Logger
***Since Hashicorp's switch to BSL, I stopped using Nomad and am thus no longer maintaining this project***
This is a simple Go application that polls the [Nomad](https://www.nomadproject.io/) API for all allocations on a certain host,
and then updates the config of you log shipper of choice.
## Usage
```
Usage: nomad-logger [--nomad-addr NOMAD-ADDR] [--nomad-allocs-dir NOMAD-ALLOCS-DIR] [--nomad-node-id NOMAD-NODE-ID] [--reload-cmd RELOAD-CMD] [--log-shipper LOG-SHIPPER] [--fluentbit-conf-file FLUENTBIT-CONF-FILE] [--fluentbit-tag-prefix FLUENTBIT-TAG-PREFIX] [--promtail-targets-file PROMTAIL-TARGETS-FILE]
Options:
--nomad-addr NOMAD-ADDR
The address of the Nomad API [default: http://localhost:4646, env: NOMAD_ADDR]
--nomad-allocs-dir NOMAD-ALLOCS-DIR
The location of the Nomad allocations data. Used to set the path to the logfiles [default: /var/lib/nomad/alloc, env: NOMAD_ALLOCS_DIR]
--nomad-node-id NOMAD-NODE-ID
The ID of the Nomad node to collect logs for. If empty, we'll suppose this also runs in as a nomad job, and the available env vars will be used to determine the Node ID [env: NOMAD_NODE_ID]
--reload-cmd RELOAD-CMD
Optional command to execute after logshipper config has changed. Usefull to signal a service to reload it's config. Valid for fluentbit logshipper. [env: RELOAD_CMD]
--log-shipper LOG-SHIPPER
The logshipper to use. Options: fluentbit, promtail [default: promtail, env: LOG_SHIPPER]
--fluentbit-conf-file FLUENTBIT-CONF-FILE [default: /etc/fluent-bit/nomad.conf, env: FLUENTBIT_CONF_FILE]
--fluentbit-tag-prefix FLUENTBIT-TAG-PREFIX [default: nomad, env: FLUENTBIT_TAG_PREFIX]
--promtail-targets-file PROMTAIL-TARGETS-FILE
The promtail file_sd_config file where the generated config can be written. Will be completely overwritten, so don't put anything else there. [default: /etc/promtail/nomad.yaml, env: PROMTAIL_TARGETS_FILE]
--help, -h display this help and exit
```
### Example
There is an example nomad job file in `examples/nomad-job.hcl`.
If this is the only job you are running on that node,
than the resulting promtail `file_sd_config` file will look something like `examples/promtail-nomad.yaml`.
## Log shippers
### Promtail
https://grafana.com/docs/loki/latest/clients/promtail/
Will create/update a `file_sd_config` for promtail to use.
Promtail will watch this file for changes, so no need to signal promtail.
### Fluentbit
https://fluentbit.io/
`[INPUT]` and `[FILTER]` stanza's will be written into a dedicated file,
which can be `@INCLUDE`'ed from your main fluentbit config file.
Fluentbit does not watch it's config files.
So you either have to write something that watches to config file,
or use the `--reload-cmd` flag to execute a command every time the config file changes.
(examples: `--reload-cmd 'systemctl restart fluent-bit'`, `--reload-cmd 'touch a-canary-file-watched-by-a-wrapper'`)
## Installing/Building
You can just `go install github.com/dmaes/nomad-logger@latest` or `git clone` and `go build` this.
There is also a container (`ghcr.io/dmaes/nomad-logger`) that you can use.
This container is build for every commit, you can either use the commit sha or `latest` as tag.