https://github.com/darklang/honeycomb-logger
An nginx container that proxies to the honeycomb api and also logs events to stdout
https://github.com/darklang/honeycomb-logger
Last synced: 10 months ago
JSON representation
An nginx container that proxies to the honeycomb api and also logs events to stdout
- Host: GitHub
- URL: https://github.com/darklang/honeycomb-logger
- Owner: darklang
- License: other
- Created: 2020-05-21T23:41:33.000Z (about 6 years ago)
- Default Branch: master
- Last Pushed: 2020-05-21T23:47:20.000Z (about 6 years ago)
- Last Synced: 2025-06-14T19:09:34.519Z (12 months ago)
- Language: Lua
- Size: 6.84 KB
- Stars: 0
- Watchers: 4
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Honeycomb Logger
## Goals
We'd like to be able to send the logs that go to Honeycomb elsewhere (for
longer-term storage); we want this split to happen _after_ the Honeycomb client
(whether it's honeytail or honeycomb-kubernetes-agent or a beeline) does any
processing.
Thus, we send events both to Honeycomb and to stdout for capture by other
log-collection tools.
## Use
We provide Dockerfile containing an nginx proxy that does this.
You must provide the container with a `DATASETS_TO_LOG` env var. It may be
either a pipe-delimited list of datasets:
```
dataset1|dataset2|dataset3
```
or a match-all:
```
.*
```
Then, override your honeycomb client's `API Host` to point at the nginx proxy,
on port 8010; depending on your client, this might be an env var, a key in yaml
or INI config files, or a flag. (`honeytail` uses `--api_host`, for instance.)
Having done that:
- data will be proxied to `https://api.honeycomb.io:443`
- events sent to honeycomb (either as single events to /1/events/:dataset or in
a batch to /1/batch/:dataset) will be logged, and prefixed with the name of
the dataset
For example (see below), posting 4 events to a dataset
called 'sandbox', stdout from the container looks like:
```
sandbox [{"data":{"bar":3},"time":"2020-05-15T18:27:53.167828126Z"},{"data":{"alpha":"a","beta":"b"},"time":"2020-05-15T18:27:53.167971161Z"},{"data":{"foo":1},"time":"2020-05-15T18:27:53.167979722Z"},{"data":{"last":"done"},"time":"2020-05-15T18:27:53.168155905Z"}]
```
## Implementation details
Nginx can sit as a proxy between the honeycomb client and the honeycomb API.
Requests from the client may be gzipped; we use a lua plugin to inflate the body
and make that data available to nginx, which then logs it to stdout.
## Development & testing
1. To build: `docker build -t honeycomb-logger -f honeycomb-logger.Dockerfile .`
2. Run the container using `dev-stuff/runner.sh`. Note: You'll need to set a
WRITEKEY environment variable that honeytail can use.
3. Find the container name using `docker ps`, and exec into it:
`docker exec -it bash`
4. Once in the container, run `./installer.sh`.
5. You can now run `./honeytail-runner.sh` to send the contents of `sample-data.json` to honeycomb. You will need to use your api key for `--writekey`, and you may wish to change which `--dataset` you send to.
6. Expected output from the container is shown above; `honeytail-runner.sh` also
has some output logs, including a line indicating that you successfully sent
4 events: `number sent by response status code=map[202:4] total attempted sends=4`.