{"id":16842550,"url":"https://github.com/trentm/go-ecslog","last_synced_at":"2025-06-14T13:03:02.186Z","repository":{"id":49177727,"uuid":"337314299","full_name":"trentm/go-ecslog","owner":"trentm","description":"`ecslog` CLI to pretty-print and filter log files in ecs-logging format","archived":false,"fork":false,"pushed_at":"2022-02-10T05:59:25.000Z","size":2665,"stargazers_count":8,"open_issues_count":18,"forks_count":2,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-06-14T13:02:50.023Z","etag":null,"topics":["cli","ecs-logging","logging"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/trentm.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-02-09T06:33:49.000Z","updated_at":"2025-02-19T17:29:45.000Z","dependencies_parsed_at":"2022-09-26T19:53:29.020Z","dependency_job_id":null,"html_url":"https://github.com/trentm/go-ecslog","commit_stats":null,"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/trentm/go-ecslog","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/trentm%2Fgo-ecslog","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/trentm%2Fgo-ecslog/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/trentm%2Fgo-ecslog/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/trentm%2Fgo-ecslog/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/trentm","download_url":"https://codeload.github.com/trentm/go-ecslog/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/trentm%2Fgo-ecslog/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":259820787,"owners_count":22916544,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cli","ecs-logging","logging"],"created_at":"2024-10-13T12:46:36.857Z","updated_at":"2025-06-14T13:03:02.139Z","avatar_url":"https://github.com/trentm.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ecslog\n\n`ecslog` is a CLI for pretty-printing and filtering log files in\n[ecs-logging](https://www.elastic.co/guide/en/ecs-logging/overview/master/intro.html)\nformat. It is a single (Go-implemented) binary, with builds for Linux, Mac, and\nWindows. Its default styling should play nice with both light and\ndark-background terminals.\n\n![ecslog demo screenshot](./docs/img/demo-light-and-dark.png)\n\n# Install\n\n- For homebrew users:\n\n        brew install trentm/tap/ecslog\n        # OR 'brew upgrade trentm/tap/ecslog' if you have it already\n\n- Or download a pre-built binary package from [the releases page](https://github.com/trentm/go-ecslog/releases)\n  and copy to somewhere on your PATH.\n\n- Or you can build from source via:\n\n        git clone git@github.com:trentm/go-ecslog.git\n        cd go-ecslog\n        make  # produces \"./ecslog\", a single binary you can put on your PATH\n        ./ecslog --version\n\nThen, try it on a demo log file:\n\n    curl -s https://raw.githubusercontent.com/trentm/go-ecslog/main/demo.log \\\n        | ecslog\n\n# Introduction\n\n`ecslog` pretty-prints log data in the given file argument(s):\n\n    ecslog /var/log/some.log [other-log-files...]\n\nor passed in on stdin\n\n    node server.js | ecslog\n    tail -f /var/log/some.log | ecslog\n    docker logs -f my-container | ecslog\n\nThe default behaviour can be customised by CLI options (use `ecslog --help` to\nlist all options) and/or a `~/.ecslog.toml` config file (see\n[Configuration](#configuration) below).  The rest of this section describes some\nfeatures of ecslog.\n\n\n## Level filtering\n\nUse `-l LEVEL-NAME` to only show log records at that level or above.\n\n![ecslog level filtering](./docs/img/level-filtering.png)\n\n([ECS](https://www.elastic.co/guide/en/ecs/current/ecs-log.html#field-log-level)\ndoes not mandate level names, so `ecslog` uses [a best-effort ordering](https://github.com/trentm/go-ecslog/blob/v0.4.0/internal/ecslog/ecslog.go#L150-L168)\nof level names in common usage.)\n\n\n## KQL filtering\n\nUse `-k KQL` to filter log records using [KQL](https://www.elastic.co/guide/en/kibana/current/kuery-query.html), as you would in Kibana.\n\n![ecslog KQL filtering](./docs/img/kql-filtering.png)\n\nSome examples:\n\n- Filter on the \"message\" field by default: `ecslog ./examples/apm-server.log -k '*ILM*'`\n- Filter on logger name (the `log.logger` field in ECS): `ecslog ./examples/apm-server.log -k 'log.logger: pipeline'`\n- Filter on slow requests: `ecslog ./example/apm-server.log -k 'event.duration \u003e 500000'`\n\nNote that this is a subset of KQL and necessarily slightly adapted for use on log files without an Elasticsearch mapping for field types. See [internal/kqlog/README.md](./internal/kqlog/README.md) for details.\n\n\n## Include/exclude fields from rendering\n\nSometimes it can help to focus by eliding some distracting fields. Use `-x FIELD,FIELD,...`\nto exclude named fields from the rendered output.\n\n![ecslog exclude fields](./docs/img/exclude-fields.png)\n\nOr use `-i FIELD,FIELD,...` to exclude all extra fields (those after the title\nline) **except** those given.\n\n\n## `@timestamp` diff highlighting\n\nBy default, `ecslog` will highlight the change in a log record's `@timestamp`\nfrom the previous log record. With the \"default\" formatter, the changed part\nof the timestamp is underlined. For example:\n\n![ecslog @timestamp diff highlighting](./docs/img/timestamp-diff-highlighting.png)\n\nThis can be turned off with the [`timestampShowDiff=false` config var](#config-timestampshowdiff).\n\n\n## `ecsLenient` for almost-ecs-logging format logs\n\nThe [ecs-logging spec](https://github.com/elastic/ecs-logging/blob/master/spec/spec.json)\nmandates all of the `ecs.version`, `log.level`, and `@timestamp` fields. Given\nthat ecs-logging is a relatively new spec, some applications, in the process of\nmigrating, are *almost* conformant. Add `ecsLenient=true` to `~/.ecslog.toml` to\nonly require that log lines have *one* of these fields to be considered an\necs-logging record.\n\n\u003c!--\n- elasticsearch: https://github.com/elastic/elasticsearch/issues/73256\n- kibana: Missing log.level (because hapi \"tags\"). I haven't opened an issue.\n- apm-server: https://github.com/elastic/beats/issues/25922\n--\u003e\n\n\n## Output formats\n\n`ecslog` has multiple output formats for rendering ECS logs that may be selected\nvia the `-f, --format NAME` option. Some formats are *lossy*, i.e. do not render\nall fields, typically for compactness. Formats labelled as \"lossless\" have one\nexception: the \"ecs.version\" field is typically not rendered.\n\n- `default`: A default lossless that format renders each log record with a\n  \"title line\" -- which includes core and common fields -- followed by all\n  remaining extra fields. Roughly:\n\n  ```\n  [@timestamp] LOG.LEVEL (log.logger/service.name on host.hostname): message\n      extraKey1: extraValue1-as-multiline-jsonish\n      extraKey2: extraValue2-as-multiline-jsonish\n  ```\n\n  where \"multiline jsonish\" means 4-space-indented JSON with the one special\n  case that multiline string values are printed indented and with newlines.\n  For example, `error.stack_trace` in the following:\n\n  ```\n  [2021-02-11T06:24:53.251Z]  WARN (myapi on purple.local): something went wrong\n      error: {\n          \"type\": \"Error\"\n          \"message\": \"boom\"\n          \"stack_trace\":\n              Error: boom\n                  at pino/examples/express-simple.js:67:15\n                  at Layer.handle [as handle_request] (pino/node_modules/express/lib/router/layer.js:95:5)\n                  ...\n  ```\n\n  The \"key: value\" format, for simple values, has the benefit of being usable\n  as-is as a KQL filter with the `-k KQL` option.  The format of the title line\n  may change in future versions.\n\n- `compact`: A lossless format similar to \"default\", but attempts are made\n  to make the \"extraKey\" info more compact by balancing multiline JSON with\n  80-column output.\n\n- `ecs`: The native/raw ECS format, ndjson.\n\n- `simple`: A *lossy* format that simply renders `LOG.LEVEL: message`. If extra\n  fields (other than the core \"@timestamp\" and \"ecs.version\" fields) are being\n  elided, a ellipsis is appended to the line.\n\n\n## `--strict` to filter out non-ecs-logging lines\n\nBy default `ecslog` will pass through non-ecs-logging lines unchanged, which is\nuseful for mixed-format logs that are typically in the real world -- for example,\nstructured logging mixed with debug printfs. However, sometimes it can be\nuseful to limit to just ecs-logging lines. Use the `--strict` option for this.\n\nOne use case for this is in a pipeline that will process JSON logs, say, with\n`jq`. For example, this will filter an example \"apm-server\" log file to\nrecords with the 'event.duration' field (`-k 'event.duration: *'`), exclude\nnon-ecs-logging records (`--strict`), format as raw JSON (`-f ecs`) and then\npipe to `jq` to extract the single JSON field:\n\n```shell\n$ ecslog examples/apm-server.log -k 'event.duration: *' --strict -f ecs | jq '.\"event.duration\"'\n166823\n104904\n76471\n90210\n149853\n10226197680\n...\n```\n\n\n# Configuration\n\nAny of the following `ecslog` options can be set in a `~/.ecslog.toml` file.\nSee https://toml.io/ for TOML syntax information.  The `--no-config` option can\nbe used to ignore `~/.ecslog.toml`, if there is one.\n\nAn example config:\n\n```toml\nformat=\"compact\"\nmaxLineLen=32768\necsLenient=true\n```\n\n### config: format\n\nSet the output format name (a string, equivalent of `-f, --format` option).\nValid values are: \"default\" (the default), \"compact\", \"ecs\", \"simple\".\n\n```toml\nformat=\"default\"\n```\n\n### config: color\n\nA color mode string for whether output should be colorized.  Valid values are:\n\"auto\" (the default), \"yes\", \"no\". \"auto\" will colorize if the output stream\nis a TTY.\n\n```toml\ncolor=\"auto\"\n```\n\n### config: maxLineLen\n\nSet the maximum number of bytes long for a single line that will be considered\nfor processing. Longer lines will be treated as if they are not ecs-logging\nrecords.  Valid values are: -1 (to use the default 16384), or a value between 1\nand 1048576 (inclusive).\n\n```toml\nmaxLineLen=16384\n```\n\n### config: ecsLenient\n\nSome JSON logs are \"ECS compatible\" in that they attempt to follow [ECS general\nguidelines](https://www.elastic.co/guide/en/ecs/current/ecs-guidelines.html) --\nhave a `@timestamp`, use the specified data types for ECS fields, set\n`ecs.version` -- but are not \"ecs-logging compliant\" because they are missing\none or more of `@timestamp`, `ecs.version`, or `log.level`.\n\nBy default `ecslog` will skip rendering for any log line that does not have\nthose three fields. Set `ecsLenient` to true to tell `ecslog` to attempt to\nrendering any log record that has **at least one** of these fields.\n\n```toml\necsLenient=false\n```\n\n### config: timestampShowDiff\n\nIf coloring the output (see [config: color](#config-color) above), by default\n`ecslog` will style the change in the timestamp from the preceding log record.\nSet this config var to `false` to turn off this styling.\n\n```toml\ntimestampShowDiff=true\n```\n\n\n# Bugs\n\nIf you find a crash or some other issue with `ecslog`, please\n[create an issue](https://github.com/trentm/go-ecslog/issues/new).\n\nThe `ECSLOG_DEBUG` environment variable can be set to get some internal limited\ndebugging information on stderr. For example:\n\n    ECSLOG_DEBUG=1 ecslog ...\n\nInternal debug logging is disabled if `ECSLOG_DEBUG` is unset, or is set\nto one of: the empty string, \"0\", or \"false\".\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftrentm%2Fgo-ecslog","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftrentm%2Fgo-ecslog","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftrentm%2Fgo-ecslog/lists"}