{"id":19257312,"url":"https://github.com/logzio/docker-collector-logs","last_synced_at":"2025-04-21T15:31:19.670Z","repository":{"id":39653863,"uuid":"170496900","full_name":"logzio/docker-collector-logs","owner":"logzio","description":null,"archived":false,"fork":false,"pushed_at":"2025-03-11T07:44:36.000Z","size":109,"stargazers_count":16,"open_issues_count":13,"forks_count":12,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-04-01T14:14:37.735Z","etag":null,"topics":["integration"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/logzio.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-02-13T11:31:51.000Z","updated_at":"2025-03-11T07:44:40.000Z","dependencies_parsed_at":"2024-04-24T14:12:13.311Z","dependency_job_id":"867c198e-ebef-44bb-9672-53422f8fa567","html_url":"https://github.com/logzio/docker-collector-logs","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/logzio%2Fdocker-collector-logs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/logzio%2Fdocker-collector-logs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/logzio%2Fdocker-collector-logs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/logzio%2Fdocker-collector-logs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/logzio","download_url":"https://codeload.github.com/logzio/docker-collector-logs/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250080611,"owners_count":21371539,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["integration"],"created_at":"2024-11-09T19:09:30.013Z","updated_at":"2025-04-21T15:31:19.382Z","avatar_url":"https://github.com/logzio.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# docker-collector-logs\n\ndocker-collector-logs is a Docker container that uses Filebeat to collect logs from other Docker containers and forward those logs to your Logz.io account.\n\nTo use this container, you'll set environment variables in your `docker run` command.\ndocker-collector-logs uses those environment variables to generate a valid Filebeat configuration for the container.\ndocker-collector-logs mounts docker.sock and the Docker logs directory to the container itself, allowing Filebeat to collect the logs and metadata.\n\ndocker-collector-logs ships logs only.\nIf you want to ship metrics to Logz.io, see [docker-collector-metrics](https://github.com/logzio/docker-collector-metrics).\n\n**Note:**\n- Upgrading to a newer version of a docker-collector-logs while it is already running will cause it to resend logs that are within the `ignoreOlder` timeframe. You can minimize log duplicates by setting the `ignoreOlder` parameter of the new docker to a lower value (for example, `20m`).\n- This integration supports **ARM architecture** as of `logzio/docker-collector-logs:0.1.6`.\n\n\n\n## docker-collector-logs setup\n\n### 1. Pull the Docker image\n\nDownload the logzio/docker-collector-logs image:\n\n```shell\ndocker pull logzio/docker-collector-logs\n```\n\n### 2. Run the container\n\nFor a complete list of options, see the parameters below the code block.👇\n\n```shell\ndocker run --name docker-collector-logs \\\n--env LOGZIO_TOKEN=\"\u003cACCOUNT-TOKEN\u003e\" \\\n-v /var/run/docker.sock:/var/run/docker.sock:ro \\\n-v /var/lib/docker/containers:/var/lib/docker/containers \\\nlogzio/docker-collector-logs\n```\n\n#### Parameters\n\n| Parameter              | Description                                                                                                                                                                                                                                                                                                                                                                                                       |\n|------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| **LOGZIO_TOKEN**       | **Required**. Your Logz.io account token. Replace `\u003cACCOUNT-TOKEN\u003e` with the [token](https://app.logz.io/#/dashboard/settings/general) of the account you want to ship to.                                                                                                                                                                                                                                        |\n| **LOGZIO_REGION**      | **Default**: US region.\u003cbr\u003e Logz.io region code to ship the logs to. This region code changes depending on the region your account is hosted in. For example, accounts in the EU region have region code `eu`.\u003cbr /\u003e For more information, see [Account region](https://docs.logz.io/user-guide/accounts/account-region.html) on the Logz.io Docs.                                                                |\n| **LOGZIO_TYPE**        | **Default**: Docker image name \u003cbr\u003e The log type you'll use with this Docker. This is shown in your logs under the `type` field in Kibana. \u003cbr\u003e Logz.io applies parsing based on `type`.                                                                                                                                                                                                                          |\n| **LOGZIO_CODEC**       | **Default**: `plain`\u003cbr\u003e Set to `json` if shipping JSON logs. Otherwise, set to `plain`.                                                                                                                                                                                                                                                                                                                          |\n| **matchContainerName** | Comma-separated list of containers you want to collect the logs from. If a container's name partially matches a name on the list, that container's logs are shipped. Otherwise, its logs are ignored. \u003cbr /\u003e **Note**: Can't be used with `skipContainerName`                                                                                                                                                     |\n| **skipContainerName**  | Comma-separated list of containers you want to ignore. If a container's name partially matches a name on the list, that container's logs are ignored. Otherwise, its logs are shipped. \u003cbr /\u003e **Note**: Can't be used with `matchContainerName`                                                                                                                                                                   |\n| **additionalFields**   | Include additional fields with every message sent, formatted as `\"fieldName1=fieldValue1;fieldName2=fieldValue2\"`. \u003cbr /\u003e To use an environment variable, format as `\"fieldName1=fieldValue1;fieldName2=$ENV_VAR_NAME\"`. In that case, the environment variable should be the only value in the field. In case the environment variable can't be resolved, the field will be omitted.                             |\n| **excludeLines**       | Comma-separated list of regular expressions to match the lines that you want Filebeat to exclude. \u003cbr /\u003e **Note**: Does not behave well with regular expressions containing commas `,`                                                                                                                                                                                                                            |\n| **includeLines**       | Comma-separated list of regular expressions to match the lines that you want Filebeat to include. \u003cbr /\u003e **Note**: Does not behave well with regular expressions containing commas `,`                                                                                                                                                                                                                            |\n| **ignoreOlder**        | **Default** `3h` \u003cbr\u003e Logs older than this will be ignored                                                                                                                                                                                                                                                                                                                                                        |\n| **renameFields**       | Rename fields with every message sent, formatted as `\"oldName,newName;oldName2,newName2\"`. \u003cbr /\u003e To use an environment variable, format as `\"oldName,newName;oldName2,$ENV_VAR_NAME\"`. In that case, the environment variable should be the only value in the field. In case the environment variable can't be resolved, the field will be omitted.                                                              |\n| **HOSTNAME**           | **Default** `''` \u003cbr\u003e Insert your host name if you want it to appear under the field `agent.name`. If no value entered,  `agent.name` will show the container's id.                                                                                                                                                                                                                                               |\n| **multilinePattern**   | **Default** `''` \u003cbr\u003e Insert your regex pattern. See [Filebeat's official documentation](https://www.elastic.co/guide/en/beats/filebeat/7.12/multiline-examples.html#multiline) for further info.                                                                                                                                                                                                                 |\n| **multilineNegate**    | **Default** `'false'` \u003cbr\u003e Insert `'true'` to negate the pattern. \u003cbr /\u003e **Note**: Cannot be used without multilinePattern. See [Filebeat's official documentation](https://www.elastic.co/guide/en/beats/filebeat/7.12/multiline-examples.html#multiline) for further info.                                                                                                                                      |\n| **multilineMatch**     | **Default** `'after'` \u003cbr\u003e  Specifies how Filebeat combines matching lines into an event. The settings are `after` or `before`. The behavior of these settings depends on what you specify for negate. \u003cbr /\u003e **Note**: Cannot be used without multilinePattern. See [Filebeat's official documentation](https://www.elastic.co/guide/en/beats/filebeat/7.12/multiline-examples.html#multiline) for further info. |\n| **LOG_LEVEL**          | **Default** `info`. Set log level for the collector. Allowed values are: `debug`, `info`, `warning`, `error`.                                                                                                                                                                                                                                                                                                     |\n| **INPUT_ENCODING**     | **Default** `utf-8`. Here is a full list of [valid encodings](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-container.html#_encoding) you can use.                                                                                                                                                                                                                                        |\n\n\n**Note**: By default, logs from `docker-collector-logs` and `docker-collector-metrics` containers are ignored.\n\n### 3. Check Logz.io for your logs\n\nSpin up your Docker containers if you haven’t done so already. Give your logs a few minutes to get from your system to ours, and then open [Kibana](https://app.logz.io/#/dashboard/kibana).\n\n### Change log\n\n\n- 0.4.0:\n  - Migrate to Filebeat OSS 8.5.3.\n  - Upgrade base image to python 3.12.\n- 0.3.0:\n  - Upgrade to Filebeat 8.3.3.\n  - Allow configuring encoding for input (`INPUT_ENCODING`).\n  - Remove print of version number.\n- 0.2.0:\n  - Allow changing log level.\n  - Upgrade base image to python 3.11.\n\n\n\u003cdetails\u003e\n  \u003csummary markdown=\"span\"\u003e Expand to check old versions \u003c/summary\u003e\n\n- 0.1.6:\n    Support ARM architecture.\n- 0.1.5:\n    Added rename processors.\n- 0.1.4:\n    - **BREAKING CHANGES**:\n        - Upgrade to Filebeat 7.12.1.\n    - Added multiline variables for support of multiline patterns.\n- 0.1.3: Processors were moved to root level of the yml to allow rename of filebeat metadata fields.\n- 0.1.2: Add rename processor (\"renameFields\") to specify a list of fields to rename.\n- 0.1.1: Fixed script to match Filebeat 7.9 changes.\n- 0.1.0:\n    - **BREAKING CHANGES**:\n        - Upgrade to Filebeat 7.9.0.\n        - Deprecated `LOGZIO_URL`. Use `LOGZIO_REGION` instead.\n    - Update default_filebeat.yml configuration to match newer Filebeat version.\n    - Support adding hostname.\n- 0.0.6: Updated new public SSL certificate in Docker image \u0026 Filebeat configuration.\n- 0.0.4: Added options to include or exclude lines\n- 0.0.3: Support additional fields\n- 0.0.2: Add an option to configure logzio_codec and logzio_type\n\n\u003c/details\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flogzio%2Fdocker-collector-logs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flogzio%2Fdocker-collector-logs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flogzio%2Fdocker-collector-logs/lists"}