{"id":25829387,"url":"https://github.com/majst01/fluent-bit-go-redis-output","last_synced_at":"2025-06-18T03:06:07.095Z","repository":{"id":64307341,"uuid":"111925631","full_name":"majst01/fluent-bit-go-redis-output","owner":"majst01","description":"Fluent-Bit go redis output plugin","archived":false,"fork":false,"pushed_at":"2022-11-14T06:58:41.000Z","size":78,"stargazers_count":22,"open_issues_count":1,"forks_count":16,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-03T08:42:56.573Z","etag":null,"topics":["cloud-native","fluentd","fluentd-bit","golang","log-forwarder","logging","logstash","plugin","redis"],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/majst01.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-11-24T14:14:14.000Z","updated_at":"2025-01-09T20:22:58.000Z","dependencies_parsed_at":"2023-01-15T11:00:23.680Z","dependency_job_id":null,"html_url":"https://github.com/majst01/fluent-bit-go-redis-output","commit_stats":null,"previous_names":[],"tags_count":10,"template":false,"template_full_name":null,"purl":"pkg:github/majst01/fluent-bit-go-redis-output","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/majst01%2Ffluent-bit-go-redis-output","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/majst01%2Ffluent-bit-go-redis-output/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/majst01%2Ffluent-bit-go-redis-output/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/majst01%2Ffluent-bit-go-redis-output/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/majst01","download_url":"https://codeload.github.com/majst01/fluent-bit-go-redis-output/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/majst01%2Ffluent-bit-go-redis-output/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":260477922,"owners_count":23015064,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cloud-native","fluentd","fluentd-bit","golang","log-forwarder","logging","logstash","plugin","redis"],"created_at":"2025-02-28T18:58:27.255Z","updated_at":"2025-06-18T03:06:02.081Z","avatar_url":"https://github.com/majst01.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# fluent-bit redis output plugin\n\n[![Build Status](https://travis-ci.org/majst01/fluent-bit-go-redis-output.svg?branch=master)](https://travis-ci.org/majst01/fluent-bit-go-redis-output)\n[![codecov](https://codecov.io/gh/majst01/fluent-bit-go-redis-output/branch/master/graph/badge.svg)](https://codecov.io/gh/majst01/fluent-bit-go-redis-output)\n[![Go Report Card](https://goreportcard.com/badge/majst01/fluent-bit-go-redis-output)](https://goreportcard.com/report/github.com/majst01/fluent-bit-go-redis-output)\n\nThis plugin is used to have redis output from fluent-bit. You can use fluent-bit redis instead of logstash in a configuration\nwhere you have a redis and optional stunnel in front of your elasticsearch infrastructure. \n\nThe configuration typically looks like:\n\n```graphviz\nfluent-bit --\u003e stunnel --\u003e redis \u003c-- logstash --\u003e elasticsearch\n```\n\nIf you have multiple elastic search servers, each covered with a redis cache in front it might look like this:\n\n```graphviz\n           /-\u003e stunnel --\u003e redis \u003c-- logstash --\u003e elasticsearch \n           |\nfluent-bit --\u003e stunnel --\u003e redis \u003c-- logstash --\u003e elasticsearch\n           |\n           \\-\u003e stunnel --\u003e redis \u003c-- logstash --\u003e elasticsearch\n```\n\n# Usage\n\n```bash\ndocker run -it --rm -v /path/to/fluent-bit.conf:/fluent-bit/etc/fluent-bit.conf majst01/fluent-bit-go-redis-output\n```\n\n## Building\n\n```bash\ndocker build --no-cache --tag fluent-bit-go-redis-output .\ndocker run -it --rm -v /path/to/fluent-bit.conf:/fluent-bit/etc/fluent-bit.conf fluent-bit-go-redis-output\n```\n\n### Configuration Options\n\n| Key           | Description                                    | Default        |\n| --------------|------------------------------------------------|----------------|\n| Hosts         | Host(s) of redis servers, whitespace separated ip/host:port | 127.0.0.1:6379 |\n| Password      | Optional redis password for all redis instances | \"\" |\n| DB            | redis database (integer)  | 0 |\n| UseTLS        | connect to redis with tls | False |\n| TlsSkipVerify | if tls is configured skip tls certificate validation for self signed certificates | True |\n| Key           | the key where to store the entries in redis | \"logstash\" |\n\n\nExample:\n\nadd this section to fluent-bit.conf\n\n```properties\n[Output]\n    Name redis\n    Match *\n    UseTLS true\n    TLSSkipVerify true\n    # if port is ommited, 6379 is used\n    Hosts 172.17.0.1 172.17.0.1:6380 172.17.0.1:6381 172.17.0.1:6382 172.17.0.1:6383\n    Password homer\n    DB 0\n    Key elastic-logstash\n```\n\n## Useful links\n\n### Redis format\n\n- [logrus-redis-hook](https://github.com/rogierlommers/logrus-redis-hook/blob/master/logrus_redis.go)\n\n### Logstash Redis Output\n\n- [logstash-redis-docu](https://github.com/logstash-plugins/logstash-output-redis/blob/master/docs/index.asciidoc)\n\n## TODO\n\n### Strategies for redis connection error handling\n\n1. crash on connection errors\n\nGiven a list of 4 Redis databases, we pick on start a random one, if during operation this fails we panic and on restart the next hopefully working is selected.\n\n1. rely on FLB_RETRY\n\nWith a list of redis databases we can create a list of pools, one pool per database and instead of doing a pool.Get(), call list.Get() with selects the next random redis database. If a failure occurs return FLB_RETRY and the library will retry.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmajst01%2Ffluent-bit-go-redis-output","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmajst01%2Ffluent-bit-go-redis-output","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmajst01%2Ffluent-bit-go-redis-output/lists"}