{"id":13747427,"url":"https://github.com/dwbutler/logstash-logger","last_synced_at":"2025-05-14T20:11:06.869Z","repository":{"id":5938985,"uuid":"7159368","full_name":"dwbutler/logstash-logger","owner":"dwbutler","description":"Ruby logger that writes logstash events","archived":false,"fork":false,"pushed_at":"2024-04-22T20:48:16.000Z","size":386,"stargazers_count":456,"open_issues_count":31,"forks_count":120,"subscribers_count":10,"default_branch":"master","last_synced_at":"2025-04-20T16:55:57.692Z","etag":null,"topics":["json","logging","logging-library","logstash","redis","ruby","tcp","udp"],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dwbutler.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2012-12-14T03:18:17.000Z","updated_at":"2025-02-08T12:42:21.000Z","dependencies_parsed_at":"2024-06-18T12:24:15.830Z","dependency_job_id":"a3c02657-2de2-4c31-89d4-034c001bbea5","html_url":"https://github.com/dwbutler/logstash-logger","commit_stats":{"total_commits":314,"total_committers":32,"mean_commits":9.8125,"dds":"0.23885350318471332","last_synced_commit":"b8f5403c44150f10d15b01133f8b6d1e9eb31806"},"previous_names":[],"tags_count":43,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dwbutler%2Flogstash-logger","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dwbutler%2Flogstash-logger/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dwbutler%2Flogstash-logger/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dwbutler%2Flogstash-logger/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dwbutler","download_url":"https://codeload.github.com/dwbutler/logstash-logger/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254219374,"owners_count":22034397,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["json","logging","logging-library","logstash","redis","ruby","tcp","udp"],"created_at":"2024-08-03T06:01:28.489Z","updated_at":"2025-05-14T20:11:04.025Z","avatar_url":"https://github.com/dwbutler.png","language":"Ruby","funding_links":[],"categories":["Ruby"],"sub_categories":[],"readme":"# LogStashLogger\n[![Build Status](https://travis-ci.org/dwbutler/logstash-logger.svg?branch=master)](https://travis-ci.org/dwbutler/logstash-logger) [![Code Climate](https://codeclimate.com/github/dwbutler/logstash-logger/badges/gpa.svg)](https://codeclimate.com/github/dwbutler/logstash-logger) [![codecov.io](http://codecov.io/github/dwbutler/logstash-logger/coverage.svg?branch=master)](http://codecov.io/github/dwbutler/logstash-logger?branch=master) [![Gem Version](https://badge.fury.io/rb/logstash-logger.svg)](https://badge.fury.io/rb/logstash-logger)\n\nLogStashLogger extends Ruby's `Logger` class to log directly to\n[Logstash](https://www.elastic.co/products/logstash).\nIt supports writing to various outputs in logstash JSON format. This is an improvement over\nwriting to a file or syslog since Logstash can receive the structured data directly.\n\n## Features\n\n* Can write directly to a logstash listener over a UDP or TCP/SSL connection.\n* Can write to a file, Redis, Kafka, Kinesis, Firehose, a unix socket, syslog, stdout, or stderr.\n* Logger can take a string message, a hash, a `LogStash::Event`, an object, or a JSON string as input.\n* Events are automatically populated with message, timestamp, host, and severity.\n* Writes in logstash JSON format, but supports other formats as well.\n* Can write to multiple outputs.\n* Log messages are buffered and automatically re-sent if there is a connection problem.\n* Easily integrates with Rails via configuration.\n\n## Installation\n\nAdd this line to your application's Gemfile:\n\n    gem 'logstash-logger'\n\nAnd then execute:\n\n    $ bundle\n\nOr install it yourself as:\n\n    $ gem install logstash-logger\n\n## Usage Examples\n\n```ruby\nrequire 'logstash-logger'\n\n# Defaults to UDP on 0.0.0.0\nlogger = LogStashLogger.new(port: 5228)\n\n# Specify host and type (UDP or TCP) explicitly\nudp_logger = LogStashLogger.new(type: :udp, host: 'localhost', port: 5228)\ntcp_logger = LogStashLogger.new(type: :tcp, host: 'localhost', port: 5229)\n\n# Other types of loggers\nfile_logger = LogStashLogger.new(type: :file, path: 'log/development.log', sync: true)\nunix_logger = LogStashLogger.new(type: :unix, path: '/tmp/sock')\nsyslog_logger = LogStashLogger.new(type: :syslog)\nredis_logger = LogStashLogger.new(type: :redis)\nkafka_logger = LogStashLogger.new(type: :kafka)\nstdout_logger = LogStashLogger.new(type: :stdout)\nstderr_logger = LogStashLogger.new(type: :stderr)\nio_logger = LogStashLogger.new(type: :io, io: io)\n\n# Use a different formatter\ncee_logger = LogStashLogger.new(\n  type: :tcp,\n  host: 'logsene-receiver-syslog.sematext.com',\n  port: 514,\n  formatter: :cee_syslog\n)\n\ncustom_formatted_logger = LogStashLogger.new(\n  type: :redis,\n  formatter: MyCustomFormatter\n)\n\nlambda_formatted_logger = LogStashLogger.new(\n  type: :stdout,\n  formatter: -\u003e(severity, time, progname, msg) { \"[#{progname}] #{msg}\" }\n)\n\nruby_default_formatter_logger = LogStashLogger.new(\n  type: :file,\n  path: 'log/development.log',\n  formatter: ::Logger::Formatter\n)\n\n# Send messages to multiple outputs. Each output will have the same format.\n# Syslog cannot be an output because it requires a separate logger.\nmulti_delegating_logger = LogStashLogger.new(\n  type: :multi_delegator,\n  outputs: [\n    { type: :file, path: 'log/development.log' },\n    { type: :udp, host: 'localhost', port: 5228 }\n  ])\n\n# Balance messages between several outputs.\n# Works the same as multi delegator, but randomly chooses an output to send each message.\nbalancer_logger = LogStashLogger.new(\n  type: :balancer,\n  outputs: [\n    { type: :udp, host: 'host1', port: 5228 },\n    { type: :udp, host: 'host2', port: 5228 }\n  ])\n\n# Send messages to multiple loggers.\n# Use this if you need to send different formats to different outputs.\n# If you need to log to syslog, you must use this.\nmulti_logger = LogStashLogger.new(\n  type: :multi_logger,\n  outputs: [\n    { type: :file, path: 'log/development.log', formatter: ::Logger::Formatter },\n    { type: :tcp, host: 'localhost', port: 5228, formatter: :json }\n  ])\n\n# The following messages are written to UDP port 5228:\n\nlogger.info 'test'\n# {\"message\":\"test\",\"@timestamp\":\"2014-05-22T09:37:19.204-07:00\",\"@version\":\"1\",\"severity\":\"INFO\",\"host\":\"[hostname]\"}\n\nlogger.error '{\"message\": \"error\"}'\n# {\"message\":\"error\",\"@timestamp\":\"2014-05-22T10:10:55.877-07:00\",\"@version\":\"1\",\"severity\":\"ERROR\",\"host\":\"[hostname]\"}\n\nlogger.debug message: 'test', foo: 'bar'\n# {\"message\":\"test\",\"foo\":\"bar\",\"@timestamp\":\"2014-05-22T09:43:24.004-07:00\",\"@version\":\"1\",\"severity\":\"DEBUG\",\"host\":\"[hostname]\"}\n\nlogger.warn LogStash::Event.new(message: 'test', foo: 'bar')\n# {\"message\":\"test\",\"foo\":\"bar\",\"@timestamp\":\"2014-05-22T16:44:37.364Z\",\"@version\":\"1\",\"severity\":\"WARN\",\"host\":\"[hostname]\"}\n\n# Tagged logging\nlogger.tagged('foo') { logger.fatal('bar') }\n# {\"message\":\"bar\",\"@timestamp\":\"2014-05-26T20:35:14.685-07:00\",\"@version\":\"1\",\"severity\":\"FATAL\",\"host\":\"[hostname]\",\"tags\":[\"foo\"]}\n```\n\n## URI Configuration\nYou can use a URI to configure your logstash logger instead of a hash. This is useful in environments\nsuch as Heroku where you may want to read configuration values from the environment. The URI scheme\nis `type://host:port/path?key=value`. Some sample URI configurations are given below.\n\n```\nudp://localhost:5228\ntcp://localhost:5229\nunix:///tmp/socket\nfile:///path/to/file\nredis://localhost:6379\nkafka://localhost:9092\nstdout:/\nstderr:/\n```\n\nPass the URI into your logstash logger like so:\n\n```ruby\n# Read the URI from an environment variable\nlogger = LogStashLogger.new(uri: ENV['LOGSTASH_URI'])\n```\n\n## Logstash Listener Configuration\n\nIn order for logstash to correctly receive and parse the events, you will need to\nconfigure and run a listener that uses the `json_lines` codec. For example, to receive\nevents over UDP on port 5228:\n\n```ruby\ninput {\n  udp {\n    host =\u003e \"0.0.0.0\"\n    port =\u003e 5228\n    codec =\u003e json_lines\n  }\n}\n```\n\nFile and Redis inputs should use the `json` codec instead. For more information\nread the [Logstash docs](https://www.elastic.co/guide/en/logstash/current/plugins-codecs-json_lines.html).\n\nSee the [samples](https://github.com/dwbutler/logstash-logger/tree/master/samples) directory for more configuration samples.\n\n## SSL\n\nIf you are using TCP then there is the option of adding an SSL certificate to the options hash on initialize.\n\n```ruby\nLogStashLogger.new(type: :tcp, port: 5228, ssl_certificate: \"/path/to/certificate.crt\")\n```\n\nThe SSL certificate and key can be generated using\n\n    openssl req -x509 -batch -nodes -newkey rsa:2048 -keyout logstash.key -out logstash.crt\n\nYou can also enable SSL without a certificate:\n\n```ruby\nLogStashLogger.new(type: :tcp, port: 5228, ssl_enable: true)\n```\n\nSpecify an SSL context to have more control over the behavior. For example,\nset the verify mode:\n\n```ruby\nctx = OpenSSL::SSL::SSLContext.new\nctx.set_params(verify_mode: OpenSSL::SSL::VERIFY_NONE)\nLogStashLogger.new(type: :tcp, port: 5228, ssl_context: ctx)\n```\n\nThe following Logstash configuration is required for SSL:\n\n```ruby\ninput {\n  tcp {\n    host =\u003e \"0.0.0.0\"\n    port =\u003e 5228\n    codec =\u003e json_lines\n    ssl_enable =\u003e true\n    ssl_cert =\u003e \"/path/to/certificate.crt\"\n    ssl_key =\u003e \"/path/to/key.key\"\n  }\n}\n```\n\n### Hostname Verification\n\nHostname verification is enabled by default. Without further configuration,\nthe hostname supplied to `:host` will be used to verify the server's certificate\nidentity.\n\nIf you don't pass an `:ssl_context` or pass a falsey value to the\n`:verify_hostname` option, hostname verification will not occur.\n\n#### Examples\n\n**Verify the hostname from the `:host` option**\n\n```ruby\nctx = OpenSSL::SSL::SSLContext.new\nctx.cert = '/path/to/cert.pem'\nctx.verify_mode = OpenSSL::SSL::VERIFY_PEER\n\nLogStashLogger.new \\\n  type: :tcp,\n  host: 'logstash.example.com'\n  port: 5228,\n  ssl_context: ctx\n```\n\n**Verify a hostname different from the `:host` option**\n\n```ruby\nLogStashLogger.new \\\n  type: :tcp,\n  host: '1.2.3.4'\n  port: 5228,\n  ssl_context: ctx,\n  verify_hostname: 'server.example.com'\n```\n\n**Explicitly disable hostname verification**\n\n```ruby\nLogStashLogger.new \\\n  type: :tcp,\n  host: '1.2.3.4'\n  port: 5228,\n  ssl_context: ctx,\n  verify_hostname: false\n```\n\n## Custom Log Fields\n\n`LogStashLogger` by default will log a JSON object with the format below.\n\n```json\n{\n  \"message\":\"Some Message\",\n  \"@timestamp\":\"2015-01-29T10:43:32.196-05:00\",\n  \"@version\":\"1\",\n  \"severity\":\"INFO\",\n  \"host\":\"hostname\"\n}\n```\n\nSome applications may need to attach additional metadata to each message.\nThe `LogStash::Event` can be manipulated directly by specifying a `customize_event` block in the `LogStashLogger` configuration.\n\n```ruby\nconfig = LogStashLogger.configure do |config|\n  config.customize_event do |event|\n    event[\"other_field\"] = \"some_other_value\"\n  end\nend\n```\n\nThis configuration would result in the following output.\n\n```json\n{\n    \"message\": \"Some Message\",\n    \"@timestamp\": \"2015-01-29T10:43:32.196-05:00\",\n    \"@version\": \"1\",\n    \"severity\": \"INFO\",\n    \"host\": \"hostname\",\n    \"other_field\": \"some_other_value\"\n}\n```\n\nThis block has full access to the event, so you can remove fields, modify\nexisting fields, etc. For example, to remove the default timestamp:\n\n```ruby\nconfig = LogStashLogger.configure do |config|\n  config.customize_event do |event|\n    event.remove('@timestamp')\n  end\nend\n```\n\nYou can also customize events on a per-logger basis by passing a callable object\n(lambda or proc) to the `customize_event` option when creating a logger:\n\n```ruby\nLogStashLogger.new(customize_event: -\u003e(event){ event['other_field'] = 'other_field' })\n``` \n\n## Buffering / Automatic Retries\n\nFor devices that establish a connection to a remote service, log messages are buffered internally\nand flushed in a background thread. If there is a connection problem, the\nmessages are held in the buffer and automatically resent until it is successful.\nOutputs that support batch writing (Redis and Kafka) will write log messages in bulk from the\nbuffer. This functionality is implemented using a fork of\n[Stud::Buffer](https://github.com/jordansissel/ruby-stud/blob/master/lib/stud/buffer.rb).\nYou can configure its behavior by passing the following options to LogStashLogger:\n\n* :buffer_max_items - Max number of items to buffer before flushing. Defaults to 50.\n* :buffer_max_interval - Max number of seconds to wait between flushes. Defaults to 5.\n* :drop_messages_on_flush_error - Drop messages when there is a flush error. Defaults to false.\n* :drop_messages_on_full_buffer - Drop messages when the buffer is full. Defaults to true.\n* :sync - Flush buffer every time a message is received (blocking). Defaults to false.\n* :buffer_flush_at_exit - Flush messages when exiting the program. Defaults to true.\n* :buffer_logger - Logger to write buffer debug/error messages to. Defaults to none.\n\nYou can turn buffering off by setting `sync = true`.\n\nPlease be aware of the following caveats to this behavior:\n\n * It's possible for duplicate log messages to be sent when retrying. For outputs like Redis and\n   Kafka that write in batches, the whole batch could get re-sent. If this is a problem, you\n   can add a UUID field to each event to uniquely identify it. You can either do this\n   in a `customize_event` block, or by using logstash's\n   [UUID filter](https://www.elastic.co/guide/en/logstash/current/plugins-filters-uuid.html).\n * It's still possible to lose log messages. Ruby won't detect a TCP/UDP connection problem\n   immediately. In my testing, it took Ruby about 4 seconds to notice the receiving end was down\n   and start raising exceptions. Since logstash listeners over TCP/UDP do not acknowledge received\n   messages, it's not possible to know which log messages to re-send.\n * When `sync` is turned off, Ruby may buffer data internally before writing to\n   the IO device. This is why you may not see messages written immediately to a\n   UDP or TCP socket, even though LogStashLogger's buffer is periodically flushed.\n\n## Full Buffer\n\nBy default, messages are discarded when the buffer gets full. This can happen\nif the output source is down for too long or log messages are being received\ntoo quickly. If your application suddenly terminates (for example, by SIGKILL or a power outage),\nthe whole buffer will be lost.\n\nYou can make message loss less likely by increasing `buffer_max_items`\n(so that more events can be held in the buffer), and decreasing `buffer_max_interval` (to wait\nless time between flushes). This will increase memory pressure on your application as log messages\naccumulate in the buffer, so make sure you have allocated enough memory to your process.\n\nIf you don't want to lose messages when the buffer gets full, you can set\n`drop_messages_on_full_buffer = false`. Note that if the buffer gets full, any\nincoming log message will block, which could be undesirable.\n\n## Sync Mode\n\nAll logger outputs support a `sync` setting. This is analogous to the \"sync mode\" setting on Ruby IO\nobjects. When set to `true`, output is immediately flushed and is not buffered internally. Normally,\nfor devices that connect to a remote service, buffering is a good thing because\nit improves performance and reduces the likelihood of errors affecting the program. For these devices,\n`sync` defaults to `false`, and it is recommended to leave the default value.\nYou may want to turn sync mode on for testing, for example if you want to see\nlog messages immediately after they are written.\n\nIt is recommended to turn sync mode on for file and Unix socket outputs. This\nensures that log messages from different threads or proceses are written correctly on separate lines.\n\nSee [#44](https://github.com/dwbutler/logstash-logger/issues/44) for more details.\n\n## Error Handling\n\nIf an exception occurs while writing a message to the device, the exception is\nlogged using an internal logger. By default, this logs to $stderr. You can\nchange the error logger by setting `LogStashLogger.configuration.default_error_logger`, or by passsing\nyour own logger object in the `:error_logger` configuration key when\ninstantiating a LogStashLogger.\n\n## Logger Silencing\n\nLogStashLogger provides support for Rails-style logger silencing. The\nimplementation was extracted from Rails, but has no dependencies, so it can be\nused outside of a Rails app. The interface is the same as in Rails:\n\n```ruby\nlogger.silence(temporary_level) do\n  ...\nend\n```\n\n## Custom Logger Class\n\nBy default, LogStashLogger creates a logger that extends Ruby's built in `Logger` class.\nIf you require a different logger implementation, you can use a different class\nby passing in the class with the `logger_class` option.\n\nNote that for syslog, the `Syslog::Logger` class is required and cannot be\nchanged.\n\n## Rails Integration\n\nSupports Rails 4.2 and 5.x.\n\nBy default, every Rails log message will be written to logstash in `LogStash::Event` JSON format.\n\nFor minimal, more-structured logstash events, try one of the following gems:\n\n* [lograge](https://github.com/roidrage/lograge)\n* [yarder](https://github.com/rurounijones/yarder)\n\nCurrently these gems output a JSON string, which LogStashLogger then parses.\nFuture versions of these gems could potentially have deeper integration with LogStashLogger\n(e.g. by directly writing `LogStash::Event` objects).\n\n### Rails Configuration\n\nAdd the following to your `config/environments/production.rb`:\n\n#### Common Options\n\n```ruby\n# Optional, Rails sets the default to :info\nconfig.log_level = :debug\n\n# Optional, Rails 4 defaults to true in development and false in production\nconfig.autoflush_log = true\n\n# Optional, use a URI to configure. Useful on Heroku\nconfig.logstash.uri = ENV['LOGSTASH_URI']\n\n# Optional. Defaults to :json_lines. If there are multiple outputs,\n# they will all share the same formatter.\nconfig.logstash.formatter = :json_lines\n\n# Optional, the logger to log writing errors to. Defaults to logging to $stderr\nconfig.logstash.error_logger = Logger.new($stderr)\n\n# Optional, max number of items to buffer before flushing. Defaults to 50\nconfig.logstash.buffer_max_items = 50\n\n# Optional, max number of seconds to wait between flushes. Defaults to 5\nconfig.logstash.buffer_max_interval = 5\n\n# Optional, drop message when a connection error occurs. Defaults to false\nconfig.logstash.drop_messages_on_flush_error = false\n\n# Optional, drop messages when the buffer is full. Defaults to true\nconfig.logstash.drop_messages_on_full_buffer = true\n```\n\n#### UDP\n```ruby\n# Optional, defaults to '0.0.0.0'\nconfig.logstash.host = 'localhost'\n\n# Optional, defaults to :udp.\nconfig.logstash.type = :udp\n\n# Required, the port to connect to\nconfig.logstash.port = 5228\n```\n\n#### TCP\n\n```ruby\n# Optional, defaults to '0.0.0.0'\nconfig.logstash.host = 'localhost'\n\n# Required, the port to connect to\nconfig.logstash.port = 5228\n\n# Required\nconfig.logstash.type = :tcp\n\n# Optional, enables SSL\nconfig.logstash.ssl_enable = true\n```\n\n#### Unix Socket\n\n```ruby\n# Required\nconfig.logstash.type = :unix\n\n# Required\nconfig.logstash.path = '/tmp/sock'\n```\n\n#### Syslog\n\nIf you're on Ruby 1.9, add `Syslog::Logger` v2 to your Gemfile:\n\n    gem 'SyslogLogger', '2.0'\n\nIf you're on Ruby 2+, `Syslog::Logger` is already built into the standard library.\n\n```ruby\n# Required\nconfig.logstash.type = :syslog\n\n# Optional. Defaults to 'ruby'\nconfig.logstash.program_name = 'MyApp'\n\n# Optional default facility level. Only works in Ruby 2+\nconfig.logstash.facility = Syslog::LOG_LOCAL0\n```\n\n#### Redis\n\nAdd the redis gem to your Gemfile:\n\n    gem 'redis'\n\n```ruby\n# Required\nconfig.logstash.type = :redis\n\n# Optional, will default to the 'logstash' list\nconfig.logstash.list = 'logstash'\n\n# All other options are passed in to the Redis client\n# Supported options include host, port, path, password, url\n# Example:\n\n# Optional, Redis will default to localhost\nconfig.logstash.host = 'localhost'\n\n# Optional, Redis will default to port 6379\nconfig.logstash.port = 6379\n```\n\n#### Kafka\n\nAdd the poseidon gem to your Gemfile:\n\n    gem 'poseidon'\n\n```ruby\n# Required\nconfig.logstash.type = :kafka\n\n# Optional, will default to the 'logstash' topic\nconfig.logstash.path = 'logstash'\n\n# Optional, will default to the 'logstash-logger' producer\nconfig.logstash.producer = 'logstash-logger'\n\n# Optional, will default to localhost:9092 host/port\nconfig.logstash.hosts = ['localhost:9092']\n\n# Optional, will default to 1s backoff\nconfig.logstash.backoff = 1\n\n```\n\n#### Kinesis\n\nAdd the aws-sdk gem to your Gemfile:\n\n    # aws-sdk \u003e= 3.0\n    gem 'aws-sdk-kinesis'\n\n    # aws-sdk \u003c 3.0\n    gem 'aws-sdk'\n\n```ruby\n# Required\nconfig.logstash.type = :kinesis\n\n# Optional, will default to the 'logstash' stream\nconfig.logstash.stream = 'my-stream-name'\n\n# Optional, will default to 'us-east-1'\nconfig.logstash.aws_region = 'us-west-2'\n\n# Optional, will default to the AWS_ACCESS_KEY_ID environment variable\nconfig.logstash.aws_access_key_id = 'ASKASKHLD12341'\n\n# Optional, will default to the AWS_SECRET_ACCESS_KEY environment variable\nconfig.logstash.aws_secret_access_key = 'ASKASKHLD1234123412341234'\n\n```\n\n#### Firehose\n\nAdd the aws-sdk gem to your Gemfile:\n\n    # aws-sdk \u003e= 3.0\n    gem 'aws-sdk-firehose'\n\n    # aws-sdk \u003c 3.0\n    gem 'aws-sdk'\n\n```ruby\n# Required\nconfig.logstash.type = :firehose\n\n# Optional, will default to the 'logstash' delivery stream\nconfig.logstash.stream = 'my-stream-name'\n\n# Optional, will default to AWS default region config chain\nconfig.logstash.aws_region = 'us-west-2'\n\n# Optional, will default to AWS default credential provider chain\nconfig.logstash.aws_access_key_id = 'ASKASKHLD12341'\n\n# Optional, will default to AWS default credential provider chain\nconfig.logstash.aws_secret_access_key = 'ASKASKHLD1234123412341234'\n\n```\n\n#### File\n\n```ruby\n# Required\nconfig.logstash.type = :file\n\n# Optional, defaults to Rails log path\nconfig.logstash.path = 'log/production.log'\n```\n\n#### IO\n\n```ruby\n# Required\nconfig.logstash.type = :io\n\n# Required\nconfig.logstash.io = io\n```\n\n#### Multi Delegator\n\n```ruby\n# Required\nconfig.logstash.type = :multi_delegator\n\n# Required\nconfig.logstash.outputs = [\n  {\n    type: :file,\n    path: 'log/production.log'\n  },\n  {\n    type: :udp,\n    port: 5228,\n    host: 'localhost'\n  }\n]\n```\n\n#### Multi Logger\n\n```ruby\n# Required\nconfig.logstash.type = :multi_logger\n\n# Required. Each logger may have its own formatter.\nconfig.logstash.outputs = [\n  {\n    type: :file,\n    path: 'log/production.log',\n    formatter: ::Logger::Formatter\n  },\n  {\n    type: :udp,\n    port: 5228,\n    host: 'localhost'\n  }\n]\n```\n\n### Logging HTTP request data\n\nIn web applications, you can log data from HTTP requests (such as headers) using the\n[RequestStore](https://github.com/steveklabnik/request_store) middleware. The following\nexample assumes Rails.\n\n```ruby\n# in Gemfile\ngem 'request_store'\n```\n\n```ruby\n# in application.rb\nLogStashLogger.configure do |config|\n  config.customize_event do |event|\n    event[\"session_id\"] = RequestStore.store[:load_balancer_session_id]\n  end\nend\n```\n\n```ruby\n# in app/controllers/application_controller.rb\nbefore_filter :track_load_balancer_session_id\n\ndef track_load_balancer_session_id\n  RequestStore.store[:load_balancer_session_id] = request.headers[\"X-LOADBALANCER-SESSIONID\"]\nend\n```\n\n## Cleaning up resources when forking\n\nIf your application forks (as is common with many web servers) you will need to\nmanage cleaning up resources on your LogStashLogger instances. The instance method\n`#reset` is available for this purpose. Here is sample configuration for\nseveral common web servers used with Rails:\n\nPassenger:\n```ruby\n::PhusionPassenger.on_event(:starting_worker_process) do |forked|\n  Rails.logger.reset\nend\n```\n\nPuma:\n```ruby\n# In config/puma.rb\non_worker_boot do\n  Rails.logger.reset\nend\n```\n\nUnicorn\n```ruby\n# In config/unicorn.rb\nafter_fork do |server, worker|\n  Rails.logger.reset\nend\n```\n\n## Ruby Compatibility\n\nVerified to work with:\n\n* MRI Ruby 2.2 - 2.5\n* JRuby 9.x\n* Rubinius\n\nRuby versions \u003c 2.2 are EOL'ed and no longer supported.\n\n## What type of logger should I use?\n\nIt depends on your specific needs, but most applications should use the default (UDP). Here are the advantages and\ndisadvantages of each type:\n\n* UDP is faster than TCP because it's asynchronous (fire-and-forget). However, this means that log messages could get dropped.\n  This is okay for many applications.\n* TCP verifies that every message has been received via two-way communication. It also supports SSL for secure transmission\n  of log messages over a network. This could slow your app down to a crawl if the TCP listener is under heavy load.\n* A file is simple to use, but you will have to worry about log rotation and running out of disk space.\n* Writing to a Unix socket is faster than writing to a TCP or UDP port, but only works locally.\n* Writing to Redis is good for distributed setups that generate tons of logs. However, you will have another moving part and\n  have to worry about Redis running out of memory.\n* Writing to stdout is only recommended for debugging purposes.\n\nFor a more detailed discussion of UDP vs TCP, I recommend reading this article:\n[UDP vs. TCP](http://gafferongames.com/networking-for-game-programmers/udp-vs-tcp/)\n\n## Troubleshooting\n\n### Logstash never receives any logs\nIf you are using a device backed by a Ruby IO object (such as a file, UDP socket, or TCP socket), please be aware that Ruby\nkeeps its own internal buffer. Despite the fact that LogStashLogger buffers\nmessages and flushes them periodically, the data written to the IO object can\nbe buffered by Ruby internally indefinitely, and may not even write until the\nprogram terminates. If this bothers you or you need to see log messages\nimmediately, your only recourse is to set the `sync: true` option.\n\n### JSON::GeneratorError\nYour application is probably attempting to log data that is not encoded in a valid way. When this happens, Ruby's\nstandard JSON library will raise an exception. You may be able to overcome this by swapping out a different JSON encoder\nsuch as Oj. Use the [oj_mimic_json](https://github.com/ohler55/oj_mimic_json) gem to use Oj for JSON generation.\n\n### No logs getting sent on Heroku\nHeroku recommends installing the [rails_12factor](https://github.com/heroku/rails_12factor) so that logs get sent to STDOUT.\nUnfortunately, this overrides LogStashLogger, preventing logs from being sent to their configured destination. The solution\nis to remove `rails_12factor` from your Gemfile.\n\n### Logging eventually stops in production\nThis is most likely not a problem with LogStashLogger, but rather a different gem changing the log level of `Rails.logger`.\nThis is especially likely if you're using a threaded server such as Puma, since gems often change the log level of\n`Rails.logger` in a non thread-safe way. See [#17](https://github.com/dwbutler/logstash-logger/issues/17) for more information.\n\n### Sometimes two lines of JSON log messages get sent as one message\nIf you're using UDP output and writing to a logstash listener, you are most likely encountering a bug in the UDP implementation\nof the logstash listener. There is no known fix at this time. See [#43](https://github.com/dwbutler/logstash-logger/issues/43)\nfor more information.\n\n### Errno::EMSGSIZE - Message too long\nA known drawback of using TCP or UDP is the 65535 byte limit on total message size. To workaround\nthis issue, you will have to truncate the message by setting the max message size:\n\n```ruby\nLogStashLogger.configure do |config|\n  config.max_message_size = 2000\nend\n```\n\nThis will truncate only the `message` field of the LogStash Event. So make sure\nyou set the max message size significantly less than 65535 bytes to make room\nfor other fields.\n\n## Breaking changes\n\n### Version 0.25+\n\nRails 3.2, MRI Ruby \u003c 2.2, and JRuby 1.7 are no longer supported, since they have been\nEOL'ed. If you are on an older version of Ruby, you will need to use 0.24 or below.\n\n### Version 0.5+\n * The `source` event key has been replaced with `host` to better match the latest logstash.\n * The `(host, port, type)` constructor has been deprecated in favor of an options hash constructor.\n\n### Version 0.4+\n`LogStash::Event` uses the v1 format starting version 1.2+. If you're using the v1, you'll need to install\nLogStashLogger version 0.4+. This is not backwards compatible with the old `LogStash::Event` v1.1.5, which uses\nthe v0 format.\n\n### Version 0.3+\nEarlier versions of this gem (\u003c= 0.2.1) only implemented a TCP connection.\nNewer versions (\u003e= 0.3) also implement UDP, and use that as the new default.\nPlease be aware if you are using the default constructor and still require TCP, you should add an additional argument:\n\n```ruby\n# Now defaults to UDP instead of TCP\nlogger = LogStashLogger.new('localhost', 5228)\n# Explicitly specify TCP instead of UDP\nlogger = LogStashLogger.new('localhost', 5228, :tcp)\n```\n\n## Contributors\n* [David Butler](https://github.com/dwbutler)\n* [pctj101](https://github.com/pctj101)\n* [Gary Rennie](https://github.com/Gazler)\n* [Nick Ethier](https://github.com/nickethier)\n* [Arron Mabrey](https://github.com/arronmabrey)\n* [Jan Schulte](https://github.com/schultyy)\n* [Kurt Preston](https://github.com/KurtPreston)\n* [Chris Blatchley](https://github.com/chrisblatchley)\n* [Felix Bechstein](https://github.com/felixb)\n* [Vadim Kazakov](https://github.com/yads)\n* [Anil Rhemtulla](https://github.com/AnilRh)\n* [Nikita Vorobei](https://github.com/Nikita-V)\n* [fireboy1919](https://github.com/fireboy1919)\n* [Mike Gunderloy](https://github.com/ffmike)\n* [Vitaly Gorodetsky](https://github.com/vitalis)\n* [Courtland Caldwell](https://github.com/caldwecr)\n* [Bibek Shrestha](https://github.com/bibstha)\n* [Alex Ianus](https://github.com/aianus)\n* [Craig Read](https://github.com/Catharz)\n* [glaszig](https://github.com/glaszig)\n* [Bin Lan](https://github.com/lanxx019)\n* [Joao Fernandes](https://github.com/jcmfernandes)\n* [CoolElvis](https://github.com/coolelvis)\n* [Sergey Pyankov](https://github.com/esergion)\n* [Alec Hoey](https://github.com/alechoey)\n* [Alexey Krasnoperov](https://github.com/AlexeyKrasnoperov)\n* [Gabriel de Oliveira](https://github.com/gdeoliveira)\n* [Vladislav Syabruk](https://github.com/SeTeM)\n* [Matus Vacula](https://github.com/matus-vacula)\n\n## Contributing\n\n1. Fork it\n2. Create your feature branch (`git checkout -b my-new-feature`)\n3. Commit your changes (`git commit -am 'Add some feature'`)\n4. Push to the branch (`git push origin my-new-feature`)\n5. Create new Pull Request\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdwbutler%2Flogstash-logger","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdwbutler%2Flogstash-logger","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdwbutler%2Flogstash-logger/lists"}