{"id":19404938,"url":"https://github.com/samesystem/logstasher","last_synced_at":"2025-10-30T05:07:10.661Z","repository":{"id":80684193,"uuid":"179621052","full_name":"samesystem/logstasher","owner":"samesystem","description":null,"archived":false,"fork":false,"pushed_at":"2019-04-05T04:55:13.000Z","size":249,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-06-03T15:48:24.948Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/samesystem.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-04-05T04:52:12.000Z","updated_at":"2019-04-05T04:55:15.000Z","dependencies_parsed_at":null,"dependency_job_id":"3c9e0e72-4c57-4175-acf1-15af62ea42e8","html_url":"https://github.com/samesystem/logstasher","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/samesystem/logstasher","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/samesystem%2Flogstasher","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/samesystem%2Flogstasher/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/samesystem%2Flogstasher/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/samesystem%2Flogstasher/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/samesystem","download_url":"https://codeload.github.com/samesystem/logstasher/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/samesystem%2Flogstasher/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":281748716,"owners_count":26554822,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-30T02:00:06.501Z","response_time":61,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-10T11:36:43.612Z","updated_at":"2025-10-30T05:07:10.645Z","avatar_url":"https://github.com/samesystem.png","language":"Ruby","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Logstasher [![Gem Version](https://badge.fury.io/rb/logstasher.svg)](https://badge.fury.io/rb/logstasher) [![Build Status](https://travis-ci.org/shadabahmed/logstasher.svg?branch=master)](https://secure.travis-ci.org/shadabahmed/logstasher)\n### Awesome Logging for Rails !!\n\nThis gem is heavily inspired from [lograge](https://github.com/roidrage/lograge), but it's focused on one thing and one thing only. That's making your logs awesome like this:\n\n[![Awesome Logs](https://f.cloud.github.com/assets/830679/2407078/dcde03e8-aa82-11e3-85ac-8c5b3a86676e.png)](https://f.cloud.github.com/assets/830679/2407078/dcde03e8-aa82-11e3-85ac-8c5b3a86676e.png)\n\nHow it's done ?\n\nBy, using these awesome tools:\n* [Logstash](http://logstash.net) - Store and index your logs\n* [Kibana](http://kibana.org/) - for awesome visualization. This is optional though, and you can use any other visualizer\n\nUpdate: Logstash now includes Kibana build in, so no need to separately install. Logstasher has been tested with **logstash version 1.4.2** with `file input` and `json codec`.\n\nSee [quickstart](#quick-setup-for-logstash) for quickly setting up logstash\n\n## About logstasher\n\nThis gem purely focuses on how to generate logstash compatible logs i.e. *logstash json event format*,  without any overhead. Infact, logstasher logs to a separate log file named `logstash_\u003cenvironment\u003e.log`.\nThe reason for this separation:\n * To have a pure json log file\n * Prevent any logger messages(e.g. info) getting into our pure json logs\n\nBefore **logstasher** :\n\n```\nStarted GET \"/login\" for 10.109.10.135 at 2013-04-30 08:59:01 -0400\nProcessing by SessionsController#new as HTML\n[ActiveJob] [TestJob] [61d6e87a-875d-4255-9424-cab7d5ff208c] Performing TestJob from Test(default) with arguments: 0, 1\n  Rendered sessions/new.html.haml within layouts/application (4.3ms)\n  Rendered shared/_javascript.html.haml (0.6ms)\n  Rendered shared/_flashes.html.haml (0.2ms)\n  Rendered shared/_header.html.haml (52.9ms)\n  Rendered shared/_title.html.haml (0.2ms)\n  Rendered shared/_footer.html.haml (0.2ms)\nCompleted 200 OK in 532ms (Views: 62.4ms | ActiveRecord: 0.0ms | ND API: 0.0ms)\n```\n\nAfter **logstasher**:\n\n```json\n{\"job_id\":\"61d6e87a-875d-4255-9424-cab7d5ff208c\",\"queue_name\":\"Test(default)\",\"job_class\":\"ExampleJob\",\"job_args\":[1,0],\n\"exception\":[\"ZeroDivisionError\",\"divided by 0\"],\"duration\":3.07,\"request_id\":\"61d6e87a-875d-4255-9424-cab7d5ff208c\",\n\"source\":\"unknown\",\"tags\":[\"job\",\"perform\",\"exception\"],\"@timestamp\":\"2016-03-29T16:14:32.837Z\",\"@version\":\"1\"}\n\n{\"@source\":\"unknown\",\"@tags\":[\"request\"],\"@fields\":{\"method\":\"GET\",\"path\":\"/\",\"format\":\"html\",\"controller\":\"file_servers\"\n,\"action\":\"index\",\"status\":200,\"duration\":28.34,\"view\":25.96,\"db\":0.88,\"ip\":\"127.0.0.1\",\"route\":\"file_servers#index\",\n\"parameters\":\"\",\"ndapi_time\":null,\"uuid\":\"e81ecd178ed3b591099f4d489760dfb6\",\"user\":\"shadab_ahmed@abc.com\",\n\"site\":\"internal\"},\"@timestamp\":\"2013-04-30T13:00:46.354500+00:00\"}\n```\n\nBy default, the older format rails request logs are disabled, though you can enable them.\n\n## Installation\n\nIn your Gemfile:\n```ruby\ngem 'logstasher'\n```\n\n### Configure your `\u003cenvironment\u003e.rb` e.g. `development.rb`\n```ruby\n# Enable the logstasher logs for the current environment\nconfig.logstasher.enabled = true\n\n# Each of the following lines are optional. If you want to selectively disable log subscribers.\nconfig.logstasher.controller_enabled = false\nconfig.logstasher.mailer_enabled = false\nconfig.logstasher.record_enabled = false\nconfig.logstasher.view_enabled = false\nconfig.logstasher.job_enabled = false\n\n# This line is optional if you do not want to suppress app logs in your \u003cenvironment\u003e.log\nconfig.logstasher.suppress_app_log = false\n\n# This line is optional, it allows you to set a custom value for the @source field of the log event\nconfig.logstasher.source = 'your.arbitrary.source'\n\n# This line is optional if you do not want to log the backtrace of exceptions\nconfig.logstasher.backtrace = false\n\n# This line is optional, defaults to log/logstasher_\u003cenvironment\u003e.log\nconfig.logstasher.logger_path = 'log/logstasher.log'\n\n# This line is optional, loaded only if the value is truthy\nconfig.field_renaming = {\n    old_field_name =\u003e new_field_name,\n}\n\n```\n\n## Optionally use config/logstasher.yml (overrides `\u003cenvironment\u003e.rb`)\n\nHas the same optional fields as the `\u003cenvironment\u003e.rb`. You can specify common configurations that are then overriden by environment specific configurations:\n```yml\ncontroller_enabled: true\nmailer_enabled: false\nrecord_enabled: false\njob_enabled: false\nview_enabled: true\nsuppress_app_log: false\ndevelopment:\n  enabled: true\n  record_enabled: true\nproduction:\n  enabled: true\n  mailer_enabled: true\n  view_enabled: false\n```\n## Logging params hash\n\nLogstasher can be configured to log the contents of the params hash.  When enabled, the contents of the params hash (minus the ActionController internal params)\nwill be added to the log as a deep hash.  This can cause conflicts within the Elasticsearch mappings though, so should be enabled with care.  Conflicts will occur\nif different actions (or even different applications logging to the same Elasticsearch cluster) use the same params key, but with a different data type (e.g. a\nstring vs. a hash).  This can lead to lost log entries.  Enabling this can also significantly increase the size of the Elasticsearch indexes.\n\nTo enable this, add the following to your `\u003cenvironment\u003e.rb`\n```ruby\n# Enable logging of controller params\nconfig.logstasher.log_controller_parameters = true\n```\n\n## Adding custom fields to the log\n\nSince some fields are very specific to your application for e.g. *user_name*, so it is left upto you, to add them. Here's how to add those fields to the logs:\n```ruby\n# Create a file - config/initializers/logstasher.rb\n\nif LogStasher.enabled?\n  LogStasher.add_custom_fields do |fields|\n    # This block is run in application_controller context,\n    # so you have access to all controller methods\n    fields[:user] = current_user \u0026\u0026 current_user.mail\n    fields[:site] = request.path =~ /^\\/api/ ? 'api' : 'user'\n\n    # If you are using custom instrumentation, just add it to logstasher custom fields\n    LogStasher.custom_fields \u003c\u003c :myapi_runtime\n  end\n\n  LogStasher.add_custom_fields_to_request_context do |fields|\n    # This block is run in application_controller context,\n    # so you have access to all controller methods\n    # You can log custom request fields using this block\n    fields[:user] = current_user \u0026\u0026 current_user.mail\n    fields[:site] = request.path =~ /^\\/api/ ? 'api' : 'user'\n  end\nend\n```\n## Logging ActionMailer events\n\nLogstasher can easily log messages from `ActionMailer`, such as incoming/outgoing e-mails and e-mail content generation (Rails \u003e= 4.1).\nThis functionality is automatically enabled. Since the relationship between a concrete HTTP request and a mailer invocation is lost\nonce in an `ActionMailer` instance method, global (per-request) state is kept to correlate HTTP requests and events from other parts\nof rails, such as `ActionMailer`. Every time a request is invoked, a `request_id` key is added which is present on every `ActionMailer` event.\n\nNote: Since mailers are executed within the lifetime of a request, they will show up in logs prior to the actual request.\n\n## Logging ActiveJob events\n\nLogstasher can also easily log messages from `ActiveJob` (Rails \u003e= 4.2).\nThis functionality is automatically enabled. The `request_id` is set to the Job ID when the job is\nperformed, and then reverted back to its previous value once the job is complete. Imagine this\nscenario:\n\n* Web request starts (sets `request_id` to some value)\n* Job is enqueued because of the web request (the same web `request_id` is used)\n* Job is performing starts (pretend non-asynchronous adapter or perform_now was used)\n* `request_id` is set to the job id. This is important because for asynchronous jobs, there's no way\n  to remember the original `request_id`\n* Now, you can add your own detailed logging to the job, and the `request_id` can be used\n* Once the job completes, the `request_id` is reverted and other SQL and View log lines will use\n  that same old `request_id` again.\n\n## Listening to `ActiveSupport::Notifications` events\n\nIt is possible to listen to any `ActiveSupport::Notifications` events and store arbitrary data to be included in the final JSON log entry:\n```ruby\n# In config/initializers/logstasher.rb\n\n# Watch calls the block with the same arguments than any ActiveSupport::Notification, plus a store\nLogStasher.watch('some.activesupport.notification') do |name, start, finish, id, payload, store|\n  # Do something\n  store[:count] = 42\nend\n```\n\nWould change the log entry to:\n\n```json\n{\"@source\":\"unknown\",\"@tags\":[\"request\"],\"@fields\":{\"method\":\"GET\",\"path\":\"/\",\"format\":\"html\",\"controller\":\"file_servers\",\"action\":\"index\",\"status\":200,\"duration\":28.34,\"view\":25.96,\"db\":0.88,\"ip\":\"127.0.0.1\",\"route\":\"file_servers#index\", \"parameters\":\"\",\"ndapi_time\":null,\"uuid\":\"e81ecd178ed3b591099f4d489760dfb6\",\"user\":\"shadab_ahmed@abc.com\", \"site\":\"internal\",\"some.activesupport.notification\":{\"count\":42}},\"@timestamp\":\"2013-04-30T13:00:46.354500+00:00\"}\n```\n\nThe store exposed to the blocked passed to `watch` is thread-safe, and reset after each request.\nBy default, the store is only shared between occurences of the same event.\nYou can easily share the same store between different types of notifications, by assigning them to the same event group:\n\n```ruby\n# In config/initializers/logstasher.rb\n\nLogStasher.watch('foo.notification', event_group: 'notification') do |*args, store|\n  # Shared store with 'bar.notification'\nend\n\nLogStasher.watch('bar.notification', event_group: 'notification') do |*args, store|\n  # Shared store with 'foo.notification'\nend\n```\n\n## Quick Setup for Logstash\n\nFollow the instructions at [logstash documentation](https://www.elastic.co/guide/en/logstash/index.html) to setup logstash.\nStart logstash with the following command:\n```\nbin/logstash -f quickstart.conf\n```\n\n## Versions\nAll versions require Rails 3.0.x and higher and Ruby 1.9.2+. Tested on Rails 4 and Ruby 2.0\n\n## Development\n - Install dependencies:\n   export RAILS_VERSION=4.2\n   bundle install --without guard --path=${BUNDLE_PATH:-vendor/bundle}\n - Run tests - `rake`\n - Generate test coverage report - `rake coverage`. Coverage report path - coverage/index.html\n\n## Copyright\n\nCopyright (c) 2016 Shadab Ahmed, released under the MIT license\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsamesystem%2Flogstasher","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsamesystem%2Flogstasher","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsamesystem%2Flogstasher/lists"}