Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/kineticcafe/marlowe

Marlowe makes it easier to trace a request through all of your application logs
https://github.com/kineticcafe/marlowe

correlation-id rack rails ruby

Last synced: about 1 month ago
JSON representation

Marlowe makes it easier to trace a request through all of your application logs

Awesome Lists containing this project

README 

        
= Marlowe, Your Request Sleuth



code :: https://github.com/KineticCafe/marlowe/

issues :: https://github.com/KineticCafe/marlowe/issues

docs :: http://www.rubydoc.info/github/KineticCafe/marlowe/master

continuous integration :: {Build Status}[https://travis-ci.org/KineticCafe/marlowe]



== Description



{Marlowe}[https://github.com/KineticCafe/marlowe] is a Rack middleware that

extracts or creates a request ID using a pre-defined header, permitting request

correlation across multiple services.



When using Rails, Marlowe automatically adds itself to the middleware before

Rails::Rack::Logger.



As of Marlowe 3.0, a Faraday middleware is provided (require 'marlowe/faraday').



=== Upgrading



==== from Marlowe 2.0



In Marlowe 2.0, configuration was entirely specified in Rails configuration or

in the Rack +use+ clause. With the addition of a Faraday middleware for use with

Marlowe, centralization of the configuration is advisable.



The existing configuration mechanisms will work (and create instance

configurations if provided), but it is instead recommended to use the global

configuration:



  # Compatibility with Marlowe 1.0

  Marlowe.configure do |config|

    config.header = 'Correlation-ID'

    config.handler = :simple

    config.return = false

  end



Configuration is static for Marlowe::Middleware or Marlowe::Faraday instances

and changes to Marlowe configuration only affect *new* instances. If the

configuration options are provided in the middleware +use+ clause, they will

override the configured values.



  Marlowe.configure do |config|

    config.header = 'Correlation-ID'

    config.handler = :simple

    config.return = false

  end

  use Marlowe::Middleware, handler: :clean

  # The handler will be the :clean handler for the used middleware.



==== from Marlowe 1.x



In Marlowe 1.0, the correlation header was called Correlation-Id;

since then, Rails 5 and other tools and frameworks (such as Phoenix) have

standardized on the header X-Request-Id. Marlowe 2.0 changes to this

header by default.



To keep complete compatibility with Marlowe 1.0, the following should be used:



  # Rails

  config.marlowe_header = 'Correlation-Id'

  config.marlowe_handler = :simple

  config.marlowe_return = false



  # Rack

  use Marlowe::Middleware, header: 'Correlation-Id', handler: :simple, return: false



== Configuration



Marlowe has three main configuration options: the request ID header, the

request ID handler, and the request ID return. The options may be provided to

the Rack +use+ command as a keyword option, or set in a corresponding

marlowe_option configuration variable in Rails.



=== Request ID Header



Specifies the header to be used for the request correlation ID. Defaults to

X-Request-Id.



    # Rails

    config.marlowe_header = 'Correlation-Id'

    # OR: config.marlowe_correlation_header = 'Correlation-Id'



    # Rack

    use Marlowe::Middleware, header: 'Correlation-Id'

    # OR: use Marlowe::Middleware, correlation_header: 'Correlation-Id'



Marlowe will convert this to an appropriate HTTP header (in the Rack +env+

parameter, the above header would be represented as

env['HTTP_CORRELATION_ID']).



=== Request ID Handler



Specifies the method for sanitizing or generating the request correlation ID.

Values can be :clean (the default, which limits incoming correlation

IDs to 255 alphanumeric-or-dash characters), :simple (does not limit

incoming correlation IDs), or a proc to transform or generate a correlation ID.



In all cases, if a correlation request ID is not handled, a UUID will be

generated.



    # Rails

    config.marlowe_handler = :simple

    config.marlowe_handler = ->(req_id) {

      req_id.try(:reverse) || SecureRandom.uuid

    }



    # Rack

    use Marlowe::Middleware, handler: :simple

    use Marlowe::Middleware, handler: ->(req_id) {

      req_id.try(:reverse) || SecureRandom.uuid

    }



=== Request ID Return



If +true+ (the default), the request correlation ID will be returned to the

client in the same header that it was provided in.



    # Rails

    config.marlowe_return = false



    # Rack

    use Marlowe::Middleware, return: false



=== Using Marlowe with Rails 5



Rails 5 includes the ActionDispatch::RequestId middleware, reducing

the need for Marlowe. Marlowe is more configurable than the Rails 5 default, so

set +marlowe_replace_action_dispatch_request_id+ to true to have

Marlowe::Middleware will replace ActionDispatch::RequestId:



  # Rails only

  config.marlowe_replace_action_dispatch_request_id = true



== Accessing the Correlation ID



The correlation id can be accessed throughout the application by accessing the

{RequestStore}[https://github.com/steveklabnik/request_store] storage.



  RequestStore[:correlation_id]



== Logging



For a Rails application, you simply need to change the log formatter to one of

the provided ones. Correlated versions of both the SimpleFormatter and

Formatter are included.



  # config/environments/development.rb

  Rails.application.configure do

    config.log_formatter = Marlowe::SimpleFormatter.new

  end



To create your own formatter, you'll need to access the RequestStore storage.

You can use this pattern if you've rolled your own logger/formatter:



  # lib/correlated_formatter.rb

  require 'request_store'



  class CorrelatedSimpleFormatter < ActiveSupport::Logger::SimpleFormatter

    def call(severity, timestamp, progname, msg)

      "[#{RequestStore.store[:correlation_id]}] #{super}"

    end

  end



=== Lograge



As {lograge}[https://github.com/roidrage/lograge] supplies its own formatter,

you will need to do something a little different:



  # config/application.rb



  class Application < Rails::Application

    config.before_initialize do

      ...

      # use lograge for all request logs

      config.lograge.enabled = true

      config.lograge.custom_options = lambda do |event|

        { correlation_id: RequestStore[:correlation_id] }

      end

    end

  end



=== SemanticLogger



As {semantic_logger}[https://github.com/rocketjob/semantic_logger] provides its

own formatters this should be added as a tag. The best way that I can see to do

this is to capture the +correlation_id+ in an +on_log+ event:



  # config/initializers/semantic_logger.rb



  SemanticLogger.on_log do |log|

    if RequestStore[:correlation_id]

      log.named_tags[:correlation_id] = RequestStore[:correlation_id]

    end

  end



== Clients



Catching and creating the correlation ID is a great all on its own, but to

really take advantage of the correlation in a service based architecture you'll

need to pass the request ID to the next service in the change.



Here's an example with {Faraday}[https://github.com/lostisland/faraday]:



    require 'faraday'

    require 'faraday_middleware'

    require 'marlowe/faraday'



    conn = Faraday.new(url: 'https://example.org/') do |conn|

      conn.request :marlowe

      conn.request :json



      conn.response :json

      conn.adapter Faraday.default_adapter

    end



== Install



Add Marlowe to your Gemfile:



    gem 'marlowe', '~> 2.0'



Or manually install:



  $ gem install marlowe



== Marlowe Semantic Versioning



Marlowe uses a {Semantic Versioning}[http://semver.org/] scheme with one

significant change:



* When PATCH is zero (+0+), it will be omitted from version references.



Additionally, the major version will generally be reserved for plug-in

infrastructure changes.



== Community and Contributing



Marlowe welcomes your contributions as described in

{Contributing.md}[https://github.com/KineticCafe/marlowe/blob/master/Contributing.md].

This project, like all Kinetic Cafe {open source

projects}[https://github.com/KineticCafe], is under the Kinetic Cafe Open

Source {Code of Conduct}[https://github.com/KineticCafe/code-of-conduct].