{"id":15033186,"url":"https://github.com/sorentwo/knuckles","last_synced_at":"2025-06-19T04:09:08.313Z","repository":{"id":32124044,"uuid":"35696590","full_name":"sorentwo/knuckles","owner":"sorentwo","description":":punch: High performance cached object serialization","archived":false,"fork":false,"pushed_at":"2016-07-09T04:16:52.000Z","size":204,"stargazers_count":67,"open_issues_count":0,"forks_count":2,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-06-12T18:46:32.222Z","etag":null,"topics":["ruby","serialization"],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sorentwo.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-05-15T21:01:37.000Z","updated_at":"2024-11-22T01:10:47.000Z","dependencies_parsed_at":"2022-09-11T19:22:51.988Z","dependency_job_id":null,"html_url":"https://github.com/sorentwo/knuckles","commit_stats":null,"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/sorentwo/knuckles","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sorentwo%2Fknuckles","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sorentwo%2Fknuckles/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sorentwo%2Fknuckles/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sorentwo%2Fknuckles/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sorentwo","download_url":"https://codeload.github.com/sorentwo/knuckles/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sorentwo%2Fknuckles/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":260684178,"owners_count":23046103,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ruby","serialization"],"created_at":"2024-09-24T20:20:20.491Z","updated_at":"2025-06-19T04:09:03.291Z","avatar_url":"https://github.com/sorentwo.png","language":"Ruby","readme":"[![Build Status](https://travis-ci.org/sorentwo/knuckles.svg?branch=master)](https://travis-ci.org/sorentwo/knuckles)\n[![Coverage Status](https://coveralls.io/repos/github/sorentwo/knuckles/badge.svg?branch=master)](https://coveralls.io/github/sorentwo/knuckles?branch=master)\n[![Code Climate](https://codeclimate.com/github/sorentwo/knuckles/badges/gpa.svg)](https://codeclimate.com/github/sorentwo/knuckles)\n[![Inline Docs](http://inch-ci.org/github/sorentwo/knuckles.svg?branch=master)](http://inch-ci.org/github/sorentwo/knuckles)\n\n# Knuckles (Because Sonic was Taken)\n\nKnuckles is a performance focused data serialization pipeline. More simply, it\ntries to serialize models into large JSON payloads as quickly as possible.\n\n### What's it all about?\n\n* Emphasis on caching as a composable operation\n* Reduced object instantiation\n* Complete instrumentation of every discrete operation\n* Entirely agnostic, can be dropped into any project and integrated over time\n* Minimal runtime dependencies\n* Explicit serializer view API with as little overhead and no DSL\n\n### Is It Better?\n\nKnuckles is absolutely faster and has a lower memory overhead than uncached\nor cached usage of `ActiveModelSerializers`, and significantly faster than\ncached use of `ActiveModelSerializers` with the [perforated][perforated] gem.\n\nHere are performance and memory comparisons for an endpoint that has been cached\nwith Perforated and with Knuckles. All measurments were done with production\nsettings over the local network.\n\n|                | average | longest | shortest | allocated | retained |\n| -------------- | ------- | ------- | -------- | --------- | -------- |\n| perforated/ams | 230ms   | 560ms   | 190ms    | 148,735   | 18,203   |\n| knuckles/ams\t | 30ms    | 60ms    | 20ms     | 19,603    | 136      |\n\nThese are measurements for a sizable payload with hundreds of associated\nrecords.\n\n[perforated]: https://github.com/sorentwo/perforated\n\n## Installation\n\nAdd this line to your application's Gemfile:\n\n```ruby\ngem \"knuckles\"\n```\n\n## Configuration\n\nThere isn't a hard dependency on `Oj` or `Readthis`, but you'll find they are\ndrastically more performant.\n\n```ruby\nrequire \"activesupport\"\nrequire \"oj\"\nrequire \"readthis\"\n\nKnuckles.configure do |config|\n  config.cache = Readthis::Cache.new(\n    marshal: Oj,\n    compress: true,\n    driver: :hiredis\n  )\n\n  config.keygen = Readthis::Expanders\n  config.serializer = Oj\nend\n```\n\nWith the top level module configured it is simple to jump right into rendering,\nbut we'll look at configuring the pipeline first.\n\n## Understanding and Using Pipelines\n\nKnuckles renders and serializes data through a series of stages composed into a\npipeline. Stages can easily be added or removed to control how data is\ntransformed. Here is a breakdown of the default stages and what their role is\nwithin the pipeline.\n\n#### Fetcher\n\nThe fetcher is responsible for bulk retrieval of data from the cache. Fetching\nis done using a single `read_multi` operation, which is multiplexed in caches\nlike Redis or MemCached.\n\n```ruby\npipeline = Knuckles::Pipeline.new\n\npipeline.call(posts)\n```\n\n#### Hydrator\n\nModels that couldn't be retrieved from the cache will then be hydrated, a\nprocess where the stripped down model that was given for fetching is replaced\nwith a full model with preloaded associations. The behavior of the hydrator\nstage is entirely controlled by passing a Proc as the `hydrate` option. If the\n`hydrate` proc is omitted hydration will be skipped. Skipping hydration is\nuseful if you want a simplified pipeline where full models and their\nassociations are preloaded before starting serialization.\n\nSee `Knuckles::Active::Hydrator` for an alternative `ActiveRecord` specific\nhydrator. If you are using Knuckles within a Rais app, this is probably the\nhydration stage you want to use.\n\n```ruby\n# Using the standard hydrator\npipeline.call(posts, hydrator: -\u003e (model) { model.fetch })\n\n# Using active hydrator with a relation that has a `prepared` scope\npipeline.call(posts, relation: posts.prepared)\n```\n\n#### Renderer\n\nAfter un-cached models have been hydrated they can be rendered. Rendering is\nsynonymous with converting a model to a hash, like calling `as_json` on an\n`ActiveRecord` model. Knuckles provides a minimal (but fast) view module that\ncan be used with the rendering step. Alternatively, if you're migrating from\n`ActiveModelSerializers` you can pass in an AMS class instead.\n\n```ruby\n# Using Knuckles::View\npipeline.call(models, view: PostView)\n\n# Using ActiveModelSerializer\npipeline.call(models, view: PostSerializer)\n```\n\n#### Writer\n\nAfter un-cached models have been serialized they are ready to be cached for\nfuture retrieval. Each fully serialized model is written to the cache in a\nsingle `write_multi` operation if available (using Readthis, for example). Only\npreviously un-cached data will be written to the cache, making the writer a\nno-op when all of the data was cached initially.\n\n#### Enhancer\n\nThe enhancer modifies rendered data using proc passed through options. The\nenhancer stage is critical to customizing the final output. For example, if\nstaff should have confidential data that regular users can't see you can enhance\nthe final values. Another use of enhancers is personalizing an otherwise generic\nresponse.\n\n```ruby\n# Removing staff only content from the rendered data\npipeline.call(posts,\n  scope: current_user,\n  enhancer: lambda do |result, options|\n    scope = options[:scope]\n\n    unless scope.staff?\n      result.delete_if { |key, _| key == \"confidential\" }\n    end\n\n    result\n  end\n)\n```\n\n#### Combiner\n\nThe combiner stage merges all of the individually rendered results into a single\nhash. The output of this stage is a single object, ready to be serialized.\n\n#### Dumper\n\nThe dumping process combines de-duplication and actual serialization. For every\ntop level key that is an array all of the children will have uniqueness\nenforced. For example, if you had rendered a collection of posts that shared the\nsame author, you will only have a single author object serialized. Be aware that\nthe uniqueness check relies on the presence of an `id` key rather than full\nobject comparisons.\n\nDumping is the final stage of the pipeline. At this point you have a single\nserialized payload in the format of your choice (JSON by default), ready to send\nback as a response.\n\n## Customizing Pipelines\n\nPipelines stages can be removed, swapped out or otherwise tuned. An array of\nstages can be passed when building a new pipeline. Here is an example of\ncreating a customized pipeline without any caching, hydration, or enhancing:\n\n```ruby\nKnuckles::Pipeline.new(stages: [\n  Knuckles::Stages::Renderer,\n  Knuckles::Stages::Combiner,\n  Knuckles::Stages::Dumper\n])\n```\n\nOr, perhaps you want to use the active hydrator instead:\n\n```ruby\nKnuckles::Pipeline.new(stages: [\n  Knuckles::Stages::Fetcher,\n  Knuckles::Active::Hydrator,\n  Knuckles::Stages::Renderer,\n  Knuckles::Stages::Writer,\n  Knuckles::Stages::Enhancer,\n  Knuckles::Stages::Combiner,\n  Knuckles::Stages::Dumper\n])\n```\n\nNote that once the pipeline is initialized the stages are frozen to prevent\nmodification.\n\n## Defining Views for Rendering\n\nWhile you can use Knuckles with other serializers, you can also use the provided\nview layer. Knuckles views are simple templates that let you build up data and\nrelations. They look like this:\n\n```ruby\nmodule ScoutView\n  extend Knuckles::View\n\n  def self.root\n    :scouts\n  end\n\n  def self.data(object, options)\n    {id: object.id, email: object.email, name: object.name}\n  end\n\n  def self.relations(object, options)\n    {things: has_many(object.things, ThingView)}\n  end\nend\n```\n\nSee `Knuckles::View` for more usage details.\n\n## Rendering in Rails\n\nOne driving factor of Knuckles is that code should be explicit. As a result\nthere isn't a default Railtie that will integrate Knuckles into the\n`ActiveController` rendering process for you. Luckily there isn't much to\nsetting up a new pipeline for rendering. Add this to your\n`ApplicationController` or an API specific controller:\n\n```ruby\ndef knuckles_render(relation, options)\n  Knuckles::Pipeline.new.call(relation, options)\nend\n```\n\nNow you can easily render responses:\n\n```ruby\ndef index\n  posts = posts.published.paginate(pagination_params)\n\n  render json: knuckles_render(\n    posts.select(:id, :updated_at),\n    relation: posts.prepared,\n    view: PostView,\n    scope: current_user,\n  )\nend\n```\n\n## Contributing\n\n1. Fork it ( https://github.com/sorentwo/knuckles/fork )\n2. Create your feature branch (`git checkout -b my-new-feature`)\n3. Commit your changes (`git commit -am 'Add some feature'`)\n4. Push to the branch (`git push origin my-new-feature`)\n5. Create a new Pull Request\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsorentwo%2Fknuckles","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsorentwo%2Fknuckles","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsorentwo%2Fknuckles/lists"}