{"id":16746546,"url":"https://github.com/shopify/bootsnap","last_synced_at":"2025-05-12T13:06:51.149Z","repository":{"id":19350936,"uuid":"81134943","full_name":"Shopify/bootsnap","owner":"Shopify","description":"Boot large Ruby/Rails apps faster","archived":false,"fork":false,"pushed_at":"2025-05-07T08:19:45.000Z","size":3066,"stargazers_count":2695,"open_issues_count":9,"forks_count":185,"subscribers_count":437,"default_branch":"main","last_synced_at":"2025-05-12T13:06:45.473Z","etag":null,"topics":["computers","gem","performance","ruby"],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Shopify.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.txt","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2017-02-06T21:22:34.000Z","updated_at":"2025-05-12T11:33:25.000Z","dependencies_parsed_at":"2023-02-14T10:45:28.740Z","dependency_job_id":"33448367-2778-4e29-b5d8-c9c2d24a8122","html_url":"https://github.com/Shopify/bootsnap","commit_stats":{"total_commits":505,"total_committers":101,"mean_commits":5.0,"dds":0.6653465346534653,"last_synced_commit":"cae219a7aa57370951e36c010db6e8e2408543e7"},"previous_names":[],"tags_count":97,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Shopify%2Fbootsnap","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Shopify%2Fbootsnap/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Shopify%2Fbootsnap/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Shopify%2Fbootsnap/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Shopify","download_url":"https://codeload.github.com/Shopify/bootsnap/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253745151,"owners_count":21957317,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["computers","gem","performance","ruby"],"created_at":"2024-10-13T02:06:49.559Z","updated_at":"2025-05-12T13:06:51.101Z","avatar_url":"https://github.com/Shopify.png","language":"Ruby","readme":"# Bootsnap [![Actions Status](https://github.com/Shopify/bootsnap/workflows/ci/badge.svg)](https://github.com/Shopify/bootsnap/actions)\n\nBootsnap is a library that plugs into Ruby, with optional support for `YAML` and `JSON`,\nto optimize and cache expensive computations. See [How Does This Work](#how-does-this-work).\n\n#### Performance\n\n- [Discourse](https://github.com/discourse/discourse) reports a boot time reduction of approximately\n  50%, from roughly 6 to 3 seconds on one machine;\n- One of our smaller internal apps also sees a reduction of 50%, from 3.6 to 1.8 seconds;\n- The core Shopify platform -- a rather large monolithic application -- boots about 75% faster,\n  dropping from around 25s to 6.5s.\n* In Shopify core (a large app), about 25% of this gain can be attributed to `compile_cache_*`\n  features; 75% to path caching. This is fairly representative.\n\n## Usage\n\nThis gem works on macOS and Linux.\n\nAdd `bootsnap` to your `Gemfile`:\n\n```ruby\ngem 'bootsnap', require: false\n```\n\nIf you are using Rails, add this to `config/boot.rb` immediately after `require 'bundler/setup'`:\n\n```ruby\nrequire 'bootsnap/setup'\n```\n\nNote that bootsnap writes to `tmp/cache` (or the path specified by `ENV['BOOTSNAP_CACHE_DIR']`),\nand that directory *must* be writable. Rails will fail to\nboot if it is not. If this is unacceptable (e.g. you are running in a read-only container and\nunwilling to mount in a writable tmpdir), you should remove this line or wrap it in a conditional.\n\n**Note also that bootsnap will never clean up its own cache: this is left up to you. Depending on your\ndeployment strategy, you may need to periodically purge `tmp/cache/bootsnap*`. If you notice deploys\ngetting progressively slower, this is almost certainly the cause.**\n\nIt's technically possible to simply specify `gem 'bootsnap', require: 'bootsnap/setup'`, but it's\nimportant to load Bootsnap as early as possible to get maximum performance improvement.\n\nYou can see how this require works [here](https://github.com/Shopify/bootsnap/blob/main/lib/bootsnap/setup.rb).\n\nIf you are not using Rails, or if you are but want more control over things, add this to your\napplication setup immediately after `require 'bundler/setup'` (i.e. as early as possible: the sooner\nthis is loaded, the sooner it can start optimizing things)\n\n```ruby\nrequire 'bootsnap'\nenv = ENV['RAILS_ENV'] || \"development\"\nBootsnap.setup(\n  cache_dir:            'tmp/cache',          # Path to your cache\n  ignore_directories:   ['node_modules'],     # Directory names to skip.\n  development_mode:     env == 'development', # Current working environment, e.g. RACK_ENV, RAILS_ENV, etc\n  load_path_cache:      true,                 # Optimize the LOAD_PATH with a cache\n  compile_cache_iseq:   true,                 # Compile Ruby code into ISeq cache, breaks coverage reporting.\n  compile_cache_yaml:   true,                 # Compile YAML into a cache\n  compile_cache_json:   true,                 # Compile JSON into a cache\n  readonly:             true,                 # Use the caches but don't update them on miss or stale entries.\n)\n```\n\n**Protip:** You can replace `require 'bootsnap'` with `BootLib::Require.from_gem('bootsnap',\n'bootsnap')` using [this trick](https://github.com/Shopify/bootsnap/wiki/Bootlib::Require). This\nwill help optimize boot time further if you have an extremely large `$LOAD_PATH`.\n\nNote: Bootsnap and [Spring](https://github.com/rails/spring) are orthogonal tools. While Bootsnap\nspeeds up the loading of individual source files, Spring keeps a copy of a pre-booted Rails process\non hand to completely skip parts of the boot process the next time it's needed. The two tools work\nwell together.\n\n### Environment variables\n\n`require 'bootsnap/setup'` behavior can be changed using environment variables:\n\n- `BOOTSNAP_CACHE_DIR` allows to define the cache location.\n- `DISABLE_BOOTSNAP` allows to entirely disable bootsnap.\n- `DISABLE_BOOTSNAP_LOAD_PATH_CACHE` allows to disable load path caching.\n- `DISABLE_BOOTSNAP_COMPILE_CACHE` allows to disable ISeq and YAML caches.\n- `BOOTSNAP_READONLY` configure bootsnap to not update the cache on miss or stale entries.\n- `BOOTSNAP_LOG` configure bootsnap to log all caches misses to STDERR.\n- `BOOTSNAP_STATS` log hit rate statistics on exit. Can't be used if `BOOTSNAP_LOG` is enabled.\n- `BOOTSNAP_IGNORE_DIRECTORIES` a comma separated list of directories that shouldn't be scanned.\n  Useful when you have large directories of non-ruby files inside `$LOAD_PATH`.\n  It defaults to ignore any directory named `node_modules`.\n\n### Environments\n\nAll Bootsnap features are enabled in development, test, production, and all other environments according to the configuration in the setup. At Shopify, we use this gem safely in all environments without issue.\n\nIf you would like to disable any feature for a certain environment, we suggest changing the configuration to take into account the appropriate ENV var or configuration according to your needs.\n\n### Instrumentation\n\nBootsnap cache misses can be monitored though a callback:\n\n```ruby\nBootsnap.instrumentation = -\u003e(event, path) { puts \"#{event} #{path}\" }\n```\n\n`event` is either `:hit`, `:miss`, `:stale` or `:revalidated`.\nYou can also call `Bootsnap.log!` as a shortcut to log all events to STDERR.\n\nTo turn instrumentation back off you can set it to nil:\n\n```ruby\nBootsnap.instrumentation = nil\n```\n\n## How does this work?\n\nBootsnap optimizes methods to cache results of expensive computations, and can be grouped\ninto two broad categories:\n\n* [Path Pre-Scanning](#path-pre-scanning)\n    * `Kernel#require` and `Kernel#load` are modified to eliminate `$LOAD_PATH` scans.\n* [Compilation caching](#compilation-caching)\n    * `RubyVM::InstructionSequence.load_iseq` is implemented to cache the result of ruby bytecode\n      compilation.\n    * `YAML.load_file` is modified to cache the result of loading a YAML object in MessagePack format\n      (or Marshal, if the message uses types unsupported by MessagePack).\n    * `JSON.load_file` is modified to cache the result of loading a JSON object in MessagePack format\n\n### Path Pre-Scanning\n\n*(This work is a minor evolution of [bootscale](https://github.com/byroot/bootscale)).*\n\nUpon initialization of bootsnap or modification of the path (e.g. `$LOAD_PATH`),\n`Bootsnap::LoadPathCache` will fetch a list of requirable entries from a cache, or, if necessary,\nperform a full scan and cache the result.\n\nLater, when we run (e.g.) `require 'foo'`, ruby *would* iterate through every item on our\n`$LOAD_PATH` `['x', 'y', ...]`,  looking for `x/foo.rb`, `y/foo.rb`, and so on. Bootsnap instead\nlooks at all the cached requirables for each `$LOAD_PATH` entry and substitutes the full expanded\npath of the match ruby would have eventually chosen.\n\nIf you look at the syscalls generated by this behaviour, the net effect is that what would\npreviously look like this:\n\n```\nopen  x/foo.rb # (fail)\n# (imagine this with 500 $LOAD_PATH entries instead of two)\nopen  y/foo.rb # (success)\nclose y/foo.rb\nopen  y/foo.rb\n...\n```\n\nbecomes this:\n\n```\nopen y/foo.rb\n...\n```\n\nThe following diagram flowcharts the overrides that make the `*_path_cache` features work.\n\n![Flowchart explaining\nBootsnap](https://cloud.githubusercontent.com/assets/3074765/24532120/eed94e64-158b-11e7-9137-438d759b2ac8.png)\n\nBootsnap classifies path entries into two categories: stable and volatile. Volatile entries are\nscanned each time the application boots, and their caches are only valid for 30 seconds. Stable\nentries do not expire -- once their contents has been scanned, it is assumed to never change.\n\nThe only directories considered \"stable\" are things under the Ruby install prefix\n(`RbConfig::CONFIG['prefix']`, e.g. `/usr/local/ruby` or `~/.rubies/x.y.z`), and things under the\n`Gem.path` (e.g. `~/.gem/ruby/x.y.z`) or `Bundler.bundle_path`. Everything else is considered\n\"volatile\".\n\nIn addition to the [`Bootsnap::LoadPathCache::Cache`\nsource](https://github.com/Shopify/bootsnap/blob/main/lib/bootsnap/load_path_cache/cache.rb),\nthis diagram may help clarify how entry resolution works:\n\n![How path searching works](https://cloud.githubusercontent.com/assets/3074765/25388270/670b5652-299b-11e7-87fb-975647f68981.png)\n\n\nIt's also important to note how expensive `LoadError`s can be. If ruby invokes\n`require 'something'`, but that file isn't on `$LOAD_PATH`, it takes `2 *\n$LOAD_PATH.length` filesystem accesses to determine that. Bootsnap caches this\nresult too, raising a `LoadError` without touching the filesystem at all.\n\n### Compilation Caching\n\n*(A more readable implementation of this concept can be found in\n[yomikomu](https://github.com/ko1/yomikomu)).*\n\nRuby has complex grammar and parsing it is not a particularly cheap operation. Since 1.9, Ruby has\ntranslated ruby source to an internal bytecode format, which is then executed by the Ruby VM. Since\n2.3.0, Ruby [exposes an API](https://docs.ruby-lang.org/en/master/RubyVM/InstructionSequence.html) that\nallows caching that bytecode. This allows us to bypass the relatively-expensive compilation step on\nsubsequent loads of the same file.\n\nWe also noticed that we spend a lot of time loading YAML and JSON documents during our application boot, and\nthat MessagePack and Marshal are *much* faster at deserialization than YAML and JSON, even with a fast\nimplementation. We use the same strategy of compilation caching for YAML and JSON documents, with the\nequivalent of Ruby's \"bytecode\" format being a MessagePack document (or, in the case of YAML\ndocuments with types unsupported by MessagePack, a Marshal stream).\n\nThese compilation results are stored in a cache directory, with filenames generated by taking a hash\nof the full expanded path of the input file (FNV1a-64).\n\nWhereas before, the sequence of syscalls generated to `require` a file would look like:\n\n```\nopen    /c/foo.rb -\u003e m\nfstat64 m\nclose   m\nopen    /c/foo.rb -\u003e o\nfstat64 o\nfstat64 o\nread    o\nread    o\n...\nclose   o\n```\n\nWith bootsnap, we get:\n\n```\nopen      /c/foo.rb -\u003e n\nfstat64   n\nclose     n\nopen      /c/foo.rb -\u003e n\nfstat64   n\nopen      (cache) -\u003e m\nread      m\nread      m\nclose     m\nclose     n\n```\n\nThis may look worse at a glance, but underlies a large performance difference.\n\n*(The first three syscalls in both listings -- `open`, `fstat64`, `close` -- are not inherently\nuseful. [This ruby patch](https://bugs.ruby-lang.org/issues/13378) optimizes them out when coupled\nwith bootsnap.)*\n\nBootsnap writes a cache file containing a 64 byte header followed by the cache contents. The header\nis a cache key including several fields:\n\n* `version`, hardcoded in bootsnap. Essentially a schema version;\n* `ruby_platform`, A hash of `RUBY_PLATFORM` (e.g. x86_64-linux-gnu) variable.\n* `compile_option`, which changes with `RubyVM::InstructionSequence.compile_option` does;\n* `ruby_revision`, A hash of `RUBY_REVISION`, the exact version of Ruby;\n* `size`, the size of the source file;\n* `mtime`, the last-modification timestamp of the source file when it was compiled; and\n* `data_size`, the number of bytes following the header, which we need to read it into a buffer.\n\nIf the key is valid, the result is loaded from the value. Otherwise, it is regenerated and clobbers\nthe current cache.\n\n### Putting it all together\n\nImagine we have this file structure:\n\n```\n/\n├── a\n├── b\n└── c\n    └── foo.rb\n```\n\nAnd this `$LOAD_PATH`:\n\n```\n[\"/a\", \"/b\", \"/c\"]\n```\n\nWhen we call `require 'foo'` without bootsnap, Ruby would generate this sequence of syscalls:\n\n\n```\nopen    /a/foo.rb -\u003e -1\nopen    /b/foo.rb -\u003e -1\nopen    /c/foo.rb -\u003e n\nclose   n\nopen    /c/foo.rb -\u003e m\nfstat64 m\nclose   m\nopen    /c/foo.rb -\u003e o\nfstat64 o\nfstat64 o\nread    o\nread    o\n...\nclose   o\n```\n\nWith bootsnap, we get:\n\n```\nopen      /c/foo.rb -\u003e n\nfstat64   n\nclose     n\nopen      /c/foo.rb -\u003e n\nfstat64   n\nopen      (cache) -\u003e m\nread      m\nread      m\nclose     m\nclose     n\n```\n\nIf we call `require 'nope'` without bootsnap, we get:\n\n```\nopen    /a/nope.rb -\u003e -1\nopen    /b/nope.rb -\u003e -1\nopen    /c/nope.rb -\u003e -1\nopen    /a/nope.bundle -\u003e -1\nopen    /b/nope.bundle -\u003e -1\nopen    /c/nope.bundle -\u003e -1\n```\n\n...and if we call `require 'nope'` *with* bootsnap, we get...\n\n```\n# (nothing!)\n```\n\n## Precompilation\n\nIn development environments the bootsnap compilation cache is generated on the fly when source files are loaded.\nBut in production environments, such as docker images, you might need to precompile the cache.\n\nTo do so you can use the `bootsnap precompile` command.\n\nExample:\n\n```bash\n$ bundle exec bootsnap precompile --gemfile app/ lib/ config/\n```\n\n## When not to use Bootsnap\n\n*Alternative engines*: Bootsnap is pretty reliant on MRI features, and parts are disabled entirely on alternative ruby\nengines.\n\n*Non-local filesystems*: Bootsnap depends on `tmp/cache` (or whatever you set its cache directory\nto) being on a relatively fast filesystem. If you put it on a network mount, bootsnap is very likely\nto slow your application down quite a lot.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshopify%2Fbootsnap","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fshopify%2Fbootsnap","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshopify%2Fbootsnap/lists"}