{"id":13587332,"url":"https://github.com/dimo414/bash-cache","last_synced_at":"2025-03-16T22:30:30.876Z","repository":{"id":62722699,"uuid":"234518385","full_name":"dimo414/bash-cache","owner":"dimo414","description":"Transparent caching layer for bash functions; particularly useful for functions invoked as part of your prompt.","archived":false,"fork":false,"pushed_at":"2023-01-05T01:58:35.000Z","size":91,"stargazers_count":77,"open_issues_count":8,"forks_count":4,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-03-16T06:11:21.104Z","etag":null,"topics":["bash","cache","decorator","memoization","profilegem","prompt","shell","terminal"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dimo414.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-01-17T09:42:58.000Z","updated_at":"2025-03-15T15:43:56.000Z","dependencies_parsed_at":"2023-02-03T04:55:11.338Z","dependency_job_id":null,"html_url":"https://github.com/dimo414/bash-cache","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dimo414%2Fbash-cache","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dimo414%2Fbash-cache/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dimo414%2Fbash-cache/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dimo414%2Fbash-cache/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dimo414","download_url":"https://codeload.github.com/dimo414/bash-cache/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243944287,"owners_count":20372787,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bash","cache","decorator","memoization","profilegem","prompt","shell","terminal"],"created_at":"2024-08-01T15:06:09.736Z","updated_at":"2025-03-16T22:30:30.624Z","avatar_url":"https://github.com/dimo414.png","language":"Shell","funding_links":[],"categories":["Shell","\u003ca name=\"utility\"\u003e\u003c/a\u003eUtilities"],"sub_categories":[],"readme":"# Bash Cache\n\nBash Cache provides a transparent mechanism for caching, or memoizing, long-running Bash functions.\nAlthough it can be used for scripting its motivating purpose is to cache the results of expensive\ncommands for display in your terminal prompt.\n\nOriginally part of [ProfileGem](http://git.mwdiamond.com/profilegem) and\n[prompt.gem](http://git.mwdiamond.com/prompt.gem), this functionality has been pulled out\ninto a standalone utility.\n\nThis library has also inspired [`bkt`](http://git.mwdiamond.com/bkt), a standalone binary for\ncaching subprocess invocations. If bash-cache doesn't fit your use case see if `bkt` does.\n\n## Installation\n\nSimply `source bash-cache.sh` into your script or shell.\n\n## Usage\n\n```\nbc::cache FUNCTION TTL REFRESH [ENV_VARS ...]\n```\n\nTo cache a function pass its name to `bc::cache`, along with the amount of time the cached results\nshould persist. This function [decorates](https://en.wikipedia.org/wiki/Decorator_pattern) an\nexisting Bash function, wrapping it with a caching layer that temporarily retains the output and\nexit status of the backing function.\n\nBy default the cache is keyed off the function arguments (meaning `some_func`, `some_func bar`, and\n`some_func baz` are each cached separately).\n\nCached data **is shared across processes** by default; see below for ways to change this.\n\nSome example usages can be seen in the\n[prompt.gem project](https://github.com/dimo414/prompt.gem/blob/master/env_functions.sh).\n\n### Fidelity\n\nThere are many existing command-caching utilities and patterns in the wild, however the cached\nbehavior is typically incomplete (often only caching stdout). bash-cache strives to provide\nhigh-fidelity caching, such that cached results are as close to indiscernable as possible.\n\n* stdout and stderr are both cached, and output separately to stdout and stderr respectively\n* output is lossless; many implementations can't handle trailing whitespace or `nul` bytes\n* exit status code is preserved\n* positional arguments are respected; naive implementations may conflate `foo bar baz` (two args)\n  and `foo 'bar baz'` (one arg with whitespace)\n\n### Cache durations\n\nEach cached result is associated with two durations; the *TTL* deadline and the *refresh* deadline.\nDurations can be specified in (s)econds, (m)inutes, (h)ours, and (d)ays, for example `30s`, `1d`,\nor `1h24m5s`. \n\n* Once a cached result exceeds its TTL it is eligible for cleanup, and will shortly be removed.\n  Note that until it is cleaned up the cached data may still be returned from the cache.\n  `1m` is a recommended TTL duration for functions that will be surfaced in a prompt.\n* If a cached result exceeds its refresh deadline it will be asynchronously updated when the\n  function is invoked. The cached data will continue to be used until the refresh completes.\n  `10s` is a recommended refresh duration for functions that will be surfaced in a prompt.\n\n### Customizing the cache key\n\nIf your function depends on additional state, such as the current working directory, you'll want to\nensure the cache is keyed off that state, in addition to the function's arguments. To do so pass\nany relevant environment variable names to `bc::cache` after the function name.\n\n* `PWD` is often used in order to cache a function based on the current working directory.\n* `$` is less common, but can be used to isolate a function's cache to the current process. Note\n  you'll need to single-quote this argument (`'$'`).\n\n### Example usage\n\nYou can invoke `bc::cache` at any time, however you're encouraged to do so immediately following\nthe function definition as a form of self-documentation, similar to\n[Python's `@decorator` notation](https://en.wikipedia.org/wiki/Python_syntax_and_semantics#Decorators):\n\n```shell\nmy_expensive_function() {\n  ...\n} \u0026\u0026 bc::cache my_expensive_function 1m 10s PWD\n```\n\nNotice in this example `PWD` is specified, meaning the cache will key off the current working\ndirectory in addition to any arguments to the function.\n\n### Performance\n\nCached data is stored on-disk, which means accessing the cache will typically be *much* slower than\ndirectly executing many simple commands. Generally speaking, operations which benefit from caching\nare accessing the disk themselves or doing network I/O. You should benchmark your functions with and\nwithout caching (see `bc::benchmark`) to ensure you see a meaningful improvement before deciding to\ncache a particular function.\n\nCaching performance can differ drastically across machines. Notably, if the cache directory (under\n`/tmp` or `TMPDIR` by default) is on a [`tmpfs`](https://en.wikipedia.org/wiki/Tmpfs) partition or a\nsolid-state drive performance will be significantly better than caching to a spinning disk.\n\n### Calling the original function\n\nThe original function is renamed to `bc::orig::[FUNCTION_NAME]` (e.g.\n`bc::orig::my_expensive_function`). This can be used to bypass caching if needed.\n\n### Manually refreshing the cache\n\nTwo other functions, `bc::warm::[FUNCTION_NAME]` and `bc::force::[FUNCTION_NAME]`, are provided to\nupdate the cache on demand. Both unconditionally execute and cache the backing function, but\ndiffer in how they are intended to be used.\n\n`bc::warm::[FUNCTION_NAME]` refreshes a cached invocation asynchronously and silently, returning\ncontrol to the caller immediately. This can be used to ensure invocations are freshly cached\nbefore the output is needed. For example, prompt.gem can\n[warm cached functions displayed in the prompt](https://github.com/dimo414/prompt.gem/blob/7e6b5d2a7d773d5aa44880c5425a4b12c1dc066f/callback_functions.sh#L61)\nbefore the prompt is actually constructed, allowing the functions to be refreshed concurrently\nrather than one at a time.\n\nBy contrast `bc::force::[FUNCTION_NAME]` forces a synchronous cache refresh, blocking until the\nbacking function completes and outputing the cached contents exactly like calling\n`[FUNCTION_NAME]`.\n\n### Cleanup\n\nA cleanup task is run regularly to remove stale cache data, however no attempt is made to clean up\nthe cache directory on exit since by design the cache can be shared by multiple processes. By\ndefault, cached data is stored in a temp directory that the OS will clean up from time to time\n(generally on reboot), but if you override the cache directory via `BC_CACHE_DIR` you may want to\nclean up the directory yourself.\n\n**Note:** cached data is cleaned up asynchronously, therefore data may persist longer than the\nspecified TTL duration.\n\n### Locking\n\nBy design the caching provided by bash-cache is racy - concurrent invocations may or may not end up\nreusing the same cached value. For most cases (idempotent functions, to be precise) this should be\nsufficient.\n\nFor cases where concurrent calls to the backing function are problematic, use `bc::locked_cache`\ninstead of `bc::cache`. This behaves identically to `bc::cache` but uses an advisory mutex lock to\nprevent concurrent invocations of the backing function.\n\nNote that needing mutual-exclusion is a **strong** signal that you should be using a more powerful\nlanguage than Bash, and that the locking bash-cache provides is\n[advisory](https://en.wikipedia.org/wiki/File_locking#In_Unix-like_systems) and best-effort only.\n\n## Other Functions\n\n### `bc::benchmark`\n\nBenchmarks a function without caching enabled, and with a cold and warm cache. This allows you to\nsee the overhead introduced by Bash Cache and decide if it's beneficial for your function.\n\nThis function runs in a subshell against a clean cache directory, and works for any function - you\ndo not need to have previously called `bc::cache`.\n\n`bc::benchmark_memoize` provides the same basic benchmarking for `bc::memoize`.\n\n### `bc::copy_function`\n\nThis helper function copies an existing function to a new name. This can be used to decorate or\nreplace a function by first copying the function and then defining a new function with the original\nname. This is how `bc::cache` overwrites the function being decorated.\n\nIf desired you can stop caching a particular function by copying the `bc::orig::...` function back\nto its original name:\n\n```shell\nbc::copy_function bc::orig::my_expensive_function my_expensive_function\n```\n\n### `bc::on` and `bc::off`\n\nEnables or disables caching process-wide. If `bc::off` is called all cached functions will delegate\nimmediately to the original function they decorate and will not attempt to use cached data or\ncache new data. Call `bc::on` to re-enable caching.\n\n## Configuration\n\n### Use an isolated cache directory\n\nBy default bash-cache stores cached output in a user-specific directory under `/tmp` or the path\nspecified by `TMPDIR`. To use a different path as the cache root set `BC_CACHE_DIR` before sourcing\n`bash-cache.sh`. This is useful if you're using Bash Cache across multiple scripts, as you could\notherwise run into namespace collisions (e.g. two scripts caching different functions with the same\nname).\n\n## Copyright and License\n\nCopyright 2012-2020 Michael Diamond\n\nThis program is free software: you can redistribute it and/or modify\nit under the terms of the GNU General Public License as published by\nthe Free Software Foundation, either version 3 of the License, or\n(at your option) any later version.\n\nThis program is distributed in the hope that it will be useful,\nbut WITHOUT ANY WARRANTY; without even the implied warranty of\nMERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the\nGNU General Public License for more details.\n\nYou should have received a copy of the GNU General Public License\nalong with this program.  If not, see \u003chttp://www.gnu.org/licenses/\u003e.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdimo414%2Fbash-cache","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdimo414%2Fbash-cache","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdimo414%2Fbash-cache/lists"}