{"id":29273680,"url":"https://github.com/simplifi/pay_day_loan","last_synced_at":"2025-07-05T02:36:34.466Z","repository":{"id":56449312,"uuid":"66086180","full_name":"simplifi/pay_day_loan","owner":"simplifi","description":"Framework for building on-demand caching.  Fast cache now!","archived":false,"fork":false,"pushed_at":"2024-02-08T19:02:24.000Z","size":126,"stargazers_count":6,"open_issues_count":0,"forks_count":2,"subscribers_count":59,"default_branch":"master","last_synced_at":"2025-05-02T04:48:05.136Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/simplifi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-08-19T13:57:24.000Z","updated_at":"2024-03-26T07:09:49.000Z","dependencies_parsed_at":"2022-08-15T19:00:35.533Z","dependency_job_id":null,"html_url":"https://github.com/simplifi/pay_day_loan","commit_stats":null,"previous_names":[],"tags_count":13,"template":false,"template_full_name":null,"purl":"pkg:github/simplifi/pay_day_loan","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simplifi%2Fpay_day_loan","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simplifi%2Fpay_day_loan/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simplifi%2Fpay_day_loan/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simplifi%2Fpay_day_loan/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/simplifi","download_url":"https://codeload.github.com/simplifi/pay_day_loan/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simplifi%2Fpay_day_loan/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263671878,"owners_count":23494056,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-07-05T02:36:33.765Z","updated_at":"2025-07-05T02:36:34.454Z","avatar_url":"https://github.com/simplifi.png","language":"Elixir","funding_links":[],"categories":[],"sub_categories":[],"readme":"# PayDayLoan\n\n[![Build Status](https://travis-ci.org/simplifi/pay_day_loan.svg?branch=master)](https://travis-ci.org/simplifi/pay_day_loan)\n[![Coverage Status](https://coveralls.io/repos/github/simplifi/pay_day_loan/badge.svg)](https://coveralls.io/github/simplifi/pay_day_loan)\n[![Hex.pm version](https://img.shields.io/hexpm/v/pay_day_loan.svg?style=flat-square)](https://hex.pm/packages/pay_day_loan)\n[![Hex.pm downloads](https://img.shields.io/hexpm/dt/pay_day_loan.svg?style=flat-square)](https://hex.pm/packages/pay_day_loan)\n[![License](https://img.shields.io/hexpm/l/pay_day_loan.svg?style=flat-square)](https://hex.pm/packages/pay_day_loan)\n[![API Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](http://hexdocs.pm/pay_day_loan/)\n\nFast cache now!\n\nThis project provides a framework for building on-demand caching in Elixir. \nIt provides a synchronous API to a cache that is loaded asynchronously.\nThe cache itself may be backed in any way that you choose, though the default\nis to use an ETS table backend that has several built-in features for managing\nthe mapping of keys to process ids (e.g., a process registry).  You have the\noption of implementing your own backend using Redis, mnesia, a single process,\netc.\n\nPDL is designed for low-latency access to cache elements after they\nare initially loaded and gives you a framework to minimize load time\nby performing batch loads.  This works very well with data streaming\napplications that have multiple workers processing events in parallel\nand are sharing cache state across workers.\n\nThink of PDL as a cache \"frontend\".  In a typical application, we may want to\nload data from a database and cache it for fast lookup later.  PDL provides\na \"frontend\" so that `MyCache.get(some_id)` will automatically make sure that\nthe data corresponding to `some_id` is loaded into the cache and will return\nthe value once it is available (or time out if the load takes too long).  It\nbatches the loading of data so that you can take advantage of, e.g., database\nqueries that fetch multiple records in one call.\n\nThe actual storage of the data is done by a cache \"backend\".  PDL provides a\ndefault backend via `PayDayLoan.EtsBackend` that is quite flexible.  You can,\nhowever, implement your own backend using the `PayDayLoan.Backend` behaviour.\nThis is useful for using an external service (e.g., Redis) as a cache backend.\nSee the examples below.\n\n**NOTE** `_pid` functions (e.g., `PayDayLoan.get_pid/2`) are deprecated and\nhave been removed.  These functions can be replaced with their non-`_pid`\nequivalents.  `get_pid` is replaced with `get`, `peek_pid` is replaced\nwith `peek`, and `with_pid` is replaced with `with_value`.  0.3.0 was the last\nrelease that included the `_pid` functions.\n\n## Key ideas\n\n* Presents a synchronous API for asynchronous cache loading\n* The cache consists of key-value pairs\n* Provides a default backend for storing values in an ETS table but allows\n  arbitrary backend implementations\n* Tries very hard not to use process messaging in the main lookup API\n  because that can be a bottleneck.  Uses ETS tables for state management.\n* Encourages bulk queries for cache loading.\n* Provides hooks for instrumentation\n  \n## Example usage: Default backend\n\n``` elixir\n# cache wrapper module - this wraps the PDL functions so that they\n#   make sense within the context of your application\ndefmodule MyCache do\n  # defines MyCache.pay_day_loan/0 (and alias pdl/0),\n  #    which is set up with defaults and the supplied callback module\n  use PayDayLoan, callback_module: MyCacheLoader\n\n  # optionally pass in other arguments to override defaults, e.g.,\n  #   use PayDayLoan, callback_module: MyCacheLoader, batch_size: 100\n  \n  # also defines pass-through functions for the PayDayLoan module -\n  #  e.g., `MyCache.get(key)` is a pass-through to\n  #   `MyCache.get(MyCache.pdl(), key)`\nend\n\n# cache loader callback module - this will, for example, execute database\n#   queries and turn the results into cache elements (e.g., Agent or\n#   GenServer processes)\ndefmodule MyCacheLoader do\n  @behaviour PayDayLoan.Loader\n \n  def key_exists?(key) do\n    # should return true if the key exists -\n    #   e.g., if \"SELECT count(1) FROM some_table WHERE id = #{key}\" returns \u003e 0\n  end\n\n  def bulk_load(keys) do\n    # code to look up records for keys in database (or whatever)\n    #  should return a list of tuples of the format\n    #  [{key, load_datum}]\n  end\n  \n  def new(key, load_datum) do\n    # note these are three separate examples - your callback will not do\n    #   all three\n\n    # if we are using processes:\n    Agent.start_link(fn -\u003e load_datum end)\n\n    # if we want to store a callback:\n    {:ok, fn -\u003e {:ok, load_datum} end}\n\n    # if we want to store the bare value\n    {:ok, load_datum}\n  end\n  \n  def refresh(existing_value, key, load_datum) do\n    # note these are three separate examples - your callback will not do\n    #   all three\n\n    # if we are using proccesses, the existing_value is the pid of the\n    #   already-started process\n    pid = existing_value\n    Agent.update(pid, fn(_cached_datum) -\u003e load_datum end)\n    # we need to return the pid back\n    {:ok, pid}\n\n    # or we could stop the existing pid and replace it with a new one\n    Agent.stop(pid)\n    Agent.start_link(fn -\u003e load_datum end)\n\n    # or if we stored a callback\n    {:ok, cached_datum} = existing_value.()\n    Logger.info(\"Replacing #{inspect cached_datum} with #{inspect load_datum}\")\n    {:ok, fn -\u003e {:ok, load_datum} end}\n\n    # or to store the new datum as a bare value\n    {:ok, load_datum}\n  end\nend\n\n# Add PDL to your existing supervision tree so that everything initializes properly\ndefmodule MyOTPApp do\n  use Application \n\n  # existing Application.start callback\n  def start(_type, _args) do\n    my_supervisor_children = [\n      # ... existing children specs\n      PayDayLoan.supervisor_specification(MyCache.pdl)\n    ]\n    \n    # for example\n    Supervisor.start_link(my_supervisor_children, supervisor_opts)\n  end\nend\n\n# synchronous API - behind the scenes will add the key (1) to the\n#   load state table and the asynchronous loader will include that\n#   in its next load cycle - this call does not return until either\n#   the cache is loaded (via new above) or the request times out\n{:ok, value} = MyCache.get(1)\n```\n\n## Example usage: Process backend (e.g., Redis connection)\n\n``` elixir\n# cache wrapper module - this wraps the PDL functions so that they\n#   make sense within the context of your application\ndefmodule MyCache do\n  # same as above but we specify a `backend` module and disable the\n  #  cache monitor, we also specify a `backend_payload` so that we can\n  #  specify a unique identifier for the backend process \n  use(\n    PayDayLoan,\n    callback_module: MyCacheLoader,\n    backend: MyCacheBackend,\n    backend_payload: :my_cache,\n    cache_monitor: false # we won't be storing pids\n  )\nend\n\n# same ideas as above but the new/refresh callbacks are different\ndefmodule MyCacheLoader do\n  @behaviour PayDayLoan.Loader\n \n  def key_exists?(key) do\n    # should return true if the key exists -\n    #   e.g., if \"SELECT count(1) FROM some_table WHERE id = #{key}\" returns \u003e 0\n  end\n\n  def bulk_load(keys) do\n    # code to look up records for keys in database (or whatever)\n    #  should return a list of tuples of the format\n    #  [{key, load_datum}]\n  end\n  \n  def new(key, load_datum) do\n    # we could modify the data here, but we are just going to store it raw\n    {:ok, load_datum}\n  end\n  \n  def refresh(_existing_value, key, load_datum) do\n    # we could merge the existing value and the load_datum or we could modify\n    #  before we store, but we're just going to replace\n    {:ok, load_datum}\n  end\nend\n\n# backend behaviour implementation\ndefmodule MyCacheBackend do\n  @behaviour PayDayLoan.Backend\n\n  # this shows an example of how we might use a single process backend, using\n  # Redis is very similar - the process would be Redis connection and the\n  # various callbacks would use Redis commands\n\n  def start_link(name), do: Agent.start_link(fn -\u003e %{} end, name: __name)\n\n  # nothing to do for setup\n  def setup(_pdl), do: :ok\n\n  # this would be a little more involved with redis - you could use the KEYS\n  #   command and then MGET but with a large cache, that approach is not\n  #   advised.  SCAN can be used with larger caches.\n  def reduce(pdl, acc0, reducer) do\n    Agent.get(pdl.backend_payload, fn(m) -\u003e Enum.reduce(m, acc0, reducer) end)\n  end\n\n  # with redis this could be a call to DBSIZE\n  def size(pdl), do: Agent.get(pdl.backend_payload, \u0026map_size/1) \n\n  # with redis this could be a call to the KEYS command\n  def keys(pdl), do: Agent.get(pdl.backend_payload, \u0026Map.keys/1)\n\n  # see comments on the reduce command\n  def values(pdl), do: Agent.get(pdl.backend_payload, \u0026Map.values/1)\n\n  # this should be a simple GET command in redis\n  def get(pdl, key) do\n    case Agent.get(pdl.backend_payload, fn(m) -\u003e Map.get(m, key) end) do\n      nil -\u003e {:error, :not_found}\n      v -\u003e {:ok, v}\n    end\n  end\n\n  # with redis you could use SET here\n  def put(pdl, key, val) do\n    Agent.update(pdl.backend_payload, fn(m) -\u003e Map.put(m, key, \"V#{val}\") end)\n  end\n\n  # corresponds to redis DEL\n  def delete(pdl, key) do\n    Agent.update(pdl.backend_payload, fn(m) -\u003e Map.delete(m, key) end)\n  end\nend\n\n# Add PDL to your existing supervision tree so that everything initializes properly\ndefmodule MyOTPApp do\n  use Application \n\n  # existing Application.start callback\n  def start(_type, _args) do\n    my_supervisor_children = [\n      # start the backend with the payload as its name\n      worker(MyCacheBackend, [MyCache.pdl().backend_payload]),\n      # ... existing children specs\n      PayDayLoan.supervisor_specification(MyCache.pdl)\n    ]\n    \n    # for example\n    Supervisor.start_link(my_supervisor_children, supervisor_opts)\n  end\nend\n\n# synchronous API - behind the scenes will add the key (1) to the\n#   load state table and the asynchronous loader will include that\n#   in its next load cycle - this call does not return until either\n#   the cache is loaded (via new above) or the request times out\n{:ok, value} = MyCache.get(1)\n```\n\n## Logging \u0026 Instrumentation\n\nThe `use` macro accepts an `event_loggers` option, which should be a list of\nfunctions that take two arguments.  When certain events occur, each of these\nfunctions will be called with an event atom and the key requested.  The events\nare\n\n* `:timed_out` - Timed out while loading cache.\n* `:disappeared` - Key was marked as `:loaded` but the backend did not return a value\n* `:failed` - The loader failed to load a value for the key\n* `:cache_miss` - A requested value was not already cached\n* `:no_key` - The loaded says this key does not exist\n\nExample usage:\n\n```elixir\ndefmodule CacheEventLogger do\n  require Logger\n\n  def log(event, key) do\n    Logger.debug(\"Requesting key #{inspect key} caused event #{inspect event}\")\n  end\nend\n\ndefmodule CacheEventStats do\n  def log(event, key) do\n    # update a statsd counter, etc.\n  end\nend\n\ndefmodule MyCache do\n  use PayDayLoan, event_loggers: [\u0026CacheEventLogge.log/2, \u0026CacheEventStats.log/2]\nend\n```\n\nThe `PayDayLoan.load_state_stats/1` function returns the count of keys in each\nload state and is also useful for instrumentation.\n\n## Development \u0026 Contributing\n\nThe usual Elixir and github contribution workflows apply.  Pull requests are welcome!\n\n```bash\nmix deps.get\nmix compile\nmix test\n```\n\n## License\n\nSee [LICENSE.txt](LICENSE.txt)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsimplifi%2Fpay_day_loan","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsimplifi%2Fpay_day_loan","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsimplifi%2Fpay_day_loan/lists"}