{"id":21206188,"url":"https://github.com/rtmigo/filememo_py","last_synced_at":"2026-01-05T18:05:22.441Z","repository":{"id":46321256,"uuid":"367473325","full_name":"rtmigo/filememo_py","owner":"rtmigo","description":"Stores the results of expensive function calls and returns the cached result when the same inputs occur again","archived":false,"fork":false,"pushed_at":"2022-02-09T13:12:47.000Z","size":131,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-01T08:49:18.661Z","etag":null,"topics":["cache","file","function","memoization","memoization-library","memoize","memoize-decorator","method","package","permanent","persistent","pickle","python"],"latest_commit_sha":null,"homepage":"https://pypi.org/project/filememo/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rtmigo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-05-14T20:32:10.000Z","updated_at":"2022-07-18T19:49:46.000Z","dependencies_parsed_at":"2022-09-02T01:51:22.241Z","dependency_job_id":null,"html_url":"https://github.com/rtmigo/filememo_py","commit_stats":null,"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rtmigo%2Ffilememo_py","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rtmigo%2Ffilememo_py/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rtmigo%2Ffilememo_py/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rtmigo%2Ffilememo_py/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rtmigo","download_url":"https://codeload.github.com/rtmigo/filememo_py/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245580767,"owners_count":20638864,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cache","file","function","memoization","memoization-library","memoize","memoize-decorator","method","package","permanent","persistent","pickle","python"],"created_at":"2024-11-20T20:54:44.445Z","updated_at":"2026-01-05T18:05:22.354Z","avatar_url":"https://github.com/rtmigo.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# [filememo](https://github.com/rtmigo/filememo_py#readme)\n\nFile-based **memoization** decorator. Caches the results of expensive function\ncalls. Retains the cached results between program restarts.\n\nCI tests are done in Python 3.8, 3.9 and 3.10 on macOS, Ubuntu and\nWindows.\n\n---\n\nThe function can be *expensive* because it is slow, or uses a lot of system\nresources, or literally makes a request to a paid API.\n\nThe `memoize` decorator returns the cached result when the same function called\nwith the same arguments. Thus, the function is expensive only once and\ninexpensive thereafter.\n\nFor example, the simplest cache for downloaded data can be set like this:\n\n``` python3\n@memoize\ndef downloaded(url):\n    return requests.get(url)\n    \ndownloaded(\"http://example.net/aaa\")  # downloads data\ndownloaded(\"http://example.net/bbb\")  # downloads data\ndownloaded(\"http://example.net/aaa\")  # gets data from cache   \n```\n\nData is saved to the file system using\n[pickledir](https://pypi.org/project/pickledir/). Even after the program\nrestart, the cached results will be in place.\n\n``` python3\n# gets data from cache after restart\ndownloaded(\"http://example.net/aaa\")     \n```\n\n# Install\n\n``` bash\n$ pip3 install filememo\n```\n\n# Use\n\n``` python3\nfrom filememo import memoize\n\n@memoize\ndef long_running_function(a, b, c):\n    return compute()\n\n# the following line actually computes the value only\n# when the program runs for the first time. On subsequent \n# runs, the value is read from the file\nx = long_running_function(1, 2, 3)\n```\n\n## Function arguments\n\nThe results depend on both the function and its arguments. All results are\ncached separately.\n\n``` python3\n@memoize\ndef that_function(a, b, c):\n    return compute(a, b, c)\n\n@memoize\ndef other_function(a, b):\n    return compute(a, b)\n\n# the following calls will cache three different values \ny1 = that_function(1, 2, 3)  \ny2 = that_function(30, 20, 40)\ny3 = other_function(1, 2)\n\n# the way the arguments are set is also important, as is their order. \n# Therefore, the following calls are cached as three different ones\ny4 = other_function(1, b=2)\ny5 = other_function(a=1, b=2)\ny6 = other_function(b=2, a=1)\n```\n\n## Cache directory\n\nIf `dir_path` is not specified, the cached data is stored in the directory\nreturned by\nthe [`gettempdir`](https://docs.python.org/3/library/tempfile.html#tempfile.gettempdir)\n. However, there is a high probability that the cache stored there will not\nsurvive a reboot. And even a certain probability that the system does not have a\ntemporary directory, so the current directory will be considered temporary.\n\nTo better control the situation, you can set a specific directory for storing\ncaches.\n\n``` python3\n@memoize(dir_path='/var/tmp/myfuncs')\ndef function(a, b):\n    return a+b\n    \n# it's ok if different functions share the same directory    \n@memoize(dir_path='/var/tmp/myfuncs')\ndef other_func():\n    return compute()\n```\n\n## Expiration date\n\nThe `max_age` argument sets two conditions at once:\n\n- if the result is not yet in the cache (and we will add it now), then it will\n  live in the cache no longer than `max_age`. After that it will be\n  automatically deleted\n- if the result is already in the cache, then we only use it if its age is less\n  than `max_age`. Otherwise, the function will be run again, and the result will\n  be replaced with a new one\n\n``` python3\n@memoize(max_age = datetime.timedelta(minutes=5))\ndef function(a, b):\n    return compute()\n```\n\n## Data version\n\nWhen you specify `version`, all results with different versions are considered\noutdated.\n\nSay you have the following function:\n\n``` python3\n@memoize(version=1)\ndef function(a, b):\n    return a + b\n```\n\nYou changed your mind, and now the function should return the product of numbers\ninstead of the sum. But the cache already contains the previous results with the\nsums. In this case, you can just change\n`version`. Previous results will not be returned.\n\n``` python3\n@memoize(version=2)\ndef function(a, b):\n    return a * b\n```\n\nNote that all **other** than the current version are deprecated, regardless of\nwhether their value is greater or less. If you used `version=10`, and then\nstarted using `version=9`, then 9 is considered current, and 10 is obsolete.\n\n## Exceptions\n\nIf the decorated function throws an exception, the error is considered\npermanent. The exception is stored in the cache and will be raised every time.\n\n``` python3\nfrom filememo import memoize, FunctionException\n\n@memoize\ndef divide(a, b):\n    return a / b\n\ntry:\n    # tryng to run the function for the first time\n    divide(1, 0)\nexcept FunctionException as e:\n    print(f\"Error: {e.inner}\")      \n\ntry:\n    # not actually running again, getting error from cache\n    divide(1, 0)\nexcept FunctionException as e:\n    print(f\"Cached error: {e.inner}\")      \n```\n\nThe `exceptions_max_age = None` argument will prevent exceptions from being\ncached. Each error will be considered a one-time error.\n\n``` python3\n@memoize(exceptions_max_age = None)\ndef download(url):\n    return http_get(url)\n    \nwhile True:\n    try:\n        download('http://sample.net/path')\n        break\n    except FunctionException:\n        time.sleep(1)\n        # will retry        \n```\n\nYou can also set the expiration time for cached exceptions. It may differ from\nthe caching time of the data itself.\n\n``` python3\n# keep downloaded data for a day, remember connection errors for 5 minutes\n\n@memoize(max_age = datetime.timedelta(days: 1)\n         exceptions_max_age = datetime.timedelta(minutes: 5))\ndef download(url):\n    return http_get(url)\n```\n\n## In-memory caching\n\nEach call to a function decorated with `@memoize` results in I/O operations. If\nyour absolute priority is performance, then even reading from the disk cache can\nbe considered expensive. Although `filememo` does not attempt to cache the read\ndata in memory, this functionality is easy to achieve by combining decorators.\n\n``` python3\nfrom functools import lru_cache\nfrom filememo import memoize\n\n@lru_cache\n@memoize\ndef too_expensive():\n    return compute()\n```\n\nIn this example, the `filememo` disk cache will be used to store the results\nbetween program runs, while the `functools` RAM cache will store the results\nbetween function calls.\n\nIf the data is already in disk cache, and the program is just started, then\ncalling `too_expensive()` for the first time will read the result from disk.\nFurther calls to `too_expensive()` will return the result from memory.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frtmigo%2Ffilememo_py","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frtmigo%2Ffilememo_py","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frtmigo%2Ffilememo_py/lists"}