{"id":13449308,"url":"https://github.com/keithrozario/lambda-cache","last_synced_at":"2025-03-22T15:31:12.521Z","repository":{"id":46333728,"uuid":"250934543","full_name":"keithrozario/lambda-cache","owner":"keithrozario","description":"Python utility for caching in Lambda Functions","archived":false,"fork":false,"pushed_at":"2020-06-24T01:49:25.000Z","size":353,"stargazers_count":38,"open_issues_count":2,"forks_count":7,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-18T13:19:25.376Z","etag":null,"topics":["aws","aws-lambda","aws-para","aws-secretsmanager","cache","caching","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/keithrozario.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-03-29T02:03:37.000Z","updated_at":"2024-10-26T22:27:19.000Z","dependencies_parsed_at":"2022-07-22T10:47:59.589Z","dependency_job_id":null,"html_url":"https://github.com/keithrozario/lambda-cache","commit_stats":null,"previous_names":["keithrozario/lambda_cache"],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/keithrozario%2Flambda-cache","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/keithrozario%2Flambda-cache/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/keithrozario%2Flambda-cache/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/keithrozario%2Flambda-cache/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/keithrozario","download_url":"https://codeload.github.com/keithrozario/lambda-cache/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244978586,"owners_count":20541880,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","aws-lambda","aws-para","aws-secretsmanager","cache","caching","python"],"created_at":"2024-07-31T06:00:35.296Z","updated_at":"2025-03-22T15:31:12.007Z","avatar_url":"https://github.com/keithrozario.png","language":"Python","readme":"\u003ch1 align=\"center\"\u003e Lambda Cache \u003c/h1\u003e\n\u003ch2 align=\"center\"\u003e Simple Caching for Lambda Functions\u003c/h2\u003e\n\n![PackageStatus](https://img.shields.io/pypi/status/lambda-cache) ![PyPI version](https://img.shields.io/pypi/v/lambda-cache) ![Downloads](https://img.shields.io/pypi/dw/lambda-cache)\n\n![PythonSupport](https://img.shields.io/static/v1?label=python\u0026message=3.6%20|%203.7%20|%203.8\u0026color=blue?style=flat-square\u0026logo=python) ![License: MIT](https://img.shields.io/github/license/keithrozario/lambda-cache) ![Documentation Status](https://readthedocs.org/projects/lambda-cache/badge/?version=latest)\n\n![Test](https://github.com/keithrozario/lambda-cache/workflows/Test/badge.svg?branch=release) [![Coverage Status](https://coveralls.io/repos/github/keithrozario/lambda-cache/badge.svg?branch=release)](https://coveralls.io/github/keithrozario/lambda-cache?branch=release) [![Codacy Badge](https://api.codacy.com/project/badge/Grade/ad70a44cb3e54d7ba600edc5fa89635c)](https://www.codacy.com/manual/keithrozario/lambda-cache?utm_source=github.com\u0026amp;utm_medium=referral\u0026amp;utm_content=keithrozario/lambda-cache\u0026amp;utm_campaign=Badge_Grade) [![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black) \n\n# Introduction\n\n\u003cp align=\"center\"\u003e\u003cimg src='docs/images/lambda_cache.png'\u003e\u003c/p\u003e\n\n_lambda-cache_ helps you cache data in your Lambda function from one invocation to another. It utilizes the internal memory of the lambda function's [execution context](https://docs.aws.amazon.com/lambda/latest/dg/runtimes-context.html) to store data across multiple invocations, which:\n\n* Reduces load on back-end systems\n* Reduces the execution time of the lambda\n* Guarantees that functions will reference latest data after cache expiry\n\n_lambda-cache_ is purpose-built for AWS Lambda functions, and prioritizes simplicity as our design gaol. It currently supports SSM Parameters, Secrets from Secrets Manager and S3 Objects.\n\n# Installation\n\nInclude the package in your function zip-file artifact using:\n\n    $ pip install lambda-cache -t /path/of/function\n\nRefer to [installation guide](https://lambda-cache.readthedocs.io/en/latest/install/) for other options.\n\n# Usage\n\nThe official [user guide](https://lambda-cache.readthedocs.io/en/latest/user_guide/) has more info.\n\n\u003c!-- vscode-markdown-toc --\u003e\n* [SSM - Parameter Store](#SSM-ParameterStore)\n\t* [Cache single parameter](#Cachesingleparameter)\n\t* [Change cache expiry](#Changecacheexpiry)\n\t* [Change cache entry settings](#Changecacheentrysettings)\n\t* [Cache multiple parameters](#Cachemultipleparameters)\n\t* [Decorator stacking](#Decoratorstacking)\n\t* [Cache invalidation](#Cacheinvalidation)\n\t* [Return Values](#ReturnValues)\n* [Secrets Manager](#SecretsManager)\n\t* [Cache single secret](#Cachesinglesecret)\n\t* [Change Cache expiry](#ChangeCacheexpiry)\n\t* [Change Cache entry settings](#ChangeCacheentrysettings)\n\t* [Decorator stacking](#Decoratorstacking-1)\n\t* [Cache Invalidation](#CacheInvalidation)\n\t* [Return Values](#ReturnValues-1)\n* [S3](#S3)\n\t* [Cache a single file](#Cacheasinglefile)\n\t* [Change Cache expiry](#ChangeCacheexpiry-1)\n\t* [Check file before download](#Checkfilebeforedownload)\n\n## \u003ca name='SSM-ParameterStore'\u003e\u003c/a\u003eSSM - Parameter Store\n\n### \u003ca name='Cachesingleparameter'\u003e\u003c/a\u003eCache single parameter\n\nTo cache a parameter from ssm, decorate your handler function like so:\n\n```python\nfrom lambda_cache import ssm\n\n@ssm.cache(parameter='/production/app/var')\ndef handler(event, context):\n    var = getattr(context,'var')\n    response = do_something(var)\n    return response\n```\nAll invocations of this function over in the next minute will reference the parameter from the function's internal cache, rather than making a network call to ssm. After one minute has lapsed, the the next invocation will invoke `get_parameter` to refresh the cache. The parameter value will be injected into the `context` object of your lambda handler for retrieval.\n\n### \u003ca name='Changecacheexpiry'\u003e\u003c/a\u003eChange cache expiry\n\nThe default `max_age_in_seconds` settings is 60 seconds (1 minute), it defines the maximum age of a parameter that is acceptable to the handler function. Cache entries older than this, will be refreshed. To set a longer cache duration (e.g 5 minutes), change the setting:\n\n```python\nfrom lambda_cache import ssm\n\n@ssm.cache(parameter='/production/app/var', max_age_in_seconds=300)\ndef handler(event, context):\n    var = getattr(context,'var')\n    response = do_something(var)\n    return response\n```\n\n_Note: The caching logic runs only at invocation, regardless of how long the function runs. A 15 minute lambda function will not refresh the parameter, unless explicitly refreshed using `get_entry` method (described later). The library is primary interested in caching 'across' invocations rather than 'within' an invocation_\n\n### \u003ca name='Changecacheentrysettings'\u003e\u003c/a\u003eChange cache entry settings\n\nThe default name of the parameter is the string after the last slash('/') character of its name. This means `/production/app/var` and `test/app/var` both resolve to just `var`. To over-ride this default, use the `entry_name` setting like so:\n\n```python\nfrom lambda_cache import ssm\n\n@ssm.cache(parameter='/production/app/var', entry_name='new_var')\ndef handler(event, context):\n    var = getattr(context,'new_var')\n    response = do_something(var)\n    return response\n```\n\n### \u003ca name='Cachemultipleparameters'\u003e\u003c/a\u003eCache multiple parameters\n\nTo cache multiple entries at once, pass a list of parameters to the parameter argument. This method groups all the parameter values in one python dictionary, stored in the Lambda Context under the `entry_name`. \n\n_Note: When using this method, `entry_name` is a required parameter, if not present a `NoEntryNameError` exception is thrown._\n\n```python\nfrom lambda_cache import ssm\n\n@ssm.cache(parameter=['/app/var1', '/app/var2'], entry_name='parameters')\ndef handler(event, context):\n    var1 = getattr(context,'parameters').get('var1')\n    var2 = getattr(context,'parameters').get('var2')\n    response = do_something(var)\n    return response\n```\n\nUnder the hood, we use the `get_parameters` API call for boto3, which translate to a single network call for multiple parameters. You can group all parameters types in a single call, including `String`, `StringList` and `SecureString`. `StringList` will return as a list, while all other types will return as plain-text strings. The library does not support returning `SecureString` parameters in encrypted form, and will only return plain-text strings regardless of String type.\n\n_Note: for this method to work, ensure you have both `ssm:GetParameter` **and** `ssm:GetParameters` (with the 's' at the end) in your function's permission policy_\n\n### \u003ca name='Decoratorstacking'\u003e\u003c/a\u003eDecorator stacking\n\nIf you wish to cache multiple parameters with different expiry times, stack the decorators. In this example, `var1` will be refreshed every 30 seconds, `var2` will be refreshed after 60.\n\n```python\n@ssm.cache(parameter='/production/app/var1', max_age_in_seconds=30)\n@ssm.cache(parameter='/production/app/var2', max_age_in_seconds=60)\ndef handler(event, context):\n    var1 = getattr(context,'var1')\n    var2 = getattr(context,'var2')\n    response = do_something(var)\n    return response\n```\n_Note: Decorator stacking performs one API call per decorator, which might result is slower performance_\n\n### \u003ca name='Cacheinvalidation'\u003e\u003c/a\u003eCache invalidation\n\nIf you require a fresh value at some point of the code, you can force a refresh using the `ssm.get_entry` function, and setting the `max_age_in_seconds` argument to 0.\n\n```python\nfrom lambda_cache import ssm\n\n@ssm.cache(parameter='/prod/var')\ndef handler(event, context):\n\n    if event.get('refresh'):\n        # refresh parameter\n        var = ssm.get_entry(parameter='/prod/var', max_age_in_seconds=0)\n    else:\n        var = getattr(context,'var')\n    \n    response = do_something(var)\n    return response\n```\n\nYou may also use `ssm.get_entry` to get a parameter entry from anywhere in your functions code.\n\nTo only get parameter once in the lifetime of the function, set `max_age_in_seconds` to some arbitary large number ~36000 (10 hours).\n\n### \u003ca name='ReturnValues'\u003e\u003c/a\u003eReturn Values\n\nCaching supports `String`, `SecureString` and `StringList` parameters with no change required (ensure you have `kms:Decrypt` permission for `SecureString`). For simplicity, `StringList` parameters are automatically converted into list (delimited by comma), while `String` and `SecureString` both return the single string value of the parameter.\n\n## \u003ca name='SecretsManager'\u003e\u003c/a\u003eSecrets Manager\n\n### \u003ca name='Cachesinglesecret'\u003e\u003c/a\u003eCache single secret\n\nSecret support is similar, but uses the `secret.cache` decorator.\n\n```python\nfrom lambda_cache import secrets_manager\n\n@secrets_manager.cache(name='/prod/db/conn_string')\ndef handler(event, context):\n    conn_string = getattr(context,'conn_string')\n    return context\n```\n\n\n### \u003ca name='ChangeCacheexpiry'\u003e\u003c/a\u003eChange Cache expiry\n\nThe default `max_age_in_seconds` settings is 60 seconds (1 minute), it defines how long a parameter should be kept in cache before it is refreshed from ssm. To configure longer or shorter times, modify this argument like so:\n\n```python\nfrom lambda_cache import secrets_manager\n\n@secrets_manager.cache(name='/prod/db/conn_string', max_age_in_seconds=300)\ndef handler(event, context):\n    var = getattr(context,'conn_string')\n    response = do_something(var)\n    return response\n```\n\n_Note: The caching logic runs only at invocation, regardless of how long the function runs. A 15 minute lambda function will not refresh the parameter, unless explicitly refreshed using get_cache_ssm. The library is primary interested in caching 'across' invocation rather than 'within' an invocation_\n\n### \u003ca name='ChangeCacheentrysettings'\u003e\u003c/a\u003eChange Cache entry settings\n\nThe name of the secret is simply shortened to the string after the last slash('/') character of the secret's name. This means `/prod/db/conn_string` and `/test/db/conn_string` resolve to just `conn_string`. To over-ride this default, use `entry_name`:\n\n```python\nfrom lambda_cache import secrets_manager\n\n@secrets_manager.cache(name='/prod/db/conn_string', entry_name='new_var')\ndef handler(event, context):\n    var = getattr(context,'new_var')\n    response = do_something(var)\n    return response\n```\n\n### \u003ca name='Decoratorstacking-1'\u003e\u003c/a\u003eDecorator stacking\n\nIf you wish to cache multiple secrets, you can use decorator stacking.\n\n```python\n@secrets_manager.cache(name='/prod/db/conn_string', max_age_in_seconds=30)\n@secrets_manager.cache(name='/prod/app/elk_username_password', max_age_in_seconds=60)\ndef handler(event, context):\n    var1 = getattr(context,'conn_string')\n    var2 = getattr(context,'elk_username_password')\n    response = do_something(var)\n    return response\n```\n\n_Note: Decorator stacking performs one API call per decorator, which might result is slower performance._\n\n### \u003ca name='CacheInvalidation'\u003e\u003c/a\u003eCache Invalidation\n\nTo invalidate a secret, use the `get_entry`, setting the `max_age_in_seconds=0`.\n```python\nfrom lambda_cache import secrets_manager\n\n@secrets_manager.cache(name='/prod/db/conn_string')\ndef handler(event, context):\n\n    try:\n        response = db_connect()\n    except AuthenticationError:\n        var = secrets_manager.get_entry(name='/prod/db/conn_string', max_age_in_seconds=0)\n        response = db_connect()\n\n    return response\n```\n\n### \u003ca name='ReturnValues-1'\u003e\u003c/a\u003eReturn Values\n\nSecrets Manager supports both string and binary secrets. For simplicity we will cache the secret in the format it is stored. It is up to the calling application to process the return as Binary or Strings.\n\n## \u003ca name='S3'\u003e\u003c/a\u003eS3\n\nS3 support is considered _experimental_ for now, but withing the python community we see a lot of folks pull down files from S3 for use in AI/ML models.\n\nFiles downloaded from s3 are automatically stored in the `/tmp` directory of the lambda function. This is the only writable directory within lambda, and has a 512MB of storage space. \n\n### \u003ca name='Cacheasinglefile'\u003e\u003c/a\u003eCache a single file\nTo download a file from S3 use the the same decorator pattern:\n\n```python\nfrom lambda_cache import s3\n\n@s3.cache(s3Uri='s3://bucket_name/path/to/object.json')\ndef s3_download_entry_name(event, context):\n\n    # Object from S3 automatically saved to /tmp directory\n    with open(\"/tmp/object.json\") as file_data:\n        status = json.loads(file_data.read())['status']\n\n    return status\n```\n\n### \u003ca name='ChangeCacheexpiry-1'\u003e\u003c/a\u003eChange Cache expiry\n\nThe default `max_age_in_seconds` settings is 60 seconds (1 minute), it defines how long a file should be kept in `/tmp` before it is refreshed from S3. To configure longer or shorter times, modify this argument like so:\n\n```python\nfrom lambda_cache import s3\n\n@s3.cache(s3Uri='s3://bucket_name/path/to/object.json', max_age_in_seconds=300)\ndef s3_download_entry_name(event, context):\n    with open(\"/tmp/object.json\") as file_data:\n        status = json.loads(file_data.read())['status']\n\n    return status\n```\n\n_Note: The caching logic runs only at invocation, regardless of how long the function runs. A 15 minute lambda function will not refresh the object, unless explicitly refreshed using `s3.get_entry`. The library is primary interested in caching 'across' invocation rather than 'within' an invocation_\n\n### \u003ca name='Checkfilebeforedownload'\u003e\u003c/a\u003eCheck file before download\n\nBy default, _lambda_cache_ will download the file once at cache has expired, however, to save on network bandwidth (and possibly time), we can set the `check_before_download` parameter to True. This will check the age of the object in S3 and download **only** if the object has changed since the last download. \n\n```python\nfrom lambda_cache import s3\n\n@s3.cache(s3Uri='s3://bucket_name/path/to/object.json', max_age_in_seconds=300, check_before_download=True)\ndef s3_download_entry_name(event, context):\n    with open(\"/tmp/object.json\") as file_data:\n        status = json.loads(file_data.read())['status']\n\n    return status\n```\n\n_Note: we use the GetHead object call to verify the objects `last_modified_date`. This simplifies the IAM policy of the function, as it still only requires the `s3:GetObject` permission. However, this is still a GET requests, and will be charged as such, for smaller objects it might be cheaper to just download the object_\n\n# Credit\n\nProject inspired by:\n* [SSM-Cache](https://github.com/alexcasalboni/ssm-cache-python)\n* [middy](https://github.com/middyjs/middy)\n","funding_links":[],"categories":["Python"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkeithrozario%2Flambda-cache","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkeithrozario%2Flambda-cache","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkeithrozario%2Flambda-cache/lists"}