{"id":19035749,"url":"https://github.com/paylogic/dogpile_filesystem","last_synced_at":"2025-04-23T18:54:29.147Z","repository":{"id":53122104,"uuid":"170706797","full_name":"paylogic/dogpile_filesystem","owner":"paylogic","description":"File System backend for Dogpile Cache","archived":false,"fork":false,"pushed_at":"2024-01-17T13:37:12.000Z","size":76,"stargazers_count":3,"open_issues_count":4,"forks_count":3,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-04-18T04:12:52.351Z","etag":null,"topics":["caching","filesystem-cache","locking-strategies","python-library"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/paylogic.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.rst","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":"AUTHORS.rst","dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-02-14T14:45:52.000Z","updated_at":"2020-09-13T12:11:03.000Z","dependencies_parsed_at":"2024-11-08T21:52:03.726Z","dependency_job_id":null,"html_url":"https://github.com/paylogic/dogpile_filesystem","commit_stats":{"total_commits":91,"total_committers":2,"mean_commits":45.5,"dds":0.02197802197802201,"last_synced_commit":"5a9f3599b203918b6bf37fb930082c83922c51db"},"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paylogic%2Fdogpile_filesystem","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paylogic%2Fdogpile_filesystem/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paylogic%2Fdogpile_filesystem/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paylogic%2Fdogpile_filesystem/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/paylogic","download_url":"https://codeload.github.com/paylogic/dogpile_filesystem/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249961533,"owners_count":21352091,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["caching","filesystem-cache","locking-strategies","python-library"],"created_at":"2024-11-08T21:51:59.554Z","updated_at":"2025-04-23T18:54:29.127Z","avatar_url":"https://github.com/paylogic.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# File System Backends for Dogpile Cache\n\n[![PyPI](https://img.shields.io/pypi/v/dogpile_filesystem.svg?style=flat)](https://pypi.org/project/dogpile-filesystem/)\n[![Build Status](https://travis-ci.org/paylogic/dogpile_filesystem.svg?branch=master)](https://travis-ci.org/paylogic/dogpile_filesystem)\n[![codecov](https://codecov.io/gh/paylogic/dogpile_filesystem/branch/master/graph/badge.svg)](https://codecov.io/gh/paylogic/dogpile_filesystem)\n![](https://img.shields.io/pypi/pyversions/dogpile_filesystem.svg?style=flat)\n![](https://img.shields.io/pypi/implementation/dogpile_filesystem.svg?style=flat)\n[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)\n\n\n\nFilesystem-based backends for dogpile cache.\n\nThe generic variant of the backend, `paylogic.filesystem`, will accept any picklable value and it will store it in the file system.\n\nThe raw variant `paylogic.raw_filesystem` will only work with file-like values and it will avoid the pickling phase. This is useful when you are generating a big file and you don't want to keep in memory the contents of this file.\n\nBoth variants use [fcntl.lockf](https://docs.python.org/3.7/library/fcntl.html#fcntl.lockf) operations, therefore it is compatible with  UNIX-like systems only.\nThe lockf system call allows to allocate an arbitrary number of locks using the same file, avoiding problems that arise when deleting lock files.\n\n\n## Installation\nInstall with pip:\n\n`$ pip install dogpile_filesystem`\n\n## Usage\n### Generic variant\nConfigure a region to use `paylogic.filesystem`:\n```python\nfrom dogpile.cache import make_region\nimport datetime\n\nregion = make_region().configure(\n    'paylogic.filesystem',\n    arguments = {\n        \"base_dir\": \"/path/to/cachedir\",  # Make sure this directory is only for this region\n        # Optional parameters\n        \"cache_size\": 1024**3,  # Defaults to 1 Gb\n        \"expiration_time\": datetime.timedelta(seconds=30),  # Defaults to no expiration\n        \"distributed_lock\": True,  # Defaults to true\n    }\n)\n\n@region.cache_on_arguments()\ndef my_function(args):\n    return 42\n```\n\n### Raw variant\nConfigure a region to use dogpile_filesystem:\n```python\nfrom dogpile.cache import make_region\nimport datetime\nimport tempfile\n\nregion = make_region().configure(\n    'paylogic.raw_filesystem',\n    arguments = {\n        \"base_dir\": \"/path/to/cachedir\",  # Make sure this directory is only for this region\n        # Optional parameters\n        \"cache_size\": 1024**3,  # Defaults to 1 Gb\n        \"file_movable\": True,  # Whether the backend can freely move the file.\n                               # When True, the backend will move the file to the cache\n                               # directory directly using os.rename(file.name).\n                               # When False (default), the content of the file will be copied to\n                               # the cache directory.\n        \"expiration_time\": datetime.timedelta(seconds=30),  # Defaults to no expiration\n        \"distributed_lock\": True,  # Defaults to true\n    }\n)\n\n@region.cache_on_arguments()\ndef big_file_operation(args):\n    # When using `file_movable=True`, we must make sure that NamedTemporaryFile does not delete the file on close,\n    # otherwise it will complain that it cannot find the file.\n    f = tempfile.NamedTemporaryFile(delete=False)\n    # fill the file\n    f.flush()\n    f.seek(0)\n    return f\n```\n\n## Development\nInstall the dev requirements and the project in development mode:\n\n`$ pip install -r requirements_dev.txt -e .`\n\nRun tests:\n\n`$ pytest tests`\n\nOptionally run tests for all supported configurations:\n\n`$ tox`\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpaylogic%2Fdogpile_filesystem","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpaylogic%2Fdogpile_filesystem","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpaylogic%2Fdogpile_filesystem/lists"}