{"id":18403688,"url":"https://github.com/ydylla/fcache","last_synced_at":"2026-02-16T01:02:59.496Z","repository":{"id":57710276,"uuid":"511661446","full_name":"ydylla/fcache","owner":"ydylla","description":"fcache is a file based persistent blob cache.","archived":false,"fork":false,"pushed_at":"2025-03-16T16:31:54.000Z","size":102,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-16T17:30:03.619Z","etag":null,"topics":["cache","file-cache","lru-cache"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ydylla.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-07-07T20:11:31.000Z","updated_at":"2025-03-16T16:36:50.000Z","dependencies_parsed_at":"2024-11-06T03:11:20.813Z","dependency_job_id":"68988ad1-85f5-4b28-887d-35cc29dd2d99","html_url":"https://github.com/ydylla/fcache","commit_stats":null,"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ydylla%2Ffcache","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ydylla%2Ffcache/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ydylla%2Ffcache/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ydylla%2Ffcache/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ydylla","download_url":"https://codeload.github.com/ydylla/fcache/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247612507,"owners_count":20966768,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cache","file-cache","lru-cache"],"created_at":"2024-11-06T02:47:53.166Z","updated_at":"2026-02-16T01:02:59.491Z","avatar_url":"https://github.com/ydylla.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# fcache\n![coverage](https://img.shields.io/badge/coverage-91%25-green) [![go-report-card](https://goreportcard.com/badge/github.com/ydylla/fcache)](https://goreportcard.com/report/github.com/ydylla/fcache) [![reference](https://pkg.go.dev/badge/github.com/ydylla/fcache.svg)](https://pkg.go.dev/github.com/ydylla/fcache)\n\nfcache is a file based persistent blob cache. It can be used to bring remote files closer to applications.\n\n\n## Features\n* Least recently used (LRU) -ish eviction strategy\n* Optional time to live (TTL) per entry\n* Configurable size limit\n* Request coalescing, to atomically query and insert entries and avoid [cache stampedes](https://en.wikipedia.org/wiki/Cache_stampede)\n* Persistent: reloads cache state from disk after restart\n* Usage statistics\n* Bring your own key hashing algorithm (for example [xxHash](https://github.com/cespare/xxhash))\n\n\n## Installation\n```shell\ngo get github.com/ydylla/fcache\n```\n\n## Usage\nTo build a new cache use the [Builder](https://pkg.go.dev/github.com/ydylla/fcache#Builder):\n```go\ncache, err := fcache.Builder(\"/tmp/fcache\", 10*fcache.GiB).Build()\n```\nThe main functions are documented at the [Cache](https://pkg.go.dev/github.com/ydylla/fcache#Cache) interface.\n\n## How it Works\nEach cache entry is saved in its own file. On a successful get query the cache responds with a `io.ReadSeekCloser` backed by an `io.File`.\nThe caller is responsible for closing the file handle after he is done reading.\nOn each insert the old file is removed and a new one is created.\nEviction happens in background after insert and only if the amount of time specified by the eviction interval has passed.\n\n\n## Limitations\n* This cache has only limited Windows support, since it assumes it is possible to delete files that are still open.  \n  On Windows that is not the case. When you hold the `io.ReadSeekCloser` from a get query open over a long period of time and do a `Put` or `Delete` on the same key you will receive an error.\n\n* If you save millions of small entries the actual storage usage will be higher than the configured limit, due to how most file systems work.\n\n\n## Examples\n### Simple example\nThis is a simple insert \u0026 query example.\n```go\npackage main\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"github.com/cespare/xxhash/v2\"\n\t\"github.com/ydylla/fcache\"\n\t\"io\"\n\t\"time\"\n)\n\nfunc main() {\n\t// build a new cache\n\tcache, err := fcache.Builder(\"/tmp/fcache\", 10*fcache.GiB).Build()\n\tif err != nil {\n\t\tfmt.Println(\"builder failed to initialize the cache:\", err)\n\t\treturn\n\t}\n\n\t// prepare test key and data\n\tkey := xxhash.Sum64String(\"test\")\n\tdata := []byte(\"Hello World\")\n\n\t// insert entry without expiration (ttl)\n\tinfo, err := cache.Put(key, data, 0)\n\tif err != nil {\n\t\tfmt.Println(\"insert failed:\", err)\n\t\treturn\n\t}\n\n\tfmt.Printf(\"Cache entry was modified at %s\\n\", info.Mtime.Format(time.RFC3339))\n\n\t// query the cache\n\treader, info, err := cache.GetReader(key)\n\tif err != nil \u0026\u0026 !errors.Is(err, fcache.ErrNotFound) {\n\t\tfmt.Println(\"get failed for some reason:\", err)\n\t\treturn\n\t}\n\tdefer reader.Close() // remember to close the reader\n\n\tbuf, _ := io.ReadAll(reader)\n\tfmt.Printf(\"received '%s' form cache\\n\", buf)\n}\n\n```\n\n### Coalescing example\nThis example demonstrates the usage of `GetReaderOrPut` which is used to ensure that the cache fill operation is only executed once.\n```go\npackage main\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"github.com/cespare/xxhash/v2\"\n\t\"github.com/ydylla/fcache\"\n\t\"io\"\n\t\"net/http\"\n)\n\n// download downloads an url at most once, even if called concurrently\nfunc download(cache fcache.Cache, url string) (reader io.ReadSeekCloser, info *fcache.EntryInfo, hit bool, err error) {\n\t// calculate the key\n\tkey := xxhash.Sum64String(url)\n\t// atomically insert \u0026 query\n\treturn cache.GetReaderOrPut(key, 0, fcache.FillerFunc(func(key uint64, sink io.Writer) (written int64, err error) {\n\t\tfmt.Println(\"downloading\", url)\n\t\tresponse, err := http.Get(url)\n\t\tif err != nil {\n\t\t\treturn 0, err // abort insert\n\t\t}\n\t\tdefer response.Body.Close()\n\t\tif response.StatusCode != 200 {\n\t\t\treturn 0, errors.New(response.Status)\n\t\t}\n\t\t// copy response body into cache \u0026 report how many bytes where written\n\t\treturn io.Copy(sink, response.Body)\n\t}))\n}\n\nfunc main() {\n\t// build a new cache\n\tcache, err := fcache.Builder(\"/tmp/fcache\", 10*fcache.GiB).Build()\n\tif err != nil {\n\t\tfmt.Println(\"builder failed to initialize the cache:\", err)\n\t\treturn\n\t}\n\n\turl := \"https://example.org\"\n\n\treader, _, hit, err := download(cache, url)\n\tif err != nil {\n\t\tfmt.Println(\"download http request or cache query failed:\", err)\n\t\treturn\n\t}\n\tdefer reader.Close() // remember to close the reader\n\n\tbuf, _ := io.ReadAll(reader)\n\tfmt.Println(\"loaded\", url, \"from cache:\", hit)\n\tfmt.Println(string(buf[:63]))\n}\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fydylla%2Ffcache","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fydylla%2Ffcache","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fydylla%2Ffcache/lists"}