{"id":13830715,"url":"https://github.com/twitter/fatcache","last_synced_at":"2025-09-28T21:30:54.135Z","repository":{"id":6182514,"uuid":"7412800","full_name":"twitter/fatcache","owner":"twitter","description":"Memcache on SSD","archived":true,"fork":false,"pushed_at":"2021-11-01T18:32:51.000Z","size":738,"stargazers_count":1301,"open_issues_count":11,"forks_count":178,"subscribers_count":207,"default_branch":"master","last_synced_at":"2024-09-27T03:22:17.635Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/twitter.png","metadata":{"files":{"readme":"README.md","changelog":"ChangeLog","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2013-01-02T20:29:16.000Z","updated_at":"2024-09-11T15:35:41.000Z","dependencies_parsed_at":"2022-09-01T22:10:51.251Z","dependency_job_id":null,"html_url":"https://github.com/twitter/fatcache","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/twitter%2Ffatcache","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/twitter%2Ffatcache/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/twitter%2Ffatcache/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/twitter%2Ffatcache/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/twitter","download_url":"https://codeload.github.com/twitter/fatcache/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":234563138,"owners_count":18853060,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-04T10:01:06.519Z","updated_at":"2025-09-28T21:30:48.816Z","avatar_url":"https://github.com/twitter.png","language":"C","readme":"# fatcache \n\n[![status: retired](https://opensource.twitter.dev/status/retired.svg)](https://opensource.twitter.dev/status/#retired)\n[![Build Status](https://travis-ci.org/twitter/fatcache.png?branch=master)](https://travis-ci.org/twitter/fatcache)\n\nfatcache is no longer actively maintained.  See [twitter/pelikan](https://github.com/twitter/pelikan) for our latest caching work.\n\n**fatcache** is memcache on SSD. Think of fatcache as a cache for your big data.\n\n## Overview\n\nThere are two ways to think of SSDs in system design. One is to think of SSD as an extension of disk, where it plays the role of making disks fast and the other is to think of them as an extension of memory, where it plays the role of making memory fat. The latter makes sense when persistence (non-volatility) is unnecessary and data is accessed over the network. Even though memory is thousand times faster than SSD, network connected SSD-backed memory makes sense, if we design the system in a way that network latencies dominate over the SSD latencies by a large factor.\n\nTo understand why network connected SSD makes sense, it is important to understand the role distributed memory plays in large-scale web architecture. In recent years, terabyte-scale, distributed, in-memory caches have become a fundamental building block of any web architecture. In-memory indexes, hash tables, key-value stores and caches are increasingly incorporated for scaling throughput and reducing latency of persistent storage systems. However, power consumption, operational complexity and single node DRAM cost make horizontally scaling this architecture challenging. The current cost of DRAM per server increases dramatically beyond approximately 150 GB, and power cost scales similarly as DRAM density increases.\n\nFatcache extends a volatile, in-memory cache by incorporating SSD-backed storage.\n\nSSD-backed memory presents a viable alternative for applications with large workloads that need to maintain high hit rate for high performance. SSDs have higher capacity per dollar and lower power consumption per byte, without degrading random read latency beyond network latency.\n\nFatcache achieves performance comparable to an in-memory cache by focusing on two design criteria:\n\n- Minimize disk reads on cache hit\n- Eliminate small, random disk writes\n\nThe latter is important due to SSDs' unique write characteristics. Writes and in-place updates to SSDs degrade performance due to an erase-and-rewrite penalty and garbage collection of dead blocks. Fatcache batches small writes to obtain consistent performance and increased disk lifetime.\n\nSSD reads happen at a page-size granularity, usually 4 KB. Single page read access times are approximately 50 to 70 usec and a single [commodity SSD](http://ark.intel.com/products/56569/Intel-SSD-320-Series-600GB-2_5in-SATA-3Gbs-25nm-ML) can sustain nearly 40K read IOPS at a 4 KB page size. 70 usec read latency dictates that disk latency will overtake typical network latency after a small number of reads. Fatcache reduces disk reads by maintaining an in-memory index for all on-disk data.\n\n## Batched Writes\n\nThere have been attempts to use an SSD as a swap layer to implement SSD-backed memory. This method degrades write performance and SSD lifetime with many small, random writes. Similar issues occur when an SSD is simply mmaped.\n\nTo minimize the number of small, random writes, fatcache treats the SSD as a log-structured object store. All writes are aggregated in memory and written to the end of the circular log in batches - usually multiples of 1 MB.\n\nBy managing an SSD as a log-structured store, no disk updates are in-place and objects won't have a fixed address on disk. To locate an object, fatcache maintains an in-memory index. An on-disk object without an index entry is a candidate for garbage collection, which occurs during capacity-triggered eviction.\n\n## In-memory index\n\nFatcache maintains an in-memory index for all data stored on disk. An in-memory index serves two purposes:\n\n- Cheap object existence checks\n- On-disk object address storage\n\nAn in-memory index is preferable to an on-disk index to minimize disk lookups to locate and read an object. Furthermore, in-place index updates become complicated by an SSD's unique write characteristics. An in-memory index avoids these shortcomings and ensures there are no disk accesses on cache miss and only a single disk access on cache hit.\n\nMaintaining an in-memory index of all on-disk data requires a compact representation of the index. The fatcache index has the following format:\n\n```c\nstruct itemx {\n  STAILQ_ENTRY(itemx) tqe;    /* link in index / free q */\n  uint8_t             md[20]; /* sha1 message digest */\n  uint32_t            sid;    /* owner slab id */\n  uint32_t            offset; /* item offset from owner slab base */\n  rel_time_t          expiry; /* expiry in secs */\n  uint64_t            cas;    /* cas */\n} __attribute__ ((__packed__));\n```\n\nEach index entry contains both object-specific information (key name, \u0026c.) and disk-related information (disk address, \u0026c.). The entries are stored in a chained hash table. To avoid long hash bin traversals, the number of hash bins is fixed to the expected number of index entries.\n\nTo further reduce the memory consumed by the index, we store the SHA-1 hash of the key in each index entry, instead of the key itself. The SHA-1 hash acts as the unique identifier for each object. The on-disk object format contains the complete object key and value. False positives from SHA-1 hash collisions are detected after object retrieval from the disk by comparison with the requested key. If there are collisions on the write path, new objects with the same hash key simply overwrite previous objects.\n\nThe index entry (struct itemx) on a 64-bit system is 48 bytes in size. It is possible to further reduce index entry size to 32 bytes, if CAS is unsupported, MD5 hashing is used, and the next pointer is reduced to 4 bytes.\n\nAt this point, it is instructive to consider the relative size of fatcache's index and the on-disk data. With a 44 byte index entry, an index consuming 48 MB of memory can address 1M objects. If the average object size is 1 KB, then a 48 MB index can address 1 GB of on-disk storage - a 23x memory overcommit. If the average object size is 500 bytes, then a 48 MB index can address 500 MB of SSD - a 11x memory overcommit. Index size and object size relate in this way to determine the addressable capacity of the SSD.\n\n## Build\n\nTo build fatcache from a [distribution tarball](http://code.google.com/p/fatcache/downloads/list):\n\n    $ ./configure\n    $ make\n    $ sudo make install\n\nTo build fatcache from a [distribution tarball](http://code.google.com/p/fatcache/downloads/list) in _debug mode_:\n\n    $ CFLAGS=\"-ggdb3 -O0\" ./configure --enable-debug=full\n    $ make\n    $ sudo make install\n\nTo build fatcache from source with _debug logs enabled_ and _assertions disabled_:\n\n    $ git clone git@github.com:twitter/fatcache.git\n    $ cd fatcache\n    $ autoreconf -fvi\n    $ ./configure --enable-debug=log\n    $ make\n    $ src/fatcache -h\n\n## Help\n\n    Usage: fatcache [-?hVdS] [-o output file] [-v verbosity level]\n               [-p port] [-a addr] [-e hash power]\n               [-f factor] [-n min item chunk size] [-I slab size]\n               [-i max index memory[ [-m max slab memory]\n               [-z slab profile] [-D ssd device] [-s server id]\n\n    Options:\n      -h, --help                  : this help\n      -V, --version               : show version and exit\n      -d, --daemonize             : run as a daemon\n      -S, --show-sizes            : print slab, item and index sizes and exit\n      -o, --output=S              : set the logging file (default: stderr)\n      -v, --verbosity=N           : set the logging level (default: 6, min: 0, max: 11)\n      -p, --port=N                : set the port to listen on (default: 11211)\n      -a, --addr=S                : set the address to listen on (default: 0.0.0.0)\n      -e, --hash-power=N          : set the item index hash table size as a power of two (default: 20)\n      -f, --factor=D              : set the growth factor of slab item sizes (default: 1.25)\n      -n, --min-item-chunk-size=N : set the minimum item chunk size in bytes (default: 84 bytes)\n      -I, --slab-size=N           : set slab size in bytes (default: 1048576 bytes)\n      -i, --max-index-memory=N    : set the maximum memory to use for item indexes in MB (default: 64 MB)\n      -m, --max-slab-memory=N     : set the maximum memory to use for slabs in MB (default: 64 MB)\n      -z, --slab-profile=S        : set the profile of slab item chunk sizes (default: n/a)\n      -D, --ssd-device=S          : set the path to the ssd device file (default: n/a)\n      -s, --server-id=I/N         : set fatcache instance to be I out of total N instances (default: 0/1)\n\n## Performance\n\n- Initial performance results are available [here](https://github.com/twitter/fatcache/blob/master/notes/performance.md).\n\n## Future Work\n\n- fatcache deals with two kinds of IOs - disk IO and network IO. Network IO in fatcache is async, but disk IO is sync. It is recommended to run multiple instances of fatcache on a single machine to exploit CPU and SSD parallelism. However, by making disk IO async (using libaio, perhaps), it would be possible for a single instance to completely exploit all available SSD device parallelism.\n- observability in fatcache through stats\n\n## Issues and Support\n\nHave a bug or question? Please create an issue here on GitHub!\n\nhttps://github.com/twitter/fatcache/issues\n\n## Contributors\n\n* Manju Rajashekhar ([@manju](https://twitter.com/manju))\n* Yao Yue ([@thinkingfish](https://twitter.com/thinkingfish))\n\n## License\n\nCopyright 2013 Twitter, Inc.\n\nLicensed under the Apache License, Version 2.0: http://www.apache.org/licenses/LICENSE-2.0\n","funding_links":[],"categories":["Memcached forks and evolutions","C"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftwitter%2Ffatcache","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftwitter%2Ffatcache","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftwitter%2Ffatcache/lists"}