{"id":26228238,"url":"https://github.com/florents-tselai/spat","last_synced_at":"2026-04-29T12:36:31.278Z","repository":{"id":273066300,"uuid":"897305409","full_name":"Florents-Tselai/spat","owner":"Florents-Tselai","description":"Redis-like In-Memory DB Embedded in Postgres","archived":false,"fork":false,"pushed_at":"2025-02-14T07:55:35.000Z","size":51,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-10T02:02:26.859Z","etag":null,"topics":["cache","postgresql","redis","sql","valkey"],"latest_commit_sha":null,"homepage":"","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Florents-Tselai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":["Florents-Tselai"]}},"created_at":"2024-12-02T12:01:46.000Z","updated_at":"2025-02-14T07:48:27.000Z","dependencies_parsed_at":"2025-01-18T12:41:53.730Z","dependency_job_id":null,"html_url":"https://github.com/Florents-Tselai/spat","commit_stats":null,"previous_names":["florents-tselai/spat"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Florents-Tselai%2Fspat","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Florents-Tselai%2Fspat/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Florents-Tselai%2Fspat/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Florents-Tselai%2Fspat/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Florents-Tselai","download_url":"https://codeload.github.com/Florents-Tselai/spat/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243293723,"owners_count":20268139,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cache","postgresql","redis","sql","valkey"],"created_at":"2025-03-12T20:39:44.016Z","updated_at":"2025-12-25T13:19:54.958Z","avatar_url":"https://github.com/Florents-Tselai.png","language":"C","funding_links":["https://github.com/sponsors/Florents-Tselai"],"categories":[],"sub_categories":[],"readme":"# spat: Redis-like In-Memory DB Embedded in Postgres\n\n![GitHub Repo stars](https://img.shields.io/github/stars/Florents-Tselai/spat)\n[![Github](https://img.shields.io/static/v1?label=GitHub\u0026message=Repo\u0026logo=GitHub\u0026color=green)](https://github.com/Florents-Tselai/spat)\n[![Build Status](https://github.com/Florents-Tselai/spat/actions/workflows/build.yml/badge.svg)](https://github.com/Florents-Tselai/spat/actions)\n[![Docker Pulls](https://img.shields.io/docker/pulls/florents/spat)](https://hub.docker.com/r/florents/spat)\n[![License](https://img.shields.io/github/license/Florents-Tselai/spat?color=blue)](https://github.com/Florents-Tselai/spat?tab=AGPL-3.0-1-ov-file#readme)\n\n\u003e [!CAUTION]\n\u003e This is stil in **alpha** and not production ready!\n\u003e Read notes on [ACID](#ACID) below.\n\n**spat** is a Redis-like in-memory data structure server embedded in Postgres.\nData is stored in Postgres shared memory.\nThe data model is key-value.\nKeys are strings, but values can be strings, lists, sets, or hashes.\n\n```sql\nSELECT SPSET('key', 'value');\nSELECT SPGET('key'); -- value\n\nSELECT LPUSH('list1', 'elem1');\nSELECT LPUSH('list1', 'elem2');\nSELECT LPOP('list1'); -- elem2\n\nSELECT SADD('set1', 'elem1', 'elem2');\nSELECT SISMEMBER('set1', 'elem1'); -- t\n\nSELECT HSET('h1', 'f1', 'Hello');\nSELECT HGET('h1', 'f1'); -- Hello\n```\n\nWith **spat**:\n- You don't need to maintain an external caching server. This greatly reduces complexity.\n- You can cache and share expensive or static data across SQL queries without having to model them relationally.\n- You can express powerful logic by embedding data structures like lists and sets\n  in your SQL queries.\n- You can reduce your infrastructure costs by reusing server resources.\n\n## Getting Started\n\nTo quickly get a Spat instance up and running, pull and run the latest Docker image:\n\n```bash\ndocker run --name spat -e POSTGRES_PASSWORD=password florents/spat:pg17\n```\n\nThis will start a Spat instance with default user postgres and password password. You can then connect to the database using psql:\n\n```bash\ndocker exec -it spat psql -U postgres\n```\n\nThen install the extension\n\n```tsql\nCREATE EXTENSION spat;\n```\n\nFor other installation optionse see [Installation](#Installation)\n\n## Usage\n\n\u003e [!TIP]\n\u003e Development follows roughly TDD principles,\n\u003e thus, the best and most up-to-date documentation are the test cases in [test/sql](test/sql)\n\nspat `key`s are always `text`.\n\nValues can be strings (aka `text` Postgres type) or data structures containing strings (sets, lists etc.).\nYou can, however, pass `anyelement` to `SPSET,` and the value will be stored as a string according to the type's textual representation.\n\nFor example, to cache a `jsonb` object and get it back, you can do something like:\n\n```sql\nSELECT SPSET('k', '{\"a\": {\"b\": {\"c\": 1}}}'::jsonb);\n\nSELECT SPGET('k')::text::jsonb;\n```\n\nIf the value stored for a key is not a string,\nit will return a human-friendly representation of the value.\nUsually the data structure type and its current size.\n\n### Strings\n\n`SPSET(key, value[, echo bool, ttl interval]) → spval`\n\nSet key to hold the value.\nIf key already holds a value, it is overwritten, regardless of its type.\nAny previous time to live associated with the key is discarded on successful SET operation.\nThe optional `ttl` sets the TTL interval.\nReturns the value if `echo` is set. If not (the default), the function returns NULL.\n\n`SPGET(key) → text`\n\nGet the value of key.\nIf the key does not exist NULL is returned.\nAn error is returned if the value stored at key is not a string, because GET only handles string values.\n\n### Sets\n\n`SADD(key, member) → int`\n\nAdd the specified members to the set stored at key.\nSpecified members that are already a member of this set are ignored.\nIf key does not exist, a new set is created before adding the specified members.\nReturns the number of elements that were added to the set, not including all the elements already present in the set.\nAn error is returned when the value stored at key is not a set\n\n`SISMEMBER(key, member) → bool`\n\nReturns if `member` is a member of the set stored at `key`.\n\n`SREM(key, member) → int`\n\nRemove the specified members from the set stored at key.\nSpecified members that are not a member of this set are ignored.\nIf key does not exist, it is treated as an empty set and this command returns 0.\nReturns the number of members that were removed from the set, not including non existing members.\nAn error is returned when the value stored at key is not a set.\n\n`SCARD(key) → int`\n\nReturns the set cardinality (number of elements) of the set stored at key,\nor 0 if the key does not exist.\n\n### Lists\n\n`LPUSH(key, element) → int`\n\nInsert all the specified values at the head of the list stored at key.\nIf key does not exist, it is created as empty list before performing the push operations.\nReturns the length of the list after the push operation.\nWhen key holds a value that is not a list, an error is returned.\n\n`LPOP(key, element) → text`\n\nRemoves and returns the first elements of the list stored at key.\nReturns NULL if the key does not exist.\n\n`LLEN(key) → int`\n\nReturns the length of the list stored at key. If key does not exist, it is interpreted as an empty list and 0 is returned. An error is returned when the value stored at key is not a list.\n\n### Hashes\n\n`HSET(key, field, value)`\n\nSets the specified fields to their respective values in the hash stored at key.\nThis command overwrites the values of specified fields that exist in the hash. If key doesn't exist, a new key holding a hash is created.\n\n`HGET(key, field) → text`\n\nReturns the value associated with field in the hash stored at key.\n\n### Generic\n\n`SPTYPE(key) → text`\n\nReturns the string representation of the type of the value stored at key.\nThe different types that can be returned are: string, list, set, and hash.\nReturns `NULL` when key doesn't exist.\n\n`DEL(key) → boolean`\n\nRemove an entry by key.\nReturns true if the key was found and the corresponding entry was removed.\n\n`TTL(key) → interval`\n\nReturns the TTL interval of a key\n\n`GETEXPIREAT(key) → timestamptz`\n\nShorthand for `GETEXPIREAT(key) - NOW()`\n\n`SP_DB_NITEMS() → integer`\n\nReturns the current number of entries in the database.\n\n`SP_DB_SIZE_BYTES() → bigint`\n\n`SP_DB_SIZE() → text`\n\nSize of the database in bytes and in human-friendly text.\n\n### Multiple DBs\n\nA spat database is just a segment of Postgres' memory addressable by a name.\nYou can switch between different databases (namespaces really),\nby setting the `spat.db` GUC during a session.\nSubsequent operations will apply to that db only.\n\n```tsql\nSET spat.db = 'db1';\n```\n\nOnce done you can switch back to `spat-default`.\n\n```tsql\nSET spat.db = 'spat-default';\n```\n\n## Installation\n\nCompile and install the extension (supports Postgres 17+)\n\n```sh\ncd /tmp\ngit clone https://github.com/Florents-Tselai/spat.git\ncd spat\nmake\nmake install # may need sudo\n```\n### MurmurHash3\n\nTo use the MurmurHash3 hashing algorithm instead of Postgres' default (`tag_hash`)\n\n``` shell\nmake all install PG_CPPFLAGS=-DSPAT_MURMUR3=1\n```\n\nYou can also install it with [Docker](#docker)\n\n### Docker\n\n```sh\ndocker pull florents/spat\n# or with explicit version\ndocker pull florents/spat:0.1.0a4-pg17\n```\n\n## ACID\nSince spat operates in PostgreSQL **shared memory**,\nit does not follow standard PostgreSQL **transactional semantics** (e.g., **MVCC, WAL, rollbacks**).\nInstead, it behaves similarly to an **in-memory key-value store** like **Redis**.\n\n### Atomicity\n\nSpat operations take effect **immediately** and cannot be rolled back.\n\n```sql\nBEGIN;\nSELECT SPSET('foo', 'bar');\nROLLBACK;\nSELECT SPGET('foo'); -- Will still return 'bar'\n```\n* Unlike regular PostgreSQL transactions, a ROLLBACK does not undo changes in spat.\n* Once a key is set, it remains in memory until explicitly deleted.\n\n### Consistency\n\n* spat ensures **internal consistency** by using **per-entry locks** to prevent data corruption during concurrent updates.\n* However, since it **bypasses WAL logging and MVCC**,\nits updates do not integrate with **PostgreSQL’s consistency guarantees**.\n\n### Isolation\n\nPostgreSQL **transactions do not isolate shared memory changes** like regular tables\n\n**Session 1**\n```sql\nBEGIN;\nSELECT SPSET('key', 'A');\n-- Transaction remains open...\n```\n\n**Session 2**\n```sql\nBEGIN;\nSELECT SPGET('key'); -- Returns 'A', even though Session 1 hasn't committed\n```\n\n* In a standard database, **Session 2** would not see uncommitted changes from **Session 1**.\n* With spat, changes are immediately **visible across all sessions**.\n\n### Durability\n\n* Since spat is **entirely in shared memory, data is lost on restart**.\n* There is no disk persistence (yet), meaning:\n  * PostgreSQL crash or restart wipes all spat data.\n  * Unlike standard tables, spat **does not survive beyond the current instance.**\n\n### Concurrency\n\n* **Per-key locks** ensure that **only one session modifies a given key at a time**.\n* Multiple readers are allowed, but **a writer will block other writes**.\n\n## Motivation\n\nThe goal is not to completely replace or recreate Redis within Postgres.\nRedis, however, has been proven to be (arguably) a tool that excels in the “20-80” rule:\nMost use 20% of its available functionality to support the 80% of use cases.\n\nI aim to provide Redis-like semantics and data structures within SQL,\noffering good enough functionality to support that critical 20% of use cases.\nThis approach simplifies state and data sharing across queries without the need to manage a separate cache service alongside the primary database.\n\n## Background\n\nSpat relies on the two following features of Postgres\n\n- PG10 Introduced dynamic shared memory areas (DSA) in [13df76a](https://github.com/postgres/postgres/commit/13df76a)\n- PG17 Introduced the dynamic shared memory registry in [8b2bcf3](https://github.com/postgres/postgres/commit/8b2bcf3)\n\nInternally, it stores its data in a `dshash`:\nThis is an open hashing hash table with a linked list at each table entry.\nIt supports dynamic resizing to prevent the linked lists from growing too long on average.\nCurrently, only growing is supported: the hash table never becomes smaller.\n\n## FAQ\n\n**What is Spat?**\n\nSpat is a Redis-like in-memory data structure server embedded in PostgreSQL, utilizing PostgreSQL's dynamic shared memory (DSA) to store and manage key-value pairs.\n\n**How does Spat differ from Redis?**\n\nUnlike Redis, which is a standalone in-memory data store, Spat operates entirely within PostgreSQL.\n\n**Can I use Spat for caching?**\n\nYes. Since Spat stores data in memory, it is well-suited for caching scenarios where frequent access to key-value data is required.\n\n**Does Spat persist data?**\n\nNo. Currently, Spat does not provide built-in persistence, meaning data is lost when PostgreSQL is restarted.\n\n**Can I use Spat with multiple PostgreSQL sessions?**\n\nYes. Since Spat operates in shared memory, multiple PostgreSQL sessions can read and write to Spat concurrently.\n\n**Is there a limit to how much data Spat can store?**\n\nSpat’s storage capacity depends on PostgreSQL’s shared memory configuration (shared_buffers, work_mem, etc.).\nYou can adjust these settings based on your workload.\n\n**How ACID-compliant is this ?**\n\nEnough. But maybe not enough for you. See [ACID](#ACID)\n\n[//]: # (\u003cimg src=\"test/bench/plot.png\" width=\"50%\"/\u003e)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fflorents-tselai%2Fspat","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fflorents-tselai%2Fspat","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fflorents-tselai%2Fspat/lists"}