{"id":17364354,"url":"https://github.com/arcward/keyquarry","last_synced_at":"2026-05-13T18:38:01.693Z","repository":{"id":209360103,"uuid":"723803026","full_name":"arcward/keyquarry","owner":"arcward","description":"In-memory key-value store in Go, with database-backed snapshotting, and telemetry. Uses GRPC.","archived":false,"fork":false,"pushed_at":"2024-02-04T23:23:33.000Z","size":293,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-04-20T19:03:32.513Z","etag":null,"topics":["go","golang","grpc","grpc-go","key-value","key-value-store","rpc"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/arcward.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-11-26T19:59:47.000Z","updated_at":"2025-01-13T16:54:07.000Z","dependencies_parsed_at":"2023-11-26T23:22:49.224Z","dependency_job_id":"df938f23-1429-41db-b582-1094512fe57c","html_url":"https://github.com/arcward/keyquarry","commit_stats":null,"previous_names":["arcward/gokv"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/arcward/keyquarry","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arcward%2Fkeyquarry","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arcward%2Fkeyquarry/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arcward%2Fkeyquarry/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arcward%2Fkeyquarry/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/arcward","download_url":"https://codeload.github.com/arcward/keyquarry/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arcward%2Fkeyquarry/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32995909,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-13T13:14:54.681Z","status":"ssl_error","status_checked_at":"2026-05-13T13:14:51.610Z","response_time":115,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["go","golang","grpc","grpc-go","key-value","key-value-store","rpc"],"created_at":"2024-10-15T20:07:57.469Z","updated_at":"2026-05-13T18:38:01.673Z","avatar_url":"https://github.com/arcward.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# keyquarry\r\n\r\nAn in-memory key-value store with gRPC API.\r\n\r\n## Quickstart\r\n\r\n### Build\r\n\r\n```shell\r\n$ make build\r\nbuilt ./dist/bin/keyquarry\r\n```\r\n\r\n### Run the server\r\n\r\nBy default, the server listens on `localhost:33969`, without TLS:\r\n\r\n```shell\r\n$ ./dist/bin/keyquarry serve\r\ntime=2023-11-26T15:13:50.083-05:00 level=INFO msg=\"starting server\" address=:33969 config.ssl-certfile=\"\" config.ssl-keyfile=\"\" config.backup=\"\" config.MaxNumberOfKeys=0 config.DefaultMaxValueSize=1000000 config.History.RevisionLimit=5\r\n```\r\n\r\nAlternatively, you can specify a unix socket like:\r\n\r\n```shell\r\n$ ./dist/bin/keyquarry serve --listen unix:///tmp/keyquarry.sock\r\n```\r\n\r\nOr bind to all interfaces, accepting connections from anywhere:\r\n\r\n```shell\r\n$ ./dist/bin/keyquarry serve --listen :39969\r\n```\r\n\r\n### Client usage\r\n\r\nSet a key `foo` to value `bar`:\r\n\r\n```shell\r\n$ ./dist/bin/keyquarry client set foo bar\r\n{\"success\":true,\"isNew\":true}\r\n```\r\n\r\nGet the value of `foo`:\r\n\r\n```shell\r\n$ ./dist/bin/keyquarry client get foo\r\nbar\r\n```\r\n\r\nUpdate the value of `foo`:\r\n\r\n```shell\r\n$ ./dist/bin/keyquarry client set foo baz\r\n$ ./dist/bin/keyquarry client get foo\r\nbaz\r\n```\r\n\r\nGet metadata for `foo`, indenting the output with two spaces:\r\n\r\n```shell\r\n$ ./dist/bin/keyquarry client --indent=2 info foo\r\n{\r\n  \"key\":\"foo\",\r\n  \"hash\":16101355973854746,\r\n  \"created\":{\"seconds\":1704058237,\"nanos\":591124661},\r\n  \"version\":1,\r\n  \"size\":3,\r\n  \"content_type\":\"text/plain; charset=utf-8\"\r\n  }\r\n```\r\n\r\nNote: `version` is incremented whenever the value of a key is updated, based\r\non the hash of the value.\r\n\r\nLock `foo`, fail to update its value, then unlock and delete `foo`:\r\n\r\n```shell\r\n$ ./dist/bin/keyquarry client lock foo 30s\r\n{\"success\":true}\r\n$ ./dist/bin/keyquarry client --client-id=bar set foo baz\r\ntime=2023-11-29T14:57:53.813-05:00 level=ERROR source=/home/edward/sdk/keyquarry/cmd/root.go:498 msg=\"rpc error: code = PermissionDenied desc = key is locked\"\r\nexit status 1\r\n$ ./dist/bin/keyquarry client unlock foo\r\n{\"success\":true}\r\n$ ./dist/bin/keyquarry client delete foo\r\n{\"deleted\":true}\r\n```\r\n\r\nBy default, `--client-id` is set as `{user}@{hostname}`. You can override this.\r\nOnly the client ID that locked a key can unlock it, extend the lock timeout,\r\nor get/set the value.\r\n\r\n### Backup/restore\r\n\r\nYou can persist the current state of the server to a SQLite or Postgres\r\ndatabase. First, set `KEYQUARRY_SNAPSHOT_ENABLED=true`.\r\n\r\nSet  `KEYQUARRY_SNAPSHOT_DATABASE` environment  variable to your connection \r\nstring (ex: `sqlite:///path/to/file.db` or \r\n`postgres://user:password@host:port/dbname`).\r\n\r\nBy default, the server will load this state on start (if any snapshots exist),\r\nand save its state on shutdown. To disable loading on start, set\r\n`KEYQUARRY_START_FRESH=true`. To create new snapshots periodically,\r\nset `KEYQUARRY_SNAPSHOT_INTERVAL` with a duration (ex: `5m`). If no interval\r\nis specified, a snapshot will only be created on shutdown.\r\n\r\nOn startup, it will look for the most recent snapshot record based on `KEYQUARRY_NAME` (by \r\ndefault, set as `{user}@{host}`), attempting to restore from that data.\r\n\r\n## Methods\r\n\r\n### Register\r\n\r\nAdds a new client ID to the server. This is used to identify the client\r\nin other methods for the purpose of locking/unlocking keys. Subsequent\r\ncalls of other methods will fail if the client ID is not registered,\r\nor included in the request metadata as `client_id`.\r\n\r\n### Set\r\n\r\nCreates a new `key` with the provided `value`. If the key already exists,\r\nit will be updated. \r\n\r\nIf `lock_duration` is provided, the key will be locked for that duration, \r\nand automatically unlocked after it has passed.\r\n\r\nIf `lifespan` is provided, the key will be deleted after that duration. The\r\nlifespan will be extended if the key is updated before it expires. Subsequent\r\ncalls to `Set` with a lifespan will reset the lifespan to the given duration.\r\nThis takes precedence over `lock_duration`, in that the key may be deleted\r\nprior to being automatically unlocked, if the lifespan is shorter than the\r\nlock duration.\r\n\r\nIf a key is locked, only the `client_id` that set the lock can update the key.\r\n\r\n### Lock\r\n\r\nLocks a key for the given duration. If the key is already locked, the lock\r\nwill be extended by the given duration. If the key is not already locked, a new\r\nlock is created. After the lock duration has passed, the key will be\r\nautomatically unlocked.\r\n\r\nLocked keys can only be updated by the same `client_id` that created the lock.\r\n\r\nIf the lock duration is shorter than the lifespan of the key, the key may be\r\ndeleted prior to the lock expiring.\r\n\r\nCreating/updating a lock will not reset the lifespan of the key.\r\n\r\n### Unlock\r\n\r\nUnlocks a key. If the key is not locked, nothing will happen. Only the\r\nsame `client_id` that created the lock can unlock the key.\r\n\r\nUnlocking a key will not reset the lifespan of the key.\r\n\r\n### Get\r\n\r\nGets the value of a key. If the key does not exist, or the key is\r\ncurrently locked by a different `client_id`, this will return an error.\r\n\r\n### Inspect\r\n\r\nInspect returns metadata about a key. If the key does not exist, this will\r\nreturn an error. \r\n\r\nIf `include_value` is true, the value of the key will be included in the response,\r\nunless the key is currently locked by another `client_id`, in which case an error\r\nwill be returned.\r\n\r\nIf `include_metrics` is true, additional access/set/lock data will be included in \r\nthe response.\r\n\r\n### Delete\r\n\r\nDeletes a key. If the key does not exist, or is currently locked\r\nby a different `client_id`, this will return an error.\r\n\r\n### Exists\r\n\r\nIndicates whether a key exists.\r\n\r\n### Pop\r\n\r\nGets the value of a key, and deletes the key. If the key does not exist, or is\r\ncurrently locked by a different `client_id`, this will return an error.\r\n\r\n### ListKeys\r\n\r\nReturns a list of keys. Results can be filtered with `pattern`, which,\r\nif provided, must be a valid regex for `regexp.MatchString`. Limit the\r\nnumber of results with `limit` (0 for no limit).\r\n\r\n### WatchStream\r\n\r\nReturns a stream of events. Watch specific keys with `keys` (if empty, all\r\nkeys will be watched). Filter for specific events with `events` (if empty,\r\nall events will be watched).\r\n\r\n### WatchKeyValue\r\n\r\nReturns a stream which will emit the value of a key whenever it is updated.\r\nThe current value of the key will be emitted immediately. Additional metadata\r\nabout the key is also provided. Values will only be sent when a key is\r\ncreated, updated, deleted, expired or expunged. Use `KeyEvent` to determine\r\nwhether the value was actually updated, and when the key no longer exists.\r\n\r\nIf the key is locked by another `client_id`, values will not be sent until\r\nthe key is unlocked. For an `Unlock` event, the current value will be\r\nsent immediately.\r\n\r\nFor `Deleted`, `Expired` or `Expunged` events, the value of the key at\r\nthe time of the event will be sent, whether or not it has changed.\r\n\r\n### GetRevision\r\n\r\nGets the value of a key for a specific revision. If the key does not exist,\r\nor the revision does not exist, this will return an error.\r\n\r\n### GetKeyMetric\r\n\r\nReturns metrics related to a key. If the key does not exist, this will return\r\nan error.\r\n\r\n## Server configuration\r\n\r\nExample YAML configuration:\r\n\r\n```yaml\r\nlisten_address: localhost:33969\r\nname: foo@bar\r\ngraceful_shutdown_timeout: 30s\r\n\r\nlog_level: NOTICE\r\nlog_json: false\r\nlog_events: false\r\n\r\nevent_stream_buffer_size: 1000\r\nevent_stream_send_timeout: 1s\r\nevent_stream_subscriber_limit: 0\r\n\r\nsnapshot:\r\n  database: postgres://keyquarry:keyquarry@localhost:5432/keyquarry\r\n  enabled: true\r\n  interval: 5m\r\n\r\nmonitor_address: localhost:33970\r\nmetrics: false\r\nexpvar: false\r\npprof: false\r\ntrace: false\r\nservice_name: keyquarry\r\n\r\nrevision_limit: 5\r\nmax_key_length: 1024\r\nmax_value_size: 1000000\r\nmin_lock_duration: 5s\r\nmax_lock_duration: 5m\r\nmin_lifespan: 5s\r\n\r\nmax_keys: 10000\r\nprune_interval: 1h\r\nprune_at: 10000\r\nprune_to: 9000\r\neager_prune_at: 20000\r\neager_prune_to: 15000\r\n```\r\n\r\n### List of options\r\n\r\nEach of these options can be set as an environment variable, or in a YAML\r\nconfiguration file. The environment variable name is the same as the option,\r\nprefixed with `KEYQUARRY_`, uppercased, with any period replaced by an underscore. \r\nFor example, `listen_address` can be set as `KEYQUARRY_LISTEN_ADDRESS` in the\r\nenvironment, while `snapshot.database` would be `KEYQUARRY_SNAPSHOT_DATABASE`\r\nin the environment.\r\n\r\n- `name`\r\n- `service_name`\r\n- `listen_address`\r\n- `graceful_shutdown_timeout`\r\n- `log_level`\r\n- `log_json`\r\n- `log_events`\r\n- `event_stream_buffer_size`\r\n- `event_stream_send_timeout`\r\n- `event_stream_subscriber_limit`\r\n- `monitor_address`\r\n- `prometheus`\r\n- `expvar`\r\n- `pprof`\r\n- `trace`\r\n- `max_key_length`\r\n- `max_keys`\r\n- `max_value_size`\r\n- `min_lock_duration`\r\n\r\n## Docker\r\n\r\n### Build the image\r\n\r\n```shell\r\n$ docker build -t keyquarry:dev .\r\n```\r\n\r\n(Or `docker compose build server`)\r\n\r\n### Run the server, snapshot DB and jaeger\r\n\r\n```shell\r\n$ docker compose up -d\r\n```\r\n\r\nThe compose file enables telemetry by default (`KEYQUARRY_TRACE=true`), and traces\r\ncan be viewed in jaeger at `http://localhost:16686`\r\n\r\nYou can monitor events with the client with `docker compose logs -f client`.\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Farcward%2Fkeyquarry","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Farcward%2Fkeyquarry","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Farcward%2Fkeyquarry/lists"}