{"id":29182557,"url":"https://github.com/dipan-ck/atomdb","last_synced_at":"2026-04-12T22:06:28.282Z","repository":{"id":301617195,"uuid":"1008655176","full_name":"dipan-ck/atomdb","owner":"dipan-ck","description":"Redis-compatible in-memory key-value database built in Go.","archived":false,"fork":false,"pushed_at":"2025-06-27T19:47:33.000Z","size":3,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-06-27T20:35:57.459Z","etag":null,"topics":["databse","go","in-memory-database","key-value-store","redis"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dipan-ck.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-06-25T22:19:00.000Z","updated_at":"2025-06-27T19:47:36.000Z","dependencies_parsed_at":"2025-06-27T20:35:58.954Z","dependency_job_id":"e00c8146-1821-42d2-8a25-e8a81ca2acdd","html_url":"https://github.com/dipan-ck/atomdb","commit_stats":null,"previous_names":["dipan-ck/atomdb"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/dipan-ck/atomdb","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dipan-ck%2Fatomdb","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dipan-ck%2Fatomdb/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dipan-ck%2Fatomdb/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dipan-ck%2Fatomdb/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dipan-ck","download_url":"https://codeload.github.com/dipan-ck/atomdb/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dipan-ck%2Fatomdb/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263029214,"owners_count":23402354,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["databse","go","in-memory-database","key-value-store","redis"],"created_at":"2025-07-01T20:06:41.136Z","updated_at":"2025-10-29T23:07:47.583Z","avatar_url":"https://github.com/dipan-ck.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AtomDB 🚀\n\n**AtomDB** is a Redis‑compatible, in‑memory key‑value database written in Go.  \nIt’s designed as a learning project and demonstrates how a simple database server works under the hood.\n\n---\n\n## 🔎 Table of Contents\n\n1. [Why AtomDB?](#why-atomdb)  \n2. [Features](#features)  \n3. [Getting Started](#getting-started)  \n4. [Core Components](#core-components)  \n   - [1. TCP Server (`server.go`)](#1-tcp-server-servergo)  \n   - [2. RESP Parser (`parser.go`)](#2-resp-parser-parsergo)  \n   - [3. Command Router (`route.go`)](#3-command-router-routego)  \n   - [4. In‑Memory Store (`store.go`)](#4-in‑memory-store-storego)  \n   - [5. TTL Expiry (`ttl.go`)](#5-ttl-expiry-ttlgo)  \n   - [6. LRU Eviction (`lru.go`)](#6-lru-eviction-lrugo)  \n5. [Data Structures](#data-structures)  \n6. [How Commands Work (Example)](#how-commands-work-example)  \n7. [Diagrams](#diagrams)  \n8. [Future Improvements](#future-improvements)  \n9. [License](#license)\n\n## Why AtomDB?\n\nI built AtomDB to learn:\n\n- How Redis’s RESP protocol works  \n- Basic TCP servers in Go with concurrent clients  \n- Simple caching (LRU) and expiry (TTL) logic  \n- Thread‑safe in‑memory storage  \n\n\n\n## Features\n\n- 🔐 **AUTH**: per‑user secret key  \n- 💾 **SET / GET**: store and retrieve strings  \n- ⏲️ **EXPIRE**: set key time‑to‑live  \n- ♻️ **LRU**: evict least‑recently‑used keys when \u003e30  \n- ⚡ **RESP protocol**: same wire format as Redis  \n- 🧵 **Concurrency**: goroutines + mutexes\n\n\n\n## Getting Started\n\n1. **Clone Repo**\n\n   ```bash\n   git clone https://github.com/your-user/atomdb.git\n   cd atomdb\n   ```\n2. **Run the server**\n\n   ```bash\n   go run main.go\n   ```\n3. **Connect with Redis CLI**\n\n   ```bash\n   redis-cli -p 6300\n   ```\n4. **Authenticate**\n\n   ```bash\n   AUTH mySecret123\n   ```\n\n## Core Components\n\n### 1. TCP Server (`server.go`)\n\n* Listens on port (default `:6300`)\n* Accepts connections, spawns a goroutine per client\n* Tracks each client in a `client` struct:\n\n  ```go\n  type client struct {\n    conn            net.Conn      // network socket\n    reader          *bufio.Reader // RESP reader\n    isAuthenticated bool           // auth flag\n    remoteAddr      string         // client address\n    secretKey       string         // user’s secret key\n    LRU             *LRUList       // per‑user LRU cache\n  }\n  ```\n* Starts a background TTL watcher\n* Main loop:\n\n  1. Read raw RESP bytes\n  2. Parse into `[]string`\n  3. If not authenticated → only `AUTH` allowed\n  4. Else → pass to `HandleCommand`\n\n  ---\n### 2. RESP Parser (`parser.go`)\n\nThe RESP (Redis Serialization Protocol) parser is a crucial component of our database, responsible for interpreting the data sent by Redis clients. Here's a detailed breakdown of how it works:\n\n```go\nfunc ReadRESP(reader *bufio.Reader) ([]byte, error) { … }\nfunc RespParsing(input []byte) ([]string, error) { … }\n```\n\n#### How Redis Sends Data\nRedis uses the RESP protocol to communicate with clients. Commands are sent as arrays, where each element is a bulk string. For example, the command `SET name Alice` is sent as:\n\n```\n*3\n$3\nSET\n$4\nname\n$5\nAlice\n```\n\n- `*3` indicates an array of three elements.\n- `$3` followed by `SET` indicates a bulk string of length 3.\n- Similarly, `$4` and `$5` indicate the lengths of the subsequent strings `name` and `Alice`.\n\n#### Breaking Data into Slices\nThe `RespParsing` function processes this input to extract the command and its arguments:\n\n1. **Read the Array Header**: The function starts by reading the `*\u003ccount\u003e` header to determine the number of elements.\n2. **Parse Each Element**: For each element, it reads the `$\u003clen\u003e` header to know how many bytes to read for the actual data.\n3. **Store in Slice**: Each parsed element is stored in a slice, resulting in a structure like `[]string{\"SET\", \"name\", \"Alice\"}`.\n\nThis parsing allows the server to understand and execute commands sent by the client, maintaining the expected behavior.\n\n\n        \n\n### 3. Command Router (`route.go`)\n\nRoutes commands based on first token:\n\n```go\nswitch cmd {\ncase \"SET\":\n  SetKey(...)\ncase \"GET\":\n  GetKey(...)\ncase \"EXPIRE\":\n  SetTTL(...)\ndefault:\n  ERR unknown command\n}\n```\n\n* Checks argument count\n* Sends Redis‑style replies:\n\n  * `+OK `\n  * \\`-\\$len\n\n\\`\n\n* `-ERR \u003cmessage\u003e `\n\n---\n\n### 4. In‑Memory Store (`store.go`)\n\nGlobal map:\n\n```go\nvar globalStore = map[string]map[string]string{}\n// secretKey → (key → value)\n```\n\n* **SetKey**:\n\n  * Locks with `mut.Lock()`\n  * Creates user map if missing\n  * Updates value\n  * Calls `AddNode` on LRU\n* **GetKey**:\n\n  * RLocks with `mut.RLock()`\n  * Reads value\n  * Calls `RecentlyUsed` on LRU\n\n---\n\n### 5. TTL Expiry (`ttl.go`)\n\nTracks expirations:\n\n```go\nvar TTLmap = map[string]map[string]time.Time{}\n// secretKey → (key → expiryTime)\n```\n\n* **SetTTL**: parse seconds, store expiry\n* **TTLWatcher**:\n\n  * Runs every second\n  * Locks TTL and store\n  * Deletes expired keys from `globalStore` and LRU\n  * Unlocks\n\n---\n\n### 6. LRU Eviction (`lru.go`)\n\nPer‑user LRU cache:\n\n```go\ntype Node struct {\n  prev, next *Node\n  key        string\n}\ntype LRUList struct {\n  head, tail *Node\n  count      int\n  LRUnodeMap map[string]*Node\n}\n```\n\n* **AddNode**:\n\n  * If count ≥30: remove `tail` (delete from store)\n  * Insert new node at `head`\n* **RecentlyUsed**: move accessed node to `head`\n* **RemoveNode**: remove node when TTL fires\n\n\n\n\n\n\n## Data Structures\n\n### User Store\n![User Store Diagram](./diagram/user_store.svg)\n\n| Feature     | Go Type                                 | Purpose                      |\n| ----------- | --------------------------------------- | ---------------------------- |\n| User Store  | `map[string]map[string]string`          | Store data per `secretKey`   |\n\nThe User Store is a nested map structure that organizes data by user authentication secrets. Each user (identified by their `secretKey`) has their own isolated key-value store. This provides data isolation between different clients connecting to the server. The implementation uses a global variable `globalStore` protected by a mutex for thread-safe concurrent access.\n\n\n### TTL Tracker\n![TTL Tracker Diagram](./diagram/ttl_watcher.svg)\n\n| Feature     | Go Type                                 | Purpose                      |\n| ----------- | --------------------------------------- | ---------------------------- |\n| TTL Tracker | `map[string]map[string]time.Time`       | Track key expirations        |\n\nThe TTL Tracker maintains expiration times for keys. It uses a nested map structure similar to the User Store, where each user's keys are mapped to their expiration timestamps. A background goroutine (`TTLWatcher`) periodically checks for expired keys and removes them from both the User Store and the LRU Cache. This implements Redis-like key expiration functionality.\n\n### LRU Cache\n![LRU Cache Diagram](./diagram/lru_list.svg)\n\n| Feature     | Go Type                                 | Purpose                      |\n| ----------- | --------------------------------------- | ---------------------------- |\n| LRU Cache   | Doubly linked list + `map[string]*Node` | O(1) eviction operations     |\n\nThe LRU (Least Recently Used) Cache implements an efficient memory management strategy. It uses a doubly-linked list to maintain the order of key access, with the most recently used keys at the head and least recently used at the tail. A map provides O(1) lookups to list nodes. When memory limits are reached (count ≥ 30), the least recently used keys are evicted. The implementation includes operations for adding nodes, marking nodes as recently used, and removing nodes when keys expire.\n\n### Synchronization\n\n| Feature     | Go Type                                 | Purpose                      |\n| ----------- | --------------------------------------- | ---------------------------- |\n| Sync        | `sync.RWMutex`                          | Safe concurrent access       |\n\nMutexes ensure thread-safe access to shared data structures. The implementation uses read-write mutexes (`sync.RWMutex`) to allow concurrent reads but exclusive writes, optimizing performance while maintaining data consistency in a multi-client environment.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdipan-ck%2Fatomdb","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdipan-ck%2Fatomdb","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdipan-ck%2Fatomdb/lists"}