{"id":13902451,"url":"https://github.com/xrrocha/kmemimg","last_synced_at":"2026-01-26T09:41:04.601Z","repository":{"id":75304968,"uuid":"487907163","full_name":"xrrocha/kmemimg","owner":"xrrocha","description":"Simple Kotlin implementation of Memory Image pattern","archived":false,"fork":false,"pushed_at":"2022-05-12T17:38:50.000Z","size":1708,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-08-07T22:33:46.832Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Kotlin","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/xrrocha.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2022-05-02T16:05:00.000Z","updated_at":"2023-08-17T15:48:17.000Z","dependencies_parsed_at":"2023-06-06T02:15:34.741Z","dependency_job_id":null,"html_url":"https://github.com/xrrocha/kmemimg","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xrrocha%2Fkmemimg","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xrrocha%2Fkmemimg/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xrrocha%2Fkmemimg/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xrrocha%2Fkmemimg/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/xrrocha","download_url":"https://codeload.github.com/xrrocha/kmemimg/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":226310598,"owners_count":17604610,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-06T22:01:09.267Z","updated_at":"2026-01-26T09:41:04.574Z","avatar_url":"https://github.com/xrrocha.png","language":"Kotlin","funding_links":[],"categories":["Kotlin"],"sub_categories":[],"readme":"# Memory Image in Kotlin\n\n\u003e When people start an enterprise application, one of the earliest questions is \"how do we talk to the database\".\n\u003e These days they may ask a slightly different question:\n\u003e \"what kind of database should we use - relational or one of these NOSQL databases?\".\n\u003e \n\u003e _But there's another question to consider: \"should we use a database at all?_\"\n\u003e\n\u003e -- [Martin Fowler](https://www.martinfowler.com/)\n\u003e\n\n![TL;DR](docs/tl-dr.png)\nThis article presents a [simple Kotlin implementation](https://github.com/xrrocha/kmemimg) of the\n[Memory Image](https://www.martinfowler.com/bliki/MemoryImage.html) architectural pattern:\n\n![TL;DR](docs/mini-kmemimg.png)\n\n```kotlin\n// Skeletal Kotlin TL;DR for the truly impatient:\nclass MemoryImageProcessor(private val system: Any, \n                           private val eventSourcing: EventSourcing) {\n    init {\n        synchronized(this) {\n            eventSourcing.replay\u003cCommand\u003e { command -\u003e command.applyTo(system) }\n        }\n    }\n\n    fun execute(command: Command): Unit = synchronized(this) {\n        TxManager.begin()\n        try {\n            command.applyTo(system)\n            eventSourcing.append(command)\n        } catch (e: Exception) {\n            TxManager.rollback()\n            throw e\n        }\n    }\n\n    fun execute(query: Query): Any? = query.extractFrom(system)\n}\n```\n\n## What is Memory Image?\nMemory image provides a reliable persistence mechanism where all application data resides _safely_ in main memory.\nYes: good ole', volatile, random-access memory.\n\nProvided you domain model fits in main memory (which is cheap and abundant nowadays) this approach yields significant\nbenefits over the established database-centric approach:\n\n- Substantially faster transaction and query processing times as the system operates at RAM speeds!\n- No [object-relation impedance mismatch](https://en.wikipedia.org/wiki/Object%E2%80%93relational_impedance_mismatch)\n  to speak of as objects only reside natively in memory. No implementation-level \n  [ORM](https://en.wikipedia.org/wiki/Object%E2%80%93relational_mapping) limitations bubbling up to the design\n- Much richer domain models leveraging advanced language and platform features, unhampered by persistence concerns.\n  [DDD](https://en.wikipedia.org/wiki/Domain-driven_design) nirvana; the cure for\n  [anemic domain models](https://en.wikipedia.org/wiki/Anemic_domain_model) 😉\n\n## Hmm... Please Elaborate\n\nRather than persisting domain entities as such (as is done, typically, on a database) in the memory image approach\nwhat gets persisted is _the sequence of application events that modify the state of the entities in the first place_.\n\nConsider the following minimalistic bank domain model:\n\n![Bank Domain](docs/bank-domain.png)\n\nHere, a bank holds a collection of accounts each having a balance that changes in time as it responds to events such as:\n\n- deposits\n- withdrawals\n- transfers\n\nEach such an event can be modeled as a \n[_command_](https://en.wikipedia.org/wiki/Command_pattern) that, when applied to an account, modifies the account's\nbalance to reflect the banking operation embodied by the command.\n\nThis could be modeled as:\n\n![Bank](docs/bank.png)\n\nLet's take a look at a progression of commands and the evolving system state resulting from their successive\napplication:\n\n![Bank](docs/command-progression.png)\n\nThe key idea behind the memory image pattern is:\n\n\u003e Serialize all state-modifying commands on persistent storage.\n\u003e \n\u003e Reconstruct in-memory application state by replaying the deserialized commands onto an empty initial state\n\nSomewhat paradoxically, entity classes themselves don't need to be persisted! (But they might, for snapshotting, as \nmentioned below.)\n\nIf your application data fits into available memory and your event history fits into available disk space then \nyou're in business.\n\n## Memory Image Processor\n\nA _memory image processor_ consumes a stream of application commands by applying each incoming command to a mutable \nobject (here referred to as the _system_).\n\n![Bank](docs/kmemimg-1.png)\n\nBecause applying commands in memory is so fast and cheap, _a memory image processor can run on a single thread_ and \nconsume incoming command sequentially with no need to provide concurrent access to its system. This removes \nmuch of the complexity traditionally associated with transactions as conflicts arising from concurrent mutation \nsimply do not occur.\n\n![Bank](docs/kmemimg-2.png)\n\nIndividual command application, however, _can_ fail in the midst of a transactional sequence of mutations so the memory \nimage processor still needs a way to rollback partial changes and restore system integrity in the face of invalid data\nand constraint violations.\n\n![Bank](docs/kmemimg-3.png)\n\nIncoming commands should only be serialized upon successful completion. Obviously, if command serialization\nfails, the memory image processor will stop processing further commands until command serialization is\nrestored.\n\n![Bank](docs/kmemimg-4.png)\n\nLastly (and crucially!), a memory image processor also services queries. \n\nA _query_ is another type of event which, unlike commands, does not mutate system state. Importantly, _queries are \nserviced in multi-threaded mode_ so querying the system is efficient and concurrent. Because in-memory access \nis so fast, many queries can be satisfied without indexing. However, special-purpose, in-memory indexing can be \neasily implemented as dictated by application requirements.\n\n![Bank](docs/kmemimg-5.png)\n\nThe following class diagram puts it all together:\n\n![Memory Image](docs/kmemimg.png)\n\n👉 Because application restarts entail reprocessing the (potentially large) history of all mutating commands,\n_system snapshotting_ can be used to serialize the entire in-memory state on demand. This enables faster restarts\nat the expense of losing the ability to time-travel.\n\n## Memory Image Processor in Kotlin\n\nThe above class diagram is materialized in Kotlin as:\n\n```kotlin\ninterface Command { fun applyTo(system: Any) }\n\ninterface Query { fun extractFrom(system: Any): Any? }\n\ninterface EventSourcing {\n    fun append(event: Any)\n    fun \u003cE\u003e replay(eventConsumer: (E) -\u003e Unit)\n}\n\ninterface TxManager {\n  fun begin()\n  fun \u003cT\u003e remember(who: Any, what: String, value: T, undo: (T) -\u003e Unit)\n  fun rollback()\n  companion object: TxManager { ... } // thread-local tx\n}\n\nclass MemoryImageProcessor(private val system: Any, \n                           private val eventSourcing: EventSourcing) {\n    init {\n        // Replay previously serialized events to restore in-memory system state\n        synchronized(this) {\n            // Any failure during initialization will be propagated\n            eventSourcing.replay\u003cCommand\u003e { command -\u003e command.applyTo(system) }\n        }\n    }\n\n    // Apply incoming command to system\n    fun execute(command: Command): Unit = synchronized(this) { // Single-threaded\n        TxManager.begin()\n        try {\n            command.applyTo(system) // Try and apply command\n            try {\n                eventSourcing.append(command) // Serialize; retry internally if needed\n            } catch (e: Exception) {\n                // Note: no attempt to rollback: this is unrecoverable\n                logger.severe(\"Error persisting command: ${e.message ?: e.toString()}\")\n                // No further processing; start over when serialization is restored\n                throw e\n            }\n        } catch (e: Exception) {\n            TxManager.rollback() // Undo any partial mutation\n            val errorMessage = \"Error executing command: ${e.message ?: e.toString()}\"\n            // It's (kinda) ok for a command to fail\n            // Re-throw as «CommandApplicationException» and go on\n            throw CommandApplicationException(\"Error executing command: ${e.message}\", e)\n        }\n    }\n\n    // Run incoming query on system\n    fun execute(query: Query): Any? = query.extractFrom(system) // Can be multi-threaded\n}\n```\n\n## Simple Example: Bank Domain Model\n\nTo exercise the above memory image processor, let's revisit our bank domain model:\n\n![Bank](docs/bank.png)\n\nThis is implemented in Kotlin as:\n\n```kotlin\n/* 1) Domain entities: Bank and Account */\ntypealias Amount = BigDecimal\n\ndata class Bank(val accounts: MutableMap\u003cString, Account\u003e = HashMap())\n\ndata class Account(val id: String, val name: String) {\n    var balance: Amount by TxDelegate(initialValue = Amount.ZERO) { \n      // tiggers rollback on validation failure\n      it \u003e= Amount.ZERO \n    }\n}\n\n/* 2) Application commands: CreateAccount, Deposit, Withdrawal, Transfer */\ndata class CreateAccount(val id: String, val name: String) : BankCommand {\n    override fun applyTo(bank: Bank) {\n        bank.accounts[id] = Account(id, name)\n    }\n}    \ndata class Deposit(override val accountId: String, val amount: Amount) : AccountCommand {\n    override fun applyTo(account: Account) {\n        account.balance += amount\n    }\n}\ndata class Withdrawal(override val accountId: String,val amount: Amount) : AccountCommand {\n    override fun applyTo(account: Account) {\n        account.balance -= amount\n    }\n}\ndata class Transfer(val fromAccountId: String, val toAccountId: String, val amount: Amount) : BankCommand {\n    override fun applyTo(bank: Bank) {\n        Deposit(toAccountId, amount).applyTo(bank)\n        Withdrawal(fromAccountId, amount).applyTo(bank)\n    }\n}\n```\n\n## Simple Example: Testing The Processor\n\nThe following test exercises the memory image processor using the same sequence of commands outlined above: \n\n```kotlin\nval bank1 = Bank()\nval memimg1 = MemImg(bank1, eventSourcing)\n\nmemimg1.execute(CreateAccount(\"janet\", \"Janet Doe\"))\nassertEquals(Amount.ZERO, bank1.accounts[\"janet\"]!!.balance)\n\nmemimg1.execute(Deposit(\"janet\", Amount(100)))\nassertEquals(Amount(100), bank1.accounts[\"janet\"]!!.balance)\n\nmemimg1.execute(Withdrawal(\"janet\", Amount(10)))\nassertEquals(Amount(90), bank1.accounts[\"janet\"]!!.balance)\n\nmemimg1.execute(CreateAccount(\"john\", \"John Doe\"))\nassertEquals(Amount.ZERO, bank1.accounts[\"john\"]!!.balance)\n\nmemimg1.execute(Deposit(\"john\", Amount(50)))\nassertEquals(Amount(50), bank1.accounts[\"john\"]!!.balance)\n\nmemimg1.execute(Transfer(\"janet\", \"john\", Amount(20)))\nassertEquals(Amount(70), bank1.accounts[\"janet\"]!!.balance)\nassertEquals(Amount(70), bank1.accounts[\"john\"]!!.balance)\n\nmemimg1.close()\n\nval bank2 = Bank()\nval memimg2 = MemImg(bank2, eventSourcing)\n// Look ma: system state restored from empty initial state via event sourcing!\nassertEquals(Amount(70), bank2.accounts[\"janet\"]!!.balance)\nassertEquals(Amount(70), bank2.accounts[\"john\"]!!.balance)\n\n// Some random query; executes at in-memory speeds\nval accountsWith70 = memimg2.execute(object : BankQuery {\n    override fun extractFrom(bank: Bank) =\n        bank.accounts.values\n            .filter { it.balance == Amount(70) }\n            .map { it.name }\n            .toSet()\n})\nassertEquals(setOf(\"Janet Doe\", \"John Doe\"), accountsWith70)\n\n// Attempt to transfer beyond means...\nval insufficientFunds = assertThrows\u003cCommandApplicationException\u003e {\n    memimg2.execute(Transfer(\"janet\", \"john\", Amount(1000)))\n}\nassertContains(insufficientFunds.message!!, \"Invalid value for Account.balance\")\n// Look ma: system state restored on failure after partial mutation\nassertEquals(Amount(70), bank2.accounts[\"janet\"]!!.balance)\nassertEquals(Amount(70), bank2.accounts[\"john\"]!!.balance)\n\nmemimg2.close()\n```\n\n## Conclusion\n\nMemory image provides a simple, straightforward way to achieve high performance and simplicity without the \ncomplications associated with persisting objects on a database (whether SQL or not.)\n\nKotlin is a uniquely expressive language in which to implement this architectural pattern.  \n\nInterested readers can inspect the working code at the [kmemimg](https://github.com/xrrocha/kmemimg) Github repository.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxrrocha%2Fkmemimg","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fxrrocha%2Fkmemimg","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxrrocha%2Fkmemimg/lists"}