{"id":18283054,"url":"https://github.com/cysharp/mastermemory","last_synced_at":"2025-05-13T00:10:34.248Z","repository":{"id":15088277,"uuid":"77537639","full_name":"Cysharp/MasterMemory","owner":"Cysharp","description":"Source Generator based Embedded Typed Readonly In-Memory Document Database for .NET and Unity.","archived":false,"fork":false,"pushed_at":"2025-03-19T06:37:39.000Z","size":4669,"stargazers_count":1658,"open_issues_count":6,"forks_count":116,"subscribers_count":48,"default_branch":"master","last_synced_at":"2025-04-23T18:47:01.951Z","etag":null,"topics":["c-sharp","memory-database","unity"],"latest_commit_sha":null,"homepage":"","language":"C#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Cysharp.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-12-28T14:14:20.000Z","updated_at":"2025-04-22T14:21:54.000Z","dependencies_parsed_at":"2022-08-03T15:09:55.069Z","dependency_job_id":"dcf8f844-b3e2-4114-ae02-ab536b52549b","html_url":"https://github.com/Cysharp/MasterMemory","commit_stats":{"total_commits":255,"total_committers":14,"mean_commits":"18.214285714285715","dds":0.2784313725490196,"last_synced_commit":"515d77ce3ab7bf2ce0d3d415c96c6d286e164638"},"previous_names":[],"tags_count":39,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Cysharp%2FMasterMemory","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Cysharp%2FMasterMemory/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Cysharp%2FMasterMemory/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Cysharp%2FMasterMemory/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Cysharp","download_url":"https://codeload.github.com/Cysharp/MasterMemory/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253843215,"owners_count":21972873,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["c-sharp","memory-database","unity"],"created_at":"2024-11-05T13:07:07.970Z","updated_at":"2025-05-13T00:10:34.219Z","avatar_url":"https://github.com/Cysharp.png","language":"C#","readme":"[![GitHub Actions](https://github.com/Cysharp/MasterMemory/workflows/Build-Debug/badge.svg)](https://github.com/Cysharp/MasterMemory/actions) [![Releases](https://img.shields.io/github/release/Cysharp/MasterMemory.svg)](https://github.com/Cysharp/MasterMemory/releases)\n\nMasterMemory\n===\nSource Generator based Embedded Typed Readonly In-Memory Document Database for .NET and Unity. \n\n![image](https://user-images.githubusercontent.com/46207/61031896-61890800-a3fb-11e9-86b7-84c821d347a4.png)\n\n**4700** times faster than SQLite and achieves zero allocation per query. Also the DB size is small. When SQLite is 3560kb then MasterMemory is only 222kb.\n\nSource Generator automatically generates a typed database structure from schemas (classes), which ensures that all queries are type-safe with full autocompletion support.\n\n![image](https://github.com/user-attachments/assets/e804fa52-f6a5-4972-a510-0b3b17a31230)\n\n![image](https://user-images.githubusercontent.com/46207/61035808-cb58e000-a402-11e9-9209-d51665d1cd56.png)\n\nThis ensures both optimal performance and excellent usability.\n\n\u003c!-- START doctoc generated TOC please keep comment here to allow auto update --\u003e\n\u003c!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE --\u003e\n## Table of Contents\n\n- [Concept](#concept)\n- [Getting Started(.NET)](#getting-startednet)\n- [Getting Started(Unity)](#getting-startedunity)\n- [DataTable configuration](#datatable-configuration)\n- [MemoryDatabase/RangeView](#memorydatabaserangeview)\n- [Extend Table](#extend-table)\n- [ImmutableBuilder](#immutablebuilder)\n- [Validator](#validator)\n- [Metadata](#metadata)\n- [Inheritance](#inheritance)\n- [Optimization](#optimization)\n- [MasterMemoryGeneratorOptions](#mastermemorygeneratoroptions)\n- [v2 -\u003e v3 migration](#v2---v3-migration)\n- [License](#license)\n\n\u003c!-- END doctoc generated TOC please keep comment here to allow auto update --\u003e\n\nConcept\n---\n\n* **Memory Efficient**, Only use underlying data memory and do aggressively string interning.\n* **Performance**, Similar as dictionary lookup.\n* **TypeSafe**, 100% Type safe by Source Generator.\n* **Fast load speed**,  MasterMemory save data by [MessagePack for C#, a fastest C# serializer](https://github.com/neuecc/MessagePack-CSharp) so load speed is blazing fast.\n* **Flexible Search**, Supports multiple key, multiple result, range/closest query.\n* **Validator**, You can define custom data validation by C#.\n* **Metadata**, To make custom importer/exporter, get the all database metadata.\n\nThese features are suitable for master data management(write-once, read-heavy) on embedded application, data analysis, game, etc. MasterMemory has better performance than any other database solutions. [PalDB](https://github.com/linkedin/PalDB) developed by LinkedIn has a similar concept(embeddable write-once key-value store), but the implementation and performance characteristics are completely different.\n\nGetting Started(.NET)\n---\nInstall the [MasterMemory](https://www.nuget.org/packages/MasterMemory) library(Runtime, Source Generator(Analyzer) via NuGet.\n\n```\ndotnet add package MasterMemory\n```\n\nPrepare the example table definition like following.\n\n```csharp\npublic enum Gender\n{\n    Male, Female, Unknown\n}\n\n// table definition marked by MemoryTableAttribute.\n// database-table must be serializable by MessagePack-CSsharp\n[MemoryTable(\"person\"), MessagePackObject(true)]\npublic record Person\n{\n    // index definition by attributes.\n    [PrimaryKey]\n    public required int PersonId { get; init; }\n\n    // secondary index can add multiple(discriminated by index-number).\n    [SecondaryKey(0), NonUnique]\n    [SecondaryKey(1, keyOrder: 1), NonUnique]\n    public required int Age { get; init; }\n\n    [SecondaryKey(2), NonUnique]\n    [SecondaryKey(1, keyOrder: 0), NonUnique]\n    public required Gender Gender { get; init; }\n\n    public required string Name { get; init; }\n}\n```\n\nData in MasterMemory is readonly, so it is recommended to use an immutable structure. While both records and classes are supported, records might be preferable as they generate more readable ToString methods.\n\nMasterMemory's Source Generator detects types marked with the `MemoryTable` attribute and automatically generates types like the following:\n\n![image](https://github.com/user-attachments/assets/e804fa52-f6a5-4972-a510-0b3b17a31230)\n\nFinally, you can regsiter and query by these files.\n\n```csharp\nusing ...; // Your project default namespace\n\n// to create database, use DatabaseBuilder and Append method.\nvar builder = new DatabaseBuilder();\nbuilder.Append(new Person[]\n{\n    new (){ PersonId = 0, Age = 13, Gender = Gender.Male,   Name = \"Dana Terry\" },\n    new (){ PersonId = 1, Age = 17, Gender = Gender.Male,   Name = \"Kirk Obrien\" },\n    new (){ PersonId = 2, Age = 31, Gender = Gender.Male,   Name = \"Wm Banks\" },\n    new (){ PersonId = 3, Age = 44, Gender = Gender.Male,   Name = \"Karl Benson\" },\n    new (){ PersonId = 4, Age = 23, Gender = Gender.Male,   Name = \"Jared Holland\" },\n    new (){ PersonId = 5, Age = 27, Gender = Gender.Female, Name = \"Jeanne Phelps\" },\n    new (){ PersonId = 6, Age = 25, Gender = Gender.Female, Name = \"Willie Rose\" },\n    new (){ PersonId = 7, Age = 11, Gender = Gender.Female, Name = \"Shari Gutierrez\" },\n    new (){ PersonId = 8, Age = 63, Gender = Gender.Female, Name = \"Lori Wilson\" },\n    new (){ PersonId = 9, Age = 34, Gender = Gender.Female, Name = \"Lena Ramsey\" },\n});\n\n// build database binary(you can also use `WriteToStream` for save to file).\nbyte[] data = builder.Build();\n\n// -----------------------\n\n// for query phase, create MemoryDatabase.\n// (MemoryDatabase is recommended to store in singleton container(static field/DI)).\nvar db = new MemoryDatabase(data);\n\n// .PersonTable.FindByPersonId is fully typed by code-generation.\nPerson person = db.PersonTable.FindByPersonId(5);\n\n// Multiple key is also typed(***And * **), Return value is multiple if key is marked with `NonUnique`.\nRangeView\u003cPerson\u003e result = db.PersonTable.FindByGenderAndAge((Gender.Female, 23));\n\n// Get nearest value(choose lower(default) or higher).\nRangeView\u003cPerson\u003e age1 = db.PersonTable.FindClosestByAge(31);\n\n// Get range(min-max inclusive).\nRangeView\u003cPerson\u003e age2 = db.PersonTable.FindRangeByAge(20, 29);\n```\n\nAll table(marked by `MemoryTableAttribute`) and methods(created by `PrimaryKeyAttribute` or `SecondaryKeyAttribute`) are typed.\n\n![image](https://user-images.githubusercontent.com/46207/61035808-cb58e000-a402-11e9-9209-d51665d1cd56.png)\n\nYou can invoke all indexed query by IntelliSense.\n\nGetting Started(Unity)\n---\nThe minimum supported Unity version will be `2022.3.12f1`, as it is necessary to support C# Incremental Source Generator(Compiler Version, 4.3.0).\n\nSince this library is provided via NuGet, install [NuGetForUnity](https://github.com/GlitchEnzo/NuGetForUnity), then navigate to Open Window from NuGet -\u003e Manage NuGet Packages, Search \"MasterMemory\" and Press Install.\n\nFirst, it is recommended to define assembly attributes in any cs file to enable the use of `init`.\n\n```csharp\n// Optional: Unity can't load default namespace to Source Generator\n// If not specified, 'MasterMemory' will be used by default,\n// but you can use this attribute if you want to specify a different namespace.\n[assembly: MasterMemoryGeneratorOptions(Namespace = \"MyProj\")]\n\n// Optional: If you want to use init keyword, copy-and-paste this.\nnamespace System.Runtime.CompilerServices\n{\n    internal sealed class IsExternalInit { }\n}\n```\n\nEverything else is the same as the standard .NET version. While the `required` keyword can't be used since it's from C# 11, using `init` alone is sufficient to guarantee immutability.\n\n```csharp\npublic enum Gender\n{\n    Male, Female, Unknown\n}\n\n// table definition marked by MemoryTableAttribute.\n// database-table must be serializable by MessagePack-CSsharp\n[MemoryTable(\"person\"), MessagePackObject(true)]\npublic record Person\n{\n    // index definition by attributes.\n    [PrimaryKey]\n    public int PersonId { get; init; }\n\n    // secondary index can add multiple(discriminated by index-number).\n    [SecondaryKey(0), NonUnique]\n    [SecondaryKey(1, keyOrder: 1), NonUnique]\n    public int Age { get; init; }\n\n    [SecondaryKey(2), NonUnique]\n    [SecondaryKey(1, keyOrder: 0), NonUnique]\n    public Gender Gender { get; init; }\n\n    public string Name { get; init; }\n}\n```\n\nAlso, for use with IL2CPP, you need to add the generated `MasterMemoryResolver` to MessagePack's Resolver. If you need other generated Resolvers, such as those from [MagicOnion](https://github.com/Cysharp/MagicOnion), please add and compose them here.\n\n```csharp\npublic static class Initializer\n{\n    [RuntimeInitializeOnLoadMethod(RuntimeInitializeLoadType.BeforeSceneLoad)]\n    public static void SetupMessagePackResolver()\n    {\n        // Create CompositeResolver\n        StaticCompositeResolver.Instance.Register(new[]{\n            MasterMemoryResolver.Instance, // set MasterMemory generated resolver\n            StandardResolver.Instance      // set default MessagePack resolver\n        });\n\n        // Create options with resolver\n        var options = MessagePackSerializerOptions.Standard.WithResolver(StaticCompositeResolver.Instance);\n\n        // Optional: as default.\n        MessagePackSerializer.DefaultOptions = options;\n    }\n}\n```\n\nDataTable configuration\n---\nElement type of datatable must be marked by `[MemoryTable(tableName)]`, datatable is generated from marked type. `string tableName` is saved in database binary, you can rename class name if tableName is same.\n\n`[PrimaryKey(keyOrder = 0)]`, `[SecondaryKey(indexNo, keyOrder)]`, `[NonUnique]` can add to public property, `[PrimaryKey]` must use in MemoryTable, `[SecondaryKey]` is option.\n\nBoth `PrimaryKey` and `SecondaryKey` can add to multiple properties, it will be generated `***And***And***...`. `keyOrder` is order of column names, default is zero(sequential in which they appear).\n\n```csharp\n[MemoryTable(\"sample\"), MessagePackObject(true)]\npublic class Sample\n{\n    [PrimaryKey]\n    public int Foo { get; set; }\n    [PrimaryKey]\n    public int Bar { get; set; }\n}\n\ndb.Sample.FindByFooAndBar((int Foo, int Bar))\n\n// ----\n\n[MemoryTable(\"sample\"), MessagePackObject(true)]\npublic class Sample\n{\n    [PrimaryKey(keyOrder: 1)]\n    public int Foo { get; set; }\n    [PrimaryKey(keyOrder: 0)]\n    public int Bar { get; set; }\n}\n\ndb.Sample.FindByBarAndFoo((int Bar, int Foo))\n```\n\nDefault of `FindBy***` return type is single(if not found, returns `null`). It means key is unique by default. If mark `[NonUnique]` in same AttributeList, return type is `RangeView\u003cT\u003e`(if not found, return empty).\n\n```csharp\n[MemoryTable(\"sample\"), MessagePackObject(true)]\npublic class Sample\n{\n    [PrimaryKey, NonUnique]\n    public int Foo { get; set; }\n    [PrimaryKey, NonUnique]\n    public int Bar { get; set; }\n}\n\nRangeView\u003cSample\u003e q = db.Sample.FindByFooAndBar((int Foo, int Bar))\n```\n\n```csharp\n[MemoryTable(\"sample\"), MessagePackObject(true)]\npublic class Sample\n{\n    [PrimaryKey]\n    [SecondaryKey(0)]\n    public int Foo { get; set; }\n    [SecondaryKey(0)]\n    [SecondaryKey(1)]\n    public int Bar { get; set; }\n}\n\ndb.Sample.FindByFoo(int Foo)\ndb.Sample.FindByFooAndBar((int Foo, int Bar))\ndb.Sample.FindByBar(int Bar)\n```\n\n`[StringComparisonOption]` allow to configure how compare if key is string. Default is `Ordinal`.\n\n```csharp\n[MemoryTable(\"sample\"), MessagePackObject(true)]\npublic class Sample\n{\n    [PrimaryKey]\n    [StringComparisonOption(StringComparison.InvariantCultureIgnoreCase)]\n    public string Foo { get; set; }\n}\n```\n\nIf computation property exists, add `[IgnoreMember]` of MessagePack should mark.\n\n```csharp\n[MemoryTable(\"person\"), MessagePackObject(true)]\npublic class Person\n{\n    [PrimaryKey]\n    public int Id { get;}\n\n    public string FirstName { get; }\n    public string LastName { get; }\n\n    [IgnoreMember]\n    public string FullName =\u003e FirstName + LastName;\n}\n```\n\nMemoryDatabase/RangeView\n---\nIn default, `MemoryDatabase` do all string data automatically interning(see: [Wikipedia/String interning](https://en.wikipedia.org/wiki/String_interning)). If multiple same string value exists in database(ex: \"goblin\",\"goblin\", \"goblin\", \"goblin\", \"goblin\"....), standard database creates string value per query or store multiple same values. But MasterMemory stores single string value reference, it can save much memory if data is denormalized.\n\nUse intern or not is selected in constructor. If you want to disable automatically interning, use `internString:false`.\n\n`MemoryDatabase(byte[] databaseBinary, bool internString = true, MessagePack.IFormatterResolver formatterResolver = null, int maxDegreeOfParallelism = 1)`.\n\nMemoryDatabase has three(or four) query methods.\n\n* `T|RangeView\u003cT\u003e` FindBy***(TKey key)\n* bool TryFindBy***(TKey key, out T result)\n* `T|RangeView\u003cT\u003e` FindClosestBy***(TKey key, bool selectLower = true)\n* `RangeView\u003cT\u003e` FindRangeBy***(TKey min, TKey max, bool ascendant = true)\n\nIf index key is unique, generates `FindBy***` and `TryFindBy***` methods and then `FindBy***` throws `KeyNotFoundException` when key is not found.\n\n`By***` is generated by `PrimaryKey` and `SecondaryKey` defines.\n\nAnd has some utility properties.\n\n* `int` Count\n* `RangeView\u003cT\u003e` All\n* `RangeView\u003cT\u003e` AllReverse\n* `RangeView\u003cT\u003e` SortBy***\n* `T[] GetRawDataUnsafe()`\n\n`struct RangeView\u003cT\u003e : IEnumerable\u003cT\u003e` is the view of database elements. It has following property/method.\n\n* `T` [int index]\n* `int` Count\n* `T` First\n* `T` Last\n* `RangeView\u003cT\u003e` Reverse\n* `IEnumerator\u003cT\u003e` GetEnumerator()\n\nExtend Table\n---\nGenerated table class is defined partial class so create same namespace and class name's partial class on another file, you can add your custom method to generated table.\n\nTable class also defined partial `OnAfterConstruct` method, it called after table has been constructed. You can use it to store custom data to field after all data has been constructed.\n\n```csharp\n// create MonsterTable.Partial.cs\n\npublic sealed partial class MonsterTable\n{\n    int maxHp;\n#pragma warning disable CS0649\n    readonly int minHp;\n#pragma warning restore CS0649    \n\n    // called after constructed\n    partial void OnAfterConstruct()\n    {\n        maxHp = All.Select(x =\u003e x.MaxHp).Max();\n        // you can use Unsafe.AsRef to set readonly field\n        Unsafe.AsRef(minHp) = All.Select(x =\u003e x.MaxHp).Min();\n    }\n    \n    // add custom method other than standard Find method\n    public IEnumerable\u003cMonster\u003e GetRangedMonster(int arg1)\n    {\n        return All.Where....();\n    }\n}\n```\n\nImmutableBuilder\n---\nIf you want to add/modify data to loaded database, you can use `ToImmutableBuilder` method.\n\n```csharp\n// Create ImmutableBuilder from original database.\nvar builder = db.ToImmutableBuilder();\n\n// Add Or Replace compare with PrimaryKey\nbuilder.Diff(addOrReplaceData);\n\n// Remove by PrimaryKey\nbuilder.RemovePerson(new[] { 1, 10, 100 });\n\n// Replace all data\nbuilder.ReplaceAll(newData);\n\n// Finally create new database\nMemoryDatabase newDatabase = builder.Build();\n\n// If you want to save new database, you can convert to MemoryDatabase-\u003eDatabaseBuilder\nvar newBuilder = newDatabase.ToDatabaseBuilder();\nvar newBinary = newBuilder.Build(); // or use WriteToStream\n```\n\nMemoryDatabase's reference can use as snapshot.\n\n```csharp\n// 1 game per 1 instance\npublic class GameRoom\n{\n    MemoryDatabase database;\n\n    // The reference is a snapshot of the timing of game begins.\n    public GameRoom(MemoryDatabase database)\n    {\n        this.database = database;\n    }\n}\n```\n\nValidator\n---\nYou can validate data by `MemoryDatabase.Validate` method. In default, it check unique key(data duplicated) and you can define custom validate logics.\n\n```csharp\n// Implements IValidatable\u003cT\u003e to targeted validation\n[MemoryTable(\"quest_master\"), MessagePackObject(true)]\npublic class Quest : IValidatable\u003cQuest\u003e\n{\n    // If index is Unique, validate duplicate in default.\n    [PrimaryKey]\n    public int Id { get; }\n    public string Name { get; }\n    public int RewardId { get; }\n    public int Cost { get; }\n\n    void IValidatable\u003cQuest\u003e.Validate(IValidator\u003cQuest\u003e validator)\n    {\n        // get the external reference table\n        var items = validator.GetReferenceSet\u003cItem\u003e();\n\n        // Custom if logics.\n        if (this.RewardId \u003e 0)\n        {\n            // RewardId must exists in Item.ItemId\n            items.Exists(x =\u003e x.RewardId, x =\u003e x.ItemId);\n        }\n\n        // Range check, Cost must be 10..20\n        validator.Validate(x =\u003e x.Cost \u003e= 10);\n        validator.Validate(x =\u003e x.Cost \u003c= 20);\n\n        // In this region, only called once so enable to validate overall of tables.\n        if (validator.CallOnce())\n        {\n            var quests = validator.GetTableSet();\n            // Check unique othe than index property.\n            quests.Where(x =\u003e x.RewardId != 0).Unique(x =\u003e x.RewardId);\n        }\n    }\n}\n\n[MemoryTable(\"item_master\"), MessagePackObject(true)]\npublic class Item\n{\n    [PrimaryKey]\n    public int ItemId { get; }\n}\n\nvoid Main()\n{\n    var db = new MemoryDatabase(bin);\n\n    // Get the validate result.\n    var validateResult = db.Validate();\n    if (validateResult.IsValidationFailed)\n    {\n        // Output string format.\n        Console.WriteLine(validateResult.FormatFailedResults());\n\n        // Get the raw FaildItem[]. (.Type, .Message, .Data)\n        // validateResult.FailedResults\n    }\n}\n```\n\nFollowing is list of validation methods.\n\n```csharp\n// all void methods are assert function, it stores message to ValidateResult if failed.\ninterface IValidator\u003cT\u003e\n{\n    ValidatableSet\u003cT\u003e GetTableSet();\n    ReferenceSet\u003cT, TRef\u003e GetReferenceSet\u003cTRef\u003e();\n    void Validate(Expression\u003cFunc\u003cT, bool\u003e\u003e predicate);\n    void Validate(Func\u003cT, bool\u003e predicate, string message);\n    void ValidateAction(Expression\u003cFunc\u003cbool\u003e\u003e predicate);\n    void ValidateAction(Func\u003cbool\u003e predicate, string message);\n    void Fail(string message);\n    bool CallOnce();\n}\n\nclass ReferenceSet\u003cTElement, TReference\u003e\n{\n    IReadOnlyList\u003cTReference\u003e TableData { get; }\n    void Exists\u003cTProperty\u003e(Expression\u003cFunc\u003cTElement, TProperty\u003e\u003e elementSelector, Expression\u003cFunc\u003cTReference, TProperty\u003e\u003e referenceElementSelector);\n    void Exists\u003cTProperty\u003e(Expression\u003cFunc\u003cTElement, TProperty\u003e\u003e elementSelector, Expression\u003cFunc\u003cTReference, TProperty\u003e\u003e referenceElementSelector, EqualityComparer\u003cTProperty\u003e equalityComparer);\n}\n\nclass ValidatableSet\u003cTElement\u003e\n{\n    IReadOnlyList\u003cTElement\u003e TableData { get; }\n    void Unique\u003cTProperty\u003e(Expression\u003cFunc\u003cTElement, TProperty\u003e\u003e selector);\n    void Unique\u003cTProperty\u003e(Expression\u003cFunc\u003cTElement, TProperty\u003e\u003e selector, IEqualityComparer\u003cTProperty\u003e equalityComparer);\n    void Unique\u003cTProperty\u003e(Func\u003cTElement, TProperty\u003e selector, string message);\n    void Unique\u003cTProperty\u003e(Func\u003cTElement, TProperty\u003e selector, IEqualityComparer\u003cTProperty\u003e equalityComparer, string message);\n    void Sequential(Expression\u003cFunc\u003cTElement, SByte|Int16|Int32|...\u003e\u003e selector, bool distinct = false);\n    ValidatableSet\u003cTElement\u003e Where(Func\u003cTElement, bool\u003e predicate);\n}\n```\n\nMetadata\n---\nYou can get the table-info, properties, indexes by metadata api. It helps to make custom importer/exporter application.\n\n```csharp\nvar metaDb = MemoryDatabase.GetMetaDatabase();\nforeach (var table in metaDb.GetTableInfos())\n{\n    // for example, generate CSV header\n    var sb = new StringBuilder();\n    foreach (var prop in table.Properties)\n    {\n        if (sb.Length != 0) sb.Append(\",\");\n\n        // Name can convert to LowerCamelCase or SnakeCase.\n        sb.Append(prop.NameSnakeCase);\n    }\n    File.WriteAllText(table.TableName + \".csv\", sb.ToString(), new UTF8Encoding(false));\n}\n```\n\nIf creates console-app, our [ConsoleAppFramework](https://github.com/Cysharp/ConsoleAppFramework/) can easy to make helper applications.\n\nHere is sample of reading and creating dynamic from csv. `builder.AppendDynamic` and `System.Runtime.Serialization.FormatterServices.GetUninitializedObject` will help it.\n\n```csharp\nvar csv = @\"monster_id,name,max_hp\n1,foo,100\n2,bar,200\";\nvar fileName = \"monster\";\n\nvar builder = new DatabaseBuilder();\n\nvar meta = MemoryDatabase.GetMetaDatabase();\nvar table = meta.GetTableInfo(fileName);\n\nvar tableData = new List\u003cobject\u003e();\n\nusing (var ms = new MemoryStream(Encoding.UTF8.GetBytes(csv)))\nusing (var sr = new StreamReader(ms, Encoding.UTF8))\nusing (var reader = new TinyCsvReader(sr))\n{\n    while ((reader.ReadValuesWithHeader() is Dictionary\u003cstring, string\u003e values))\n    {\n        // create data without call constructor\n        // use System.Runtime.CompilerServices.RuntimeHelpers.GetUninitializedObject instead on .NET 8\n        var data = System.Runtime.Serialization.FormatterServices.GetUninitializedObject(table.DataType);\n\n        foreach (var prop in table.Properties)\n        {\n            if (values.TryGetValue(prop.NameSnakeCase, out var rawValue))\n            {\n                var value = ParseValue(prop.PropertyInfo.PropertyType, rawValue);\n                if (prop.PropertyInfo.SetMethod == null)\n                {\n                    throw new Exception(\"Target property does not exists set method. If you use {get;}, please change to { get; private set; }, Type:\" + prop.PropertyInfo.DeclaringType + \" Prop:\" + prop.PropertyInfo.Name);\n                }\n                prop.PropertyInfo.SetValue(data, value);\n            }\n            else\n            {\n                throw new KeyNotFoundException($\"Not found \\\"{prop.NameSnakeCase}\\\" in \\\"{fileName}.csv\\\" header.\");\n            }\n        }\n\n        tableData.Add(data);\n    }\n}\n\n// add dynamic collection.\nbuilder.AppendDynamic(table.DataType, tableData);\n\nvar bin = builder.Build();\nvar database = new MemoryDatabase(bin);\n\n    static object ParseValue(Type type, string rawValue)\n    {\n        if (type == typeof(string)) return rawValue;\n\n        if (type.IsGenericType \u0026\u0026 type.GetGenericTypeDefinition() == typeof(Nullable\u003c\u003e))\n        {\n            if (string.IsNullOrWhiteSpace(rawValue)) return null;\n            return ParseValue(type.GenericTypeArguments[0], rawValue);\n        }\n\n        if (type.IsEnum)\n        {\n            var value = Enum.Parse(type, rawValue);\n            return value;\n        }\n\n        switch (Type.GetTypeCode(type))\n        {\n            case TypeCode.Boolean:\n                // True/False or 0,1\n                if (int.TryParse(rawValue, out var intBool))\n                {\n                    return Convert.ToBoolean(intBool);\n                }\n                return Boolean.Parse(rawValue);\n            case TypeCode.Char:\n                return Char.Parse(rawValue);\n            case TypeCode.SByte:\n                return SByte.Parse(rawValue, CultureInfo.InvariantCulture);\n            case TypeCode.Byte:\n                return Byte.Parse(rawValue, CultureInfo.InvariantCulture);\n            case TypeCode.Int16:\n                return Int16.Parse(rawValue, CultureInfo.InvariantCulture);\n            case TypeCode.UInt16:\n                return UInt16.Parse(rawValue, CultureInfo.InvariantCulture);\n            case TypeCode.Int32:\n                return Int32.Parse(rawValue, CultureInfo.InvariantCulture);\n            case TypeCode.UInt32:\n                return UInt32.Parse(rawValue, CultureInfo.InvariantCulture);\n            case TypeCode.Int64:\n                return Int64.Parse(rawValue, CultureInfo.InvariantCulture);\n            case TypeCode.UInt64:\n                return UInt64.Parse(rawValue, CultureInfo.InvariantCulture);\n            case TypeCode.Single:\n                return Single.Parse(rawValue, CultureInfo.InvariantCulture);\n            case TypeCode.Double:\n                return Double.Parse(rawValue, CultureInfo.InvariantCulture);\n            case TypeCode.Decimal:\n                return Decimal.Parse(rawValue, CultureInfo.InvariantCulture);\n            case TypeCode.DateTime:\n                return DateTime.Parse(rawValue, CultureInfo.InvariantCulture);\n            default:\n                if (type == typeof(DateTimeOffset))\n                {\n                    return DateTimeOffset.Parse(rawValue, CultureInfo.InvariantCulture);\n                }\n                else if (type == typeof(TimeSpan))\n                {\n                    return TimeSpan.Parse(rawValue, CultureInfo.InvariantCulture);\n                }\n                else if (type == typeof(Guid))\n                {\n                    return Guid.Parse(rawValue);\n                }\n\n                // or other your custom parsing.\n                throw new NotSupportedException();\n        }\n    }\n\n    // Non string escape, tiny reader with header.\n    public class TinyCsvReader : IDisposable\n    {\n        static char[] trim = new[] { ' ', '\\t' };\n\n        readonly StreamReader reader;\n        public IReadOnlyList\u003cstring\u003e Header { get; private set; }\n\n        public TinyCsvReader(StreamReader reader)\n        {\n            this.reader = reader;\n            {\n                var line = reader.ReadLine();\n                if (line == null) throw new InvalidOperationException(\"Header is null.\");\n\n                var index = 0;\n                var header = new List\u003cstring\u003e();\n                while (index \u003c line.Length)\n                {\n                    var s = GetValue(line, ref index);\n                    if (s.Length == 0) break;\n                    header.Add(s);\n                }\n                this.Header = header;\n            }\n        }\n\n        string GetValue(string line, ref int i)\n        {\n            var temp = new char[line.Length - i];\n            var j = 0;\n            for (; i \u003c line.Length; i++)\n            {\n                if (line[i] == ',')\n                {\n                    i += 1;\n                    break;\n                }\n                temp[j++] = line[i];\n            }\n\n            return new string(temp, 0, j).Trim(trim);\n        }\n\n        public string[] ReadValues()\n        {\n            var line = reader.ReadLine();\n            if (line == null) return null;\n            if (string.IsNullOrWhiteSpace(line)) return null;\n\n            var values = new string[Header.Count];\n            var lineIndex = 0;\n            for (int i = 0; i \u003c values.Length; i++)\n            {\n                var s = GetValue(line, ref lineIndex);\n                values[i] = s;\n            }\n            return values;\n        }\n\n        public Dictionary\u003cstring, string\u003e ReadValuesWithHeader()\n        {\n            var values = ReadValues();\n            if (values == null) return null;\n\n            var dict = new Dictionary\u003cstring, string\u003e();\n            for (int i = 0; i \u003c values.Length; i++)\n            {\n                dict.Add(Header[i], values[i]);\n            }\n\n            return dict;\n        }\n\n        public void Dispose()\n        {\n            reader.Dispose();\n        }\n    }\n}\n```\n\nInheritance\n---\nCurrently MasterMemory does not support inheritance. Recommend way to create common method, use interface and extension method. But if you want to create common method with common cached field(made by `OnAfterConstruct`), for workaround, create abstract class and all data properties to abstract.\n\n```csharp\npublic abstract class FooAndBarBase\n{\n    // all data properties to virtual\n    public virtual int Prop1 { get; protected set; }\n    public virtual int Prop2 { get; protected set; }\n\n    [IgnoreMember]\n    public int Prop3 =\u003e Prop1 + Prop2;\n\n    public IEnumerable\u003cFooAndBarBase\u003e CommonMethod()\n    {\n        throw new NotImplementedException();\n    }\n}\n\n[MemoryTable(\"foo_table\"), MessagePackObject(true)]\npublic class FooTable : FooAndBarBase\n{\n    [PrimaryKey]\n    public override int Prop1 { get; protected set; }\n    public override int Prop2 { get; protected set; }\n}\n\n[MemoryTable(\"bar_table\"), MessagePackObject(true)]\npublic class BarTable : FooAndBarBase\n{\n    [PrimaryKey]\n    public override int Prop1 { get; protected set; }\n    public override int Prop2 { get; protected set; }\n}\n```\n\nOptimization\n---\nWhen invoking `new MemoryDatabase(byte[] databaseBinary...)`, read and construct database from binary. If binary size is large then construct performance will slow down. `MemoryDatabase` has `ctor(..., int maxDegreeOfParallelism = 1)` option in constructor to construct in parallel.\n\n```csharp\nvar database = new MemoryDatabase(bin, maxDegreeOfParallelism: Environment.ProcessorCount);\n```\n\nThe use of Parallel can greatly improve the construct performance. Recommend to use `Environment.ProcessorCount`.\n\nIf you want to reduce code size of generated code, Validator and MetaDatabase info can omit in runtime. Generated code has two symbols `DISABLE_MASTERMEMORY_VALIDATOR` and `DISABLE_MASTERMEMORY_METADATABASE`.  By defining them, can be erased from the build code.\n\nThe database generation/loading speed and size are affected by MessagePack's serialization format. Using `[MessagePackObject]` with `[Key]` attributes instead of `[MessagePackObject(true)]` can improve loading speed and reduce size. However, regarding size, since LZ4 compression is used by default, the difference may not be significant.\n\nMasterMemoryGeneratorOptions\n---\nThe Source Generator settings are configured using the assembly attribute `[MasterMemoryGeneratorOptions]`. By placing it in any file, you can configure the following settings.\n\n```csharp\n[assembly: MasterMemoryGeneratorOptions(\n    Namespace = \"MyConsoleApp\",\n    IsReturnNullIfKeyNotFound = true,\n    PrefixClassName = \"Foo\"\n)]\n```\n\n* `Namespace`: Changes the namespace of generated files. If not specified, it tries to get the `RootNamespace` set in the csproj file; if that's not available, it defaults to `MasterMemory`\n* `IsReturnNullIfKeyNotFound`: By default, the `Find` method throws a `KeyNotFoundException` when a key is not found. If set to true, the return type becomes `T?` and returns null instead\n* `PrefixClassName`: Adds a prefix to the class names of generated files. For example, `DatabaseBuilder` becomes `FooDatabaseBuilder`. This allows you to distinguish between multiple MasterMemory projects by name.\n\nv2 -\u003e v3 migration\n---\nSince there are no changes to the API, binary format, or behavior, you can migrate simply by changing the command-line tool settings to the assembly attribute `[MasterMemoryGeneratorOptions]`.\n\n* The code generator (MSBuild Task, .NET Core Global/Local Tools) has been removed and replaced with Source Generator\n* Tool options are now available through `MasterMemoryGeneratorOptions` (e.g. `-usingNamespace`)\n* The `-addImmutableConstructor` option has been completely removed; please use C#'s record or init keyword instead\n* The library is now only available through NuGet for Unity. Please use NuGetForUnity\n\nLicense\n---\nThis library is under the MIT License.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcysharp%2Fmastermemory","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcysharp%2Fmastermemory","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcysharp%2Fmastermemory/lists"}