{"id":23301424,"url":"https://github.com/firesharkstudios/butterfly-db","last_synced_at":"2025-08-22T07:31:32.116Z","repository":{"id":91266197,"uuid":"215391458","full_name":"firesharkstudios/butterfly-db","owner":"firesharkstudios","description":"Reactive database SELECTs for popular relational databases in C#","archived":false,"fork":false,"pushed_at":"2024-01-16T20:01:35.000Z","size":5986,"stargazers_count":8,"open_issues_count":1,"forks_count":3,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-11-18T04:39:05.206Z","etag":null,"topics":["butterfly","csharp","database","mysql","orm","postgres","reactive-data-streams","reactive-documents","sqlite","sqlserver"],"latest_commit_sha":null,"homepage":"","language":"C#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/firesharkstudios.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-10-15T20:33:50.000Z","updated_at":"2021-04-08T13:33:45.000Z","dependencies_parsed_at":null,"dependency_job_id":"985641fd-dba8-4ec5-9fec-64daf3faa69b","html_url":"https://github.com/firesharkstudios/butterfly-db","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/firesharkstudios%2Fbutterfly-db","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/firesharkstudios%2Fbutterfly-db/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/firesharkstudios%2Fbutterfly-db/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/firesharkstudios%2Fbutterfly-db/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/firesharkstudios","download_url":"https://codeload.github.com/firesharkstudios/butterfly-db/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":230225673,"owners_count":18193026,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["butterfly","csharp","database","mysql","orm","postgres","reactive-data-streams","reactive-documents","sqlite","sqlserver"],"created_at":"2024-12-20T10:12:23.977Z","updated_at":"2024-12-20T10:12:24.620Z","avatar_url":"https://github.com/firesharkstudios.png","language":"C#","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Butterfly.Db ![Butterfly Logo](https://raw.githubusercontent.com/firesharkstudios/Butterfly/master/img/logo-40x40.png) \n\n\u003e Reactive database SELECTs for popular relational databases in C#\n\n# Overview\n\n*Butterfly.Db* allows executing SELECTs **and** receiving data change events on \nthe SELECTs when the underyling data changes in the relational database.\n\n*Butterfly.Db* does this by parsing the SELECT statements and running\na modified version of the SELECT after each INSERT, UPDATE, or DELETE.  Although \nthis adds overhead, the modified SELECT statements are filtered by primary key\nand run quickly.\n\nThis has a few limitations...\n\n- All modifications must be executed via *Butterfly.Db* interface\n- All modifications must modify a single record at a time\n- Only tables in the FROM clause of the SELECT will detect changes (not in subqueries)\n\nEven with the limitations above, this is still a quite useful foundation to\nbuild real-time web apps.\n\nWant to push these data change events to a web client?  See the \n*Subscription API* in [Butterfly.Web](https://github.com/firesharkstudios/butterfly-web) and the *Web Client* in [Butterfly.Client](https://github.com/firesharkstudios/butterfly-client).\n\nExecuting a SELECT and receiving events when the results of the SELECT change\nis part of a *DynamicView* in *Butterfly.Db*.\n\n*Butterfly.Db* also provides a simple interface to retreive and modify data with support for transactions.\n\n*Butterfly.Db* has implementations for memory, MySQL, Postgres, SQLite, and SqlServer.\n\nSee the example at [Butterfly.Example.DbEvents](https://github.com/firesharkstudios/butterfly-server/tree/master/Butterfly.Example.DbEvents) to see a console program that subscribes to\nthe SELECT below and receives data change events when changes to the *todo* and *user* tables would modify the results of the SELECT...\n\n```\nSELECT t.id, t.name todo_name, u.name user_name\nFROM todo t \n    INNER JOIN user u ON t.user_id=u.id\nWHERE is_done=@isDoneFilter\n```\n\n# Install from Nuget\n\n| Name | Package | Install |\n| --- | --- | --- |\n| Butterfly.Db | [![nuget](https://img.shields.io/nuget/v/Butterfly.Db.svg)](https://www.nuget.org/packages/Butterfly.Db/) | `nuget install Butterfly.Db` |\n| Butterfly.Db.MySql | [![nuget](https://img.shields.io/nuget/v/Butterfly.Db.MySql.svg)](https://www.nuget.org/packages/Butterfly.Db.MySql/) | `nuget install Butterfly.Db.MySql` |\n| Butterfly.Db.Postgres | [![nuget](https://img.shields.io/nuget/v/Butterfly.Db.Postgres.svg)](https://www.nuget.org/packages/Butterfly.Db.Postgres/) | `nuget install Butterfly.Db.Postgres` |\n| Butterfly.Db.SQLite | [![nuget](https://img.shields.io/nuget/v/Butterfly.Db.SQLite.svg)](https://www.nuget.org/packages/Butterfly.Db.SQLite/) | `nuget install Butterfly.Db.SQLite` |\n| Butterfly.Db.SqlServer | [![nuget](https://img.shields.io/nuget/v/Butterfly.Db.SqlServer.svg)](https://www.nuget.org/packages/Butterfly.Db.SqlServer/) | `nuget install Butterfly.Db.SqlServer` |\n\n# Install from Source Code\n\n```git clone https://github.com/firesharkstudios/butterfly-db```\n\n# Import Dict\n\nBecause *Dictionary\u003cstring, object\u003e* is used so extensively, the following alias is defined...\n\n```cs\nusing Dict = System.Collections.Generic.Dictionary\u003cstring, object\u003e;\n```\n\n# Accessing a Database\n\nAn *IDatabase* instance allows modifying data, selecting data, and creating *DynamicViews*.\n\n```cs\nvar id = await database.InsertAndCommitAsync\u003cstring\u003e(\"todo\", new {\n    name = \"My Todo\"\n});\nawait database.UpdateAndCommitAsync(\"todo\", new {\n    id,\n    name = \"My New Todo\"\n});\nawait database.DeleteAndCommitAsync(\"todo\", id);\n\nvar name = await database.SelectValueAsync\u003cstring\u003e(\"SELECT name FROM todo\", id);\n```\n\n## Selecting Data\n\nThere are four flavors of selecting data with different return values...\n\n| Method | Description |\n| --- | --- |\n| SelectRowAsync() | Returns a single *Dict* instance |\n| SelectRowsAsync() | Returns an array of *Dict* instances |\n| SelectValueAsync\u003cT\u003e() | Returns a single value |\n| SelectValuesAsync\u003cT\u003e() | Returns an array of values |\n\nEach flavor above takes a *sql* parameter and optional *values* parameter.\n\nThe *sql* parameter can be specified in multiple ways...\n\n| Name | Example Value |\n| --- | --- |\n| Table name only | `\"todo\"` |\n| SELECT without WHERE | `\"SELECT * FROM todo\"` |\n| SELECT with WHERE | `\"SELECT * FROM todo WHERE id=@id\"` |\n\nThe *values* parameter can also be specified in multiple ways...\n\n| Name | Example Value |\n| --- | --- |\n| Anonymous type | `new { id = \"123\" }` |\n| Dictionary | `new Dict { [\"id\"] = \"123\" }` |\n| Primary Key Value | `\"123\"` |\n\nSpecific value types will also cause a WHERE clause to be rewritten as follows...\n\n| Original WHERE | Values | New WHERE |\n| --- | --- | --- |\n| WHERE test=@test | `new { test = (string)null }` | WHERE test IS NULL |\n| WHERE test!=@test | `new { test = (string)null }` | WHERE test IS NOT NULL |\n| WHERE test=@test | `new { test = new string[] {\"123\",\"456\") }` | WHERE test IN ('123', '456') |\n| WHERE test!=@test | `new { test = new string[] {\"123\",\"456\") }` | WHERE test NOT IN ('123', '456') |\n\nSo, these are all valid examples...\n\n```cs\n// Both of these effectively run SELECT * FROM employee\nDict[] allEmployees1 = await database.SelectRowsAsync(\"employee\");\nDict[] allEmployees2 = await database.SelectRowsAsync(\"SELECT * FROM employee\");\n\n// Both of these effectively run SELECT * FROM employee WHERE department_id=\"123\"_\nDict[] departmentEmployees1 = await database.SelectRowsAsync(\"employee\", new {\n    department_id = \"123\"\n});\nDict[] departmentEmployees1 = await database.SelectRowsAsync(\"employee\", new Dict {\n    [\"department_id\"] = \"123\"\n});\n\n// All three of these effectively run SELECT name FROM employee WHERE id='123'\nstring name1 = await database.SelectValueAsync\u003cstring\u003e(\"SELECT name FROM employee\", \"123\");\nstring name2 = await database.SelectValueAsync\u003cstring\u003e(\"SELECT name FROM employee\", new {\n    id = \"123\"\n});\nstring name3 = await database.SelectValueAsync\u003cstring\u003e(\"SELECT name FROM employee\", new Dict {\n    [\"id\"] = \"123\"\n});\n\n// Effectively runs SELECT * FROM employee WHERE department_id IS NULL\nDict[] rows = await database.SelectRowsAsync(\"employee\", new {\n    department_id = (string)null\n});\n\n// Effectively runs SELECT * FROM employee WHERE department_id IS NOT NULL\nDict[] rows = await database.SelectRowsAsync(\"SELECT * employee WHERE department_id!=@department_id\", new {\n    department_id = (string)null\n});\n\n// Effectively runs SELECT * FROM employee WHERE department_id IN ('123', '456')\nDict[] rows = await database.SelectRowsAsync(\"employee\", new {\n    department_id = new string[] { \"123\", \"456\"}\n});\n\n// Effectively runs SELECT * FROM employee WHERE department_id NOT IN ('123', '456')\nDict[] rows = await database.SelectRowsAsync(\"SELECT * employee WHERE department_id!=@department_id\", new {\n    department_id = new string[] { \"123\", \"456\"}\n});\n```\n## Modifying Data\n\nA *IDatabase* instance has convenience methods that create a transaction, perform a specific action, and commit the transaction as follows...\n\n```cs\n// Execute a single INSERT and return the value of the primary key\nstring id = database.InsertAndCommitAsync\u003cstring\u003e(\"employee\", new {\n\tfirst_name = \"Jim\",\n\tlast_name = \"Smith\",\n\tbalance = 0.0f,\n});\n\n// Assuming the employee table has a unique index on the id field, \n// this updates the balance field on the matching record\ndatabase.UpdateAndCommitAsync\u003cstring\u003e(\"employee\", new {\n\tid = \"123\",\n\tbalance = 0.0f,\n});\n\n// Assuming the employee table has a unique index on the id field, \n// this deletes the matching record\ndatabase.DeleteAndCommitAsync\u003cstring\u003e(\"employee\", \"123\");\n```\n\nIn addition, you can explicitly create and commit a transaction that performs multiple actions...\n\n```cs\n// If either INSERT fails, neither INSERT will be saved\nusing (ITransaction transaction = await database.BeginTransactionAsync()) {\n\tstring departmentId = transaction.InsertAsync\u003cstring\u003e(\"department\", new {\n\t\tname = \"Sales\"\n\t});\n\tstring employeeId = transaction.InsertAsync\u003cstring\u003e(\"employee\", new {\n\t\tname = \"Jim Smith\",\n\t\tdepartment_id = departmentId,\n\t});\n\n    // Don't forget to Commit the transaction\n\tawait transaction.CommitAsync();\n}\n```\n\nSometimes, it's useful to run code after a transaction is committed, this can be done using *OnCommit* to register an action that will execute after the transaction is committed.\n\n## Synchronizing Data\n\nIt's common to synchronize a set of records in the database with a new set of inputs.  \n\nThe *SynchronizeAsync* can be used to determine the right INSERT, UPDATE, and DELETE statements to synchronize two collections...\n\n```cs\n// Assumes an article_tag table with article_id and tag_name fields\npublic async Task SynchronizeTags(string articleId, string[] tagNames) {\n    // First, retrieve the existing records from the database\n    Dict[] existingRecords = database.SelectRowsAsync(\n        @\"SELECT article_id, tag_name \n        FROM article_tag \n        WHERE article_id=@articleId\",\n        new {\n            articleId\n        }\n    );\n\n    // Next, create the new records collection from the tagNames parameter\n    Dict[] newRecords = tagNames.Select(x =\u003e new Dict {\n        [\"article_id\"] = articleId,\n        [\"tag_name\"] = x,\n    }).ToArray();\n\n    // Now, execute SynchronizeAsync() to determine the right \n    // INSERT, UPDATE, and DELETE statements to make the collections match\n    using (ITransaction transaction = database.BeginTransactionAsync()) {\n        await transaction.SynchronizeAsync(\n            \"article_tag\", \n            existingRecords, \n            newRecords\n        );\n        await transaction.CommitAsync();\n    }\n}\n```\n\n## Defaults, Overrides, and Preprocessors\n\nA *IDatabase* instance allows defining...\n\n- Default Values (applies to INSERTs)\n- Override Values (applies to INSERTs and UPDATEs)\n- Input Proprocessors\n\nEach can be defined globally or per table.\n\nExamples...\n\n```cs\n// Add an id field to any INSERT with values like at_58b5fff4-322b-4fe8-b45d-386dac7a79f9\n// if INSERTing on an auth_token table\ndatabase.SetDefaultValue(\n    \"id\", \n    tableName =\u003e $\"{tableName.Abbreviate()}_{Guid.NewGuid().ToString()}\"\n);\n\n// Add a created_at field to any INSERT with the current time\ndatabase.SetDefaultValue(\"created_at\", tableName =\u003e DateTime.Now.ToUnixTimestamp());\n\n// Add an updated_at field to any INSERT or UPDATE with the current time\nthis.database.SetOverrideValue(\"updated_at\", tableName =\u003e DateTime.Now.ToUnixTimestamp());\n\n// Remap any DateTime values to UNIX timestamp values\ndatabase.AddInputPreprocessor(BaseDatabase.RemapTypeInputPreprocessor\u003cDateTime\u003e(\n    dateTime =\u003e dateTime.ToUnixTimestamp()\n));\n\n// Remap any $NOW$ values to the current UNIX timestamp\ndatabase.AddInputPreprocessor(BaseDatabase.RemapTypeInputPreprocessor\u003cstring\u003e(\n    text =\u003e text==\"$NOW$\" ? DateTime.Now.ToUnixTimestamp().ToString() : text\n));\n\n// Remap any $UPDATE_AT$ values to be the same value as the updated_at field\ndatabase.AddInputPreprocessor(BaseDatabase.CopyFieldValue(\"$UPDATED_AT$\", \"updated_at\"));\n```\n\n# Using Dynamic Views\n\n## Overview\n\nA *DynamicViewSet* allows...\n\n- Defining multiple *DynamicView* instances using a familiar SELECT syntax\n- Publishing the initial rows as a single *DataEventTransaction* instance\n- Publishing any changes as new *DataEventTransaction* instances\n\nEach *DynamicView* instance must...\n\n- Have a unique name (defaults to the first table name in the SELECT) within a *DynamicViewSet*\n- Have key field(s) that uniquely identify each row (defaults to the primary key of the first table in the SELECT) \n\nYou can use the [Butterfly.Client](https://github.com/firesharkstudios/butterfly-client) libraries to consume these *DataEventTransaction* instances to keep local javascript arrays synchronized with your server.\n\nKey limitations...\n\n- Only INSERTs, UPDATEs, and DELETEs executed via an *IDatabase* instance will trigger data change events\n- SELECT statements with UNIONs are not supported\n- SELECT statements with subqueries may not be supported depending on the type of subquery\n- SELECT statements with multiple references to the same table can only trigger updates on one of the references\n\nA *DynamicView* will execute additional modified SELECT statements on each underlying data change event.  These modified SELECT statements are designed to execute quickly (always includes a primary key of an underlying table); however, this is additional overhead that should be considered on higher traffic implementations.\n\n## Example\n\nHere is an example of creating a *DynamicViewSet* and triggering *DataEventTransaction* instances by starting the *DynamicViewSet* and by executing an INSERT...\n```cs\nvar dynamicViewSet = database.CreateAndStartDynamicViewAsync(\n    @\"SELECT t.id, t.name todo_name, u.name user_name\n    FROM todo t \n        INNER JOIN user u ON t.user_id=u.id\n    WHERE is_done=@isDoneFilter\",\n    dataEventTransaction =\u003e {\n        var json = JsonUtil.Serialize(dataEventTransaction, format: true);\n        Console.WriteLine($\"dataEventTransaction={json}\");\n    },\n    new {\n        isDoneFilter = \"Y\"\n    }\n);\ndynamicViewSet.Start();\n```\n\nThe above code would cause a *DataEventTransaction* like this to be echoed to the console...\n\n```js\ndataEventTransaction={\n  \"dateTime\": \"2018-08-24 14:25:59\",\n  \"dataEvents\": [\n    {\n      \"name\": \"todo\",\n      \"keyFieldNames\": [\n        \"id\"\n      ],\n      \"dataEventType\": \"InitialBegin\",\n      \"id\": \"f916082a-7e56-4974-8bce-9c0af0792362\"\n    },\n    {\n      \"record\": {\n        \"id\": \"t_7dcdaf99-50ab-4bd5-ab26-271974e9cc49\",\n        \"todo_name\": \"Todo #4\",\n        \"user_name\": \"Patrick\"\n      },\n      \"name\": \"todo\",\n      \"keyValue\": \"t_7dcdaf99-50ab-4bd5-ab26-271974e9cc49\",\n      \"dataEventType\": \"Initial\",\n      \"id\": \"134afc7e-a24e-448a-b800-baed7774d6d2\"\n    },\n    {\n      \"record\": {\n        \"id\": \"t_0f2c7147-317b-4f70-851c-dc906db6f2c3\",\n        \"todo_name\": \"Todo #1\",\n        \"user_name\": \"Spongebob\"\n      },\n      \"name\": \"todo\",\n      \"keyValue\": \"t_0f2c7147-317b-4f70-851c-dc906db6f2c3\",\n      \"dataEventType\": \"Initial\",\n      \"id\": \"aaa6e491-5ad4-4a2b-9891-b1d402172c46\"\n    },\n    {\n      \"record\": {\n        \"id\": \"t_e71e3d82-2153-4b1b-8fcd-29815805307b\",\n        \"todo_name\": \"Todo #2\",\n        \"user_name\": \"Spongebob\"\n      },\n      \"name\": \"todo\",\n      \"keyValue\": \"t_e71e3d82-2153-4b1b-8fcd-29815805307b\",\n      \"dataEventType\": \"Initial\",\n      \"id\": \"efea5a4b-9a9c-4bea-bc19-d6a460f27abb\"\n    },\n    {\n      \"dataEventType\": \"InitialEnd\",\n      \"id\": \"f25b8841-b9a3-4ec6-af0a-3d34687fa767\"\n    }\n  ]\n}\n```\n\nNow, let's add a record that impacts our *DynamicViewSet*...\n\n```cs\nawait database.InsertAndCommitAsync\u003cstring\u003e(\"todo\", new {\n    name = \"Task #5\",\n    user_id = spongebobId,\n    is_done = \"N\",\n});\n```\n\nThe above code would trigger the following *DataEventTransaction* to be echoed to the console...\n\n```js\ndataEventTransaction={\n  \"dateTime\": \"2018-08-24 14:25:59\",\n  \"dataEvents\": [\n    {\n      \"record\": {\n        \"id\": \"t_89378473-97ed-4e0f-9c1d-4303ef6f4d04\",\n        \"todo_name\": \"Task #5\",\n        \"user_name\": \"Spongebob\"\n      },\n      \"name\": \"todo\",\n      \"keyValue\": \"t_89378473-97ed-4e0f-9c1d-4303ef6f4d04\",\n      \"dataEventType\": \"Insert\",\n      \"id\": \"e140185e-9636-45e9-9687-a3368ad6caeb\"\n    }\n  ]\n}\n```\n\nYou can run a more robust example [here](https://github.com/firesharkstudios/butterfly-server-dotnet/blob/master/Butterfly.Example.Database/Program.cs).\n\n# Implementations\n\n## Using a Memory Database\n\n*Butterfly.Db.MemoryDatabase* database is included in *Butterfly.Db* and doesn't require installing additional packages; however, *MemoryDatabase* has these key limitattions...\n\n- Data is NOT persisted\n- SELECT statements with JOINs are NOT supported\n\nUnder the hood, the *MemoryDatabase* is using a System.Data.DataTable instance to manage the data.\n\nIn your application...\n\n```csharp\nvar database = new Butterfly.Db.Memory.MemoryDatabase();\n```\n\n## Using MySQL\n\nIn the *Package Manager Console*...\n\n```\nInstall-Package Butterfly.Db.Mysql\n```\n\nIn your application...\n\n```csharp\nvar database = new Butterfly.Db.Mysql.MySqlDatabase(\"Server=127.0.0.1;Uid=test;Pwd=test!123;Database=butterfly_db_demo\");\n```\n\n## Using Postgres\n\nIn the *Package Manager Console*...\n\n```\nInstall-Package Butterfly.Db.Postgres\n```\n\nIn your application...\n\n```csharp\nvar database = new Butterfly.Db.Postgres.PostgresDatabase(\"User ID=test;Password=test!123;Host=localhost;Port=5432;Database=test;\");\n```\n\n## Using SQLite\n\nIn the *Package Manager Console*...\n\n```\nInstall-Package Butterfly.Db.SQLite\n```\n\nIn your application...\n\n```csharp\nvar database = new Butterfly.Db.SQLite.SQLiteDatabase(\"Filename=./my_database.db\");\n```\n\n## Using MS SQL Server\n\nIn the *Package Manager Console*...\n\n```\nInstall-Package Butterfly.Db.SqlServer\n```\n\nIn your application...\n\n```csharp\nvar database = new Butterfly.Db.SqlServer.SqlServerDatabase(\"Server=localhost; Initial Catalog=Butterfly; User ID=test; Password=test!123\");\n```\n\n# Contributing\n\nIf you'd like to contribute, please fork the repository and use a feature\nbranch. Pull requests are warmly welcome.\n\n# Licensing\n\nThe code is licensed under the [Mozilla Public License 2.0](http://mozilla.org/MPL/2.0/).  \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffiresharkstudios%2Fbutterfly-db","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffiresharkstudios%2Fbutterfly-db","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffiresharkstudios%2Fbutterfly-db/lists"}