{"id":19383298,"url":"https://github.com/rebus-org/debaser","last_synced_at":"2025-04-23T21:31:51.505Z","repository":{"id":137920605,"uuid":"74262760","full_name":"rebus-org/Debaser","owner":"rebus-org","description":":smiling_imp: Lean mean SQL Server upsert machine","archived":false,"fork":false,"pushed_at":"2024-03-26T11:25:48.000Z","size":5172,"stargazers_count":46,"open_issues_count":1,"forks_count":5,"subscribers_count":7,"default_branch":"master","last_synced_at":"2024-05-01T11:48:07.876Z","etag":null,"topics":["fast","merge","sql-server","upsert"],"latest_commit_sha":null,"homepage":"","language":"C#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rebus-org.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2016-11-20T08:53:25.000Z","updated_at":"2023-12-20T01:06:57.000Z","dependencies_parsed_at":"2023-11-16T11:23:57.954Z","dependency_job_id":"3ef19e93-80b5-437f-a42e-04e81074bc96","html_url":"https://github.com/rebus-org/Debaser","commit_stats":null,"previous_names":[],"tags_count":25,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rebus-org%2FDebaser","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rebus-org%2FDebaser/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rebus-org%2FDebaser/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rebus-org%2FDebaser/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rebus-org","download_url":"https://codeload.github.com/rebus-org/Debaser/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223934071,"owners_count":17227646,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["fast","merge","sql-server","upsert"],"created_at":"2024-11-10T09:25:26.523Z","updated_at":"2024-11-10T09:25:27.245Z","avatar_url":"https://github.com/rebus-org.png","language":"C#","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Debaser\n\n![](https://raw.githubusercontent.com/rebus-org/Debaser/master/artwork/new_logo_512.png)\n\nHave you ever had the need to insert/update many rows in SQL Server? (i.e. \"upsert\" them)\n\nDid you try to use the `MERGE INTO ...` syntax then? (and did you enjoy it?)\n\nDid you know that ADO.NET has an API that allows for STREAMING rows to a temporary table in `tempdb` making SQL Server call a stored procedure for each row very quickly?\n\nDid you know that Debaser can do these things for you?\n\n## How to do it?\n\nCreate a class that looks the way you want it to look:\n\n```csharp\nclass SomeDataRow(int id, decimal number, string text)\n{\n    public int Id { get; } = id;\n    public decimal Number { get; } = number;\n\tpublic string Text { get; } = text;\n}\n```\n\nDebaser supports using a default constructor and properties with setters, or using a constructor with parameters matching the properties like this example shows.\n\nYou CAN also use C# records, but Debaser's configuration attributes (like `[DebaserKey]`, `[DebaserMapper(..)]`, etc.) must be used on properties, so C# classes using the primary constructor syntax as shown above is probably the most compact and neat way to declare your Debaser types.\n\nThen you create the `UpsertHelper` for it:\n\n```csharp\nvar upsertHelper = new UpsertHelper\u003cSomeDataRow\u003e(\"db\");\n```\n\n(where `db` in this case is the name of a connection string in the current app.config)\n\nand then you do this once:\n\n```csharp\nupsertHelper.CreateSchema();\n```\n\n(which will create a table, a table data type, and a stored procedure)\n\nand then you insert 100k rows like this:\n\n```csharp\nvar rows = Enumerable.Range(1, 100000)\n\t.Select(i =\u003e new SomeDataRow(i, i*2.3m, $\"This is row {i}\"))\n\t.ToList();\n\nvar stopwatch = Stopwatch.StartNew();\n\nawait upsertHelper.Upsert(rows);\n\nvar elapsedSeconds = stopwatch.Elapsed.TotalSeconds;\n\nConsole.WriteLine($\"Upserting {rows.Count} rows took {elapsedSeconds} - that's {rows.Count / elapsedSeconds:0.0} rows/s\");\n```\n\nwhich on my machine yields this:\n\n```csharp\nUpserting 100000 rows took 0.753394 - that's 132732.7 rows/s\n```\n\nwhich I think is fair.\n\n## Merging with existing data\n\nBy default each row is identified by its ID property. This means that any IDs matching existing rows will lead to an update instead of an insert.\n\nLet's hurl 100k rows into the database again:\n\n```csharp\nvar rows = Enumerable.Range(1, 100000)\n\t.Select(i =\u003e new SomeDataRow(i, i*2.3m, $\"This is row {i}\"))\n\t.ToList();\n\nawait upsertHelper.Upsert(rows);\n```\n\nand then let's perform some iterations where we pick 10k random rows, mutate them, and upsert them:\n\n```csharp\nvar stopwatch = Stopwatch.StartNew();\n\nvar updatedRows = rows.InRandomOrder().Take(rowsToChange)\n    .Select(row =\u003e new SomeDataRow(row.Id, row.Number + 1, string.Concat(row.Text, \"-HEJ\")));\n\nawait upsertHelper.Upsert(updatedRows);\n```\n\nwhich on my machine yields this:\n\n```csharp\nUpdating 10000 random rows in dataset of 100000 took average of 0.2 s - that's 57191.1 rows/s\n```\n\nwhich (again) I think is OK.\n\n## Maturity\n\nDebaser is fairly mature. It's been used for some serious things already, but please back your Debaser-based stuff up by some nice integration tests.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frebus-org%2Fdebaser","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frebus-org%2Fdebaser","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frebus-org%2Fdebaser/lists"}