{"id":13629508,"url":"https://github.com/d-p-y/SogePoco","last_synced_at":"2025-04-17T09:34:16.731Z","repository":{"id":226604624,"uuid":"769105805","full_name":"d-p-y/SogePoco","owner":"d-p-y","description":"SogePoco SOurce GEnerated POCOs. ORM-ish thing inspired by PetaPoco. Dotnet project.","archived":false,"fork":false,"pushed_at":"2024-03-08T11:58:25.000Z","size":182,"stargazers_count":4,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2024-08-01T22:43:48.771Z","etag":null,"topics":["dotnet","orm","sourcegenerator"],"latest_commit_sha":null,"homepage":"","language":"C#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/d-p-y.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2024-03-08T11:02:18.000Z","updated_at":"2024-03-11T00:33:14.000Z","dependencies_parsed_at":null,"dependency_job_id":"eba03ab7-bf41-4356-b0d7-8d7f908687e6","html_url":"https://github.com/d-p-y/SogePoco","commit_stats":null,"previous_names":["d-p-y/sogepoco"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/d-p-y%2FSogePoco","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/d-p-y%2FSogePoco/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/d-p-y%2FSogePoco/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/d-p-y%2FSogePoco/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/d-p-y","download_url":"https://codeload.github.com/d-p-y/SogePoco/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223751171,"owners_count":17196583,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dotnet","orm","sourcegenerator"],"created_at":"2024-08-01T22:01:12.366Z","updated_at":"2024-11-08T20:31:03.702Z","avatar_url":"https://github.com/d-p-y.png","language":"C#","funding_links":[],"categories":["Contributors Welcome for those"],"sub_categories":["1. [ThisAssembly](https://ignatandrei.github.io/RSCG_Examples/v2/docs/ThisAssembly) , in the [EnhancementProject](https://ignatandrei.github.io/RSCG_Examples/v2/docs/rscg-examples#enhancementproject) category"],"readme":"# SogePoco\n\nSogePoco is a dotnet library and source generator to interact with SQL databases in a type safe and easy manner. \nIt leverages [incremental source generators](https://github.com/dotnet/roslyn/blob/main/docs/features/incremental-generators.md) to generate [ADO.NET](https://learn.microsoft.com/en-us/dotnet/framework/data/adonet/ado-net-code-examples) commands and readers. Contents of tables is represented in auto-generated classes following plain-old-CLR-object expectations. Currently supported databases: SQL Server, Postgres, Sqlite  \nName: **So**urce **Ge**nerated **P**lain **o**ld **c**lr **o**bjects \n\n# Quickstart / Demo\n\n[![Watch the video](https://img.youtube.com/vi/Y37LvcjAdko/hqdefault.jpg)](https://www.youtube.com/embed/Y37LvcjAdko)\n\nIf you wish to repeat process yourself, use [same instructions but in text format](docs/quickstart_plan.md).\nIt works on Windows too as evidenced by tests. In Visual Studio w/wo ReSharper you may need to unload/reload projects to make \"errors\" go away.\n\n# Example\n\nAssuming that you are dealing with following sqlite database:\n\n```sql\nCREATE TABLE foo(\n    id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,\n    nullable_text TEXT,\n    not_nullable_text TEXT NOT NULL);\n\nCREATE TABLE child_of_foo(\n    id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,\n    foo_id INTEGER NOT NULL,\n    val INTEGER NULL,\n    CONSTRAINT FK_ChildOfFoo_fooId FOREIGN KEY (foo_id) REFERENCES foo(id));\n```\n\nand that you used default settings in code generator you will end up with following generated Poco classes\n\n```csharp\n\npublic class Foo {\n    public long Id {get; set;}\n    public string NotNullableText {get; set;}\n    public string? NullableText {get; set;}\n\n    public Foo(long id, string notNullableText, string? nullableText) {\n        this.Id = id;\n        this.NotNullableText = notNullableText;\n        this.NullableText = nullableText;\n    }\n\n    public Foo() : this(default(long), \"\", default(string?)) {}\n\n    public override string ToString() =\u003e @\"foo\";\n} \n\npublic class ChildOfFoo {\n    public long Id {get; set;}\n    public long FooId {get; set;}\n    public long? Val {get; set;}\n    \n    public class ForeignKeysCollection {\n        public SogePoco.Common.JoinInfo\u003cChildOfFoo,SogePocoIntroduction.Pocos.Foo\u003e Foo_by_FooId                 \n            =\u003e throw new System.Exception(\"This code should not be called in runtime, it only servers as DSL during query generation\");\n    }\n    public ForeignKeysCollection ForeignKeys =\u003e throw new System.Exception(\"This code should not be called in runtime, it only servers as DSL during query generation\");\n\n    public ChildOfFoo(long id, long fooId, long? val) {\n        this.Id = id;\n        this.FooId = fooId;\n        this.Val = val;\n    }\n\n    public ChildOfFoo() : this(default(long), default(long), default(long?)) {}\n\n    public override string ToString() =\u003e @\"child_of_foo\";\n} \n\n```\n\nThere will also be a database.cs generated with methods: `Insert(Foo)`,`Insert(ChildOfFoo)`, `Update(Foo)`, `Update(ChildOfFoo)`.\n\nIf you request query to be generated in the following way:\n\n```csharp\n\n[GenerateQueries]\npublic class InvokeGenerator {\n    public void GetFoo() =\u003e Query.Register((Foo x) =\u003e x.NotNullableText == \"something\");\n}\n\n```\n\nSogePoco will generate following code:\n\n```csharp\n\npublic static class DatabaseExtensions {    \n    public static async System.Collections.Generic.IAsyncEnumerable\u003cSogePocoIntroduction.Pocos.Foo\u003e GetFoo(\n            this SogePocoIntroduction.Database self) {\n\n        await using var cmd = self.CreateCommand();\n        cmd.Parameters.Add(\n            CreateParam(\n                cmd,\n                @\"$0\", \n                ((object?)@\"something\" ?? System.DBNull.Value)));                        \n        cmd.CommandText = @\"SELECT\n\"\"t0\"\".\"\"id\"\", \"\"t0\"\".\"\"not_nullable_text\"\", \"\"t0\"\".\"\"nullable_text\"\"\nFROM \"\"foo\"\" as \"\"t0\"\"\nWHERE \"\"t0\"\".\"\"not_nullable_text\"\" = $0;\";\n        self.LastSqlText = cmd.CommandText;\n        self.LastSqlParams = cmd.Parameters.Cast\u003cSystem.Data.Common.DbParameter\u003e().ToArray();\n        await using var rdr = await cmd.ExecuteReaderAsync();\n\n        while (await rdr.ReadAsync()) {\n            var iCol = 0;\n\n            var itm0 = new SogePocoIntroduction.Pocos.Foo(\n                id:(long)rdr.GetValue(iCol++), \n                notNullableText:(string)rdr.GetValue(iCol++), \n                nullableText:rdr.GetValue(iCol++) switch { System.DBNull =\u003e null, var x =\u003e (string?)x});\n            \n            yield return itm0;\n        }            \n    }\n    \n    private static System.Data.Common.DbParameter CreateParam(System.Data.Common.DbCommand cmd, string n, object? v) {\n        var result = cmd.CreateParameter();\n        result.ParameterName = n;\n        result.Value = v;\n        return result;\n    }\n}\n```\n\n...hence just regular contemporary async ADO.NET command with added benefit of `LastSqlText` and `LastSqlParams` inspired by PetaPoco and others.\n\n# Features\n\nFor brevity, please assume that you are dealing with same sqlite database schema as defined in former point.\n\n* request one table query `select * from Foo where nullable_text = 'abc'`\n\n```csharp\nuse SogePoco.Common;\n\n[GenerateQueries]\nclass SomeClassToHoldMyRequestsToGenerateQueries {\n    public void SomeRequestedQuery() =\u003e\t\n        Query.Register((Foo f) =\u003e f.NullableText == \"abc\");\t\t\n}\n```\n\n* to add variable sql parameters e.g. `select * from Foo where nullable_text = $0`\n\n```csharp\nuse SogePoco.Common;\n\n[GenerateQueries]\nclass SomeClassToHoldMyRequestsToGenerateQueries {\n    public void SomeRequestedQuery(string x) =\u003e\t\n        Query.Register((Foo f) =\u003e f.NullableText == x);\t\t\n}\n```\n\n* to deal with `null` easily\n\n```csharp\nuse SogePoco.Common;\n\n[GenerateQueries]\nclass Irrelevant {    \n    //to get equivalent of `select * from Foo where nullable_text is null or nullable_text = $0` use:\n    public void GetMatchingFoo(string p) =\u003e\t\n        Query.Register((Foo f) =\u003e f.NullableText == null || f.NullableText == p);\n    \n    //to get equivalent of `select * from Foo where nullable_text is not null and nullable_text = $0` use:\n    public void GetMatchingFoo(string p) =\u003e\t\n        Query.Register((Foo f) =\u003e f.NullableText != null \u0026\u0026 f.NullableText == p);\t\n}\n```\n\n* to write multi-table queries (`join`ed tables)\n\n```csharp\nuse SogePoco.Common;\n\n[GenerateQueries]\nclass Irrelevant {\n    /* \n    to get equivalent of\n        select * \n        from child_of_foo \n        join foo on foo.id = child_of_foo.foo_id \n        where child_of_foo.val = $0` \n    Notice: you don't need to specify how to join tables. This information is stored as foreign key\n    */\n    public void GetValueTupleOfChildOfFooAndFoo(int neededVal) =\u003e\t\n        Query\n            .From\u003cChildOfFoo\u003e()\n            .Join(cof =\u003e cof.ForeignKeys.Foo_by_FooId)\t\t\n            .Where( (cof,f) =\u003e cof.Val == neededVal); //no `.Select(...)` hence generated query will return Value tuple of all joined types/pocos. In this case: (ChildOfFoo, Foo)\n    \n    //you can initiate join from another type (one that doesn't have foreign key) like this:\n    public void GetValueTupleOfChildOfFooAndFoo(int prm) =\u003e\n        Query\n            .From\u003cFoo\u003e()\n            .Join((ChildOfFoo cof) =\u003e cof.ForeignKeys.Foo_by_FooId, f =\u003e f) //second parameter is very important when you have joined at least one table already\n            .Where( (cof,f) =\u003e f.Val == prm);\n    \n    public void GetValueTupleOfChildOfFooAndFoo(int v) =\u003e\t\n        Query\n            .From\u003cChildOfFoo\u003e()\n            .Join(cof =\u003e cof.ForeignKeys.Foo_by_FooId)\t\t\n            .Where( (cof,f) =\u003e cof.Val == v)\n            .Select((cof,f) =\u003e cof); //same as first query but generated query will return `ChildOfFoo` instances instead of ValueTuple: (ChildOfFoo, Foo)\n\n    //you can also use .LeftJoin() with same syntax as .Join() above to get `left join` instead of `inner join` \n}\n``` \n\n* to sort returned instances / to include `order by ... asc` or `order by ... desc` sql clause\n\n```csharp\nuse SogePoco.Common;\n\n[GenerateQueries]\nclass Second {\npublic void GetFoo() =\u003e\t\n    Query\n        .From\u003cFoo\u003e()\n        .OrderByAsc(f =\u003e f.NotNullableText);\n        \n        //for reverse order use: \n        //.OrderByDesc(f =\u003e f.NotNullableText)\n}\n```\n\n* to limit amount of returned records / to include `select top N` or `select ... limit N` sql clause\n\n```csharp\nuse SogePoco.Common;\n\n[GenerateQueries]\nclass Second {\n    public void GetFoo() =\u003e\t\n        Query\n            .From\u003cFoo\u003e()\n            .Take(1);            \n}\n```\n\n* leverage native support for sql parameters being collection  \ne.g. you want to pass `new [] {10,50,100}` to `select * from tbl where tbl.col in (10,50,100)` as a single sql parameter in query effectively turning it into `select * from tbl where tbl.col in $0`  \n\n```csharp\nuse SogePoco.Common;\n\n[GenerateQueries]\nclass Second {\n    public void GetMatchingFoo(string[] mustBeIn) =\u003e\n        Query.Register((Foo f) =\u003e mustBeIn.Contains(f.NullableText));\n}\n```\n\n\u003e [!NOTE]\n\u003e Support:  \n\u003e * In postgres, it works out of the box.  \n\u003e * In sqlite it is not supported out of the box (requires [optional extension](https://sqlite.org/carray.html)).   \n\u003e * In sql server, it works too \u003cdetails\u003e\u003csummary\u003ewith additional setup\u003c/summary\u003e\n\u003e   - add custom type (e.g. `CREATE TYPE ArrayOfInt AS TABLE(V int NULL)`)   \n\u003e   - pass custom mapping to `SqlServerMapper` in your `SogePoco*Config.cs` informing it about custom type name(s)\n\u003e More info:  \n\u003e   - see [SystemUnderTestFactory](SogePoco.Impl.Tests/PocoGeneration/SystemUnderTestFactory.cs) method `CreateSqlServer(bool withArraysSupport)`.  \n\u003e   - see usage of `Syntax.Contains` in [tests](SogePoco.Impl.Tests/QueryGeneration/TestQueryGenerationQueryParameters.cs)\n\u003e     \u003c/details\u003e\n\n* to do explicit locking in Postgres / to include e.g. `select ... for update`\n\n```csharp\nuse SogePoco.Common;\n\n[GenerateQueries]\nclass Second {\n    public void GetMatchingFoo() =\u003e\t\n        PostgresQuery  //notice: different root here\n            .From\u003cFoo\u003e()\n            .ForUpdate();\n            //or .ForShare()\n            //or .ForNoKeyUpdate()\n            //or: .WithForClause(\"update nowait or anything custom goes here\") \n}\n```\n\n**Note** blocking/locking rules in Postgres are much simpler than in Sql Server. It is advised in Sql Server to use different isolation levels instead of playing with query hints (e.g. `with rowlock`). \nSee \n[TestLockingUnrelatedTables.cs](SogePoco.Impl.Tests/DatabaseBehavior/TestLockingUnrelatedTables.cs)\n and \n[TestLockingRelatedTables.cs](SogePoco.Impl.Tests/DatabaseBehavior/TestLockingRelatedTables.cs)\nto see it yourself. That's why I decided to not implement it (yet?)\n\nThere are many more features. At this moment it's best to look at rich collection of included xunit tests to know what else is supported.\n* support for automatically incremented version fields  \nin postgres using `xmin`; in sql server using `rowversion`\n* \"rows affected\" in `update` and `delete` statement checks  \nby default generated Database class checks them. If they are not returning one, you will get exception.\nIt prevents forgotten check mistakes. If you don't like it, you can set following property to empty action (`() =\u003e {}`) to check it yourself or react differently.\n\n```csharp\nSystem.Action\u003cstring\u003e OnRowsAffectedExpectedToBeExactlyOne {set;}\n```\n* computed columns (term of sql server) / generated columns (term of postgres)  \nsogepoco understands what computed/generated columns are and is not attempting to `insert`/`update` them while still `select`-ing them \n* columns with defaults  \ndefaults are extracted and you have ability to decide whether to use Poco class instance value or default value. To use it pass 2nd parameter to generated Database class (=`defaultableColumnShouldInsert`)\nsee test in [TestLockingUnrelatedTables.cs](SogePoco.Impl.Tests/PocoGeneration/TestsInsert.cs)\n\n## Missing features, shortcomings\n\nSome things that I consider as important and would like to implement them eventually\n \n* ability to do custom joins (not following foreign keys but specify columns manually)\n* optional support for `Equals(object)` and `GetHashCode()` generation \n  so that two Pocos of the same type having same primary key value are `Equal` \n* better error reporting  \nso that errors get source code hint in Rider / Visual Studio. Today, errors show up in build tab and/or log file. \n* little things such as support for `.Take(param_instead_of_literal)`\n* support `view`s\n\n# Nuget version / status\n![SogePoco.Common NuGet Version](https://img.shields.io/nuget/v/SogePoco.Common)\n[SogePoco.Common](https://www.nuget.org/packages/SogePoco.Common/)\n\n![SogePoco.Impl NuGet Version](https://img.shields.io/nuget/v/SogePoco.Impl)\n[SogePoco.Impl](https://www.nuget.org/packages/SogePoco.Impl/)\n\n![SogePoco.Template.Postgres NuGet Version](https://img.shields.io/nuget/v/SogePoco.Template.Postgres)\n[SogePoco.Template.Postgres](https://www.nuget.org/packages/SogePoco.Template.Postgres/)\n\n![SogePoco.Template.SqlServer NuGet Version](https://img.shields.io/nuget/v/SogePoco.Template.SqlServer)\n[SogePoco.Template.SqlServer](https://www.nuget.org/packages/SogePoco.Template.SqlServer/)\n\n![SogePoco.Template.Sqlite NuGet Version](https://img.shields.io/nuget/v/SogePoco.Template.Sqlite)\n[SogePoco.Template.Sqlite](https://www.nuget.org/packages/SogePoco.Template.Sqlite/)\n\n## Github actions\n\n![assure_project_builds](https://github.com/d-p-y/SogePoco/actions/workflows/assure_project_builds.yaml/badge.svg)\n`dotnet pack` works for all libraries and templates:\n\n![assure_template_is_usable_postgres](https://github.com/d-p-y/SogePoco/actions/workflows/assure_template_is_usable_postgres.yaml/badge.svg) \npostgres template generates working POCOs and queries  \n(using postgres server instance running in podman container)? \n\n\n![assure_template_is_usable_sqlite](https://github.com/d-p-y/SogePoco/actions/workflows/assure_template_is_usable_sqlite.yaml/badge.svg)\nsqlite template generates working POCOs and queries   \n(using in memory sqlite)?\n\n\n![assure_template_is_usable_sqlserver](https://github.com/d-p-y/SogePoco/actions/workflows/assure_template_is_usable_sqlserver.yaml/badge.svg)\nsqlserver template generates working POCOs and queries   \n(using sqlserver instance running in podman container)?\n\n\n![tests_postgresql_and_common](https://github.com/d-p-y/SogePoco/actions/workflows/tests_postgresql_and_common.yaml/badge.svg)\nall tests (~100) pass when using postgres server  \n(instance running in podman container)?\n\n\n![tests_sqlserver_and_common](https://github.com/d-p-y/SogePoco/actions/workflows/tests_sqlserver_and_common.yaml/badge.svg)  \nall tests (~100) pass when using sqlserver server    \n(instance running in podman container)?\n\n\n![tests_sqlite_and_common](https://github.com/d-p-y/SogePoco/actions/workflows/tests_sqlite_and_common.yaml/badge.svg)\nall tests (~100) pass when using in memory sqlite (on ubuntu)?\n\n\n![windows_tests_sqlite_and_common](https://github.com/d-p-y/SogePoco/actions/workflows/windows_tests_sqlite_and_common.yaml/badge.svg)\nall tests (~100) pass when using in memory sqlite (on windows)?\n\n\n# High Level Design\n\n\u003cdetails\u003e\n    \u003csummary\u003eRead more about design...\u003c/summary\u003e\n\nWhole library relies on knowing: \n* which database engine is in use and \n* what is the database schema\n\nThis way, source generator doesn't need connect to db everytime it needs schema. Otherwise things would be *slow*.  \nCalling extraction process is only need when db changes e.g. after performing db migrations.  \nWhat schema extractor does can be summarized in following way:\n\n![Extraction step](docs/01_diagram_extraction.svg)\n\nWhat logically follows after extraction are two invocations of source generator.\nFirst one generates poco classes and database class. Second generates queries build using\nthose poco classes and registering them as extensions of database class.\n\nFirst invocation overview:\n![POCOs and database classes generation](docs/02_diagram_pocos_and_db_classes_generation.svg)\n\nSecond invocation overview:\n![POCO query generation](docs/03_diagram_query_generation.svg)\n\n\u003c/details\u003e\n\n# Scope of project and motivation\n\nI wanted to build a spiritual successor of [PetaPoco](https://github.com/CollaboratingPlatypus/PetaPoco), \n[AsyncPoco](https://github.com/tmenier/AsyncPoco) projects but instead of emitting MSIL. I wanted to use \n[incremental source generators](https://github.com/dotnet/roslyn/blob/main/docs/features/incremental-generators.md) which is a recommended technology for metaprogramming.\nApart from that I also wanted to:\n* restore benefits of T4 text templates that generated Poco classes from existing database \n[eg](https://github.com/CollaboratingPlatypus/PetaPoco/wiki/T4-Template-Poco-Builder) (=the \"database first\"). \nT4 is [not working in dotnet core/standard](https://developercommunity.visualstudio.com/t/support-loading-of-net-core-libraries-in-t4-templa/881587). Hence *Poco libraries had a choice to either migrate to dotnet core or drop T4 support. \n* leverage collections as sql parameters\nsuch parameters are available in database engines but *poco libraries tend to not use and instead automagicaly change into as-many-parameters-as-there-are-items-in-collection.\nIn the past [I forked AsyncPoco and added this feature](https://github.com/d-p-y/AsyncPoco) so I knew it is a valid approach\n* add ability for type safe query building\nIn the past [I've implemented it for AsyncPoco](https://github.com/d-p-y/statically-typed-poco-queries) but it relied on reflection and limited metadata inferred from POCO classes and their attributes. \nI wanted to support `join`s where its `on` clause would be autogenerated from foreign keys to prevent mistakes and be less error prone.\n\n\n## Goals\n* Everything \"as async as possible\"  \nhence usage of features such as [IAsyncEnumerable\u003cT\u003e and IAsyncDisposable](https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/proposals/csharp-8.0/async-streams)\n* support tables with both 'generated primary key' and 'manual primary key'  \nmeaning with or without auto_increment / IDENTITY / serial\n* keep it simple and avoid surprises  \nwhile Entity Framework and (N)Hibernate have much more features and cover all edge cases they do it at the cost of significant complexity. \nHidden state in entity classes and potential pitfalls such as lazy collections are some examples.  \nOn the other hand, PetaPoco (and other *Poco libraries) feel fragile as they tend to rely on strings and objects limiting type safety.  \n[linq2db](https://github.com/linq2db/linq2db) is an interesting middle ground but IMHO uses complicated syntax (e.g for joins).\n* What can be checked/computed during compilation, should be performed in comptime (instead of runtime).  \n* Avoid providing functionalities that tend to be fragile\ne.g. sogepoco has `Insert()` and `Update()` but no `Save()`. It requires guessing game that breaks for tables with\nnongenerated primary key.\n\n## Non goals\n * support tables without primary keys\n * support updating primary keys\n\n## License\n\nApache License 2.0\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fd-p-y%2FSogePoco","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fd-p-y%2FSogePoco","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fd-p-y%2FSogePoco/lists"}