{"id":22980524,"url":"https://github.com/kcartlidge/newt","last_synced_at":"2026-05-06T20:34:57.274Z","repository":{"id":145848550,"uuid":"440106317","full_name":"kcartlidge/Newt","owner":"kcartlidge","description":"Autogenerate a .Net (C#/EF Core) data project (class library with entities and data contexts) and an admin website from a Postgres database. Also creates backup SQL and Graphviz .dot diagram source.","archived":false,"fork":false,"pushed_at":"2025-01-16T22:13:53.000Z","size":137101,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-02T09:44:45.047Z","etag":null,"topics":["admin","core","csharp","dotnet","efcore","entities","entity-framework","graphviz","postgres","sql","website"],"latest_commit_sha":null,"homepage":"","language":"C#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kcartlidge.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-12-20T09:07:56.000Z","updated_at":"2025-01-16T22:11:02.000Z","dependencies_parsed_at":null,"dependency_job_id":"f3946666-dfba-43be-a570-0d78fd3b5df2","html_url":"https://github.com/kcartlidge/Newt","commit_stats":null,"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"purl":"pkg:github/kcartlidge/Newt","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kcartlidge%2FNewt","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kcartlidge%2FNewt/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kcartlidge%2FNewt/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kcartlidge%2FNewt/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kcartlidge","download_url":"https://codeload.github.com/kcartlidge/Newt/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kcartlidge%2FNewt/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32711379,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-06T19:35:05.142Z","status":"ssl_error","status_checked_at":"2026-05-06T19:35:03.996Z","response_time":117,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["admin","core","csharp","dotnet","efcore","entities","entity-framework","graphviz","postgres","sql","website"],"created_at":"2024-12-15T01:43:48.959Z","updated_at":"2026-05-06T20:34:57.263Z","avatar_url":"https://github.com/kcartlidge.png","language":"C#","funding_links":[],"categories":[],"sub_categories":[],"readme":"# NEWT\n\nAutogenerate a .Net (C#/EF Core) *data project* (class library with entities and data contexts) and an *admin website* from a Postgres database.\nAlso creates backup SQL and Graphviz `.dot` diagram source.\n\nIf a Solution doesn't exist in the main folder then a new one is created.\nThe Data and optional Web project are both added to it.\n*Existing Solution files are not updated.*\n\n*Requires dotnet 6.0.3 or later to be installed.*\n\n- [View the changelog](./CHANGELOG.md)\n\nThere are [pre-built executables](./builds) ready to run for Linux, Mac, and Windows.\n\nNote that whilst *this code repository* is AGPL there are specifically *no licensing constraints applied to generated output*.\nYou are free to apply any license you wish; generated code is entirely yours.\n\nYou *must* have the `dotnet` command installed (v6+), even if using a pre-built Newt executable, as it is used to create your new project and add packages to it.\n\n## What Newt does\n\nAdd the connection details for your Postgres database to your environment.\n\n- Newt will then ...\n    - Scan Postgres\n    - Create a new .Net class library *data project*\n    - Add the required Nuget packages for EF Core\n        - It will also restore them\n    - Create EF Core data contexts\n        - With automatic .Net class/property naming conventions\n        - An InMemory context for testing etc\n        - A Postgres context ready for use\n    - Optionally create an ASP.Net MVC Web project\n    - Create a Solution file if one doesn't already exist\n        - Add the Data and optional Web project to it\n    - Create entity models (classes) for each table\n        - With automatic .Net class/property naming conventions\n        - With data annotations for keys, types, lengths etc\n        - With comments matching those for the database columns\n        - With comments regarding where they map to\n    - Create an emergency SQL script for everything generated\n        - Tables, columns, primary/foreign keys, indexes etc\n    - Create the `.dot` source for a *Graphviz* class diagram\n        - With classes, properties, and foreign keys\n    - Create an *admin web site*\n        - Gneerate login page\n        - Generate admin pages\n            - Include column sorting and table paging\n- You can then ...\n    - Include this in your solution (or generate it in-place)\n    - Re-run any time you want an updated data project\n    - Maintain things with the admin website\n    - Or use that as the basis for an actual site\n\n## Status\n\nThis is *beta*. It works and is in active use.\nMake sure you check the database conventions in the contents (below).\n\nThere are [pre-built executables](./builds) ready to run for Linux, Mac, and Windows.\n\n---\n\n## Contents\n\n- [Running Newt](#running-newt)\n- [Database conventions](#database-conventions)\n    - [On foreign keys](#on-foreign-keys)\n- [Output](#output)\n    - [Created project files](#created-project-files)\n    - [Created context](#created-context)\n    - [Created entity](#created-entity)\n    - [Created SQL](#created-sql)\n    - [Created Graphviz](#created-graphviz)\n- [Sample Usage](#sample-usage)\n- [Copying a build to somewhere convenient](#copying-a-build-to-somewhere-convenient)\n\n---\n\nFor Newt developers only:\n\n- [Generating stand-alone builds](#generating-stand-alone-builds)\n    - [For all target systems in one go](#for-all-target-systems-in-one-go)\n    - [For a single target system](#for-a-single-target-system)\n\n## Running Newt\n\nThe command arguments (which are always displayed at runtime) are as follows.\n\n``` txt\nNEWT (build 2024-08-11)\n\nGenerates a DotNet (C#/EF Core) data access repository project from a Postgres database.\nOptionally also creates a matching MVC site for data management.\n\nIf the parent folder has no solution it will be created and the project(s) added.\nExisting solutions will NOT be updated to include new project(s).\n\nThe solution name is used if a new solution needs creating.\nNamespaces will also be used as project names/subfolders.\n\n\nUSAGE:\n\n  -env  text  * Environment variable containing the connection string  \n  -s    text  * The database schema to generate code for  \n  -f    text  * The parent folder for the solution  \n  -sln  text  * The solution name/namespace  \n  -dn   text  * The C# data project name/namespace  \n  -wn   text    The C# MVC web project name/namespace  \n  -od           Overwrite existing data project\n  -ow           Overwrite existing web project\n  -os           Overwrite existing solution\n\n  * is required\n\nEXAMPLE:\n\n  Newt -env DB_CONNSTR -s public -f ./sample -sln Sample -dn Data -wn Admin\n```\n\nNote that the destination `-f` specifies the *parent* of where it should write to.  This is usually the *Solution* folder; a sub-folder will be created according to the `-sln` name/namespace.\n\nFor this to work, you need an environment variable containing a connection string.\nThat same environment variable name will be referenced in the generated EF context.\n\n``` shell\nexport DB_CONNSTR=\"Server=127.0.0.1;Port=5432;Database=coregen;User Id=coregen;Password=coregen;\"\n```\n\n*The above is an example connection string, not a revealed secret.*\n\nThis environment variable is also used by the auto-generated EF Core data context.\n\nOn Windows if you use the `set` command rather than `export` then ensure you don't accidentally include the double quotes (`\"`) surrounding the value.  If you do, you'll get a message similar to \"*Format of the initialization string does not conform to specification starting at index ...*\" - in which case the simple solution is to use the Control Panel to set the Environment Variable value.\n\nIf you don't have a published Newt binary in your path you can run it from source:\n\n``` shell\ncd \u003csolution\u003e\ndotnet run --project Newt -- -env DB_CONNSTR -s public -f ~/sample -sln Sample -dn Data -wn Admin -od -ow\n```\n\nNote the extra `--` before `-env` which is *required* for the dotnet CLI to recognise which options it should interpret itself (those to the left) and which it should pass to Newt (those to the right).\n\nIf a Solution file doesn't alrady exist at the top level a new one is created.\n\nIf you provide a `-wn` parameter then an ASP.Net MVC Web project is also created.\nWhen a new solution has been requested any created projects are added to it.\n\nEfforts are made to not 'damage' any enclosing solution as it may be your own code.\n\nAs a precaution you are *strongly advised* to commit to source control before running to avoid overwriting things you want to keep!\n\n## Database conventions\n\nNewt scans the specified Postgres database to generate code.\nWhen it does so it needs to be able to understand, parse, and use what it finds.\nThis means certain conventions should be followed.\n\n- *Postgres* naming conventions must be used, *not* C# ones\n    - Table names are lower case snake (underscore delimited)\n        - `customer_order` not `CustomerOrder` or `customerOrder`\n    - They should also be in the singular form\n        - `author` not `authors`\n    - Columns names are *also* lower case snake (underscore delimited)\n        - `display_name` not `DisplayName` or `displayName`\n        - `id` not `ID` or `Id`\n- Only one field in table primary keys\n\nSome of these restrictions are expected to be removed in the near future.\n\n### On foreign keys\n\nForeign keys can result in lists being added to entities.\nCurrently this only works for *belongs-to* not *has-a*.\n\nFor example:\n\n- *has-a*\n    - When `Blog` has `ThemeId` foreign key, but `Theme` doesn't have `BlogId` foreign key\n    - The blog *has a* theme, but the theme does not *belong to* the blog\n    - No theme list is added to the blog as the blog doesn't own it\n- *belongs-to*\n    - When `Post` has `BlogId` foreign key\n    - The post *belongs to* the blog\n    - A `List\u003cPost\u003e` *is added* to the `Blog` as posts *belong to* blogs\n\nYou only get a list in the *parent* entity if the *child* one has a foreign key *to it*.\n\n## Output\n\nThe following represents generated output assuming a namespace of `Sample.Data`.\nThe database has an `article`, a `blog`, and a `post` table.\n\n### Created project files\n\nYou will optionally get a Solution and possibly a Web (admin) project.\nThey are standard off-the shelf projects.\n\nHere's the Data project's files/folders:\n\n```\nSample.Data/\n    Entities/\n        Article.cs\n        Blog.cs\n        Post.cs\n    SQL/\n        Postgres.sql\n    DataContext.cs\n    InMemoryDataContext.cs\n    Sample.Data.csproj\n    schema-dump.json\n```\n\nHere's an example of the Admin website.\n\n![Admin Website](./admin.png)\n\n### Created context\n\n``` cs\nusing System;\nusing Microsoft.EntityFrameworkCore;\nusing Sample.Data.Entities;\n\nnamespace Sample.Data\n{\n    public class DataContext : DbContext\n    {\n        public DbSet\u003cArticle\u003e Articles { get; set; }\n        public DbSet\u003cBlog\u003e Blogs { get; set; }\n        public DbSet\u003cPost\u003e Posts { get; set; }\n\n        protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)\n        {\n            var connectionString = Environment.GetEnvironmentVariable(\"DB_CONNSTR\");\n            optionsBuilder.UseNpgsql(connectionString);\n        }\n    }\n}\n```\n\n### Created entity\n\n(some columns/properties have been removed for brevity)\n\n``` cs\nusing System;\nusing System.Collections.Generic;\nusing System.ComponentModel;\nusing System.ComponentModel.DataAnnotations;\nusing System.ComponentModel.DataAnnotations.Schema;\n\nnamespace Sample.Data.Entities\n{\n    /// \u003csummary\u003e\n    /// Contains the definition of a single blog site.\n    /// \u003c/summary\u003e\n    [Table(\"blog\")]\n    public class Blog\n    {\n        /// \u003csummary\u003e\n        /// This line is taken from the comment in the Postgres column definition.\n        /// The line below is auto-generated for every property.\n        /// Database column `id` of type `bigint`.\n        /// \u003c/summary\u003e\n        [Key]\n        [Required]\n        [Column(\"id\")]\n        [DisplayName(\"Id\")]\n        public long Id { get; set; }\n\n        /// \u003csummary\u003e\n        /// Database column `created_at` of type `timestamp with time zone`.\n        /// \u003c/summary\u003e\n        [Required]\n        [Column(\"created_at\")]\n        [DisplayName(\"Created At\")]\n        public DateTime CreatedAt { get; set; }\n\n        /// \u003csummary\u003e\n        /// Database column `deleted_at` of type `timestamp with time zone`.\n        /// \u003c/summary\u003e\n        [Column(\"deleted_at\")]\n        [DisplayName(\"Deleted At\")]\n        public DateTime? DeletedAt { get; set; }\n\n        /// \u003csummary\u003e\n        /// For use in browser tabs, tables, lists etc.\n        /// Database column `title` of type `character varying`.\n        /// The capacity is 50.\n        /// \u003c/summary\u003e\n        [MaxLength(50)]\n        [Required]\n        [Column(\"title\")]\n        [DisplayName(\"Title\")]\n        public string Title { get; set; }\n\n        /// \u003csummary\u003e\n        /// Database column `hostname` of type `character varying`.\n        /// The capacity is 100.\n        /// \u003c/summary\u003e\n        [MaxLength(100)]\n        [Required]\n        [Column(\"hostname\")]\n        [DisplayName(\"Hostname\")]\n        public string Hostname { get; set; }\n\n        /// \u003csummary\u003e\n        /// Database column `theme_id` of type `bigint`.\n        /// \u003c/summary\u003e\n        [Required]\n        [Column(\"theme_id\")]\n        [DisplayName(\"Theme Id\")]\n        public long ThemeId { get; set; }\n\n        /// \u003csummary\u003eForeign key on Article\u003c/summary\u003e\n        public List\u003cArticle\u003e Articles { get; set; }\n\n        /// \u003csummary\u003eForeign key on Post\u003c/summary\u003e\n        public List\u003cPost\u003e Posts { get; set; }\n    }\n}\n```\n\n### Created SQL\n\n(the generated SQL is for emergency use only; it only includes what Newt uses)\n\n``` sql\nDROP TABLE public.blog CASCADE;\nCREATE TABLE IF NOT EXISTS public.blog (\n  id           BIGSERIAL NOT NULL,\n  created_at   timestamp with time zone NOT NULL,\n  deleted_at   timestamp with time zone,\n  title        character varying(50) NOT NULL,\n  display      character varying(200) NOT NULL,\n  hostname     character varying(100) NOT NULL,\n  theme_id     bigint NOT NULL DEFAULT 1,\n\n  CONSTRAINT pk_blog PRIMARY KEY (id),\n  CONSTRAINT uq_blog_hostname UNIQUE (hostname),\n  CONSTRAINT fk_blog_theme FOREIGN KEY (theme_id)\n    REFERENCES public.theme (id) MATCH SIMPLE \n    ON UPDATE NO ACTION ON DELETE NO ACTION\n);\nALTER TABLE public.blog OWNER to sample;\nCOMMENT ON TABLE public.blog IS 'Contains the definition of a single blog site.';\nCOMMENT ON COLUMN public.blog.title IS 'For use in browser tabs, tables, lists etc.';\n```\n\n### Created Graphviz\n\n[Graphviz](https://graphviz.org) source is generated in a `schema.dot` file.\nAt the top of that file are sample commands to render it.\n\nA Graphviz image generated from sample source is below, with the source itself underneath that.\nEach class lists its properties then the other classes it is linked to by foreign keys.\nFor brevity most of the classes have been manually removed from the sample source and image.\n\nClick the image below to open it directly.\nWhen using Github or similar, you may also need to click for the *Raw* file.\n\n![Sample Graphviz](./schema.svg)\n\n``` dot\n// GRAPHVIZ DEFINITION\n//\n// Example usage:\n//   dot -o schema.png -Tpng schema.dot\n//   dot -o schema.svg -Tsvg schema.dot\n\ndigraph {\n  label = \"Class diagram for Sample.Data\"\n  labelloc = \"top\"\n  fontname = \"Verdana\"\n  fontsize = 16\n  nodesep = 0.5\n  ranksep = 0.5\n  pad = 0.25\n\n  node[\n    fontname = \"Monospace\"\n    fontsize = 12\n    shape = \"Mrecord\"\n    width = 1.5\n    margin = \"0.1,0.1\"\n  ]\n\n  edge[\n    arrowhead = \"dot\"\n  ]\n\n  // Classes.\n  Blog [ label = \"{Blog | Id ................. long\\nCreatedAt ...... DateTime\\nDeletedAt ...... DateTime\\nAdminEmail .. string[100]\\nTitle ........ string[50]\\nBlurb ........ string[50]\\nHostname .... string[100]\\nCopyright ... string[100]\\nThemeId ............. int\\n | Authors\\lCategories\\lPosts\\lTags\\lThemes\\l}}\" ]\n  Post [ label = \"{Post | Id .................. long\\nBlogId .............. long\\nCreatedAt ....... DateTime\\nDeletedAt ....... DateTime\\nDisplay ...... string[100]\\nSnippet ...... string[500]\\nMarkdown .......... string\\nPublished ....... DateTime\\n | Authors\\lComments\\lTags\\l}}\" ]\n  Article [ label = \"{Article | Id .................. long\\nBlogId .............. long\\nCreatedAt ....... DateTime\\nDeletedAt ....... DateTime\\nTitle ........ string[100]\\nMarkdown .......... string\\nPublished ....... DateTime\\n | Tags\\l}}\" ]\n\n  // Foreign keys.\n  Blog -\u003e Post\n  Blog -\u003e Article\n}\n```\n\n## Sample Usage\n\nHere's how it might be registered in `Program.cs` in .Net 6.\nIt assumes the created project is named `Sample.Data` and is referenced.\n\n``` cs\n// Program.cs\n\nusing Sample.Data;\n\nbuilder.Services.AddScoped\u003cDataContext\u003e();\n```\n\n*Important note:*\nIf you added the connection string environment variable *after* starting your IDE/editor, then you will usually need to restart it.  Any debug/release runs will likely be unable to connect until you do, as the OS only makes available the environment variables that already existed at the time your IDE/editor was launched.\n\nAnd here's how it might then be used in a controller in a `Sample.UI` project.\n\n``` cs\n// ArticlesController.cs\n\nusing Microsoft.AspNetCore.Mvc;\nusing Sample.Data;\nusing Sample.Data.Entities;\n\nnamespace Sample.UI.Controllers;\n\n[ApiController]\n[Route(\"[controller]\")]\npublic class ArticlesController : ControllerBase\n{\n    private readonly ILogger\u003cArticlesController\u003e _logger;\n    private readonly DataContext _context;\n\n    public ArticlesController(ILogger\u003cArticlesController\u003e logger, DataContext context)\n    {\n        _logger = logger;\n        _context = context;\n    }\n\n    [HttpGet(Name = \"ListArticles\")]\n    public IEnumerable\u003cArticle\u003e List()\n    {\n        return _context.Articles.OrderBy(x =\u003e x.Title).ToArray();\n    }\n}\n```\n\n## Copying a build to somewhere convenient\n\nCopy a build to somewhere accessible via your system path and you can run it from anywhere.\n\nFor example (Mac):\n\n``` sh\ncd \u003csolution\u003e\nsudo rm -rf /usr/local/bin/Newt\nsudo cp builds/macos-arm64/Newt /usr/local/bin/Newt\n```\n\nAlternatively, as builds are pretty small you can include ones for relevant platforms directly into your own project's repository.\n\n---\n\n**What follows if for Newt developers only. You don't need to read this if you are only using Newt as a tool for your own projects.**\n\n## Generating stand-alone builds\n\nThe commands below should be run from within the `Newt/Newt` *project* folder, not the solution. Remember to update the build date in the `Program.cs` console output, README output capture, and the CHANGELOG.\nThere are also details available regarding [copying a build to somewhere convenient](#copying-a-build-to-somewhere-convenient).\n\n*Do not check in new builds automatically every commit*; only do so when a genuine version and release change occurs.\n\n### For all target systems in one go\n\nThere are two scripts you can run in the *project* folder.\nIf you are building on Linux/Mac use `build.sh` and on Windows use `build.bat`.\n\nLinux/Mac:\n\n```sh\ncd \u003cproject\u003e\nchmod +x build.sh   # for reference; not usually needed\n./build.sh\n```\n\nWindows:\n\n```bat\ncd \u003cproject\u003e\nbuild.bat\n```\n\nUpon completion, both scripts will advise what builds are now available.\n\n### For a single target system\n\n``` shell\n# Mac, Apple Silicon (eg M1)\ndotnet build\ndotnet publish -r osx-arm64 -c Release /p:PublishSingleFile=true /p:PublishTrimmed=true /p:IncludeNativeLibrariesForSelfExtract=true\n\n# Mac, Intel\ndotnet build\ndotnet publish -r osx-x64 -c Release /p:PublishSingleFile=true /p:PublishTrimmed=true /p:IncludeNativeLibrariesForSelfExtract=true\n\n# Linux, Intel\ndotnet build\ndotnet publish -r linux-x64 -c Release /p:PublishSingleFile=true /p:PublishTrimmed=true /p:IncludeNativeLibrariesForSelfExtract=true\n\n# Windows, Intel\ndotnet build\ndotnet publish -r win10-x64 -c Release /p:PublishSingleFile=true /p:PublishTrimmed=true /p:IncludeNativeLibrariesForSelfExtract=true\n```\n\nWhichever command you choose to run, it will tell you in its response where it has placed the binary.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkcartlidge%2Fnewt","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkcartlidge%2Fnewt","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkcartlidge%2Fnewt/lists"}