{"id":26550097,"url":"https://github.com/feedboards/feedboards.json.sqlify","last_synced_at":"2025-08-23T10:11:46.252Z","repository":{"id":283487978,"uuid":"946606882","full_name":"feedboards/Feedboards.Json.Sqlify","owner":"feedboards","description":"A library for generating SQL code based on the structure of any JSON files.","archived":false,"fork":false,"pushed_at":"2025-07-04T12:52:16.000Z","size":447,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"dev","last_synced_at":"2025-07-22T08:41:26.236Z","etag":null,"topics":["clickhouse","json","sql"],"latest_commit_sha":null,"homepage":"https://www.nuget.org/packages/Feedboards.Json.Sqlify","language":"C#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/feedboards.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-03-11T12:01:24.000Z","updated_at":"2025-07-04T12:52:20.000Z","dependencies_parsed_at":"2025-03-20T14:34:57.670Z","dependency_job_id":"f8d07bc0-3a71-49af-87a5-1a86175d8852","html_url":"https://github.com/feedboards/Feedboards.Json.Sqlify","commit_stats":null,"previous_names":["feedboards/feedboards.json.sqlify"],"tags_count":7,"template":false,"template_full_name":null,"purl":"pkg:github/feedboards/Feedboards.Json.Sqlify","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/feedboards%2FFeedboards.Json.Sqlify","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/feedboards%2FFeedboards.Json.Sqlify/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/feedboards%2FFeedboards.Json.Sqlify/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/feedboards%2FFeedboards.Json.Sqlify/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/feedboards","download_url":"https://codeload.github.com/feedboards/Feedboards.Json.Sqlify/tar.gz/refs/heads/dev","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/feedboards%2FFeedboards.Json.Sqlify/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":271555169,"owners_count":24779879,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-21T02:00:08.990Z","response_time":74,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["clickhouse","json","sql"],"created_at":"2025-03-22T07:30:14.496Z","updated_at":"2025-08-23T10:11:46.241Z","avatar_url":"https://github.com/feedboards.png","language":"C#","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Feedboards.Json.Sqlify\n\nA .NET library for converting JSON data structures into SQL schemas and tables. Currently supports ClickHouse with planned support for MSSQL, PostgreSQL, MySQL, and more. This library helps you automatically generate SQL table definitions from JSON data, handling nested structures and complex data types.\n\n## Documentation\n\n- [Error Codes](https://github.com/feedboards/Feedboards.Json.Sqlify/tree/dev/docs/v2.0.4/clickhouse/error_codes.md) - Detailed list of error codes and their meanings\n- [Exception Types](https://github.com/feedboards/Feedboards.Json.Sqlify/tree/dev/docs/v2.0.4/error_types.md) - Exception types, usage, and best practices for error handling\n\n## Features\n\n- Automatic SQL schema generation from JSON files\n- Support for nested JSON structures\n- Handles both single files and directories of JSON files\n- Configurable maximum depth for nested structures (including unlimited depth)\n- Automatic type detection and mapping to database-specific data types\n  - All date-related fields are mapped to String type for maximum compatibility\n  - Support for ClickHouse Nested type structures\n- Comprehensive error handling system with detailed error codes\n- Separate validation for SQL and JSON nesting depths\n- Planned support for multiple databases:\n  - ClickHouse (current)\n  - MSSQL (planned)\n  - PostgreSQL (planned)\n  - MySQL (planned)\n\n## Installation\n\n### Building from Source\n\n```bash\n# Clone the repository\ngit clone https://github.com/feedboards/Feedboards.Json.Sqlify.git\ncd Feedboards.Json.Sqlify\n\n# Build the solution\ndotnet build\n```\n\n### NuGet Package\n\n```bash\nNuGet\\Install-Package Feedboards.Json.Sqlify -Version 2.0.4\n```\n\n### Dependency Injection Setup\n\nThe library supports dependency injection in ASP.NET Core applications. Add the following code to your `Program.cs` or `Startup.cs`:\n\n```csharp\nusing Feedboards.Json.Sqlify.Infrastructure;\nusing Feedboards.Json.Sqlify.DTOs.ClickHouse;\n\n// In your ConfigureServices method or Program.cs\nbuilder.Services.AddFeedboardsJsonSqlify(options =\u003e\n{\n    // Configure ClickHouse options\n    options.UseCLickHouseSchema(new ClickHouseOption\n    {\n        // All properties are optional and default to null\n        PathToFolderWithJson = \"path/to/json/files\",  // Optional\n        PathToOutputFolder = \"path/to/output\",        // Optional\n\n        // Database details are optional and for future features\n        DatabaseDetails = new ClickHouseDatabaseDetails\n        {\n            Host = \"localhost\",     // Required if DatabaseDetails is provided\n            Port = 8123,            // Required if DatabaseDetails is provided\n            User = \"default\",       // Required if DatabaseDetails is provided\n            Password = \"\",          // Required if DatabaseDetails is provided\n            Database = \"default\"    // Required if DatabaseDetails is provided\n        }\n    });\n\n    // Or use minimal configuration\n    options.UseCLickHouseSchema(); // All options are optional\n});\n```\n\nThen you can inject and use the client in your services:\n\n```csharp\npublic class YourService\n{\n    private readonly IClickHouseClient _clickHouseClient;\n\n    public YourService(IClickHouseClient clickHouseClient)\n    {\n        _clickHouseClient = clickHouseClient;\n    }\n\n    public void ProcessJsonFile(string tableName)\n    {\n        try\n        {\n            // Using direct file paths (no configuration needed)\n            var sqlSchema = _clickHouseClient.GenerateSQL(\n                jsonFolder: \"path/to/input.json\",\n                tableName: tableName\n            );\n\n            // Or using configured paths\n            _clickHouseClient.GenerateSQLAndWrite(tableName);\n        }\n        catch (InvalidTableNameException ex)\n        {\n            // Handle invalid table name\n            Console.WriteLine($\"Error {ex.ErrorCode}: {ex.Message}\");\n            Console.WriteLine($\"Invalid table name: {ex.Metadata[\"TableName\"]}\");\n        }\n        catch (FeedboardsJsonSqlifyException ex)\n        {\n            // Handle any other custom exception\n            Console.WriteLine($\"Error {ex.ErrorCode}: {ex.Message}\");\n            foreach (var data in ex.Metadata)\n            {\n                Console.WriteLine($\"{data.Key}: {data.Value}\");\n            }\n        }\n    }\n}\n```\n\n### Configuration Options\n\nThe `ClickHouseOption` class supports the following configuration:\n\n```csharp\npublic class ClickHouseOption\n{\n    // Optional: Path to the folder containing JSON files\n    // Defaults to null if not provided\n    // Required only when using methods that read from files\n    public string? PathToFolderWithJson { get; set; } = null;\n\n    // Optional: Path where SQL files will be generated\n    // Defaults to null if not provided\n    // Required only when using methods that write to files\n    public string? PathToOutputFolder { get; set; } = null;\n\n    // Optional: Connection details for direct database operations\n    // Defaults to null if not provided\n    // Required only for future database integration features\n    public ClickHouseDatabaseDetails? DatabaseDetails { get; set; } = null;\n}\n\n// Database connection details class\n// All properties are required if DatabaseDetails is provided\npublic class ClickHouseDatabaseDetails\n{\n    public required string Host { get; set; }\n    public required short Port { get; set; }\n    public required string User { get; set; }\n    public required string Password { get; set; }\n    public required string Database { get; set; }\n}\n```\n\n#### When to Provide Options\n\n1. For generating SQL schema as string:\n\n   - Provide `PathToFolderWithJson` when using configuration-based methods\n   - Or use direct file path methods which don't require configuration\n\n2. For writing SQL to files:\n\n   - Provide both `PathToFolderWithJson` and `PathToOutputFolder` when using configuration-based methods\n   - Or use direct file path methods which don't require configuration\n\n3. For database operations (planned feature):\n   - `DatabaseDetails` will be required with all its properties set\n\n#### Example Usage\n\nWithout configuration (using direct paths):\n\n```csharp\nvar client = new ClickHouseClient(); // No options needed\nclient.GenerateSQL(\n    jsonFolder: \"path/to/input.json\",\n    tableName: \"my_table\"\n);\n```\n\nWith configuration:\n\n```csharp\nvar options = new ClickHouseOption\n{\n    PathToFolderWithJson = \"path/to/json/files\",\n    PathToOutputFolder = \"path/to/output\"\n};\n\nvar client = new ClickHouseClient(options);\nclient.GenerateSQL(tableName: \"my_table\");\n```\n\nWith database details (for future features):\n\n```csharp\nvar options = new ClickHouseOption\n{\n    DatabaseDetails = new ClickHouseDatabaseDetails\n    {\n        Host = \"localhost\",\n        Port = 8123,\n        User = \"default\",\n        Password = \"\",\n        Database = \"default\"\n    }\n};\n\nvar client = new ClickHouseClient(options);\n```\n\n## Quick Start\n\n```csharp\ntry\n{\n    // Initialize the client with options\n    var options = new ClickHouseOption\n    {\n        PathToFolderWithJson = \"path/to/json/files\",\n        PathToOutputFolder = \"path/to/output\"\n    };\n\n    var client = new ClickHouseClient(options);\n\n    // Generate SQL from a single JSON file with nesting validation\n    client.GenerateSQL(\n        jsonFolder: \"path/to/input.json\",\n        outputFolder: \"path/to/output.sql\",\n        tableName: \"my_table\",\n    );\n\n    // Process an entire directory with default depth (10)\n    client.GenerateSQL(\n        jsonFolder: \"path/to/json/folder\",\n        outputFolder: \"path/to/sql/folder\"\n    );\n\n        // Use unlimited depth with maxDepth: 0\n    client.GenerateSQL(\n        jsonFolder: \"path/to/input.json\",\n        outputFolder: \"path/to/output.sql\",\n        tableName: \"my_table\",\n        maxDepth: 0  // No nesting limit\n    );\n}\ncatch (FeedboardsJsonSqlifyException ex)\n{\n    Console.WriteLine($\"Error {ex.ErrorCode}: {ex.Message}\");\n    foreach (var data in ex.Metadata)\n    {\n        Console.WriteLine($\"{data.Key}: {data.Value}\");\n    }\n}\n```\n\n## JSON to SQL Mapping Example\n\nInput JSON:\n\n```json\n{\n  \"test\": [\n    {\n      \"name\": \"example\",\n      \"title\": \"test1\"\n    }\n  ]\n}\n```\n\nGenerated SQL:\n\n```sql\nCREATE TABLE IF NOT EXISTS test_table\n(\n    `test` Nested(\n        `name` String,\n        `title` String\n    )\n)\nENGINE = MergeTree()\nORDER BY tuple();\n```\n\n### Nested Structures and flatten_nested Setting\n\nWhen dealing with nested structures inside other nested structures, the library automatically adds the `flatten_nested=0` setting to ensure proper handling of complex nested data.\n\n## Error Handling\n\nThe library provides a comprehensive error handling system with detailed error codes and metadata. For detailed information about error handling, see:\n\n- [Error Codes Documentation](https://github.com/feedboards/Feedboards.Json.Sqlify/tree/dev/docs/v2.0.4/clickhouse/error_codes.md)\n- [Exception Types Documentation](https://github.com/feedboards/Feedboards.Json.Sqlify/tree/dev/docs/v2.0.4/error_types.md)\n\nExample of error handling:\n\n```csharp\ntry\n{\n    client.GenerateSQL(\"path/to/json\", \"invalid@table\");\n}\ncatch (InvalidTableNameException ex)\n{\n    Console.WriteLine($\"Error {ex.ErrorCode}: {ex.Message}\");\n    Console.WriteLine($\"Invalid table name: {ex.Metadata[\"TableName\"]}\");\n}\ncatch (InvalidJsonStructureException ex)\n{\n    Console.WriteLine($\"Error {ex.ErrorCode}: {ex.Message}\");\n    Console.WriteLine($\"JSON file: {ex.Metadata[\"JsonPath\"]}\");\n    Console.WriteLine($\"Error details: {ex.InnerException?.Message}\");\n}\ncatch (FeedboardsJsonSqlifyException ex)\n{\n    Console.WriteLine($\"Error {ex.ErrorCode}: {ex.Message}\");\n    foreach (var data in ex.Metadata)\n    {\n        Console.WriteLine($\"{data.Key}: {data.Value}\");\n    }\n}\n```\n\n## Contributing\n\nContributions are welcome! Please feel free to submit a Pull Request. For major changes, please open an issue first to discuss what you would like to change.\n\n## License\n\nThis project is licensed under the MIT License - see the [LICENSE](https://github.com/feedboards/Feedboards.Json.Sqlify/blob/dev/LICENSE) file for details.\n\n## Support\n\nFor support, please:\n\n1. Check the [documentation](https://github.com/feedboards/Feedboards.Json.Sqlify/tree/dev/docs/v2.0.4)\n2. Open an issue in the GitHub repository\n3. Contact the maintainers\n\n## Roadmap\n\n- [x] Add an advanced error system\n- [ ] Add support for MSSQL\n- [ ] Add support for PostgreSQL\n- [ ] Add support for MySQL\n- [ ] Improve type inference\n- [ ] Add support for custom type mappings\n- [ ] Add support for schema validation\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffeedboards%2Ffeedboards.json.sqlify","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffeedboards%2Ffeedboards.json.sqlify","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffeedboards%2Ffeedboards.json.sqlify/lists"}