{"id":16683531,"url":"https://github.com/marcominerva/databasegpt","last_synced_at":"2025-04-07T11:04:58.148Z","repository":{"id":199472316,"uuid":"702937895","full_name":"marcominerva/DatabaseGPT","owner":"marcominerva","description":"Query a database using natural language","archived":false,"fork":false,"pushed_at":"2024-10-16T08:37:43.000Z","size":12750,"stargazers_count":69,"open_issues_count":4,"forks_count":15,"subscribers_count":8,"default_branch":"master","last_synced_at":"2025-03-31T09:07:09.402Z","etag":null,"topics":["azure-openai","c-sharp","chatgpt","hacktoberfest","openai","post","postgresql","query-builder","sql-server"],"latest_commit_sha":null,"homepage":"https://databasegpt.azurewebsites.net/","language":"C#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/marcominerva.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-10-10T09:41:56.000Z","updated_at":"2025-02-16T09:48:44.000Z","dependencies_parsed_at":"2023-11-09T11:52:53.507Z","dependency_job_id":"20644186-440d-4527-89e8-7e535b62eb21","html_url":"https://github.com/marcominerva/DatabaseGPT","commit_stats":null,"previous_names":["marcominerva/databasegpt"],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcominerva%2FDatabaseGPT","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcominerva%2FDatabaseGPT/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcominerva%2FDatabaseGPT/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcominerva%2FDatabaseGPT/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/marcominerva","download_url":"https://codeload.github.com/marcominerva/DatabaseGPT/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247640462,"owners_count":20971557,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["azure-openai","c-sharp","chatgpt","hacktoberfest","openai","post","postgresql","query-builder","sql-server"],"created_at":"2024-10-12T14:25:10.136Z","updated_at":"2025-04-07T11:04:58.108Z","avatar_url":"https://github.com/marcominerva.png","language":"C#","funding_links":[],"categories":[],"sub_categories":[],"readme":"# DatabaseGPT\n\n[![Lint Code Base](https://github.com/marcominerva/DatabaseGPT/actions/workflows/linter.yml/badge.svg)](https://github.com/marcominerva/DatabaseGPT/actions/workflows/linter.yml)\n[![CodeQL](https://github.com/marcominerva/DatabaseGPT/actions/workflows/github-code-scanning/codeql/badge.svg)](https://github.com/marcominerva/DatabaseGPT/actions/workflows/github-code-scanning/codeql)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://github.com/marcominerva/TinyHelpers/blob/master/LICENSE)\n\nQuery a database using natural language.\n\nThanks to [Adam Buckley](https://github.com/happyadam73/tsql-chatgpt) for the [original inspiration](https://www.linkedin.com/pulse/query-your-data-azure-sql-using-natural-language-chatgpt-adam-buckley/) for this project.\n\n![](https://raw.githubusercontent.com/marcominerva/DatabaseGPT/master/assets/DatabaseGptWeb.gif)\n\n### Usage\n\nCurrently, SQL Server, PostgreSQL and SQLite databases are supported.\n\n#### Using DatabaseGpt in your project\n\nIf you want to use **DatabaseGpt** as a library in your application, you can reference the `src/DatabaseGpt/DatabaseGpt.csproj` project and the one that contains the specific implementation for your DBMS, available as `src/DatabaseGpt.\u003cDBMS\u003e/DatabaseGpt.\u003cDBMS\u003e.csproj`:\n\nDatabase|Project to include\n-|-\nSQL Server|src/DatabaseGpt.SqlServer/DatabaseGpt.SqlServer.csproj\nPostgreSQL|src/DatabaseGpt.Npgsql/DatabaseGpt.Npgsql.csproj\nSQLite|src/DatabaseGpt.Sqlite/DatabaseGpt.Sqlite.csproj\n\nAfter referencing the proper projects, you can easily initialize **DatabaseGpt** at the startup of your application.\n\n```csharp\n// ...\n\nbuilder.Services.AddDatabaseGpt(database =\u003e\n{\n    // For SQL Server.\n    database.UseConfiguration(context.Configuration)\n            .UseSqlServer(context.Configuration.GetConnectionString(\"SqlConnection\"));\n\n    // For PostgreSQL.\n    //database.UseConfiguration(context.Configuration)\n    //        .UseNpgsql(context.Configuration.GetConnectionString(\"NpgsqlConnection\"));\n\n    // For SQLite.\n    //database.UseConfiguration(context.Configuration)\n    //        .UseSqlite(context.Configuration.GetConnectionString(\"SqliteConnection\"));\n},\nchatGpt =\u003e\n{\n    chatGpt.UseConfiguration(context.Configuration);\n});\n```\n\nYou need to set the required values in the **appsettings.json** file:\n\n```\n\"ConnectionStrings\": {\n    \"SqlConnection\": \"\"             // The SQL Server connection string    \n    //\"NpgsqlConnection\": \"\"        // The PostgreSQL connection string\n    //\"SqliteConnection\": \"\"        // The SQLite connection string\n},\n\"ChatGPT\": {\n    \"Provider\": \"OpenAI\",           // Optional. Allowed values: OpenAI (default) or Azure\n    \"ApiKey\": \"\",                   // Required\n    \"Organization\": \"\",             // Optional, used only by OpenAI\n    \"ResourceName\": \"\",             // Required when using Azure OpenAI Service\n    \"AuthenticationType\": \"ApiKey\", // Optional, used only by Azure OpenAI Service. Allowed values: ApiKey (default) or ActiveDirectory\n    \"DefaultModel\": \"my-model\"      // Required  \n},\n\"DatabaseGptSettings\": {\n    \"IncludedTables\": [ ],          // Array of table names to include (in the form of \"schema.table\")\n    \"ExcludedTables\": [ ],          // Array of table names to exclude (in the form of \"schema.table\")\n    \"ExcludedColumns\": [ ],         // Array of column names to exclude (in the form of \"schema.table.column\" to exclude a specific column, or \"column\" to exclude the column in all tables)\n    \"MaxRetries\": 3                 // Max retries when the query fails\n}\n```\n\nFor more information about how to configure the ChatGPT integration, refer to the documentation of [ChatGptNet](https://github.com/marcominerva/ChatGptNet).\n\n\u003e **Note**\nIf possible, use GPT-4 models. Current experiments demonstrate that they are more accurate than GPT-3 models when generating queries.\n\n#### The Web Application\n\nThe [DatabaseGptWeb](https://github.com/marcominerva/DatabaseGPT/tree/master/samples/DatabaseGpt.Web) project is a Web Application that shows how to provide a UI for **DatabaseGPT**. It requires .NET 8.0 SDK or later. To execute it, you need to set the required values in the [appsettings.json](https://github.com/marcominerva/DatabaseGPT/blob/master/samples/DatabaseGpt.Web/appsettings.json) file, as described above.\n\nYou can find a live demo that uses the [Northwind database](https://learn.microsoft.com/dotnet/framework/data/adonet/sql/linq/downloading-sample-databases) on https://databasegpt.azurewebsites.net. It relies on a deployment with a limited quota, so be patient if you get some error when trying it :)\n\n#### The Console Application\n\n![](https://raw.githubusercontent.com/marcominerva/DatabaseGPT/master/assets/DatabaseGptConsole.gif)\n\nThe [DatabaseGptConsole](https://github.com/marcominerva/DatabaseGPT/tree/master/samples/DatabaseGptConsole) project is a Console Application that can be used to test the library. It requires .NET 8.0 SDK or later. To execute it, you need to set the required values in the [appsettings.json](https://github.com/marcominerva/DatabaseGPT/blob/master/samples/DatabaseGptConsole/appsettings.json) file, as described above.\n\nIf you just want to run the application on Windows, you can download the binaries from the [Releases section](https://github.com/marcominerva/DatabaseGPT/releases).\n\n### Configuration\n\nThe system works by using an OpenAI model to generate a SQL query from a natural language question, reading the list of the available tables with their structure. If table names and columns are well defined, the library should be able to automatically determine what tables to use and how to join them. For example:\n\n```sql\nCREATE TABLE dbo.Categories(\n\tId INT IDENTITY(1,1) NOT NULL,\n\tCategoryName NVARCHAR(15) NOT NULL\n)\n\nCREATE TABLE dbo.Suppliers(\n\tId INT IDENTITY(1,1) NOT NULL,\n\tCompanyName NVARCHAR(40) NOT NULL,\n\tContactName NVARCHAR(30) NULL\n)\n\nCREATE TABLE dbo.Products(\n\tId INT IDENTITY(1,1) NOT NULL,\n\tProductName NVARCHAR(40) NOT NULL,\n\tSupplierId INT NULL,\n\tCategoryId INT NULL,\n\tQuantityPerUnit NVARCHAR(20) NULL,\n\tUnitPrice MONEY NULL,\n\tUnitsInStock SMALLINT NULL,\n\tUnitsOnOrder SMALLINT NULL,\n\tDiscontinued BIT NOT NULL\n)\n```\n\nGiving this schema, the model will be able to infer the following information, for example:\n\n- If the user wants the name of the products, the column `ProductName` of the table `Products` must be used.\n- The `SupplierId` column in the `Products` table is a foreign key to the `Id` column in the `Suppliers` table.\n- The `CategoryId` column in the `Products` table is a foreign key to the `Id` column in the `Categories` table.\n\nIf in the schema there are tables and columns and you never want to be used, you can exclude them from the query generation process by adding them to the `ExcludedTables` and `ExcludedColumns` arrays in the [appsettings.json](https://github.com/marcominerva/DatabaseGPT/blob/master/src/DatabaseGptConsole/appsettings.json#L17-L18) file. For example:\n\n```\n\"DatabaseSettings\": {\n    \"ExcludedTables\": [ \"dbo.CheckView\" ],       \n    \"ExcludedColumns\": [ \"Timestamp\" ]         // Exclude the Timestamp column from all tables\n}\n```\n\nOn the other hand, if you want to use only a particular set of tables, you can add them to the `IncludedTables` array in the [appsettings.json](https://github.com/marcominerva/DatabaseGPT/blob/master/src/DatabaseGptConsole/appsettings.json#L16) file. For example:\n\n```json\n\"DatabaseSettings\": {\n    \"IncludedTables\": [ \"Production.Product\", \"Sales.SalesOrderHeader\" \"Sales.SalesOrderDetail\" ]\n}\n```\n\nIn some cases, some columns might contains values that have a particular meaning. For example:\n\n```sql\nCREATE TABLE dbo.Attachments(\n\tId INT IDENTITY(1,1) NOT NULL,\n\tName NVARCHAR(40) NOT NULL,\n\tStatus INT NOT NULL,\n\tPath NVARCHAR(MAX) NOT NULL\n)\n```\n\nIn this case, the `Status` column contains an integer value that represents the status of the attachment. In order to make the query generation process more accurate, you need to tell the library that the `Status` column is an enumeration. You can do this by adding some comments in the [SystemMessage.txt](https://github.com/marcominerva/DatabaseGPT/blob/master/src/DatabaseGptConsole/SystemMessage.txt) file, using a natural language:\n\n```\n- If the 'Status' column of table 'dbo.Attachments' is equals to 0, it means that the attachment has not been processed yet.\n- If the 'Status' column of table 'dbo.Attachments' is equals to 1, it means that the attachment has been processed and approved.\n- If the 'Status' column of table 'dbo.Attachments' is equals to 2, it means that the attachment has been rejected.\n```\n\nYou can add as many indications as you need. The library will use this information to generate the query.\n\n#### Retry strategy\n\nAs we know, GPT models are not perfect. Sometimes, the generated query is not valid. In this case, the library will retry to generate the query, using a different approach. The number of retries is configured in the [appsettings.json](https://github.com/marcominerva/DatabaseGPT/blob/master/src/DatabaseGptConsole/appsettings.json#L19) file:\n\n```json\n\"DatabaseSettings\": {\n    \"MaxRetries\": 3\n}\n```\n\nThe retry strategy is handled using [Polly](https://github.com/App-vNext/Polly).\n\n## Contribute\n\nThe project is constantly evolving. Contributions are welcome. Feel free to file issues and pull requests on the repo and we'll address them as we can. \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarcominerva%2Fdatabasegpt","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmarcominerva%2Fdatabasegpt","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarcominerva%2Fdatabasegpt/lists"}