{"id":26219707,"url":"https://github.com/jackd248/typo3-natural-language-query","last_synced_at":"2025-10-03T13:09:59.567Z","repository":{"id":280979863,"uuid":"933809276","full_name":"jackd248/typo3-natural-language-query","owner":"jackd248","description":"TYP3 extension, which provides a natural language query interface for TYPO3 records.","archived":false,"fork":false,"pushed_at":"2025-03-11T11:18:55.000Z","size":1658,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-16T01:57:33.906Z","etag":null,"topics":["typo3","typo3-cms-extension"],"latest_commit_sha":null,"homepage":"","language":"PHP","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jackd248.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-02-16T18:29:39.000Z","updated_at":"2025-03-11T11:18:58.000Z","dependencies_parsed_at":"2025-03-06T11:18:06.764Z","dependency_job_id":null,"html_url":"https://github.com/jackd248/typo3-natural-language-query","commit_stats":null,"previous_names":["jackd248/typo3-natural-language-query"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jackd248%2Ftypo3-natural-language-query","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jackd248%2Ftypo3-natural-language-query/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jackd248%2Ftypo3-natural-language-query/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jackd248%2Ftypo3-natural-language-query/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jackd248","download_url":"https://codeload.github.com/jackd248/typo3-natural-language-query/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249183107,"owners_count":21226141,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["typo3","typo3-cms-extension"],"created_at":"2025-03-12T14:19:01.796Z","updated_at":"2025-10-03T13:09:59.516Z","avatar_url":"https://github.com/jackd248.png","language":"PHP","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\n![Extension icon](Resources/Public/Icons/Extension.svg)\n\n# TYPO3 extension `typo3_natural_language_query`\n\n\u003c/div\u003e\n\nThis extension provides a natural language query interface for TYPO3 records.\n\n\u003e [!NOTE]  \n\u003e This extension is meant to be a proof of concept for prompt engineering and query building using OpenAI. You should probably not use this in production\n\n![Screencast](./Documentation/Images/screencast.gif)\n\n## Installation\n\nYou can install the package via composer:\n\n```bash\ncomposer require konradmichalik/typo3-natural-language-query\n```\n\n## Configuration\n\nThe following configuration options are available:\n\n### API\n\n- `key`: OpenAI API key. Must be created on https://platform.openai.com/account/api-keys.\n- `model`: The model to use for the query. See https://platform.openai.com/docs/models. (Default: `gpt-3.5-turbo-instruct`)\n\n### Database\n\n- `strict_mode`: In strict mode, an exception will be thrown if the query attempts to perform a write or alter operation on the database. (Default: `true`)\n- `connection`: The name of the database connection to use. Depending on your needs, you may want to restrict the database user to read-only access. (Default: `Default`)\n- `ignore_tables`: A comma separated list of tables (supporting wildcards) to ignore when querying the database. (Default: `sys_*, cache_*, cf_*`)\n- `ignore_fields`: A comma separated list of fields to ignore when querying the database. (Default: `password`)\n\n## Console command\n\nUse the console command to ask a natural langauge question for querying records.\n\n```bash\nvendor/bin/typo3 nlq:ask \"How many pages are available in the page tree?\"\n```\n\n![Console Command](./Documentation/Images/command.png)\n\nUse the `-v` verbose option to get more information about the query process.\n\n```bash\nvendor/bin/typo3 nlq:ask \"How many pages are available in the page tree?\" -v\n```\n\n![Console Command](./Documentation/Images/command-verbose.png)\n\n## Service\n\nYou can also use the `Solver` service to solve natural language queries in your own code.\n\n```php\n\u003c?php\n\nuse Xima\\NaturalLanguageQuery\\Service\\Solver;\n\nclass DemoController {\n\n    public function __construct(Solver $solver) {}\n    \n    public function demoFunction(string $question) {\n        $query = $this-\u003esolver-\u003esolve($question);\n        return $query-\u003eanswer;\n    }\n}\n```\n\n## Background\n\nThis extension uses the OpenAI API to solve natural language queries while build SQL queries to query the TYPO3 database.\n\nThis allows users to formulate complex database queries without writing SQL manually.\n\n### Creating the Query Prompt\n\nTo generate an SQL query from a natural language input, the extension uses structured prompting. It analyzes the user's request, enriches it with relevant metadata about the TYPO3 database structure, and sends it to the OpenAI API.\n\n```text\nGiven an input question, first create a syntactically correct {dialect} query for a TYPO3 instance to run, then look at the results of the query and return the answer. Pay attention to the correct syntax of the query. If the answer refers to one or more specific data sets, also provide the individual UIDs.\n\nUse the following format:\n\nQuestion: \"Question here\"\nDesiredTable: \"Table to use\"\nSQLQuery: \"SQL Query to run\"\nSQLResult: \"Result of the SQLQuery\"\nAnswer: \"Final answer here (including UIDs if necessary in format UID=1,2,3)\"\n```\n\nFirst try to determine the desired table to use for answering the question.\n\n```text\nFirst determine the desired table to use for answering the question.\n\"backend_layout\" (Backend Layout),\"be_dashboards\" (Dashboard),\"be_groups\" (Backend usergroup), ...\n\nQuestion: \"How many pages are available in the page tree?\"\nDesiredTable: \"\n```\n\nWhen you have determined the desired table, create a SQL query to answer the question.\n\n```text\nOnly use the following tables and columns:\n\"pages\" has columns: uid (INT),pid (INT),tstamp (INT),crdate (INT), ...\n\n\nQuestion: \"How many pages are available in the page tree?\"\nDesiredTable: \"pages\"\nSQLQuery: \"\n```\n\n### OpenAI Conversation\n\nThe structured query request is sent to the OpenAI API, which generates a corresponding SQL query. Optimized prompt engineering ensures that the generated queries are accurate and aligned with the TYPO3 database schema.\n\n### Querying the Database\n\nOnce OpenAI returns an SQL statement, the extension validates and executes it against the TYPO3 database. Strict security mechanisms are in place to prevent unauthorized or destructive operations.\n\n```sql\nSELECT COUNT(*) FROM pages;\n```\n\n### Getting the Answer\n\nThe results of the query are processed and presented in a user-friendly format.\n\n```text\nQuestion: \"How many pages are available in the page tree?\"\nDesiredTable: \"pages\"\nSQLQuery: \"SELECT COUNT(*) FROM pages\"\nSQLResult: \"[{\"COUNT(*)\":4}]\"\nAnswer: \"\n```\n\n![Diagram](./Documentation/Images/nlq-diagram.png)\n\n### Error handling\n\n#### Exceeding prompt length\n\nIf the prompt exceeds the maximum length of 4096 characters, the extension adjust the prompt to be within the limit.\n\n```text\nTry to reduce the sql result in your query while only fetching the necessary uids.\n```\n\n#### Invalid SQL query\n\nIf the SQL query fails, the extension resend the prompt for querying the sql query adding the sql error from the database.\n\n### Security Considerations\n\nTo prevent unauthorized access or data loss, the extension uses strict security measures. It restricts the query capabilities to read-only operations and ignores sensitive tables and fields by default.\n\nYou should also consider to use a separate database connection with read-only access to the TYPO3 database.\n\n### Vision\n\nThe extension could be a base implementation to extend the list module with a natural language search feature.\n\n## Development\n\nUse the following ddev command to easily install all supported TYPO3 versions for locale development.\n\n```bash\n$ ddev install\n```\n\n## Credits\n\nThis project is highly inspired by the article [Query your Laravel database using natural language](https://beyondco.de/blog/query-your-laravel-database-using-natural-language) by [Marcel Pociot](https://pociot.dev/).\n\n## License\n\nThis project is licensed\nunder [GNU General Public License 2.0 (or later)](LICENSE.md).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjackd248%2Ftypo3-natural-language-query","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjackd248%2Ftypo3-natural-language-query","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjackd248%2Ftypo3-natural-language-query/lists"}