{"id":15002603,"url":"https://github.com/xnuinside/simple-ddl-parser","last_synced_at":"2026-01-18T10:47:34.518Z","repository":{"id":38095996,"uuid":"345078823","full_name":"xnuinside/simple-ddl-parser","owner":"xnuinside","description":"Simple DDL Parser to parse SQL (HQL, TSQL, AWS Redshift, BigQuery, Snowflake and other dialects) ddl files to json/python dict with full information about columns: types, defaults, primary keys, etc. \u0026 table properties, types, domains, etc.","archived":false,"fork":false,"pushed_at":"2024-10-04T11:30:14.000Z","size":2139,"stargazers_count":195,"open_issues_count":35,"forks_count":42,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-04-07T00:12:19.085Z","etag":null,"topics":["bigquery","columns","ddl","ddl-parser","ddls","hive","hql","mssql","mysql","oracle-database","oracle-db","parser","postgresql","redshift","schemas","snowflake","sql","sql-parser","tsql","types"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/xnuinside.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.txt","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":null,"patreon":null,"open_collective":null,"ko_fi":"xnuinside","tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"otechie":null,"custom":null}},"created_at":"2021-03-06T11:39:02.000Z","updated_at":"2025-04-02T12:29:02.000Z","dependencies_parsed_at":"2023-02-12T05:01:46.905Z","dependency_job_id":"79125f4d-2558-4038-869a-5e198cceb984","html_url":"https://github.com/xnuinside/simple-ddl-parser","commit_stats":{"total_commits":335,"total_committers":14,"mean_commits":"23.928571428571427","dds":"0.18208955223880596","last_synced_commit":"b38506d3f1bf463268ae8c033325c9a2b6f10854"},"previous_names":[],"tags_count":80,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xnuinside%2Fsimple-ddl-parser","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xnuinside%2Fsimple-ddl-parser/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xnuinside%2Fsimple-ddl-parser/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xnuinside%2Fsimple-ddl-parser/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/xnuinside","download_url":"https://codeload.github.com/xnuinside/simple-ddl-parser/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254544146,"owners_count":22088807,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bigquery","columns","ddl","ddl-parser","ddls","hive","hql","mssql","mysql","oracle-database","oracle-db","parser","postgresql","redshift","schemas","snowflake","sql","sql-parser","tsql","types"],"created_at":"2024-09-24T18:51:26.348Z","updated_at":"2026-01-18T10:47:34.496Z","avatar_url":"https://github.com/xnuinside.png","language":"Python","readme":"## Simple DDL Parser\n\n![badge1](https://img.shields.io/pypi/v/simple-ddl-parser) ![badge2](https://img.shields.io/pypi/l/simple-ddl-parser) ![badge3](https://img.shields.io/pypi/pyversions/simple-ddl-parser) ![workflow](https://github.com/xnuinside/simple-ddl-parser/actions/workflows/main.yml/badge.svg)\n\nBuild with ply (lex \u0026 yacc in python). A lot of samples in 'tests/.\n\n### Is it Stable?\n\nYes, library already has about 9000+ downloads per day  - https://pypistats.org/packages/simple-ddl-parser..\n\nAs maintainer, I guarantee that any backward incompatible changes will not be done in patch or minor version. But! Pay attention that sometimes output in keywords can be changed in minor version because of fixing wrong behaviour in past.\n\n### Articles with examples\n\n1. SQL Diagram (Part 3): SQL-to-ERD with DDL: https://levelup.gitconnected.com/sql-diagram-part-3-sql-to-erd-with-ddl-4c9840ee86c3 \n\n### Updates in version 1.x\n\nThe full list of updates can be found in the Changelog below (at the end of README).\n\nVersion 1.0.0 was released due to significant changes in the output structure and a stricter approach regarding the scope of the produced output. Now, you must provide the argument 'output_mode=name_of_your_dialect' if you wish to see arguments/properties specific to a particular dialect\n\n\n### How does it work?\n\nParser supports: \n\n- SQL\n- HQL (Hive)\n- MSSQL dialect\n- Oracle dialect\n- MySQL dialect\n- PostgreSQL dialect\n- BigQuery\n- Redshift\n- Snowflake\n- SparkSQL\n- IBM DB2 dialect\n- Informix/GBase 8s dialect\n\nYou can check dialects sections after `Supported Statements` section to get more information that statements from dialects already supported by parser. If you need to add more statements or new dialects - feel free to open the issue. \n\n\n### Feel free to open Issue with DDL sample\nPay attentions that I'm adding functional tests for all supported statement, so if you see that your statement is failed and you didn't see it in the test 99,9% that I did n't have sample with such SQL statement - so feel free to open the issue and I will add support for it. \n\n**If you need some statement, that not supported by parser yet**: please provide DDL example \u0026 information about that is it SQL dialect or DB.\n\nTypes that are used in your DB does not matter, so parser must also work successfully to any DDL for SQL DB. Parser is NOT case sensitive, it did not expect that all queries will be in upper case or lower case. So you can write statements like this:\n\n```sql\n\n    Alter Table Persons ADD CONSTRAINT CHK_PersonAge CHECK (Age\u003e=18 AND City='Sandnes');\n\n```\n\nIt will be parsed as is without errors.\n\nIf you have samples that cause an error - please open the issue (but don't forget to add ddl example), I will be glad to fix it.\n\nA lot of statements and output result you can find in tests on the github - https://github.com/xnuinside/simple-ddl-parser/tree/main/tests .\n\n### How to install\n\n```bash\n\n    pip install simple-ddl-parser\n\n```\n\n## How to use\n\n### Extract additional information from HQL (\u0026 other dialects)\n\nIn some dialects like HQL there is a lot of additional information about table like, fore example, is it external table, STORED AS, location \u0026 etc. This property will be always empty in 'classic' SQL DB like PostgreSQL or MySQL and this is the reason, why by default this information are 'hidden'.\nAlso some fields hidden in HQL, because they are simple not exists in HIVE, for example 'deferrable_initially'\nTo get this 'hql' specific details about table in output please use 'output_mode' argument in run() method.\n\nexample:\n\n```python\n\n    ddl = \"\"\"\n    CREATE TABLE IF NOT EXISTS default.salesorderdetail(\n        SalesOrderID int,\n        ProductID int,\n        OrderQty int,\n        LineTotal decimal\n        )\n    PARTITIONED BY (batch_id int, batch_id2 string, batch_32 some_type)\n    LOCATION 's3://datalake/table_name/v1'\n    ROW FORMAT DELIMITED\n        FIELDS TERMINATED BY ','\n        COLLECTION ITEMS TERMINATED BY '\\002'\n        MAP KEYS TERMINATED BY '\\003'\n    STORED AS TEXTFILE\n    \"\"\"\n\n    result = DDLParser(ddl).run(output_mode=\"hql\")\n    print(result)\n```\n\nAnd you will get output with additional keys 'stored_as', 'location', 'external', etc.\n\n```python\n\n    # additional keys examples\n  {\n    ...,\n    'location': \"'s3://datalake/table_name/v1'\",\n    'map_keys_terminated_by': \"'\\\\003'\",\n    'partitioned_by': [{'name': 'batch_id', 'size': None, 'type': 'int'},\n                        {'name': 'batch_id2', 'size': None, 'type': 'string'},\n                        {'name': 'batch_32', 'size': None, 'type': 'some_type'}],\n    'primary_key': [],\n    'row_format': 'DELIMITED',\n    'schema': 'default',\n    'stored_as': 'TEXTFILE',\n    ... \n  }\n\n```\n\nIf you run parser with command line add flag '-o=hql' or '--output-mode=hql' to get the same result.\n\nPossible output_modes: ['redshift', 'spark_sql', 'mysql', 'bigquery', 'mssql', 'databricks', 'sqlite', 'vertics', 'ibm_db2', 'postgres', 'oracle', 'hql', 'snowflake', 'sql']\n\n### From python code\n\n```python\n    from simple_ddl_parser import DDLParser\n\n\n    parse_results = DDLParser(\"\"\"create table dev.data_sync_history(\n        data_sync_id bigint not null,\n        sync_count bigint not null,\n        sync_mark timestamp  not  null,\n        sync_start timestamp  not null,\n        sync_end timestamp  not null,\n        message varchar(2000) null,\n        primary key (data_sync_id, sync_start)\n    ); \"\"\").run()\n\n    print(parse_results) \n\n```\n\n### To parse from file\n\n```python\n    \n    from simple_ddl_parser import parse_from_file\n\n    result = parse_from_file('tests/sql/test_one_statement.sql')\n    print(result)\n\n```\n\n### From command line\n\nsimple-ddl-parser is installed to environment as command **sdp**\n\n```bash\n\n    sdp path_to_ddl_file\n\n    # for example:\n\n    sdp tests/sql/test_two_tables.sql\n    \n```\n\nYou will see the output in **schemas** folder in file with name **test_two_tables_schema.json**\n\nIf you want to have also output in console - use **-v** flag for verbose.\n\n```bash\n    \n    sdp tests/sql/test_two_tables.sql -v\n    \n```\n\nIf you don't want to dump schema in file and just print result to the console, use **--no-dump** flag:\n\n\n```bash\n    \n    sdp tests/sql/test_two_tables.sql --no-dump\n    \n```\n\nYou can provide target path where you want to dump result with argument **-t**, **--target**:\n\n\n```bash\n    \n    sdp tests/sql/test_two_tables.sql -t dump_results/\n    \n```\n### Get Output in JSON\n\nIf you want to get output in JSON in stdout you can use argument **json_dump=True** in method **.run()** for this\n```python\n    from simple_ddl_parser import DDLParser\n\n\n    parse_results = DDLParser(\"\"\"create table dev.data_sync_history(\n        data_sync_id bigint not null,\n        sync_count bigint not null,\n    ); \"\"\").run(json_dump=True)\n\n    print(parse_results) \n\n```\nOutput will be:\n\n```json\n[{\"columns\": [{\"name\": \"data_sync_id\", \"type\": \"bigint\", \"size\": null, \"references\": null, \"unique\": false, \"nullable\": false, \"default\": null, \"check\": null}, {\"name\": \"sync_count\", \"type\": \"bigint\", \"size\": null, \"references\": null, \"unique\": false, \"nullable\": false, \"default\": null, \"check\": null}], \"primary_key\": [], \"alter\": {}, \"checks\": [], \"index\": [], \"partitioned_by\": [], \"tablespace\": null, \"schema\": \"dev\", \"table_name\": \"data_sync_history\"}]\n```\n\n### More details\n\n`DDLParser(ddl).run()`\n.run() method contains several arguments, that impact changing output result. As you can saw upper exists argument `output_mode` that allow you to set dialect and get more fields in output relative to chosen dialect, for example 'hql'. Possible output_modes: ['redshift', 'spark_sql', 'mysql', 'bigquery', 'mssql', 'databricks', 'sqlite', 'vertics', 'ibm_db2', 'postgres', 'oracle', 'hql', 'snowflake', 'sql']\n\nAlso in .run() method exists argument `group_by_type` (by default: False). By default output of parser looks like a List with Dicts where each dict == one entity from ddl (table, sequence, type, etc). And to understand that is current entity you need to check Dict like: if 'table_name' in dict - this is a table, if 'type_name' - this is a type \u0026 etc.\n\nTo make work little bit easy you can set group_by_type=True and you will get output already sorted by types, like:\n\n```python\n\n    { \n        'tables': [all_pasrsed_tables], \n        'sequences': [all_pasrsed_sequences], \n        'types': [all_pasrsed_types], \n        'domains': [all_pasrsed_domains],\n        ...\n    }\n\n```\n\nFor example:\n\n```python\n\n    ddl = \"\"\"\n    CREATE TYPE \"schema--notification\".\"ContentType\" AS\n        ENUM ('TEXT','MARKDOWN','HTML');\n        CREATE TABLE \"schema--notification\".\"notification\" (\n            content_type \"schema--notification\".\"ContentType\"\n        );\n    CREATE SEQUENCE dev.incremental_ids\n        INCREMENT 10\n        START 0\n        MINVALUE 0\n        MAXVALUE 9223372036854775807\n        CACHE 1;\n    \"\"\"\n\n    result = DDLParser(ddl).run(group_by_type=True)\n\n    # result will be:\n\n    {'sequences': [{'cache': 1,\n                    'increment': 10,\n                    'maxvalue': 9223372036854775807,\n                    'minvalue': 0,\n                    'schema': 'dev',\n                    'sequence_name': 'incremental_ids',\n                    'start': 0}],\n    'tables': [{'alter': {},\n                'checks': [],\n                'columns': [{'check': None,\n                            'default': None,\n                            'name': 'content_type',\n                            'nullable': True,\n                            'references': None,\n                            'size': None,\n                            'type': '\"schema--notification\".\"ContentType\"',\n                            'unique': False}],\n                'index': [],\n                'partitioned_by': [],\n                'primary_key': [],\n                'schema': '\"schema--notification\"',\n                'table_name': '\"notification\"'}],\n    'types': [{'base_type': 'ENUM',\n                'properties': {'values': [\"'TEXT'\", \"'MARKDOWN'\", \"'HTML'\"]},\n                'schema': '\"schema--notification\"',\n                'type_name': '\"ContentType\"'}]}\n\n```\n\n### ALTER statements\n\nParser supports various ALTER TABLE statements:\n\n- **ADD COLUMN** - with or without COLUMN keyword\n- **DROP COLUMN** - with or without COLUMN keyword (Oracle style)\n- **MODIFY COLUMN** - with or without COLUMN keyword (Oracle style)\n- **ALTER COLUMN** - SQL Server style\n- **RENAME COLUMN**\n- **ADD FOREIGN KEY** - with or without CONSTRAINT\n- **ADD PRIMARY KEY** - with or without CONSTRAINT\n- **ADD UNIQUE**\n- **ADD CHECK**\n- **ADD DEFAULT**\n\nMultiple operations can be combined in a single ALTER statement using commas:\n\n```sql\nALTER TABLE my_table ADD col1 int, ADD col2 varchar;\nALTER TABLE my_table DROP COLUMN col1, DROP COLUMN col2;\nALTER TABLE my_table MODIFY COLUMN col1 int, MODIFY COLUMN col2 varchar;\n```\n\nAll ALTER statements are parsed and information is stored in the 'alter' key of the table's dict.\nFor example, please check alter statement tests - **tests/test_alter_statements.py**\n\n\n### More examples \u0026 tests\n\nYou can find in **tests/** folder.\n\n### Dump result in json\n\nTo dump result in json use argument .run(dump=True)\n\nYou also can provide a path where you want to have a dumps with schema with argument .run(dump_path='folder_that_use_for_dumps/')\n\n\n### Raise error if DDL cannot be parsed by Parser\n\nBy default Parser does not raise the error if some statement cannot be parsed - and just skip \u0026 produce empty output.\n\nTo change this behavior you can pass 'silent=False' argumen to main parser class, like:\n\n    DDLParser(.., silent=False)\n\n### Normalize names\n\nUse DDLParser(.., normalize_names=True)flag that change output of parser:\nIf flag is True (default 'False') then all identifiers will be returned without '[', '\"' and other delimiters that used in different SQL dialects to separate custom names from reserved words \u0026 statements.\nFor example, if flag set 'True' and you pass this input: \n\nCREATE TABLE [dbo].[TO_Requests](\n    [Request_ID] [int] IDENTITY(1,1) NOT NULL,\n    [user_id] [int]\n\nIn output you will have names like 'dbo' and 'TO_Requests', not '[dbo]' and '[TO_Requests]'.\n\n\n## Supported Statements\n\n- CREATE [OR REPLACE] TABLE [ IF NOT EXISTS ] + columns definition, columns attributes: column name + type + type size(for example, varchar(255)), UNIQUE, PRIMARY KEY, DEFAULT, CHECK, NULL/NOT NULL, REFERENCES, ON DELETE, ON UPDATE,  NOT DEFERRABLE, DEFERRABLE INITIALLY, GENERATED ALWAYS, STORED, COLLATE\n\n- STATEMENTS: PRIMARY KEY, CHECK, FOREIGN KEY in table definitions (in create table();)\n\n- ALTER TABLE STATEMENTS: ADD CHECK (with CONSTRAINT), ADD FOREIGN KEY (with CONSTRAINT), ADD UNIQUE, ADD DEFAULT FOR, ALTER TABLE ONLY, ALTER TABLE IF EXISTS; ALTER .. PRIMARY KEY; ALTER .. USING INDEX TABLESPACE; ALTER .. ADD; ALTER .. MODIFY; ALTER .. ALTER COLUMN; etc\n\n- PARTITION BY statement\n\n- CREATE SEQUENCE with words: INCREMENT [BY], START [WITH], MINVALUE, MAXVALUE, CACHE\n\n- CREATE TYPE statement:  AS TABLE, AS ENUM, AS OBJECT, INTERNALLENGTH, INPUT, OUTPUT\n\n- LIKE statement (in this and only in this case to output will be added 'like' keyword with information about table from that we did like - 'like': {'schema': None, 'table_name': 'Old_Users'}).\n\n- TABLESPACE statement\n\n- COMMENT ON statement\n\n- CREATE SCHEMA [IF NOT EXISTS] ... [AUTHORIZATION] ...\n\n- CREATE DOMAIN [AS]\n\n- CREATE [SMALLFILE | BIGFILE] [TEMPORARY] TABLESPACE statement\n\n- CREATE DATABASE + Properties parsing\n\n### SparkSQL Dialect statements\n\n- USING\n\n\n### HQL Dialect statements\n\n- PARTITIONED BY statement\n- ROW FORMAT, ROW FORMAT SERDE\n- WITH SERDEPROPERTIES (\"input.regex\" =  \"..some regex..\")\n- STORED AS (AVRO, PARQUET, etc), STORED AS INPUTFORMAT, OUTPUTFORMAT\n- COMMENT\n- LOCATION\n- FIELDS TERMINATED BY, LINES TERMINATED BY, COLLECTION ITEMS TERMINATED BY, MAP KEYS TERMINATED BY\n- TBLPROPERTIES ('parquet.compression'='SNAPPY' \u0026 etc.)\n- SKEWED BY\n- CLUSTERED BY \n\n### MySQL\n\n- ON UPDATE in column without reference \n\n#### MSSQL \n\n- CONSTRAINT [CLUSTERED]... PRIMARY KEY\n- CONSTRAINT ... WITH statement\n- PERIOD FOR SYSTEM_TIME in CREATE TABLE statement\n- ON [PRIMARY] after CREATE TABLE statement (sample in test files test_mssql_specific.py)\n- WITH statement for TABLE properties\n- TEXTIMAGE_ON statement\n- DEFAULT NEXT VALUE FOR in COLUMN DEFAULT\n\n### MSSQL / MySQL/ Oracle\n\n- type IDENTITY statement\n- FOREIGN KEY REFERENCES statement\n- 'max' specifier in column size\n- CONSTRAINT ... UNIQUE, CONSTRAINT ... CHECK, CONSTRAINT ... FOREIGN KEY, CONSTRAINT ... PRIMARY KEY\n- CREATE CLUSTERED INDEX\n- CREATE TABLE (...) ORGANIZATION INDEX \n\n### Oracle\n\n- ENCRYPT column property [+ NO SALT, SALT, USING]\n- STORAGE column property\n\n\n### PotgreSQL\n\n- INHERITS table statement - https://postgrespro.ru/docs/postgresql/14/ddl-inherit \n\n### AWS Redshift Dialect statements\n\n- ENCODE column property\n- SORTKEY, DISTSTYLE, DISTKEY, ENCODE table properties\n- CREATE TEMP / TEMPORARY TABLE\n\n- syntax like with LIKE statement:\n\n `create temp table tempevent(like event);`\n\n### Snowflake Dialect statements\n\n- CREATE .. CLONE statements for table, database and schema\n- CREATE TABLE [or REPLACE] [ TRANSIENT | TEMPORARY ] .. CLUSTER BY ..\n- CONSTRAINT .. [NOT] ENFORCED \n- COMMENT = in CREATE TABLE \u0026 CREATE SCHEMA statements\n- WITH MASKING POLICY\n- WITH TAG, including multiple tags in the same statement.\n- DATA_RETENTION_TIME_IN_DAYS\n- MAX_DATA_EXTENSION_TIME_IN_DAYS\n- CHANGE_TRACKING\n\n### BigQuery\n\n- OPTION in CREATE SCHEMA statement\n- OPTION in CREATE TABLE statement\n- OPTION in column definition statement\n\n### Informix/GBase 8s\n\nInformix is an IBM relational database. GBase 8s is a Chinese enterprise database based on Informix with Oracle compatibility features.\n\nSupported Informix-specific features:\n- Data types: SERIAL, SERIAL8, BIGSERIAL, INT8, LVARCHAR, TEXT, BYTE, BLOB, CLOB, MONEY, NCHAR, NVARCHAR\n- DATETIME with qualifiers: DATETIME YEAR TO SECOND, DATETIME YEAR TO FRACTION(n), DATETIME YEAR TO DAY\n- INTERVAL types: INTERVAL HOUR TO MINUTE, INTERVAL DAY TO DAY, INTERVAL YEAR TO MONTH\n- DEFAULT with Informix functions: TODAY, CURRENT, USER\n- Storage options: IN dbspace, EXTENT SIZE, NEXT SIZE, LOCK MODE\n\nGBase 8s Oracle-compatible features:\n- Data types: VARCHAR2, NVARCHAR2, NUMBER\n- Virtual columns: `column_name AS (expression)`, `GENERATED ALWAYS AS (expression)`\n\n### Parser settings\n\n\n#### Logging\n\n1. Logging to file\n\nTo get logging output to file you should provide to Parser 'log_file' argument with path or file name:\n\n```console\n\n    DDLParser(ddl, log_file='parser221.log').run(group_by_type=True)\n\n```\n\n2. Logging level\n\nTo set logging level you should provide argument 'log_level'\n\n```console\n\n    DDLParser(ddl, log_level=logging.INFO).run(group_by_type=True)\n\n```\n\n### Thanks for involving \u0026 contributions\n\nMost biggest 'Thanks' ever goes for contributions in parser:\nhttps://github.com/dmaresma\nhttps://github.com/cfhowes\nhttps://github.com/swiatek25\nhttps://github.com/slurpyb\nhttps://github.com/PBalsdon\n\nBig thanks for the involving \u0026 contribution with test cases with DDL samples \u0026 opening issues goes to:\n\n* https://github.com/kukigai , \n* https://github.com/kliushnichenko ,\n* https://github.com/geob3d\n\nfor help with debugging \u0026 testing support for BigQuery dialect DDLs:\n* https://github.com/ankitdata ,\n* https://github.com/kalyan939\n\n## Changelog\n**v1.8.0**\n### Important\n1. Python 3.7 and 3.8 support is deprecated. Minimum supported version is now Python 3.9.\n\n### Improvements\n1. Added support for Python 3.13.\n2. Added tox configuration for testing across multiple Python versions (3.9-3.13).\n\n\n**v1.7.1**\n### Fixes:\n1. Fix 'character set' issue - https://github.com/xnuinside/simple-ddl-parser/issues/288 \n\n\n**v1.7.0**\n### Fixes\n1. DEFAULT Value with '::' cast parsed correctly now - https://github.com/xnuinside/simple-ddl-parser/issues/286\n\n### Improvements\n1. Added support for ENUM \u0026 SET column type - https://github.com/xnuinside/simple-ddl-parser/issues/259 \n\n\n**v1.6.1**\n### Fixes\n1. #289 CREATE SCHEMA IF NOT EXISTS plus comment fail\n2. schema or db.schema location in snowflake\n\n**v1.6.0**\n### IMPORTANT:\nIn this versions there is some output changes \u0026 fixes that can break your code.\n1. Now all arguments inside brackets are parsed as separate strings in the list.\nFor example:\n `file_format = (TYPE=JSON NULL_IF=('field')` this was parsed like 'NULL_IF': \"('field')\",\n now it will be: 'NULL_IF': [\"'field'\"],\n\n2. Added separate tokens for EQ `=` and IN (previously they was parsed as IDs also - for internal info, for contributors.\n\n3. Some check statements in columns now parsed validly, also IN statements parsed as normal lists.\nSo this statement include_exclude_ind CHAR(1) NOT NULL CONSTRAINT chk_metalistcombo_logicalopr\n  CHECK (include_exclude_ind IN ('I', 'E')),\n\n\nwill produce this output:\n\n{'check': {'constraint_name': 'chk_metalistcombo_logicalopr',\n                         'statement': {'in_statement': {'in': [\"'I'\", \"'E'\"],\n                                                        'name': 'include_exclude_ind'}}},\n\n\n### Fixes\n1. DEFAULT word now is not arriving in key 'default' (it was before in some cases)\n\n### New Features\n1. Added Athena output mode and initial support - https://github.com/datacontract/datacontract-cli/issues/332\n\n\n**v1.5.4**\n### Improvements\n#### Snowflake :\n1. In Snowflake add `pattern` token for external table statement, and improve location rendering\n\n\n**v1.5.3**\n### Fixes\n\n1. In Snowflake unexpected error when STRIP_OUTER_ARRAY property in file_format statement - https://github.com/xnuinside/simple-ddl-parser/issues/276\n2.\n\n**v1.5.2**\n### Improvements\n#### MySQL\n1. Added support for COLLATE - https://github.com/xnuinside/simple-ddl-parser/pull/266/files\n\n### Fixes\n\n1. In Snowflake Fix unexpected behaviour when file_format name given - https://github.com/xnuinside/simple-ddl-parser/issues/273\n\n**v1.5.1**\n### Improvements\n#### MySQL\n\n1. Added support for INDEX statement in column definition - https://github.com/xnuinside/simple-ddl-parser/issues/253\n2.\n\n\n**v1.5.0**\n\n### Fixes\n\n1. Now, `unique` set up to column only if it was only one column in unique constraint/index. Issue - https://github.com/xnuinside/simple-ddl-parser/issues/255\n2. Fixed issue when UNIQUE KEY was identified as primary key - https://github.com/xnuinside/simple-ddl-parser/issues/253\n\n\n**v1.4.0**\n\n### Fixes\n#### BigQuery:\n1. Indexes without schema causes issues in BigQuery dialect - fixed.\n\n### Improvements\n#### Oracle:\n1. Added new output keywords in table definition - `temp` \u0026 `is_global`. Added support for create global temporary table - https://github.com/xnuinside/simple-ddl-parser/issues/182\n\n\n**v1.3.0**\n\n### Fixes\nPostgreSQL:\n1.  Timezone was moved out from type definition to keyword 'with_time_zone' it can be True (if with time zone) or False (if without)\nBigQuery:\n1. Previously Range in RANGE_BUCKETS was parsed as a columns, now this behaviour is changed and\nrange placed in own keyword - 'range' (can be array or str).\nAlso for all ``*_TRUNC PARTITIONS` like DATETIME_TRUNC, TIMESTAMP_TRUNC, etc, second argument moved to arg 'trunc_by'\n\n### Improvements\nPostgreSQL:\n1. Added support for PostgreSQL with / without time zone - https://github.com/xnuinside/simple-ddl-parser/issues/250\n\nBigQuery:\n1. Added support for GENERATE_ARRAY in RANGE_BUCKETS https://github.com/xnuinside/simple-ddl-parser/issues/183\n\n**v1.2.1**\n### Fixes\nMySQL:\n1. Fixed issue relative to auto_increment that caused empty output if auto_increment defined in table properties -\nhttps://github.com/xnuinside/simple-ddl-parser/issues/206\n\n### Improvements\nMySQL:\n1. auto_increment added as property to mysql output\n\nOracle:\n1. Added support for  constraint name in column definition - https://github.com/xnuinside/simple-ddl-parser/issues/203\n2. Added support for GENERATED (ALWAYS | (BY DEFAULT [ON NULL])) AS IDENTITY in column definition\n\nPostgreSQL:\n1. Added support for CAST statement in column GENERATE ALWAYS expression - https://github.com/xnuinside/simple-ddl-parser/issues/198\n\n\n**v1.1.0**\n### Improvements\nMySQL:\n1. Added support for INDEX statement inside table definition\n2. Added support for MySQL INVISIBLE/VISIBLE statement - https://github.com/xnuinside/simple-ddl-parser/issues/243\n\nSnowflake:\n1. Added support for cluster by statement before columns definition - https://github.com/xnuinside/simple-ddl-parser/issues/234\n\n\n**v1.0.4**\n### Improvements\n1. Support functions with schema prefix in `DEFAULT` and `CHECK` statements. https://github.com/xnuinside/simple-ddl-parser/issues/240\n### Fixes\n1. Fix for REFERENCES NOT NULL - https://github.com/xnuinside/simple-ddl-parser/issues/239\n2. Fix for snowflake stage name location format bug fix - https://github.com/xnuinside/simple-ddl-parser/pull/241\n\n**v1.0.3**\n### Improvements\n1. Fixed bug with `CREATE OR REPLACE SCHEMA`.\n2. Added support of create empty tables without columns CREATE TABLE tablename (); (valid syntax in SQL)\n\n### Snowflake\n1. Fixed bug with snowflake `stage_` fileformat option value equal a single string as `FIELD_OPTIONALLY_ENCLOSED_BY = '\\\"'`, `FIELD_DELIMITER = '|'`\n2. improve snowflake fileformat key equals value into dict. type.\n\n**v1.0.2**\n### Improvements\n1. Fixed bug with places first table property value in 'authorization' key. Now it is used real property name.\n2. Fixed typo on Databricks dialect\n3. improved equals symbols support within COMMENT statement.\n4. turn regexp into functions\n\n### MySQL Improvements\n1. UNSIGNED property after int parsed validly now\n\n### Snowflake\n1. Snowflake TAG now available on SCHEMA definitions.\n\n\n**v1.0.1**\n### Minor Fixes\n1. When using `normalize_names=True` do not remove `[]` from types like `decimal(21)[]`.\n2. When using `normalize_names=True` ensure that `\"complex\".\"type\"` style names convert to `complex.type`.\n\n**v1.0.0**\nIn output structure was done important changes that can in theory breaks code.\n\n### Important changes\n1. Important change: \n\nall custom table properties that are defined after column definition in 'CREATE TABLE' statement and relative to only one dialect (only for SparkSQL, or HQL,etc), for example, like here:\nhttps://github.com/xnuinside/simple-ddl-parser/blob/main/tests/dialects/test_snowflake.py#L767  or https://github.com/xnuinside/simple-ddl-parser/blob/main/tests/dialects/test_spark_sql.py#L133 will be saved now in property `table_properties` as dict.\nPreviously they was placed on same level of table output as `columns`, `alter`, etc. Now, they grouped and moved to key `table_properties`.\n\n2. Formatting parser result now represented by 2 classes - Output \u0026 TableData, that makes it more strict and readable.\n\n3. The output mode now functions more strictly. If you want to obtain output fields specific to a certain dialect, \nuse output_mode='snowflake' for Snowflake or output_mode='hql' for HQL, etc. \nPreviously, some keys appeared in the result without being filtered by dialect. \nFor example, if 'CLUSTER BY' was in the DDL, it would show up in the 'cluster_by' field regardless of the output mode. \nHowever, now all fields that only work in certain dialects and are not part of the basic SQL notation will only be shown \nif you choose the correct output_mode.\n\n\n### New Dialects support\n1. Added as possible output_modes new Dialects: \n- Databricks SQL like 'databricks', \n- Vertica as 'vertica', \n- SqliteFields as 'sqlite',\n- PostgreSQL as 'postgres'\n\nFull list of supported dialects you can find in dict - `supported_dialects`:\n\n`from simple_ddl_parser import supported_dialects`\n\nCurrently supported: ['redshift', 'spark_sql', 'mysql', 'bigquery', 'mssql', 'databricks', 'sqlite', 'vertics', 'ibm_db2', 'postgres', 'oracle', 'hql', 'snowflake', 'sql']\n\nIf you don't see dialect that you want to use - open issue with description and links to Database docs or use one of existed dialects.\n\n### Snowflake updates:\n1. For some reasons, 'CLONE' statement in SNOWFLAKE was parsed into 'like' key in output. Now it was changed to 'clone' - inner structure of output stay the same as previously.\n\n### MySQL updates:\n1. Engine statement now parsed correctly. Previously, output was always '='.\n\n### BigQuery updates:\n1. Word 'schema' totally removed from output. `Dataset` used instead of `schema` in BigQuery dialect.\n\n**v0.32.1**\n### Minor Fixes\n1. Removed debug print\n\n**v0.32.0**\n\n### Improvements\n1. Added support for several ALTER statements (ADD, DROP, RENAME, etc) - https://github.com/xnuinside/simple-ddl-parser/issues/215\nIn 'alter' output added several keys:\n    1. 'dropped_columns' - to store information about columns that was in table, but after dropped by alter\n    2. 'renamed_columns' - to store information about columns that was renamed\n    3. 'modified_columns' - to track alter column changes for defaults, datetype, etc. Argument stores previous columns states.\n\n### Fixes\n1. Include source column names in FOREIGN KEY references. Fix for: https://github.com/xnuinside/simple-ddl-parser/issues/196\n2. ALTER statement now will be parsed correctly if names \u0026 schemas written differently in `create table` statement and alter. \nFor example, if in create table you use quotes like \"schema_name\".\"table_name\", but in alter was schema_name.table_name - previously it didn't work, but now parser understand that it is the same table.\n\n**v0.31.3**\n### Improvements\n#### Snowflake update:\n1. Added support for Snowflake Virtual Column definition in External Column  `AS ()` statement - https://github.com/xnuinside/simple-ddl-parser/issues/218\n2. enforce support for Snowflake _FILE_FORMAT options in External Column ddl statement - https://github.com/xnuinside/simple-ddl-parser/issues/221\n\n#### Others\n1. Support for KEY statement in CREATE TABLE statements. KEY statements will now create INDEX entries in the DDL parser.\n\n\n**v0.31.2**\n### Improvements\n#### Snowflake update:\n1. Added support for Snowflake AUTOINCREMENT | IDENTITY column definitions with optional parameter `ORDER|NOORDER` statement - https://github.com/xnuinside/simple-ddl-parser/issues/213\n\n#### Common\n1. Added param 'encoding' to parse_from_file function - https://github.com/xnuinside/simple-ddl-parser/issues/142.\nDefault encoding is utf-8.\n\n\n**v0.31.1**\n### Improvements\n#### Snowflake update:\n1. Support multiple tag definitions in a single `WITH TAG` statement.\n2. Added support for Snowflake double single quotes - https://github.com/xnuinside/simple-ddl-parser/issues/208\n\n**v0.31.0**\n### Fixes:\n1. Move inline flag in regexp (issue with python 3.11) - https://github.com/xnuinside/simple-ddl-parser/pull/200\nFix for: https://github.com/xnuinside/simple-ddl-parser/issues/199\n\n### Improvements:\n1. Added `Snowflake Table DDL support of WITH MASKING POLICY column definition` - https://github.com/xnuinside/simple-ddl-parser/issues/201\n\n\n### Updates:\n1. All deps updated to the latest versions.\n\n**v0.30.0**\n### Fixes:\n\n1. IDENTITY now parsed normally as a separate column property. Issue: https://github.com/xnuinside/simple-ddl-parser/issues/184\n\n### New Features:\n\n1. IN TABLESPACE IBM DB2 statement now is parsed into 'tablespace' key. Issue: https://github.com/xnuinside/simple-ddl-parser/issues/194.\nINDEX IN also parsed to 'index_in' key.\nAdded support for ORGANIZE BY statement\n\n2. Added support for PostgreSQL INHERITS statement. Issue: https://github.com/xnuinside/simple-ddl-parser/issues/191\n\n**v0.29.1**\n### Important updates:\n1. Python 3.6 is deprecated in tests and by default, try to move to Python3.7, but better to 3.8, because 3.7 will be deprecated in 2023.\n\n### Fixes\n1. Fix for https://github.com/xnuinside/simple-ddl-parser/issues/177\n\n### Improvements\n1. Added support for Oracle 2 component size for types, like '30 CHAR'. From https://github.com/xnuinside/simple-ddl-parser/issues/176\n\n\n**v0.29.0**\n\n### Fixes\n1.  AUTOINCREMENT statement now parsed validly same way as AUTO_INCREMENT and showed up in output as 'autoincrement' property of the column\nFix for: https://github.com/xnuinside/simple-ddl-parser/issues/170\n2. Fix issue ' TypeError argument of type 'NoneType' is not iterable' on some foreigen keys https://github.com/xnuinside/simple-ddl-parser/issues/148\n\n### New Features\n\n1. Support for non-numeric column type parameters https://github.com/xnuinside/simple-ddl-parser/issues/171\nIt shows in column attribute 'type_parameters'.\n\n\n**v0.28.1**\nImprovements:\n1. Lines started with INSERT INTO statement now successfully ignored by parser (so you can keep them in ddl - they will be just skipped)\n\nFixes:\n1. Important fix for multiline comments\n\n\n**v0.28.0**\n\nImportant Changes (Pay attention):\n1. Because of parsing now AUTO_INCREMENT as a separate property of column previous output changed.\nPreviously it was parsed as a part of type like:  'INT AUTO_INCREMENT'.\nNow type will be only 'INT', but in column property you will see 'autoincrement': True.\n\nAmazing innovation:\n1. It's is weird to write in Changelog, but only in version 0.28.0 I recognize that floats that not supported by parser \u0026 it was fixed.\nThanks for the sample in the issue: https://github.com/xnuinside/simple-ddl-parser/issues/163\n\nImprovements:\nMariaDB:\n1. Added support for MariaDB AUTO_INCREMENT (from ddl here - https://github.com/xnuinside/simple-ddl-parser/issues/144)\nIf column is Auto Incremented - it indicated as 'autoincrement': True in column definition\n\nCommon:\n1. Added parsing for multiline comments in DDL with `/* */` syntax.\n2. Comments from DDL now all placed in 'comments' keyword if you use `group_by_type=` arg in parser.\n3. Added argument 'parser_settings={}' (dict type) in method  parse_from_file() - this way you can pass any arguments that you want to DDLParser (\u0026 that supported by it)\nSo, if you want to set log_level=logging.WARNING for parser - just use it as:\nparse_from_file('path_to_file', parser_settings={'log_level': logging.WARNING}). For issue: https://github.com/xnuinside/simple-ddl-parser/issues/160\n\n**v0.27.0**\n\nFixes:\n1. Fixed parsing CHECKS with IN statement - https://github.com/xnuinside/simple-ddl-parser/issues/150\n2. @# symbols added to ID token - (partially) https://github.com/xnuinside/simple-ddl-parser/issues/146\n\n\nImprovements:\n\n1. Added support for '*' in size column (ORACLE dialect) - https://github.com/xnuinside/simple-ddl-parser/issues/151\n2. Added arg 'debug' to parser, works same way as 'silent' - to get more clear error output.\n\nNew features:\n1. Added support for ORACLE 'ORGANIZATION INDEX'\n2. Added support for SparkSQL Partition by with procedure call - https://github.com/xnuinside/simple-ddl-parser/issues/154\n3. Added support for DEFAULT CHARSET statement MySQL - https://github.com/xnuinside/simple-ddl-parser/issues/153\n\n\n**v0.26.5**\n\nFixes:\n1. Parsetab included in builds.\n2. Added additional argumen log_file='path_to_file', to enable logging to file with providen name.\n\n**v0.26.4**\n\n1. Bugfix for (support CREATE OR REPLACE with additional keys like transient/temporary): https://github.com/xnuinside/simple-ddl-parser/issues/133\n\n**v0.26.3**\n\nImprovements:\n1. Added support for OR REPLACE in CREATE TABLE: https://github.com/xnuinside/simple-ddl-parser/issues/131\n2. Added support for AUTO INCREMENT in column:https://github.com/xnuinside/simple-ddl-parser/issues/130\n\n\n**v0.26.2**\n\nFixes:\n1. Fixed a huge bug for incorrect parsing lines with 'USE' \u0026 'GO' strings inside.\n2. Fixed parsing for CREATE SCHEMA for Snowlake \u0026 Oracle DDLs\n\nImprovements:\n1. Added  COMMENT statement for CREATE TABLE ddl (for SNOWFLAKE dialect support)\n2. Added  COMMENT statement for CREATE SCHEMA ddl (for SNOWFLAKE dialect support)\n\n\n**v0.26.1**\n\nFixes:\n\n1. support Multiple SERDEPROPERTIES  - https://github.com/xnuinside/simple-ddl-parser/issues/126\n2. Fix for issue with LOCATION and TBLPROPERTIES clauses in CREATE TABLE LIKE - https://github.com/xnuinside/simple-ddl-parser/issues/125\n3. LOCATION now works correctly with double quote strings\n\n\n**v0.26.0**\nImprovements:\n\n1. Added more explicit debug message on Statement errors - https://github.com/xnuinside/simple-ddl-parser/issues/116\n2. Added support for \"USING INDEX TABLESPACE\" statement in ALTER - https://github.com/xnuinside/simple-ddl-parser/issues/119\n3. Added support for IN statements in CHECKS - https://github.com/xnuinside/simple-ddl-parser/issues/121\n\nNew features:\n1. Support SparkSQL USING - https://github.com/xnuinside/simple-ddl-parser/issues/117\nUpdates initiated by ticket https://github.com/xnuinside/simple-ddl-parser/issues/120:\n2. In Parser you can use argument json_dump=True in method .run() if you want get result in JSON format.\n- README updated\n\nFixes:\n1. Added support for PARTITION BY one column without type\n2. Alter table add constraint PRIMARY KEY - https://github.com/xnuinside/simple-ddl-parser/issues/119\n3. Fix for paring SET statement - https://github.com/xnuinside/simple-ddl-parser/pull/122\n4. Fix for disappeared columns without properties - https://github.com/xnuinside/simple-ddl-parser/issues/123\n\n**v0.25.0**\n## Fixes:\n\n1. Fix for issue with 'at time zone' https://github.com/xnuinside/simple-ddl-parser/issues/112\n\n## New features:\n\n1. Added flag to raise errors if parser cannot parse statement DDLParser(.., silent=False) - https://github.com/xnuinside/simple-ddl-parser/issues/109\n2. Added flag to DDLParser(.., normalize_names=True) that change output of parser:\nif flag is True (default 'False') then all identifiers will be returned without '[', '\"' and other delimiters that used in different SQL dialects to separate custom names from reserved words \u0026 statements.\nFor example, if flag set 'True' and you pass this input:\n\nCREATE TABLE [dbo].[TO_Requests](\n    [Request_ID] [int] IDENTITY(1,1) NOT NULL,\n    [user_id] [int]\n\nIn output you will have names like 'dbo' and 'TO_Requests', not '[dbo]' and '[TO_Requests]'.\n\n**v0.24.2**\n## Fixes:\n\n1. Fix for the issue: https://github.com/xnuinside/simple-ddl-parser/issues/108 (reserved words can be used as table name after '.')\n\n\n**v0.24.1**\n\n## Fixes:\n### HQL:\n1. fields_terminated_by now parses , as \"','\", not as '' previously\n\n### Common:\n1. To output added 'if_not_exists' field in result to get availability 1-to-1 re-create ddl by metadata.\n\n**v0.24.0**\n\n## Fixes:\n\n### HQL:\n\n1. More then 2 tblproperties now are parsed correctly https://github.com/xnuinside/simple-ddl-parser/pull/104\n\n\n### Common:\n\n2. 'set' in lower case now also parsed validly.\n3. Now names like 'schema', 'database', 'table' can be used as names in CREATE DATABASE | SCHEMA | TABLESPACE | DOMAIN | TYPE statements and after INDEX and CONSTRAINT.\n4. Creation of empty tables also parsed correctly (like CREATE Table table;).\n\n## New Statements Support:\n\n### HQL:\n1. Added support for CLUSTERED BY - https://github.com/xnuinside/simple-ddl-parser/issues/103\n2. Added support for  INTO ... BUCKETS\n3. CREATE REMOTE DATABASE | SCHEMA\n\n**v0.23.0**\n\nBig refactoring: less code complexity \u0026 increase code coverage. Radon added to pre-commit hooks.\n\n### Fixes:\n\n1. Fix for issue with ALTER UNIQUE - https://github.com/xnuinside/simple-ddl-parser/issues/101\n\n### New Features\n\n1. SQL Comments string from DDL now parsed to \"comments\" key in output.\n\nPostgreSQL:\n\n2. Added support for ALTER TABLE ONLY | ALTER TABLE IF EXISTS\n\n**v0.22.5**\n### Fixes:\n\n1. Fix for issue with '\u003c' - https://github.com/xnuinside/simple-ddl-parser/issues/89\n\n\n**v0.22.4**\n### Fixes:\n\n### BigQuery:\n1. Fixed issue with parsing schemas with project in name.\n2. Added support for multiple OPTION() statements\n\n**v0.22.3**\n### Fixes:\n\n### BigQuery:\n1. CREATE TABLE statement with 'project_id' in format like project.dataset.table_name now is parsed validly.\n'project' added to output.\nAlso added support project.dataset.name format in CREATE SCHEMA and ALTER statement\n\n\n**v0.22.2**\n### Fixes:\n1. Fix for the issue: https://github.com/xnuinside/simple-ddl-parser/issues/94 (column name starts with CREATE)\n\n**v0.22.1**\n### New Features:\n\n## BigQuery:\n\n1. Added support for OPTION for full CREATE TABLE statement \u0026 column definition\n\n## Improvements:\n1. CLUSTED BY can be used without ()\n","funding_links":["https://ko-fi.com/xnuinside"],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxnuinside%2Fsimple-ddl-parser","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fxnuinside%2Fsimple-ddl-parser","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxnuinside%2Fsimple-ddl-parser/lists"}