{"id":15024187,"url":"https://github.com/shinichi-takii/ddlparse","last_synced_at":"2025-09-07T20:35:39.533Z","repository":{"id":47723432,"uuid":"116369133","full_name":"shinichi-takii/ddlparse","owner":"shinichi-takii","description":"DDL parase and Convert to BigQuery JSON schema and DDL statements","archived":false,"fork":false,"pushed_at":"2023-10-23T14:56:03.000Z","size":91,"stargazers_count":88,"open_issues_count":14,"forks_count":29,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-03-30T16:12:56.124Z","etag":null,"topics":["bigquery","ddl-parse","ddl-parser","maria","mysql","oracle","postgresql","python","redshift","sql"],"latest_commit_sha":null,"homepage":"https://pypi.org/project/ddlparse/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/shinichi-takii.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-01-05T09:46:29.000Z","updated_at":"2025-03-13T04:57:26.000Z","dependencies_parsed_at":"2024-06-18T17:32:07.385Z","dependency_job_id":"99b255e9-dd5c-41d7-ad11-78efe08914c1","html_url":"https://github.com/shinichi-takii/ddlparse","commit_stats":null,"previous_names":[],"tags_count":22,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shinichi-takii%2Fddlparse","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shinichi-takii%2Fddlparse/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shinichi-takii%2Fddlparse/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shinichi-takii%2Fddlparse/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/shinichi-takii","download_url":"https://codeload.github.com/shinichi-takii/ddlparse/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247526768,"owners_count":20953143,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bigquery","ddl-parse","ddl-parser","maria","mysql","oracle","postgresql","python","redshift","sql"],"created_at":"2024-09-24T19:59:55.001Z","updated_at":"2025-04-06T18:17:14.954Z","avatar_url":"https://github.com/shinichi-takii.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# DDL Parse\n\n[![PyPI version](https://img.shields.io/pypi/v/ddlparse.svg)](https://pypi.org/project/ddlparse/)\n[![Python version](https://img.shields.io/pypi/pyversions/ddlparse.svg)](https://pypi.org/project/ddlparse/)\n[![Travis CI Build Status](https://travis-ci.com/shinichi-takii/ddlparse.svg?branch=master)](https://travis-ci.com/shinichi-takii/ddlparse)\n[![Coveralls Coverage Status](https://coveralls.io/repos/github/shinichi-takii/ddlparse/badge.svg?branch=master)](https://coveralls.io/github/shinichi-takii/ddlparse?branch=master)\n[![codecov Coverage Status](https://codecov.io/gh/shinichi-takii/ddlparse/branch/master/graph/badge.svg)](https://codecov.io/gh/shinichi-takii/ddlparse)\n[![Requirements Status](https://requires.io/github/shinichi-takii/ddlparse/requirements.svg?branch=master)](https://requires.io/github/shinichi-takii/ddlparse/requirements/?branch=master)\n[![License](https://img.shields.io/badge/License-BSD%203--Clause-blue.svg)](https://github.com/shinichi-takii/ddlparse/blob/master/LICENSE.md)\n\n*DDL parase and Convert to BigQuery JSON schema and DDL statements module, available in Python.*\n\n----\n\n## Features\n\n- DDL parse and get table schema information.\n- Currently, only the `CREATE TABLE` statement is supported.\n- Convert to [BigQuery JSON schema](https://cloud.google.com/bigquery/docs/schemas#creating_a_json_schema_file) and [BigQuery DDL statements](https://cloud.google.com/bigquery/docs/reference/standard-sql/data-definition-language).\n- Supported databases are MySQL/MariaDB, PostgreSQL, Oracle, Redshift.\n\n## Requirement\n\n1. Python \u003e= 3.5\n1. [pyparsing](https://github.com/pyparsing/pyparsing)\n\n## Installation\n\n### Install\n\npip install:\n```bash\n$ pip install ddlparse\n```\n\ncommand install:\n```bash\n$ python setup.py install\n```\n\n### Update\n\npip update:\n```bash\n$ pip install ddlparse --upgrade\n```\n\n## Usage\n\n### Example\n\n```python\nimport json\n\nfrom ddlparse import DdlParse\n\nsample_ddl = \"\"\"\nCREATE TABLE My_Schema.Sample_Table (\n  Id integer PRIMARY KEY COMMENT 'User ID',\n  Name varchar(100) NOT NULL COMMENT 'User name',\n  Total bigint NOT NULL,\n  Avg decimal(5,1) NOT NULL,\n  Point int(10) unsigned,\n  Zerofill_Id integer unsigned zerofill NOT NULL,\n  Created_At date, -- Oracle 'DATE' -\u003e BigQuery 'DATETIME'\n  UNIQUE (NAME)\n);\n\"\"\"\n\n\n# parse pattern (1-1)\ntable = DdlParse().parse(sample_ddl)\n\n# parse pattern (1-2) : Specify source database\ntable = DdlParse().parse(ddl=sample_ddl, source_database=DdlParse.DATABASE.oracle)\n\n\n# parse pattern (2-1)\nparser = DdlParse(sample_ddl)\ntable = parser.parse()\n\nprint(\"* BigQuery Fields * : normal\")\nprint(table.to_bigquery_fields())\n\n\n# parse pattern (2-2) : Specify source database\nparser = DdlParse(ddl=sample_ddl, source_database=DdlParse.DATABASE.oracle)\ntable = parser.parse()\n\n\n# parse pattern (3-1)\nparser = DdlParse()\nparser.ddl = sample_ddl\ntable = parser.parse()\n\n# parse pattern (3-2) : Specify source database\nparser = DdlParse()\nparser.source_database = DdlParse.DATABASE.oracle\nparser.ddl = sample_ddl\ntable = parser.parse()\n\nprint(\"* BigQuery Fields * : Oracle\")\nprint(table.to_bigquery_fields())\n\n\nprint(\"* TABLE *\")\nprint(\"schema = {} : name = {} : is_temp = {}\".format(table.schema, table.name, table.is_temp))\n\nprint(\"* BigQuery Fields *\")\nprint(table.to_bigquery_fields())\n\nprint(\"* BigQuery Fields - column name to lower case / upper case *\")\nprint(table.to_bigquery_fields(DdlParse.NAME_CASE.lower))\nprint(table.to_bigquery_fields(DdlParse.NAME_CASE.upper))\n\nprint(\"* COLUMN *\")\nfor col in table.columns.values():\n    col_info = {}\n\n    col_info[\"name\"]                  = col.name\n    col_info[\"data_type\"]             = col.data_type\n    col_info[\"length\"]                = col.length\n    col_info[\"precision(=length)\"]    = col.precision\n    col_info[\"scale\"]                 = col.scale\n    col_info[\"is_unsigned\"]           = col.is_unsigned\n    col_info[\"is_zerofill\"]           = col.is_zerofill\n    col_info[\"constraint\"]            = col.constraint\n    col_info[\"not_null\"]              = col.not_null\n    col_info[\"PK\"]                    = col.primary_key\n    col_info[\"unique\"]                = col.unique\n    col_info[\"auto_increment\"]        = col.auto_increment\n    col_info[\"distkey\"]               = col.distkey\n    col_info[\"sortkey\"]               = col.sortkey\n    col_info[\"encode\"]                = col.encode\n    col_info[\"default\"]               = col.default\n    col_info[\"character_set\"]         = col.character_set\n    col_info[\"bq_legacy_data_type\"]   = col.bigquery_legacy_data_type\n    col_info[\"bq_standard_data_type\"] = col.bigquery_standard_data_type\n    col_info[\"comment\"]               = col.comment\n    col_info[\"description(=comment)\"] = col.description\n    col_info[\"bigquery_field\"]        = json.loads(col.to_bigquery_field())\n\n    print(json.dumps(col_info, indent=2, ensure_ascii=False))\n\nprint(\"* DDL (CREATE TABLE) statements *\")\nprint(table.to_bigquery_ddl())\n\nprint(\"* DDL (CREATE TABLE) statements - dataset name, table name and column name to lower case / upper case *\")\nprint(table.to_bigquery_ddl(DdlParse.NAME_CASE.lower))\nprint(table.to_bigquery_ddl(DdlParse.NAME_CASE.upper))\n\nprint(\"* Get Column object (case insensitive) *\")\nprint(table.columns[\"total\"])\nprint(table.columns[\"total\"].data_type)\n```\n\n## License\n\n[BSD 3-Clause License](https://github.com/shinichi-takii/ddlparse/blob/master/LICENSE.md)\n\n## Author\n\nShinichi Takii \u003cshinichi.takii@shaketh.com\u003e\n\n## Links\n\n- Repository : https://github.com/shinichi-takii/ddlparse\n- PyPI Package : https://pypi.org/project/ddlparse/\n\n## Special Thanks\n\n- pyparsing : https://github.com/pyparsing/pyparsing\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshinichi-takii%2Fddlparse","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fshinichi-takii%2Fddlparse","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshinichi-takii%2Fddlparse/lists"}