{"id":22903943,"url":"https://github.com/json2d/schematable","last_synced_at":"2025-04-01T07:56:51.210Z","repository":{"id":57464309,"uuid":"245552016","full_name":"json2d/schematable","owner":"json2d","description":"a library for working with SQL tables in Python","archived":false,"fork":false,"pushed_at":"2020-10-08T05:08:09.000Z","size":25,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-03-14T13:15:24.707Z","etag":null,"topics":["boilerplate-reduction","schematable-urls","sql","workflow"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/json2d.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-03-07T02:06:52.000Z","updated_at":"2021-06-10T04:21:08.000Z","dependencies_parsed_at":"2022-09-05T06:50:13.692Z","dependency_job_id":null,"html_url":"https://github.com/json2d/schematable","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/json2d%2Fschematable","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/json2d%2Fschematable/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/json2d%2Fschematable/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/json2d%2Fschematable/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/json2d","download_url":"https://codeload.github.com/json2d/schematable/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246604619,"owners_count":20804100,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["boilerplate-reduction","schematable-urls","sql","workflow"],"created_at":"2024-12-14T02:39:33.807Z","updated_at":"2025-04-01T07:56:51.187Z","avatar_url":"https://github.com/json2d.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![PyPI version](https://badge.fury.io/py/schematable.svg)](https://badge.fury.io/py/schematable)\n[![Build Status](https://travis-ci.com/json2d/schematable.svg?branch=master)](https://travis-ci.com/json2d/schematable) \n[![Coverage Status](https://coveralls.io/repos/github/json2d/schematable/badge.svg?branch=master)](https://coveralls.io/github/json2d/schematable?branch=master)\n\n# schematable\n\na Python utility library for working with SQL tables\n\n## Install\n\n```\npip install schematable\n```\n\n## Basic usage\n\nGo from zero to SQL query in seconds with near-zero boilerplate:\n\n```py\nimport schematable as st\n\nschdl = st.SchemaTable('schedule') # at minimum it needs a table name \n\nprint(schdl.db_url) # sqlite:///:memory:\nprint(schdl.engine) # \u003cclass 'sqlalchemy.engine.base.Engine'\u003e\n\ncreate_table_stmt = '''\n  create table schedule\n  (\n    start_time timestamp,\n    end_time timestamp,\n    event_name text,\n    id int not null\n      constraint schedule_pk\n        primary key\n  );\n'''\n\nschdl.engine.execute(create_table_stmt)\n\ninsert_records_stmt = '''\n  INSERT INTO schedule (start_time, end_time, event_name, id) VALUES ('1583531261000', '1583534865000', 'walk the doggy 🐶', 1);\n'''\n\ninsert_records_stmt_2 = '''\n  INSERT INTO schedule (start_time, end_time, event_name, id) VALUES ('1583708400000', '1583632800000', 'take a nap 😴', 2);\n'''\n\nschdl.engine.execute(insert_records_stmt)\nschdl.engine.execute(insert_records_stmt_2)\n\nrows = schdl.engine.execute('select event_name from schedule').fetchall()\n\nprint(rows) # ['walk the doggy 🐶', 'take a nap 😴']\n\n```\n\n## More advanced usage\n\nWith `sqlalchemy` under the hood you can connect to tables from all types of SQL databases:\n\n```py\nimport schematable as st\n\nschdl = st.SchemaTable(\n  db_url='postgres://user:password@localhost:5432/foo',\n  schema='bar',\n  table='schedule'\n)\n\n```\n\n`schematable` also integrates with `pandas` workflows like a charm, reducing the noise and friction involved with managing the arguments for their SQL related functions:\n\n```py\nimport pandas as pd\nimport datetime\n\ndf = pd.read_sql_table(schdl.table, schdl.engine, schld.schema) # neat - everything's in one place\n\ndf['event_name']\ndf['start_time'] \u003e datetime.datetime.now()\ndf['extracted_time'] = datetime.datetime.now()\n```\n\n## Cross database workflow\n\nWorking with multiple databases at the same time should be dead simple.\n\nEg. Lets say we wanted to create a new local test database on-the-fly with some data queried from our production database.\n\nHere's the `schematable` + `pandas` way:\n\n```py\nimport schematable as st\nimport pandas as pd\n\n# extract dataset from production db\n\nschdl = st.SchemaTable(\n  db_url='postgres://user:password@localhost:5432/foo',\n  schema='bar',\n  table='schedule'\n)\n\n# select everything from 2019\n\nselect_dataset_query = '''\n  select * from {schema_table} \n  where start_date between date('{from_date}') and date('{to_date}')\n'''.format(\n  schema_table=schdl.st, \n  from_date='2019-01-01',\n  to_date='2019-12-31'\n)\n\ndf = pd.read_sql(select_dataset_query, schdl.table, schdl.engine)\n\n# load dataset into (new) test db\n\ntest_schdl = st.SchemaTable(\n  db_url='sqlite:///db/test-2019.sqlite',\n  table='schedule'\n)\n\ndf.to_sql(test_schdl.table, test_schdl.engine, if_exists='replace', index=false)\n\n```\n\n## Extended URLs\n\nTake a standard db URL and append a schema and table to the end - you get what we call a  **schematable URL**, which can be parsed into a `SchemaTable` instance:\n\n```py\nfrom schematable import SchemaTable\n\nschdl = SchemaTable.parse('postgres://user:password@localhost:5432/foo#bar.schedule')\n\nprint(schdl.db_url) # postgres://user:password@localhost:5432/foo\nprint(schdl.schema) # bar\nprint(schdl.table) # schedule\n\n```\n\nThey're primarily handy for usecases involving serialization and deserialization of schematables:\n\n```env\n# staging.env\nSCHEDULE_SCHEMATABLE_URL=sqlite:///staging.db#bar.schedule\n```\n\n```py\nschdl = SchemaTable.parse(os.environ['SCHEDULE_SCHEMATABLE_URL'])\n\nprint(schdl.db_url) # sqlite:///staging.db\nprint(schdl.schema) # bar\nprint(schdl.table) # schedule\n```\n\nMore specifically, schematable URLs are composed of a `db_url`, `table`, and `schema` component where (just like the `SchemaTable` constructor) only the `table` component is required:\n\n```py\nSchemaTable.parse('schedule')\nSchemaTable.parse('bar.schedule')\nSchemaTable.parse('sqlite:///staging.db#schedule')\nSchemaTable.parse('postgres://user:password@localhost:5432/foo#bar.schedule')\n```\n\n## Boilerplate reduction\n\nHere are before-and-after snippets showing some code you won't have to write anymore using `schematable` to do your SQL things:\n\nEg. Here's checking if a table exists:\n\n#### before\n```py\nimport sqlalchemy as sa\n\nschdl_db_url = 'sqlite://'\nschdl_table = 'schedule'\nschdl_engine = sa.create_engine(schdl_db_url)\nif schdl_engine.has_table(schdl_table):\n  # do the thing\n```\n\n#### after\n```py\nimport schematable as st\n\nschl = st.parse('schedule')\nif schdl.engine.has_table(schdl.table, schdl.schema):\n  # do the thing\n```\n\n#### after next\n```py\nimport schematable as st\n\nschl = st.parse('schedule')\nif schdl.exists():\n  # do the thing\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjson2d%2Fschematable","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjson2d%2Fschematable","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjson2d%2Fschematable/lists"}