{"id":15908024,"url":"https://github.com/bytehouse-cloud/bytehouse-sqlalchemy","last_synced_at":"2025-08-14T18:32:51.459Z","repository":{"id":66502368,"uuid":"603977820","full_name":"bytehouse-cloud/bytehouse-sqlalchemy","owner":"bytehouse-cloud","description":"ByteHouse SQLAlchemy Connector","archived":false,"fork":false,"pushed_at":"2023-02-20T04:23:31.000Z","size":91,"stargazers_count":1,"open_issues_count":0,"forks_count":2,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-11-28T17:42:31.832Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bytehouse-cloud.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-02-20T04:15:56.000Z","updated_at":"2024-11-19T13:02:52.000Z","dependencies_parsed_at":null,"dependency_job_id":"23e4b11f-778f-4bb3-bb21-f9bc1c558e1a","html_url":"https://github.com/bytehouse-cloud/bytehouse-sqlalchemy","commit_stats":{"total_commits":3,"total_committers":2,"mean_commits":1.5,"dds":"0.33333333333333337","last_synced_commit":"e815aa00c9e60f793e2c42fa21f77c128830fda4"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bytehouse-cloud%2Fbytehouse-sqlalchemy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bytehouse-cloud%2Fbytehouse-sqlalchemy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bytehouse-cloud%2Fbytehouse-sqlalchemy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bytehouse-cloud%2Fbytehouse-sqlalchemy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bytehouse-cloud","download_url":"https://codeload.github.com/bytehouse-cloud/bytehouse-sqlalchemy/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":229857649,"owners_count":18135083,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-06T14:08:54.417Z","updated_at":"2024-12-15T18:24:32.181Z","avatar_url":"https://github.com/bytehouse-cloud.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ByteHouse SQLAlchemy Connector\n## Introduction\nByteHouse SQLAlchemy connector is ByteHouse dialect for `SQLAlchemy`, which is the Python SQL toolkit and object \nrelational mapper enabling application developers the full power and flexibility of SQL. The connector is built on \ntop of ByteHouse python driver which follows Python DB API 2.0 specification. The connector supports both SQLAlchemy \nCore and SQLAlchemy ORM APIs. \n## Requirements\nPython v3.6 or higher\n## Installation from PyPI\nThe latest release version can be installed from here:\n```commandline\npip install bytehouse-sqlalchemy\n```\n## Installation from github\nThe current development version can be installed from here:\n```commandline\npip install git+https://github.com/bytehouse-cloud/bytehouse-sqlalchemy@master#egg=bytehouse-driver\n```\n## Creating ByteHouse Account\nYou need to create a ByteHouse account in order to use Python Driver. You can simply create a free account with \nthe process mentioned in our official website documentation: https://docs.bytehouse.cloud/en/docs/quick-start \u003cbr/\u003e\n\nYou can also create ByteHouse account through Volcano Engine by ByteDance: https://www.volcengine.com/product/bytehouse-cloud\n## SQLAlchemy APIs\nSQLAlchemy has two distinct APIs, one building on top of the other. These APIs are `Core` and `ORM`. \n### SQLAlchemy Core\nSQLAlchemy Core APIs manage connectivity to a database, interact with database queries and results \u0026 manage programmatic\nconstruction of SQL statements. \n#### Engine Configuration\n`Engine` is the starting point of any SQLAlchemy application. `Engine` refers to a `Dialect` and `Connection Pool`, where \n`Dialect` is a `Python` object that represents information and methods that allow database operations to proceed on a \nparticular kind of database backend and a particular kind of Python driver for that database.  \u003cbr/\u003e\u003cbr/\u003e\nThe engine and its underlying connection pool do not establish the first actual DBAPI connection until the \n`Engine.connect()` method is called, or an operation which is dependent on this method such as `Engine.execute()` is \ninvoked. In this way, `Engine` and `Pool` can be said to have lazy initialization behaviour.\n##### ByteHouse Regions\nCurrently, the driver supports the following region names across different cloud providers. Alternatively, if you know\nthe host address of ByteHouse server, you can directly use host address \u0026 omit region name. \n\u003ctable\u003e\n    \u003ctr\u003e\n        \u003ctd\u003eRegion Name\u003c/td\u003e\n        \u003ctd\u003eTarget Server\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n        \u003ctd\u003eAP-SOUTHEAST-1\u003c/td\u003e\n        \u003ctd\u003egateway.aws-ap-southeast-1.bytehouse.cloud:19000\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n        \u003ctd\u003eVOLCANO-CN-NORTH-1\u003c/td\u003e\n        \u003ctd\u003ebytehouse-cn-beijing.volces.com:19000\u003c/td\u003e\n    \u003c/tr\u003e\n\u003c/table\u003e\n\n##### Construction from ByteHouse URI\n###### Region \u0026 Password Format\n*Required parameters:* `region` `account` `user` `password`\n```python\nfrom sqlalchemy import create_engine\n\nengine = create_engine(\"bytehouse:///?region={}\u0026account={}\u0026user={}\u0026password={}\u0026database={}\".\n    format($REGION, $ACCOUNT, $USER, $PASSWORD, $DATABASE))\n```\n###### Host Address \u0026 Password Format\n*Required parameters:* `host` `port` `account` `user` `password`\n```python\nfrom sqlalchemy import create_engine\n\nengine = create_engine(\"bytehouse://{}:{}/?account={}\u0026user={}\u0026password={}\u0026database={}\".\n    format($HOST, $PORT, $ACCOUNT, $USER, $PASSWORD, $DATABASE))\n```\n\u003e For API Key authentication, user is always 'bytehouse'\n###### Region \u0026 API Key Format\n*Required parameters:* `region` `password`\n```python\nfrom sqlalchemy import create_engine\n\nengine = create_engine(\"bytehouse:///?region={}\u0026user=bytehouse\u0026password={}\u0026database={}\".\n    format($REGION, $API_KEY, $DATABASE))\n```\n###### Host Address \u0026 API Key Format\n*Required parameters:* `host` `port` `password`\n```python\nfrom sqlalchemy import create_engine\n\nengine = create_engine(\"bytehouse://{}:{}/?user=bytehouse\u0026password={}\u0026database={}\".\n    format($HOST, $PORT, $API_KEY, $DATABASE))\n```\n##### Programmatic Construction\n```python\nfrom sqlalchemy.engine import URL\n\nuri = URL.create(\n    \"bytehouse\",\n    username=\"bytehouse\",\n    password=\"{}\".format($API_KEY),\n    host=\"{}\".format($HOST),\n    port=\"{}\".format($PORT),\n    database=\"{}\".format($DATABASE),\n)\nengine = create_engine(uri)\n```\n#### Working with Connections\nThe most basic function of the `Engine` is to provide access to a `Connection`, which can execute SQL statements. \nTo execute a textual statement to the database looks like:\n```python\nfrom sqlalchemy import text\n\nwith engine.connect() as connection:\n    result = connection.execute(text(\"SELECT 1\"))\n    for row in result:\n        print(row[0])\n```\nThe object returned here is known as the `CursorResult`, which refers to a DBAPI cursor. The DBAPI cursor will be \nclosed by the `CursorResult` when all of its result rows are exhausted. When the `Connection` is closed at the end of \nthe with block, the referenced DBAPI connection is released to the connection pool. \n#### Working with Transactions\nThe `Connection` object provides a `Connection.begin()` method which returns a `Transaction` object. The transaction is \ncommitted when the block completes. If an exception is raised, the transaction would be rolled back, and the exception \nwould be propagated outwards. \n```python\nwith engine.connect() as connection:\n    with connection.begin():\n        connection.execute(user_table.insert(), {\"user_id\": 7, \"user_name\": \"Jane\"})\n        connection.execute(user_table.insert(), {\"user_id\": 8, \"user_name\": \"Adam\"})\n```\n#### Database MetaData\n`MetaData` is a container object that keeps together different entities or features of a database. \n```python\nfrom sqlalchemy import MetaData\n\nmetadata_obj = MetaData()\n```\n#### Table Definition : Constructor Style\n`Table` class would represent a table where two primary arguments are the table name and the `MetaData` object which \nit will be associated with. The remaining positional arguments are Column objects describing each `Column` and engine\n(`CnchMergeTree`) definition.\n```python\nfrom sqlalchemy import Table, Column, Integer, String, func\nfrom bytehouse_sqlalchemy import engines\n\nuser_table = Table(  \n    \"user\",  \n    metadata,  \n    Column(\"user_id\", Integer, primary_key=True),  \n    Column(\"user_name\", String(16), nullable=False),  \n    engines.CnchMergeTree(  \n        order_by=func.tuple()  \n    )  \n)\n```\n#### Creating and Dropping Tables\nThe general way of creating all tables is to execute the `create_all()` method on the `MetaData` object. This method will \nfirst check the existence of each individual table, and if not found, then execute `CREATE` statements for all tables. \nSimilarly, for dropping all tables, we can execute the `drop_all()` method on the MetaData object. Creating and dropping\nindividual tables can be done via the `create()` and `drop()` methods. \n```python\nfrom sqlalchemy import Table, Column, Integer, String, func, MetaData\nfrom bytehouse_sqlalchemy import engines\n\nmetadata_obj = MetaData()\nuser_table = Table(  \n    \"user\",  \n    metadata,  \n    Column(\"user_id\", Integer, primary_key=True),  \n    Column(\"user_name\", String(16), nullable=False),  \n    engines.CnchMergeTree(  \n        order_by=func.tuple()  \n    )  \n)\nmetadata_obj.create_all(engine)\nmetadata_obj.drop_all(engine)\n```\n#### Insertion and Selection\n`Table.insert()` can be used to insert rows into the table, whereas `Table.select()` would fetch the result rows from the server.\n```python\nwith engine.connect() as connection:\n    connection.execute(user_table.insert(), {\"user_id\": 7, \"user_name\": \"Jane\"})\n    result_set = connection.execute(user_table.select())\n```\n### SQLAlchemy ORM\nSQLAlchemy ORM is built on top of SQLAlchemy Core which provides object relational mapping capabilities that allows \nusers to define Python classes mapped to database tables. It extends the Core SQL expression language to allow \nSQL queries to be composed and invoked in user defined objects. \n#### Create Engine\nThe `Engine` is a factory class that will create and maintain database connections for us, where connections are held \ninside of a Connection Pool for fast reuse. The details regarding the `Engine` are described at the beginning of this doc. \n```python\nfrom sqlalchemy import create_engine\n\nengine = create_engine(\"bytehouse:///?region={}\u0026account={}\u0026user={}\u0026password={}\u0026database={}\".\n    format($REGION, $ACCOUNT, $USER, $PASSWORD, $DATABASE))\n```\n#### Declarative Mapping\nThe `Declarative Mapping` defines a base class using the `declarative_base()` function, which returns a new base class \nfrom which new classes to be mapped may inherit from. A mapped class typically refers to a single particular database \ntable, the name of which is indicated by using the `__tablename__` class level attribute.\n```python\nfrom sqlalchemy import Column, Integer, String, func, MetaData\nfrom sqlalchemy.orm import declarative_base\nfrom bytehouse_sqlalchemy import engines\n\nmetadata = MetaData(bind=engine)\nBase = declarative_base(metadata=metadata)\n\nclass User(Base):  \n    __tablename__ = \"user_account\"  \n    id = Column(Integer, primary_key=True)  \n    name = Column(String(30))  \n    fullname = Column(String)  \n  \n    __table_args__ = (  \n        engines.CnchMergeTree(  \n            order_by=func.tuple()  \n        ),  \n    )\n```\n#### Table Creation\nUsing table metadata and engine, we can generate DDL schema \u0026 execute in ByteHouse at once using \n`Table.__table__.create()` method. \n```python\nUser.__table__.create()\n```\n#### Session and Object Persist\nWe can create objects of previously defined classes and pass them to the database using an object called `Session`, \nwhich uses the `Engine` to interact with the database. The `Session.add_all()` is used to add multiple objects at once, \nand the `Session.commit()` method would flush any pending changes to the database.\n```python\nfrom sqlalchemy.orm import Session\n\nwith Session(engine) as session:\n        spongebob = User(  \n            id=1,  \n            name=\"spongebob\",  \n            fullname=\"Spongebob Squarepants\"  \n        )  \n        sandy = User(  \n            id=2,  \n            name=\"sandy\",  \n            fullname=\"Sandy Cheeks\"  \n        )\n        session.add_all([spongebob, sandy])  \n        session.commit()\n```\n#### SELECT statement\nWe can use the `select()` method to create a new `Select` object, which can then be invoked using a `Session` object. \nOptionally, we can also use the `Select.where()` method to filter out the results.\n```python\nfrom sqlalchemy import select\n\nsession = Session(engine)\n\nstmt = select(User).where(User.name.in_([\"spongebob\", \"sandy\"]))\nfor user in session.scalars(stmt):  \n    print(user.name)\n```\n## Local Development\nChange `setup.cfg` file to include your connection credentials. For running tests locally, follow these steps:\n```commandline\npython testsrequire.py \u0026\u0026 pip install .\npython -m pytest tests/\n```\n## License\nThis project is distributed under the terms of the MIT license: http://www.opensource.org/licenses/mit-license.php","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbytehouse-cloud%2Fbytehouse-sqlalchemy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbytehouse-cloud%2Fbytehouse-sqlalchemy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbytehouse-cloud%2Fbytehouse-sqlalchemy/lists"}