{"id":25180974,"url":"https://github.com/vmagamedov/sqlconstruct","last_synced_at":"2025-05-07T06:47:12.817Z","repository":{"id":10820370,"uuid":"13096322","full_name":"vmagamedov/sqlconstruct","owner":"vmagamedov","description":"Functional approach to query database using SQLAlchemy","archived":false,"fork":false,"pushed_at":"2020-05-12T12:36:21.000Z","size":89,"stargazers_count":22,"open_issues_count":1,"forks_count":9,"subscribers_count":9,"default_branch":"master","last_synced_at":"2025-04-12T10:18:14.844Z","etag":null,"topics":["functional","performance","python","sql","sqlalchemy"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vmagamedov.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2013-09-25T14:39:26.000Z","updated_at":"2023-12-23T23:31:33.000Z","dependencies_parsed_at":"2022-08-29T11:20:19.374Z","dependency_job_id":null,"html_url":"https://github.com/vmagamedov/sqlconstruct","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vmagamedov%2Fsqlconstruct","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vmagamedov%2Fsqlconstruct/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vmagamedov%2Fsqlconstruct/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vmagamedov%2Fsqlconstruct/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vmagamedov","download_url":"https://codeload.github.com/vmagamedov/sqlconstruct/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252831178,"owners_count":21810779,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["functional","performance","python","sql","sqlalchemy"],"created_at":"2025-02-09T16:38:15.375Z","updated_at":"2025-05-07T06:47:12.802Z","avatar_url":"https://github.com/vmagamedov.png","language":"Python","readme":"============\nSQLConstruct\n============\n\n`SQLConstruct` is a functional approach to query database using `SQLAlchemy`\nlibrary. It was written to reach more speed without introducing unmaintainable\nand verbose code. On the contrary, code becomes simpler, so there are less\nchances of shooting yourself in the foot.\n\nMain problems it aims to solve:\n\n- ORM overhead in read-only ``SELECT`` queries;\n- Network traffic when loading unnecessary columns;\n- Code complexity;\n- N+1 problem.\n\nFinal\n=====\n\nYou describe what you want to get from the database:\n\n.. code-block:: python\n\n    from sqlconstruct import Construct, if_\n\n    product_struct = Construct({\n        'name': Product.name,\n        'url': url_for_product.defn(Product),\n        'image_url': if_(\n            Image.id,\n            then_=url_for_image.defn(Image, 100, 100),\n            else_=None,\n        ),\n    })\n\nAnd you get it. `SQLConstruct` knows which columns you need and how transform\nthem into suitable to use format:\n\n.. code-block:: python\n\n    \u003e\u003e\u003e product = (\n    ...     session.query(product_struct)\n    ...     .outerjoin(Product.image)\n    ...     .first()\n    ... )\n    ...\n    \u003e\u003e\u003e product.name\n    'Foo product'\n    \u003e\u003e\u003e product.url\n    '/p1-foo-product.html'\n    \u003e\u003e\u003e product.image_url\n    '//images.example.st/123-100x100-foo.jpg'\n\nFull story\n==========\n\nBasic preparations:\n\n.. code-block:: python\n\n    from sqlalchemy import create_engine\n    from sqlalchemy import Column, Integer, String, Text, ForeignKey\n    from sqlalchemy.orm import Session, relationship, eagerload\n    from sqlalchemy.ext.declarative import declarative_base\n\n    engine = create_engine('sqlite://')\n    Base = declarative_base()\n\n    class Image(Base):\n        __tablename__ = 'image'\n\n        id = Column(Integer, primary_key=True)\n        name = Column(String)\n\n    class Product(Base):\n        __tablename__ = 'product'\n\n        id = Column(Integer, primary_key=True)\n        name = Column(String)\n        image_id = Column(Integer, ForeignKey(Image.id))\n        description = Column(Text)\n\n        image = relationship(Image)\n\n    Base.metadata.create_all(engine)\n\n    session = Session(engine)\n    session.add(Product(name='Foo product', image=Image(name='Foo.jpg')))\n    session.commit()\n\n    def slugify(name):\n        # very dumb implementation, just for an example\n        return name.lower().replace(' ', '-')\n\n    def url_for_product(product):\n        return '/p{id}-{name}.html'.format(\n            id=product.id,\n            name=slugify(product.name),\n        )\n\n    def url_for_image(image, width, height):\n        return '//images.example.st/{id}-{width}x{height}-{name}'.format(\n            id=image.id,\n            width=width,\n            height=height,\n            name=slugify(image.name),\n        )\n\nUsual way:\n\n.. code-block:: python\n\n    \u003e\u003e\u003e product = (\n    ...     session.query(Product)\n    ...     .options(eagerload(Product.image))\n    ...     .first()\n    ... )\n    ...\n    \u003e\u003e\u003e product.name\n    u'Foo product'\n    \u003e\u003e\u003e url_for_product(product)\n    '/p1-foo-product.html'\n    \u003e\u003e\u003e url_for_image(product.image, 100, 100) if product.image else None\n    '//images.example.st/1-100x100-foo.jpg'\n\nDisadvantages:\n\n- ``description`` column isn't deferred, it will be loaded every time;\n- if you will mark ``description`` column as deferred, this can introduce N+1\n  problem somewhere else in your project;\n- if you forgot to ``eagerload`` ``Product.image`` you will also get N+1\n  problem;\n- you have to pass model instances as arguments everywhere in the project and\n  this tends to code complexity, because you don't know how they will be used in\n  the future;\n- model instances creation isn't cheap, CPU time grows with number of columns,\n  even if they are all deferred.\n\nInitial solution:\n\n.. code-block:: python\n\n    from sqlconstruct import Construct, apply_, if_\n\n    def url_for_product(product_id, product_name):\n        return '/p{id}-{name}.html'.format(\n            id=product_id,\n            name=slugify(product_name),\n        )\n\n    def url_for_image(image_id, image_name, width, height):\n        return '//images.example.st/{id}-{width}x{height}-{name}'.format(\n            id=image_id,\n            width=width,\n            height=height,\n            name=slugify(image_name),\n        )\n\n    product_struct = Construct({\n        'name': Product.name,\n        'url': apply_(url_for_product, args=[Product.id, Product.name]),\n        'image_url': if_(\n            Image.id,\n            then_=apply_(url_for_image, args=[Image.id, Image.name, 100, 100]),\n            else_=None,\n        ),\n    })\n\nUsage:\n\n.. code-block:: python\n\n    \u003e\u003e\u003e product = (\n    ...     session.query(product_struct)\n    ...     .outerjoin(Product.image)\n    ...     .first()\n    ... )\n    ...\n    \u003e\u003e\u003e product.name\n    u'Foo product'\n    \u003e\u003e\u003e product.url\n    '/p1-foo-product.html'\n    \u003e\u003e\u003e product.image_url\n    '//images.example.st/1-100x100-foo.jpg'\n\nAdvantages:\n\n- you're loading only what you need, no extra network traffic, no need to\n  defer/undefer columns;\n- ``url_for_product`` and ``url_for_image`` functions can't add complexity,\n  because they are forced to define all needed columns as arguments;\n- you're working with precomputed values (urls in this example).\n\nDisadvantages:\n\n- code of functions is hard to refactor and reuse, because you should specify or\n  pass all the arguments every time;\n- you should be careful with joins, because if you wouldn't specify them\n  explicitly, `SQLAlchemy` will produce cartesian product of the tables\n  (``SELECT ... FROM product, image WHERE ...``), which will return wrong\n  results and hurt your performance.\n\nTo address first disadvantage, `SQLConstruct` provides ``define`` decorator,\nwhich gives you ability to define hybrid functions to use them in different\nways:\n\n.. code-block:: python\n\n    from sqlconstruct import define\n\n    @define\n    def url_for_product(product):\n        def body(product_id, product_name):\n            return '/p{id}-{name}.html'.format(\n                id=product_id,\n                name=slugify(product_name),\n            )\n        return body, [product.id, product.name]\n\n    @define\n    def url_for_image(image, width, height):\n        def body(image_id, image_name, width, height):\n            return '//images.example.st/{id}-{width}x{height}-{name}'.format(\n                id=image_id,\n                width=width,\n                height=height,\n                name=slugify(image_name),\n            )\n        return body, [image.id, image.name, width, height]\n\nNow these functions can be used in these ways:\n\n.. code-block:: python\n\n    \u003e\u003e\u003e product = session.query(Product).first()\n    \u003e\u003e\u003e url_for_product(product)  # objective style\n    '/p1-foo-product.html'\n    \u003e\u003e\u003e url_for_product.defn(Product)  # apply_ declaration\n    \u003csqlconstruct.apply_ at 0x000000000\u003e\n    \u003e\u003e\u003e url_for_product.func(product.id, product.name)  # functional style\n    '/p1-foo-product.html'\n\nModified final ``Construct`` definition:\n\n.. code-block:: python\n\n    product_struct = Construct({\n        'name': Product.name,\n        'url': url_for_product.defn(Product),\n        'image_url': if_(\n            Image.id,\n            then_=url_for_image.defn(Image, 100, 100),\n            else_=None,\n        ),\n    })\n\nInstallation\n============\n\nTo install `SQLConstruct`, simply:\n\n.. code-block:: shell\n\n    $ pip install sqlconstruct\n\nTested `Python` versions: 2.7, 3.4, 3.8.\n\nTested `SQLAlchemy` versions: 1.0, 1.1, 1.2, 1.3.\n\nExamples above are using `SQLAlchemy` \u003e= 0.9, if you are using older versions,\nyou will have to do next changes in your project configuration:\n\n.. code-block:: python\n\n    from sqlconstruct import QueryMixin\n    from sqlalchemy.orm.query import Query as BaseQuery\n\n    class Query(QueryMixin, BaseQuery):\n        pass\n\n    session = Session(engine, query_cls=Query)\n\nFlask-SQLAlchemy:\n\n.. code-block:: python\n\n    from flask.ext.sqlalchemy import SQLAlchemy\n\n    db = SQLAlchemy(app, session_options={'query_cls': Query})\n\nor\n\n.. code-block:: python\n\n    db = SQLAlchemy(session_options={'query_cls': Query})\n    db.init_app(app)\n\nLicense\n=======\n\n`SQLConstruct` is distributed under the BSD license. See LICENSE.txt for more\ndetails.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvmagamedov%2Fsqlconstruct","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvmagamedov%2Fsqlconstruct","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvmagamedov%2Fsqlconstruct/lists"}