{"id":22281609,"url":"https://github.com/thanethomson/mlalchemy","last_synced_at":"2025-07-28T20:31:06.510Z","repository":{"id":57442257,"uuid":"80097603","full_name":"thanethomson/MLAlchemy","owner":"thanethomson","description":"Python library to convert YAML/JSON into SQLAlchemy SELECT queries","archived":true,"fork":false,"pushed_at":"2017-11-07T11:36:24.000Z","size":27,"stargazers_count":44,"open_issues_count":1,"forks_count":4,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-07-26T06:41:26.345Z","etag":null,"topics":["json","library","python","sqlalchemy","yaml"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/thanethomson.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-01-26T08:31:43.000Z","updated_at":"2025-05-17T14:39:34.000Z","dependencies_parsed_at":"2022-09-26T18:00:34.815Z","dependency_job_id":null,"html_url":"https://github.com/thanethomson/MLAlchemy","commit_stats":null,"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/thanethomson/MLAlchemy","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thanethomson%2FMLAlchemy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thanethomson%2FMLAlchemy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thanethomson%2FMLAlchemy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thanethomson%2FMLAlchemy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/thanethomson","download_url":"https://codeload.github.com/thanethomson/MLAlchemy/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thanethomson%2FMLAlchemy/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":267580457,"owners_count":24110844,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-28T02:00:09.689Z","response_time":68,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["json","library","python","sqlalchemy","yaml"],"created_at":"2024-12-03T16:19:48.206Z","updated_at":"2025-07-28T20:31:06.228Z","avatar_url":"https://github.com/thanethomson.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# MLAlchemy\n\n[![Build Status](https://travis-ci.org/thanethomson/MLAlchemy.svg?branch=master)](https://travis-ci.org/thanethomson/MLAlchemy)\n[![PyPI](https://img.shields.io/pypi/v/mlalchemy.svg)](https://pypi.python.org/pypi/mlalchemy)\n[![PyPI](https://img.shields.io/pypi/pyversions/mlalchemy.svg)](https://pypi.python.org/pypi/mlalchemy)\n\n## Overview\nMLAlchemy is a Python-based utility library aimed at allowing relatively safe\nconversion from YAML/JSON to SQLAlchemy read-only queries. One use case here is\nto allow RESTful web applications (written in Python) to receive YAML- or\nJSON-based queries for data, e.g. from a front-end JavaScript-based application.\n\nThe name \"MLAlchemy\" is an abbreviation for \"Markup Language for\nSQLAlchemy\".\n\n## Installation\nInstallation via PyPI:\n\n```bash\n\u003e pip install mlalchemy\n```\n\n## Query Examples\nTo get a feel for what MLAlchemy queries look like, take a look at the\nfollowing. **Note**: All field names are converted from `camelCase` or `kebab-case`\nto `snake_case` prior to query execution.\n\n### Example YAML Queries\nFetching all the entries from a table called `Users`:\n\n```yaml\nfrom: Users\n```\n\nLimiting the users to only those with the last name \"Michaels\":\n\n```yaml\nfrom: Users\nwhere:\n  last-name: Michaels\n```\n\nA more complex YAML query:\n\n```yaml\nfrom: Users\nwhere:\n  $or:\n    last-name: Michaels\n    first-name: Michael\n  $gt:\n    date-of-birth: 1984-01-01\n```\n\nThe raw SQL query for the above would look like:\n\n```sql\nSELECT * FROM users WHERE\n  (last_name = \"Michaels\" OR first_name = \"Michael\") AND\n  (date_of_birth \u003e \"1984-01-01\")\n```\n\n### Example JSON Queries\nThe same queries as above, but in JSON format. To fetch all entries\nin the `Users` table:\n\n```json\n{\n    \"from\": \"Users\"\n}\n```\n\nLimiting the users to only those with the last name \"Michaels\":\n\n```json\n{\n    \"from\": \"Users\",\n    \"where\": {\n        \"lastName\": \"Michaels\"\n    }\n}\n```\n\nAnd finally, the more complex query:\n\n```json\n{\n    \"from\": \"Users\",\n    \"where\": {\n        \"$or\": {\n            \"lastName\": \"Michaels\",\n            \"firstName\": \"Michael\"\n        },\n        \"$gt\": {\n            \"dateOfBirth\": \"1984-01-01\"\n        }\n    }\n}\n```\n\n## Usage\nA simple example of how to use MLAlchemy:\n\n```python\nfrom sqlalchemy import create_engine, Column, Integer, String, Date\nfrom sqlalchemy.ext.declarative import declarative_base\nfrom sqlalchemy.orm import sessionmaker\n\nfrom mlalchemy import parse_yaml_query, parse_json_query\n\nBase = declarative_base()\n\n\nclass User(Base):\n    __tablename__ = \"users\"\n\n    id = Column(Integer, primary_key=True)\n    first_name = Column(String)\n    last_name = Column(String)\n    date_of_birth = Column(Date)\n\n\n# use an in-memory SQLite database for this example\nengine = create_engine(\"sqlite:///:memory:\")\nBase.metadata.create_all(engine)\nSession = sessionmaker(bind=engine)\nsession = Session()\n\n# add a couple of dummy users\nuser1 = User(first_name=\"Michael\", last_name=\"Anderson\", date_of_birth=date(1980, 1, 1))\nuser2 = User(first_name=\"James\", last_name=\"Michaels\", date_of_birth=date(1976, 10, 23))\nuser3 = User(first_name=\"Andrew\", last_name=\"Michaels\", date_of_birth=date(1988, 8, 12))\nsession.add_all([user1, user2, user3])\nsession.commit()\n\n# we need a lookup table for MLAlchemy\ntables = {\n    \"User\": User\n}\n\n# try a simple YAML-based query first\nall_users = parse_yaml_query(\"from: User\").to_sqlalchemy(session, tables).all()\nprint(all_users)\n\n# same query, but this time in JSON\nall_users = parse_json_query(\"\"\"{\"from\": \"User\"}\"\"\").to_sqlalchemy(session, tables).all()\nprint(all_users)\n\n# a slightly more complex query\nyoung_users = parse_yaml_query(\"\"\"from: User\nwhere:\n    $gt:\n        date-of-birth: 1988-01-01\n\"\"\").to_sqlalchemy(session, tables).all()\nprint(young_users)\n```\n\n## Query Language Syntax\nAs mentioned before, queries can either be supplied in YAML format or\nin JSON format to one of the respective parsers.\n\n### `from`\nAt present, MLAlchemy can only support selecting data from a single\ntable (multi-table support is planned in future). Here, the `from`\nparameter allows you to specify the name of the table from which\nto select data.\n\n### `where`\nThe `where` parameter defines, in hierarchical fashion, the structure\nof the logical query to perform. There are 3 kinds of key types in\nthe JSON/YAML structures, as described in the following table.\n\n| Kind            | Description                                           | Options                                                                    |\n| --------------- | ----------------------------------------------------- | -------------------------------------------------------------------------- |\n| **Operators**   | Logical (boolean) operators for combining sub-clauses | `$and`, `$or`, `$not`                                                      |\n| **Comparators** | Comparative operators for comparing fields to values  | `$eq`, `$gt`, `$gte`, `$lt`, `$lte`, `$like`, `$neq`, `$in`, `$nin`, `$is` |\n| **Field Names** | The name of a field in the `from` table               | (Depends on table)                                                         |\n\n### `order-by` (YAML) or `orderBy` (JSON)\nProvides the ordering for the resulting query. Must either be a single\nfield name or a list of field names, with the direction specifier in\nfront of the field name. For example:\n\n```yaml\n# Order by \"field2\" in ascending order\norder-by: field2\n```\n\nAnother example:\n\n```yaml\n# Order by \"field2\" in *descending* order\norder-by: \"-field2\"\n```\n\nA more complex example:\n\n```yaml\n# Order first by \"field1\" in ascending order, then by \"field2\" in\n# descending order\norder-by:\n    - field1\n    - \"-field2\"\n```\n\n### `offset`\nSpecifies the number of results to skip before providing results. If\nnot specified, no results are skipped.\n\n### `limit`\nSpecifies the maximum number of results to return. If not specified,\nthere will be no limit to the number of returned results.\n\n## Query Examples\n\n### Example 1: Simple Query\nThe following is an example of a relatively simple query in YAML format:\n\n```yaml\nfrom: SomeTable\nwhere:\n    - $gt:\n        field1: 5\n    - $lt:\n        field2: 3\norder-by:\n    - field1\noffset: 2\nlimit: 10\n```\n\nThis would translate into the following SQLAlchemy query:\n\n```python\nfrom sqlalchemy.sql.expression import and_\n\nsession.query(SomeTable).filter(\n    and_(SomeTable.field1 \u003e 5, SomeTable.field2 \u003c 3)\n) \\\n    .order_by(SomeTable.field1) \\\n    .offset(2) \\\n    .limit(10)\n```\n\n### Example 2: Slightly More Complex Query\nThe following is an example of a more complex query in YAML format:\n\n```yaml\nfrom: SomeTable\nwhere:\n    - $or:\n        field1: 5\n        field2: something\n    - $not:\n        $like:\n            field3: \"else%\"\n```\n\nThis would translate into the following SQLAlchemy query:\n\n```python\nfrom sqlalchemy.sql.expression import and_, or_, not_\n\nsession.query(SomeTable) \\\n    .filter(\n        and_(\n            or_(\n                SomeTable.field1 == 5,\n                SomeTable.field2 == \"something\"\n            ),\n            not_(\n                SomeTable.field3.like(\"else%\")\n            )\n        )\n    )\n```\n\n### Example 3: Complex JSON Query\nThe following is an example of a relatively complex query in\nJSON format:\n\n```json\n{\n    \"from\": \"SomeTable\",\n    \"where\": [\n        {\n            \"$or\": [\n                {\"field1\": 10},\n                {\n                    \"$gt\": {\n                        \"field2\": 5\n                    }\n                }\n            ],\n            \"$and\": [\n                {\"field3\": \"somevalue\"},\n                {\"field4\": \"othervalue\"},\n                {\n                    \"$or\": {\n                        \"field5\": 5,\n                        \"field6\": 6\n                    }\n                }\n            ]\n        }\n    ],\n    \"orderBy\": [\n        \"field1\",\n        \"-field2\"\n    ],\n    \"offset\": 2,\n    \"limit\": 10\n}\n```\n\nThis query would be translated into the following SQLAlchemy code:\n\n```python\nfrom sqlalchemy.sql.expression import and_, or_, not_\n\nsession.query(SomeTable) \\\n    .filter(\n        and_(\n            or_(\n                SomeTable.field1 == 10,\n                SomeTable.field2 \u003e 5\n            ),\n            and_(\n                SomeTable.field3 == \"somevalue\",\n                SomeTable.field4 == \"othervalue\",\n                or_(\n                    SomeTable.field5 == 5,\n                    SomeTable.field6 == 6\n                )\n            )\n        )\n    ) \\\n    .order_by(SomeTable.field1, SomeTable.field2.desc()) \\\n    .offset(2) \\\n    .limit(10)\n```\n\n## License\n**The MIT License (MIT)**\n\nCopyright (c) 2017 Thane Thomson\n\nPermission is hereby granted, free of charge, to any person obtaining a copy of\nthis software and associated documentation files (the \"Software\"), to deal in\nthe Software without restriction, including without limitation the rights to\nuse, copy, modify, merge, publish, distribute, sublicense, and/or sell copies\nof the Software, and to permit persons to whom the Software is furnished to do\nso, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthanethomson%2Fmlalchemy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthanethomson%2Fmlalchemy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthanethomson%2Fmlalchemy/lists"}