{"id":17327058,"url":"https://github.com/jaypipes/sqltoast","last_synced_at":"2025-09-07T03:32:35.969Z","repository":{"id":15203712,"uuid":"17932029","full_name":"jaypipes/sqltoast","owner":"jaypipes","description":"A SQL parser written in C++","archived":false,"fork":false,"pushed_at":"2021-10-22T11:35:53.000Z","size":976,"stargazers_count":30,"open_issues_count":17,"forks_count":11,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-04-14T17:47:19.710Z","etag":null,"topics":["c-plus-plus","parser","recursive-descent-parser","sql"],"latest_commit_sha":null,"homepage":null,"language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jaypipes.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"COPYING","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2014-03-20T05:44:38.000Z","updated_at":"2025-01-17T19:01:58.000Z","dependencies_parsed_at":"2022-09-09T09:21:06.155Z","dependency_job_id":null,"html_url":"https://github.com/jaypipes/sqltoast","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/jaypipes/sqltoast","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jaypipes%2Fsqltoast","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jaypipes%2Fsqltoast/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jaypipes%2Fsqltoast/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jaypipes%2Fsqltoast/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jaypipes","download_url":"https://codeload.github.com/jaypipes/sqltoast/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jaypipes%2Fsqltoast/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":273992644,"owners_count":25203783,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-07T02:00:09.463Z","response_time":67,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["c-plus-plus","parser","recursive-descent-parser","sql"],"created_at":"2024-10-15T14:18:30.848Z","updated_at":"2025-09-07T03:32:35.930Z","avatar_url":"https://github.com/jaypipes.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# sqltoast - a parser for SQL written in C++ [![Build Status](https://travis-ci.org/jaypipes/sqltoast.svg?branch=master)](https://travis-ci.org/jaypipes/sqltoast)\n\nThe `sqltoast` library is an Apache 2 licensed C++ library designed for both\nfast and fully thread-safe parsing of SQL into a collection of structs that\nrepresent the abstract syntax tree (thus SQL to AST, or \"sqltoast\").\n\n## Overview\n\nThere is one primary function that the library publishes: ``sqltoast::parse()``.\n\nThis function comes in two flavors; one which accepts just the string to be\nparsed and one which accepts the string to be parsed along with a struct\nrepresenting parsing options and instructions for sqltoast.\n\nThe return value of ``sqltoast::parse()`` is a reference to a struct\n`sqltoast::parse_result_t` that describes the results of the parse:\n\n```c++\n\n#include \u003ciostream\u003e\n\n#include \u003csqltoast/sqltoast.h\u003e\n\nint main(int argc, char *argv[]) {\n    std::string input;\n    if (argc == 2) {\n        input.assign(args[1]);\n    } else\n        std::cout \u003c\u003c \"Supply a SQL string to parse.\" \u003c\u003c std::endl;\n        return 1;\n    }\n    sqltoast::parse_result_t res;\n    sqltoast::parse_input_t subject(input.cbegin(), input.cend());\n\n    res = sqltoast::parse(subject);\n    if (res.code == sqltoast::PARSE_OK)\n        std::cout \u003c\u003c \"Success!\" \u003c\u003c std::endl;\n    else if (res.code == sqltoast::PARSE_SYNTAX_ERROR)\n        std::cout \u003c\u003c \"Syntax error: \" \u003c\u003c res.error \u003c\u003c std::endl;\n}\n```\n\nAn important attribute of the `sqltoast::parse_result_t` struct is the\n`statements` field, which is of type\n`std::vector\u003cstd::unique_ptr\u003csqltoast::statement_t\u003e\u003e`. For each valid SQL\nstatement parsed by `sqltoast::parse()`, there will be a\n`std::unique_ptr\u003csqltoast::statement_t\u003e` subclassed struct in the `statements`\nfield.\n\nThe [example program](sqltoaster/main.cc) included in the\n[sqltoaster/](sqltoaster/) directory can be a good guide to lean about the\ncollection of `sqltoast::statement_t` structs.\n\nThe example program contains \"printer functions\" for each type of\n`sqltoast::statement_t` struct that is part of the `sqltoast` library. The\nexample program outputs in a variety of formats, including YAML, so you can,\nfor example, see a YAML document that describes the parsed\n`sqltoast::statement_t` type. Reading through the printer functions can give\nyou a good idea of the structure of these `sqltoast::statement_t` types and how\nto utilize the parsed information.\n\nAfter building the `sqltoaster` binary, let's ask it to parse a `SELECT`\nstatement and output the parsed information as a YAML document.\n\n```\nsqltoaster --yaml \"SELECT a FROM t1\"\nstatements:\n- type: SELECT\n  selected_columns:\n    - column-reference[a]\n  referenced_tables:\n    - t1\n(took 15790 nanoseconds)\n```\n\nBy examining the `sqltoaster::print::to_yaml()` function in the `sqltoaster`\nprogram, we can see how to read information about a particular\n`sqltoast::statement_t` struct that is contained in the\n`sqltoast::parse_result_t` struct's `statements` attribute.\n\n**NOTE**: I've trimmed the code below and slightly modified it (removing some\nindenting-related fluff) for brevity.\n\nFirst, let's take a look at the `sqltoaster::print::to_yaml()` function that\nreceives a `sqltoast::statement_t` object. This is our \"top-level\" YAML-output\nfunction for SQL statements.\n\n```c++\nvoid to_yaml(printer_t\u0026 ptr, std::ostream\u0026 out, const sqltoast::statement_t\u0026 stmt) {\n    switch (stmt.type) {\n        // ...\n        case sqltoast::STATEMENT_TYPE_SELECT:\n            out \u003c\u003c \"type: SELECT\";\n            {\n                const sqltoast::select_statement_t\u0026 sub =\n                    static_cast\u003cconst sqltoast::select_statement_t\u0026\u003e(stmt);\n                to_yaml(ptr, out, sub);\n            }\n            break;\n        // ...\n    }\n}\n```\n\nThe above demonstrates an important pattern deployed in the `sqltoast` library\nand a fundamental technique used by the example `sqltoaster` program to sift\nthrough the components of the parse tree.\n\nFor performance reasons, `sqltoast` is built with no run-time type\nidentification (RTTI). All type identification must be handled at compile time,\nand due to this, users of the `sqltoast` library (in this case, the example\n`sqltoaster` program) use a \"type switch\" approach to determine what class of\nstatement the supplied `sqltoast::statement_t` reference is and then use a\n`static_cast\u003c\u003e` to convert the `sqltoast::statement_t\u0026` into the appropriate\nderived type.\n\nOnce you've `static_cast\u003c\u003e`'d to a reference of the derived type, you would\noperate on that derived type reference, as shown in this code that examines the\nelements of the `sqltoast::select_statement_t` sub-type and outputs those\nelements into the YAML document.\n\n```c++\nvoid to_yaml(printer_t\u0026 ptr, std::ostream\u0026 out, const sqltoast::select_statement_t\u0026 stmt) {\n    if (stmt.distinct)\n       out \u003c\u003c \"distinct: true\";\n    out \u003c\u003c \"selected_columns:\";\n    for (const sqltoast::derived_column_t\u0026 dc : stmt.selected_columns)\n        out \u003c\u003c \"- \" \u003c\u003c dc;\n    out \u003c\u003c \"referenced_tables:\";\n    for (const std::unique_ptr\u003csqltoast::table_reference_t\u003e\u0026 tr : stmt.referenced_tables)\n        out \u003c\u003c \"- \" \u003c\u003c *tr;\n    if (stmt.where_condition)\n        out \u003c\u003c \"where:\" \u003c\u003c *stmt.where_condition;\n    if (! stmt.group_by_columns.empty()) {\n        out \u003c\u003c \"group_by:\";\n        for (const sqltoast::grouping_column_reference_t\u0026 gcr : stmt.group_by_columns)\n            out \u003c\u003c \"- \" \u003c\u003c gcr;\n    }\n    if (stmt.having_condition)\n        out \u003c\u003c \"having:\" \u003c\u003c *stmt.having_condition;\n}\n```\n\n## Repository layout\n\nThis source repository contains two directories with C++ code: the\n[libsqltoast](../libsqltoast) directory contains the source code and headers\nfor the `sqltoast` library; the [sqltoaster](../sqltoaster) directory contains\nsource files for an example program that utilizes the `sqltoast` library.\n\nFinally, there is a [tests](../tests) directory which contains various test\nfiles and test infrastructure.\n\n## Build instructions\n\n`sqltoast` uses [cmake](https://cmake.org) as its primary build toolchain.\n\n**NOTE**: If you add a new source or header file or dependency, you will need\nto edit the `CMakeLists.txt` file in either the [libsqltoast](../libsqltoast)\ndirectory or the [sqltoaster](../sqltoaster) directory.\n\nTo build the `sqltoast` library and `sqltoaster` example executable, first,\ncreate a directory to use for building:\n\n```\nBUILD_DIR=\"$(pwd)/_build\"\nmkdir -p $BUILD_DIR\n```\n\nThen change directory into your build directory and run `cmake`:\n\n```bash\ncd $BUILD_DIR\ncmake ..\n```\n\nthis will regenerate the `Makefile`s as needed and you may then build the\nlibrary and `sqltoaster` binary with a simple:\n\n```bash\nmake\n```\n\n## Running tests\n\nAfter building the library and `sqltoaster` binary, you can run tests by\nchanging into the [tests/grammar](../tests/grammar) directory and executing the\nPython test runner, like so:\n\n```\ncd tests/grammar\npython runner.py run\n```\n\nwhich will run all the grammar tests and output something like this:\n\n```\njaypipes@uberbox:~/src/github.com/jaypipes/sqltoast$ cd tests/grammar/\njaypipes@uberbox:~/src/github.com/jaypipes/sqltoast/tests/grammar$ python runner.py run\nRunning ansi-92/alter-table ... OK\nRunning ansi-92/case-expressions ... OK\nRunning ansi-92/column-definitions ... OK\nRunning ansi-92/create-schema ... OK\nRunning ansi-92/create-table ... OK\nRunning ansi-92/create-view ... OK\nRunning ansi-92/datetime-expressions ... OK\nRunning ansi-92/delete ... OK\nRunning ansi-92/drop-schema ... OK\nRunning ansi-92/drop-table ... OK\nRunning ansi-92/drop-view ... OK\nRunning ansi-92/grant ... OK\nRunning ansi-92/identifiers ... OK\nRunning ansi-92/joins ... OK\nRunning ansi-92/insert-select ... OK\nRunning ansi-92/insert-values ... OK\nRunning ansi-92/interval-expressions ... OK\nRunning ansi-92/numeric-expressions ... OK\nRunning ansi-92/predicates ... OK\nRunning ansi-92/row-value-constructors ... OK\nRunning ansi-92/string-expressions ... OK\nRunning ansi-92/table-references ... OK\nRunning ansi-92/update ... OK\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjaypipes%2Fsqltoast","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjaypipes%2Fsqltoast","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjaypipes%2Fsqltoast/lists"}