{"id":13393604,"url":"https://github.com/alash3al/sqler","last_synced_at":"2025-05-14T07:11:01.548Z","repository":{"id":41092789,"uuid":"164500360","full_name":"alash3al/sqler","owner":"alash3al","description":"write APIs using direct SQL queries with no hassle, let's rethink about SQL","archived":false,"fork":false,"pushed_at":"2024-12-28T18:30:08.000Z","size":117,"stargazers_count":2080,"open_issues_count":0,"forks_count":117,"subscribers_count":44,"default_branch":"master","last_synced_at":"2025-04-12T23:29:39.401Z","etag":null,"topics":["api-rest","baas","cockroach","mssql","mysql","postgresql","restful","sqlite3"],"latest_commit_sha":null,"homepage":"https://www.sqler.dev/","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/alash3al.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-01-07T21:38:36.000Z","updated_at":"2025-04-08T16:58:24.000Z","dependencies_parsed_at":"2024-06-20T11:01:00.620Z","dependency_job_id":"2ff7f198-862f-4739-b6e2-9e047a9b6c56","html_url":"https://github.com/alash3al/sqler","commit_stats":{"total_commits":63,"total_committers":11,"mean_commits":"5.7272727272727275","dds":"0.17460317460317465","last_synced_commit":"d7bd00eea73b297c9889caed6e63adc3359c7bb2"},"previous_names":[],"tags_count":10,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alash3al%2Fsqler","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alash3al%2Fsqler/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alash3al%2Fsqler/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alash3al%2Fsqler/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/alash3al","download_url":"https://codeload.github.com/alash3al/sqler/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254092798,"owners_count":22013292,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api-rest","baas","cockroach","mssql","mysql","postgresql","restful","sqlite3"],"created_at":"2024-07-30T17:00:56.794Z","updated_at":"2025-05-14T07:10:56.528Z","avatar_url":"https://github.com/alash3al.png","language":"Go","funding_links":[],"categories":["Misc","Projects","开源类库","Go","Open source library","postgresql"],"sub_categories":["Web and Publishing","数据库","Database"],"readme":"SQLer\n=====\n\u003e `SQL-er` is a tiny portable server enables you to write APIs using SQL query to be executed when anyone hits it, also it enables you to define validation rules so you can validate the request body/query params, as well as data transformation using simple `javascript` syntax. `sqler` uses `nginx` style configuration language ([`HCL`](https://github.com/hashicorp/hcl)) and `javascript` engine for custom expressions.\n\nTable Of Contents\n=================\n- [SQLer](#sqler)\n- [Table Of Contents](#table-of-contents)\n- [Features](#features)\n- [Quick Tour](#quick-tour)\n- [Supported DBMSs](#supported-dbmss)\n- [Docker](#docker)\n- [Configuration Overview](#configuration-overview)\n- [REST vs RESP](#rest-vs-resp)\n- [Sanitization](#sanitization)\n- [Validation](#validation)\n- [Authorization](#authorization)\n- [Data Transformation](#data-transformation)\n- [Aggregators](#aggregators)\n- [Issue/Suggestion/Contribution ?](#issuesuggestioncontribution)\n- [Author](#author)\n- [License](#license)\n\nFeatures\n========\n- Standalone with no dependencies.\n- Works with most of SQL databases out there including (`SQL Server`, `MYSQL`, `SQLITE`, `PostgreSQL`, `Cockroachdb`)\n- Built-in RESTful server\n- Built-in RESP `Redis Protocol`, you connect to `SQLer` using any `redis` client\n- Built-in `Javascript` interpreter to easily transform the result\n- Built-in Validators\n- Automatically uses prepared statements\n- Uses ([`HCL`](https://github.com/hashicorp/hcl)) configuration language\n- You can load multiple configuration files not just one, based on `unix glob` style pattern\n- Each `SQL` query could be named as `Macro`\n- Uses `Javascript` custom expressions.\n- Each macro has its own `Context` (`query params` + `body params`) as `.Input` which is `map[string]interface{}`, and `.Utils` which is a list of helper functions, currently it contains only `SQLEscape`.\n- You can define `authorizers`, an `authorizer` is just a simple webhook that enables `sqler` to verify whether the request should be done or not.\n- Trigger a `webhook` or another `macro` when a specific `macro` get executed.\n- Schedule specific macros to run at specific time using simple `cron` syntax.\n\nQuick Tour\n==========\n- You install `sqler` using the right binary for your `os` from the [releases](https://github.com/alash3al/sqler/releases) page.\n- Let's say that you downloaded `sqler_darwin_amd64`\n- Let's rename it to `sqler`, and copy it to `/usr/local/bin`\n- Now just run `sqler -h`, you will the next  \n```bash\n                         ____   ___  _\n                        / ___| / _ \\| |    ___ _ __\n                        \\___ \\| | | | |   / _ \\ '__|\n                         ___) | |_| | |__|  __/ |\n                        |____/ \\__\\_\\_____\\___|_|\n\n        turn your SQL queries into safe valid RESTful apis.\n\n\n  -config string\n        the config file(s) that contains your endpoints configs, it accepts comma seprated list of glob style pattern (default \"./config.example.hcl\")\n  -driver string\n        the sql driver to be used (default \"mysql\")\n  -dsn string\n        the data source name for the selected engine (default \"root:root@tcp(127.0.0.1)/test?multiStatements=true\")\n  -resp string\n        the resp (redis protocol) server listen address (default \":3678\")\n  -rest string\n        the http restful api listen address (default \":8025\")\n  -workers int\n        the maximum workers count (default 4)\n```\n- you can specifiy multiple files for `-config` as [configuration](#configuration-overview), i.e `-config=\"/my/config/dir/*.hcl,/my/config/dir2/*.hcl\"`\n- you need specify which driver you need and its `dsn` from the following:\n\n| Driver | DSN |\n---------| ------ |\n| `mysql`| `usrname:password@tcp(server:port)/dbname?option1=value1\u0026...`|\n| `postgres`| `postgresql://username:password@server:port/dbname?option1=value1`|\n| `sqlite3`| `/path/to/db.sqlite?option1=value1`|\n| `sqlserver` | `sqlserver://username:password@host/instance?param1=value\u0026param2=value` |\n|             | `sqlserver://username:password@host:port?param1=value\u0026param2=value`|\n|             | `sqlserver://sa@localhost/SQLExpress?database=master\u0026connection+timeout=30`|\n| `mssql` | `server=localhost\\\\SQLExpress;user id=sa;database=master;app name=MyAppName`|\n|         | `server=localhost;user id=sa;database=master;app name=MyAppName`|\n|         | `odbc:server=localhost\\\\SQLExpress;user id=sa;database=master;app name=MyAppName` |\n|         | `odbc:server=localhost;user id=sa;database=master;app name=MyAppName` |\n| `hdb` (SAP HANA) |   `hdb://user:password@host:port` |\n| `clickhouse` (Yandex ClickHouse) |   `tcp://host1:9000?username=user\u0026password=qwerty\u0026database=clicks\u0026read_timeout=10\u0026write_timeout=20\u0026alt_hosts=host2:9000,host3:9000` |\n\nSupported DBMSs\n===============\n- `MYSQL`, `TiDB`, `MariaDB`, `Percona` and any MYSQL compatible server uses `mysql` driver.\n- `PostgreSQL`, `CockroachDB` and any PostgreSQL compatible server uses `postgres` driver.\n- `SQL Server`, `MSSQL`, `ADO`, `ODBC` uses `sqlserver` or `mssql` driver.\n- `SQLITE`, uses `sqlite3` driver.\n- `HANA` (SAP), uses `hdb` driver.\n- `Clickhouse`, uses `clickhouse` driver.\n\nDocker\n======\n\u003e SQLer has a docker image called `alash3al/sqler` it is an automated build, you can use it like the following:\n\n```bash\n\n# run the help message\ndocker run --rm alash3al/sqler --help\n\n# connect to a local mysql\ndocker run --network=host alash3al/sqler -driver=mysql -dsn=usr:pass@tcp(127.0.0.1:3306)/dbname\n\n# connect to another mysql container\ndocker run -link mysql alash3al/sqler -driver=mysql -dsn=usr:pass@tcp(mysql:3306)/dbname\n\n```\n\nConfiguration Overview\n======================\n```hcl\n// create a macro/endpoint called \"_boot\",\n// this macro is private \"used within other macros\" \n// because it starts with \"_\".\n_boot {\n    // the query we want to execute\n    exec = \u003c\u003cSQL\n        CREATE TABLE IF NOT EXISTS `users` (\n            `ID` INT UNSIGNED AUTO_INCREMENT PRIMARY KEY,\n            `name` VARCHAR(30) DEFAULT \"@anonymous\",\n            `email` VARCHAR(30) DEFAULT \"@anonymous\",\n            `password` VARCHAR(200) DEFAULT \"\",\n            `time` INT UNSIGNED\n        );\n    SQL\n}\n\n// adduser macro/endpoint, just hit `/adduser` with\n// a `?user_name=\u0026user_email=` or json `POST` request\n// with the same fields.\nadduser {\n    validators {\n        user_name_is_empty = \"$input.user_name \u0026\u0026 $input.user_name.trim().length \u003e 0\"\n        user_email_is_empty = \"$input.user_email \u0026\u0026 $input.user_email.trim(' ').length \u003e 0\"\n        user_password_is_not_ok = \"$input.user_password \u0026\u0026 $input.user_password.trim(' ').length \u003e 5\"\n    }\n\n    bind {\n        name = \"$input.user_name\"\n        email = \"$input.user_email\"\n        password = \"$input.user_password\"\n    }\n\n    methods = [\"POST\"]\n\n    authorizer = \u003c\u003cJS\n        (function(){\n            log(\"use this for debugging\")\n            token = $input.http_authorization\n            response = fetch(\"http://requestbin.fullcontact.com/zxpjigzx\", {\n                headers: {\n                    \"Authorization\": token\n                }\n            })\n            if ( response.statusCode != 200 ) {\n                return false\n            }\n            return true\n        })()\n    JS\n\n    // include some macros we declared before\n    include = [\"_boot\"]\n\n    exec = \u003c\u003cSQL\n        INSERT INTO users(name, email, password, time) VALUES(:name, :email, :password, UNIX_TIMESTAMP());\n        SELECT * FROM users WHERE id = LAST_INSERT_ID();\n    SQL\n}\n\n// list all databases, and run a transformer function\ndatabases {\n    exec = \"SHOW DATABASES\"\n    cron = \"* * * * *\"\n    trigger {\n        webhook = \"http://some.url/hook\"\n    }\n}\n\n// list all tables from all databases\ntables {\n    exec = \"SELECT `table_schema` as `database`, `table_name` as `table` FROM INFORMATION_SCHEMA.tables\"\n}\n\n// a macro that aggregates `databases` macro and `tables` macro into one macro\ndatabases_tables {\n    aggregate = [\"databases\", \"tables\"]\n}\n\n```\n\nREST vs RESP\n=============\n\u003e RESTful server could be used to interact directly with i.e `mobile, browser, ... etc`, in this mode `SQLer` is protected by `authorizers`, which gives you the ability to check authorization against another 3rd-party api.  \n\u003e Each macro you add to the configuration file(s) you can access to it by issuing a http request to `/\u003cmacro-name\u003e`, every query param and json body will be passed to the macro `.Input`.\n\n\u003e RESP server is just a basic `REDIS` compatible server, you connect to it using any `REDIS` client out there, even `redis-cli`, just open `redis-cli -p 3678 list` to list all available macros (`commands`), you can execute any macro as a redis command and pass the arguments as a json encoded data, i.e `redis-cli -p 3678 adduser \"{\\\"user_name\\\": \\\"u6\\\", \\\"user_email\\\": \\\"email@tld.com\\\", \\\"user_password\\\":\\\"pass@123\\\"}\"`.\n\nSanitization\n=============\n\u003e `SQLer` uses prepared statements, you can bind inputs like the following:\n\n\n```hcl\naddpost {\n    // $input is a global variable holds all request inputs,\n    // including the http headers too (prefixed with `http_`)\n    // all http header keys are normalized to be in this form \n    // `http_x_header_example`, `http_authorization` ... etc in lower case.\n    bind {\n        title = \"$input.post_title\"\n        content = \"$input.post_content\"\n        user_id = \"$input.post_user\"\n    }\n\n    exec = \u003c\u003cSQL\n        INSERT INTO posts(user_id, title, content) VALUES(:user_id, :title, :content);\n        SELECT * FROM posts WHERE id = LAST_INSERT_ID();\n    SQL\n}\n```\n\n\nValidation\n===========\n\u003e Data validation is very easy in `SQLer`, it is all about simple `javascript` expression like this:\n\n```hcl\naddpost {\n    // if any rule returns false, \n    // SQLer will return 422 code, with invalid rules.\n    // \n    // $input is a global variable holds all request inputs,\n    // including the http headers too (prefixed with `http_`)\n    // all http header keys are normalized to be in this form \n    // `http_x_header_example`, `http_authorization` ... etc in lower case.\n    validators {\n        post_title_length = \"$input.post_title \u0026\u0026 $input.post_title.trim().length \u003e 0\"\n        post_content_length = \"$input.post_content \u0026\u0026 $input.post_content.length \u003e 0\"\n        post_user = \"$input.post_user\"\n    }\n\n    bind {\n        title = \"$input.post_title\"\n        content = \"$input.post_content\"\n        user_id = \"$input.post_user\"\n    }\n\n    exec = \u003c\u003cSQL\n        INSERT INTO posts(user_id, title, content) VALUES(:user_id, :title, :content);\n        SELECT * FROM posts WHERE id = LAST_INSERT_ID();\n    SQL\n}\n```\n\nAuthorization\n=============\n\u003e If you want to expose `SQLer` as a direct api to API consumers, you will need to add an authorization layer on top of it, let's see how to do that\n\n```hcl\naddpost {\n    authorizer = \u003c\u003cJS\n        (function(){\n            // $input is a global variable holds all request inputs,\n            // including the http headers too (prefixed with `http_`)\n            // all http header keys are normalized to be in this form \n            // `http_x_header_example`, `http_authorization` ... etc in lower case.\n            token = $input.http_authorization\n            response = fetch(\"http://requestbin.fullcontact.com/zxpjigzx\", {\n                headers: {\n                    \"Authorization\": token\n                }\n            })\n            if ( response.statusCode != 200 ) {\n                return false\n            }\n            return true\n        })()\n    JS\n}\n```\n\n\u003e using that trick, you can use any third-party Authentication service that will remove that hassle from your code.\n\nData Transformation\n====================\n\u003e In some cases we need to transform the resulted data into something more friendly to our API consumers, so I added `javascript` interpreter to `SQLer` so we can transform our data, each js code has a global variable called `$result`, it holds the result of the `exec` section, you should write your code like the following:\n\n```hcl\n// list all databases, and run a transformer function\ndatabases {\n    exec = \"SHOW DATABASES\"\n\n    transformer = \u003c\u003cJS\n        // there is a global variable called `$result`,\n        // `$result` holds the result of the sql execution.\n        (function(){\n            newResult = []\n\n            for ( i in $result ) {\n                newResult.push($result[i].Database)\n            }\n\n            return newResult\n        })()\n    JS\n}\n```\n\nAggregators\n============\n\u003e `SQLer` helps you to merge multiple macros into one to minimize the API calls number, see the example bellow\n\n```hcl\ndatabases {\n    exec = \"SHOW DATABASES\"\n\n    transformer = \u003c\u003cJS\n        // there is a global variable called `$result`,\n        // `$result` holds the result of the sql execution.\n        (function(){\n            newResult = []\n\n            for ( i in $result ) {\n                newResult.push($result[i].Database)\n            }\n\n            return newResult\n        })()\n    JS\n}\n\ntables {\n    exec = \"SELECT `table_schema` as `database`, `table_name` as `table` FROM INFORMATION_SCHEMA.tables\"\n    transformer = \u003c\u003cJS\n        (function(){\n            $ret = {}\n            for (  i in $result ) {\n                if ( ! $ret[$result[i].database] ) {\n                    $ret[$result[i].database] = [];\n                }\n                $ret[$result[i].database].push($result[i].table)\n            }\n            return $ret\n        })()\n    JS\n}\n\ndatabasesAndTables {\n    aggregate {\n        databases = \"current_databases\"\n        tables = \"current_tables\"\n    }\n}\n```\n\nIssue/Suggestion/Contribution ?\n===============================\n`SQLer` is your software, feel free to open an issue with your feature(s), suggestions, ... etc, also you can easily contribute even you aren't a `Go` developer, you can write wikis it is open for all, let's make `SQLer` more powerful.\n\nAuthor\n=======\n\u003e I'm Mohamed Al Ashaal, just a problem solver :), you can view more projects from me [here](https://github.com/alash3al), and here is my email [m7medalash3al@gmail.com](mailto:m7medalash3al@gmail.com)\n\nLicense\n========\n\u003e Copyright 2019 The SQLer Authors. All rights reserved.\n\u003e Use of this source code is governed by a Apache 2.0\n\u003e license that can be found in the [LICENSE](/License) file.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falash3al%2Fsqler","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Falash3al%2Fsqler","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falash3al%2Fsqler/lists"}