{"id":16577321,"url":"https://github.com/allwefantasy/mlsql","last_synced_at":"2025-10-29T04:32:20.758Z","repository":{"id":44871779,"uuid":"426826027","full_name":"allwefantasy/mlsql","owner":"allwefantasy","description":"New Repo: https://github.com/byzer-org/kolo-lang","archived":false,"fork":false,"pushed_at":"2021-12-16T06:36:15.000Z","size":51347,"stargazers_count":12,"open_issues_count":2,"forks_count":6,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-02-01T22:22:51.417Z","etag":null,"topics":["mlsql","ray","spark","sql"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/allwefantasy.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-11-11T00:54:38.000Z","updated_at":"2024-06-06T09:00:25.000Z","dependencies_parsed_at":"2022-09-24T22:52:14.515Z","dependency_job_id":null,"html_url":"https://github.com/allwefantasy/mlsql","commit_stats":null,"previous_names":[],"tags_count":20,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allwefantasy%2Fmlsql","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allwefantasy%2Fmlsql/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allwefantasy%2Fmlsql/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allwefantasy%2Fmlsql/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/allwefantasy","download_url":"https://codeload.github.com/allwefantasy/mlsql/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":238768449,"owners_count":19527200,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["mlsql","ray","spark","sql"],"created_at":"2024-10-11T22:10:28.324Z","updated_at":"2025-10-29T04:32:15.714Z","avatar_url":"https://github.com/allwefantasy.png","language":"JavaScript","readme":" ![CI](https://github.com/byzer-org/kolo-lang/actions/workflows/build_and_test.yml/badge.svg)   [![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)\n\n\u003cp align=\"center\"\u003e\n    \u003cimg src=\"https://raw.githubusercontent.com/byzer-org/kolo-lang/master/images/Kolo_Logo.png\" alt=\"drawing\"  width=\"200\"/\u003e\n\u003c/p\u003e\n\n## TOC   \n   * [Kolo-Lang](#kolo-lang)\n      * [Kolo Code Example](#kolo-code-example)\n      * [Kolo Architecture](#kolo-aritchitechture)\n   * [Official WebSite](#official-website)\n   * [VSCode Extension（MacOS、Linux）](#vscode-extensionmacoslinux)\n   * [Docker Sandbox](#docker-sandbox)\n      * [Pulling Sandbox Docker Image](#pulling-sandbox-docker-image)\n      * [Start Container](#start-container)\n   * [Download Kolo](#download-kolo)\n   * [Building a Distribution](#building-a-distribution)\n      * [Prerequisites](#prerequisites)\n      * [Downloading Source Code](#downloading-source-code)\n      * [Building Spark 2.4.3 Bundle](#building-spark-243-bundle)\n      * [Building Spark 3.1.1 Bundle](#building-spark-311-bundle)\n      * [Building without Chinese Analyzer](#building-without-chinese-analyzer)\n      * [Building with Aliyun OSS Support](#building-with-aliyun-oss-support)\n   * [Deploying](#deploying)\n   * [How to contribute to Kolo-Lang](#how-to-contribute-to-kolo-lang)\n   * [Contributors](#contributors)\n   * [WeChat Group](#wechat-group)\n\n## Kolo-Lang\n\n**Kolo** (former MLSQL) is a low-code, open-sourced and distributed programming language for data pipeline, analytics and AI in cloud native way.\n\n**Deisgn protocol: Everything is a table.** Kolo is a SQL-like language, to simplify data pipeline, analytics and AI, combined with built-in algorithms and extensions.\n\nWe believe that everything is a table, a simple and powerful SQL-like language can significantly reduce human efforts of data development without switching different tools.\n\n### Kolo Architecture\n\n![Kolo-lang Arch](images/Kolo-arch.jpg)\n\n### Kolo Code Example\n\n\n```sql\nload hive.`raw.stripe_discounts` as discounts;\nload hive.`raw.stripe_invoice_items` as invoice_items;\n\nselect\n        invoice_items.*,\n        case\n            when discounts.discount_type = 'percent'\n                then amount * (1.0 - discounts.discount_value::float / 100)\n            else amount - discounts.discount_value\n        end as discounted_amount\n\n    from invoice_items\n\n    left outer join discounts\n        on invoice_items.customer_id = discounts.customer_id\n        and invoice_items.invoice_date \u003e discounts.discount_start\n        and (invoice_items.invoice_date \u003c discounts.discount_end\n             or discounts.discount_end is null)\nas joined;\n\n\n\nselect\n\n        id,\n        invoice_id,\n        customer_id,\n        coalesce(discounted_amount, amount) as discounted_amount,\n        currency,\n        description,\n        created_at,\n        deleted_at\n\n    from joined\nas final;\n\n\n\nset allColumns = \"all,wow\";\n\n!if ''' split(:allColumns,\",\")[0] == \"all\" ''';\n   select * from final as final2;\n!else;\n   select id,invoice from final as final2;\n!fi;\n\nselect * from final2 as output;\n```\n\n\n## Official WebSite\n\n[https://mlsql.tech](https://mlsql.tech) [New official website is coming soon]\n\n## Notebook Support\n\n[byzer-notebook](https://github.com/byzer-org/byzer-notebook)\n\n\n## VSCode Extension（MacOS、Linux、Windows）\n\n[VSCode IDE Extension](https://github.com/byzer-org/kolo-desktop)\n\n[More document about kolo-lang vscode extension（Chinese version）](https://mlsql-docs.kyligence.io/latest/zh-hans/howtouse/mlsql_desktop_install.html)\n\n## Docker Sandbox  (With Notebook)\n\n```\nexport MYSQL_PASSWORD=${1:-root}\nexport SPARK_VERSION=${SPARK_VERSION:-3.1.1}\nexport MLSQL_VERSION=${MLSQL_VERSION:-2.2.0-SNAPSHOT}\n\ndocker run -d \\\n-p 3306:3306 \\\n-p 9002:9002 \\\n-p 9003:9003 \\\n-e MYSQL_ROOT_HOST=% \\\n-e MYSQL_ROOT_PASSWORD=\"${MYSQL_PASSWORD}\" \\\n--name mlsql-sandbox-${SPARK_VERSION}-${MLSQL_VERSION} \\\nmlsql-sandbox:${SPARK_VERSION}-${MLSQL_VERSION}\n```\n\nThen you can visit `http://127.0.0.1:9002` .\n\n\n## Download Kolo\n\n* The latest stable version is release-2.1.0\n* You can download from [Kolo Website](https://mlsql-downloads.kyligence.io/2.1.0/)\n* Spark 2.4.3/3.1.1 have been tested\n\n***Naming Convention***\n\nmlsql-engine_${spark_major_version}-${mlsql_version}.tgz\n```shell\n## Pre-built for Spark 2.4.3\nmlsql-engine_2.4-2.1.0.tar.gz\n\n## Pre-built for Spark 3.1.1\nmlsql-engine_3.0-2.1.0.tar.gz\n```  \n\n## Building a Distribution\n### Prerequisites\n- JDK 8+\n- Maven\n- Linux or MacOS\n\n### Downloading Source Code\n```shell\n## Clone the code base\ngit clone https://github.com/byzer-org/kolo-lang.git\ncd kolo-lang\n```\n\n### Building Spark 2.4.3 Bundle\n```shell\nexport MLSQL_SPARK_VERSION=2.4\n./dev/make-distribution.sh\n```\n\n### Building Spark 3.1.1 Bundle\n```shell\nexport MLSQL_SPARK_VERSION=3.0\n./dev/make-distribution.sh\n```\n### Building without Chinese Analyzer\n```shell\n## Chinese analyzer is enabled by default.\nexport ENABLE_CHINESE_ANALYZER=false\n./dev/make-distribution.sh \u003cspark_version\u003e\n```\n\n## Deploying\n1. [Download](#Download) or [build a distribution](#Build) \n2. Install Spark and set environment variable SPARK_HOME, make sure Spark version matches that of MLSQL\n3. Deploy tgz\n- Set environment variable MLSQL_HOME\n- Copy distribution tar ball over and untar it\n\n4.Start Kolo in local mode\n```shell\ncd $MLSQL_HOME\n## Run process in background\nnohup ./bin/start-local.sh 2\u003e\u00261 \u003e ./local_mlsql.log \u0026\n```\n5. Open a browser and type in http://localhost:9003, have fun.\n\nDirectory structure\n```shell\n|-- mlsql\n    |-- bin        \n    |-- conf       \n    |-- data       \n    |-- examples   \n    |-- libs       \n    |-- README.md  \n    |-- LICENSE\n    |-- RELEASE\n```\n\n## How to contribute to Kolo-Lang\n\nIf you are planning to contribute to this repository, please create an issue at [our Issue page](https://github.com/byzer-org/kolo-lang/issues)\neven if the topic is not related to source code itself (e.g., documentation, new idea and proposal).\n\nThis is an active open source project for everyone,\nand we are always open to people who want to use this system or contribute to it.\n\nFor more details about how to contribute to the Byzer Org, please refer to [How to Contribute](https://docs.byzer.org/#/kolo-lang/zh-cn/appendix/contribute)\n\n\n## Contributors\n\n\u003ca href=\"https://github.com/byzer-org/kolo-lang/graphs/contributors\"\u003e\n  \u003cimg src=\"https://contrib.rocks/image?repo=byzer-org/kolo-lang\" /\u003e\n\u003c/a\u003e\n\nMade with [contrib.rocks](https://contrib.rocks).\n\n##  WeChat Group\n\n扫码添加K小助微信号，添加成功后，发送  mlsql  这5个英文字母进群。\n\n![](https://github.com/allwefantasy/mlsql/blob/master/images/dc0f4493-570f-4660-ab41-0e487b17a517.png)\n\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fallwefantasy%2Fmlsql","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fallwefantasy%2Fmlsql","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fallwefantasy%2Fmlsql/lists"}