{"id":13958835,"url":"https://github.com/byzer-org/byzer-lang","last_synced_at":"2025-05-15T03:08:07.575Z","repository":{"id":38238888,"uuid":"57278919","full_name":"byzer-org/byzer-lang","owner":"byzer-org","description":"Byzer (former MLSQL): A low-code open-source programming language for data pipeline, analytics and AI.","archived":false,"fork":false,"pushed_at":"2024-05-29T08:24:09.000Z","size":57315,"stargazers_count":1839,"open_issues_count":120,"forks_count":545,"subscribers_count":115,"default_branch":"master","last_synced_at":"2025-04-14T03:09:34.007Z","etag":null,"topics":["bigdata","machine-learning","mlsql","sql-like-dsl"],"latest_commit_sha":null,"homepage":"https://www.byzer.org","language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/byzer-org.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-04-28T07:06:14.000Z","updated_at":"2025-04-14T01:41:33.000Z","dependencies_parsed_at":"2023-02-19T05:16:08.846Z","dependency_job_id":"eef34d74-eb55-411c-829b-52ec8fb71cce","html_url":"https://github.com/byzer-org/byzer-lang","commit_stats":null,"previous_names":["allwefantasy/streamingpro"],"tags_count":32,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/byzer-org%2Fbyzer-lang","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/byzer-org%2Fbyzer-lang/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/byzer-org%2Fbyzer-lang/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/byzer-org%2Fbyzer-lang/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/byzer-org","download_url":"https://codeload.github.com/byzer-org/byzer-lang/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254264769,"owners_count":22041794,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bigdata","machine-learning","mlsql","sql-like-dsl"],"created_at":"2024-08-08T13:01:54.085Z","updated_at":"2025-05-15T03:08:02.565Z","avatar_url":"https://github.com/byzer-org.png","language":"Scala","funding_links":[],"categories":["数据库管理系统","人工智能"],"sub_categories":["网络服务_其他"],"readme":" ![CI](https://github.com/byzer-org/byzer-lang/actions/workflows/build_and_test.yml/badge.svg)   [![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)\n\n\u003cp align=\"center\"\u003e\n    \u003cimg src=\"/images/Byzer_Logo.png\" alt=\"drawing\"  width=\"200\"/\u003e\n\u003c/p\u003e\n\n\n## Byzer-lang\n\n**Byzer** (former MLSQL) is a low-code, open-sourced and distributed programming language for data pipeline, analytics and AI in cloud native way.\n\n**Deisgn protocol: Everything is a table.** Byzer is a SQL-like language, to simplify data pipeline, analytics and AI, combined with built-in algorithms and extensions.\n\nWe believe that everything is a table, a simple and powerful SQL-like language can significantly reduce human efforts of data development without switching different tools.\n\n### Byzer Architecture\n\n![Byzer-lang Arch](images/Byzer-arch.png)\n\nYou can build a data product based on Byzer engine \u0026 Byzer-lang without interacting with computing framework directly like Spark in your data APP. Thus will simplify your data app significantly. \n\nFor example, Byzer org contributes a data app [Byzer Notebook](https://github.com/byzer-org/byzer-notebook), which provides notebook interaction \u0026 workflow GUI interaction.\n\n### BIP (Byzer Improvement Proposal)\n\nByzer project uses the [BIP](https://github.com/byzer-org/byzer-lang/wiki) for the community  collaboration, you can checkout the feature design or architecture design in BIP.\n\n### Online Trial\n\nYou can access the official website [https://www.byzer.org/](https://www.byzer.org/) and try Byzer-lang \u0026 Byzer Notebook online.\n\n\n### Download \n\nYou can download Byzer engine via:\n- [https://download.byzer.org/](https://download.byzer.org/)\n- [Byzer Docker Hub](https://hub.docker.com/u/byzer)\n- [Github Release](https://github.com/byzer-org/byzer-lang/releases)\n\nFor more details, please refer to the [docs](https://docs.byzer.org/#/byzer-lang/zh-cn/installation/README)\n\n### Install\n\nFor **dev/test** purpose, you can download [Byzer All In One Package](https://docs.byzer.org/#/byzer-lang/zh-cn/installation/server/byzer-all-in-one-deployment), extract and then execute the command below\n\n```\n$ cd {BYZER_HOME}\n$ ./bin/byzer.sh start\n```\n\n\nAnd for **production** purpose, we recommend to use [Byzer Server Package](https://docs.byzer.org/#/byzer-lang/zh-cn/installation/server/binary-installation) and deploy it on Hadoop.\n\n\nYou can also install [Byzer VSCode Extension](https://docs.byzer.org/#/byzer-lang/zh-cn/installation/vscode/byzer-vscode-extension-installation) to use Byzer-lang. \n\nFor the Docker Image or , please refer to the [docs](https://docs.byzer.org/#/byzer-lang/zh-cn/installation/README)\n\n\n\n### Byzer Code Example\n\nBelow list an example that how to process Github API as a table to get the information of Byzer Org\n\n```sql\n-- Get Github Organization Info\n\n-- set API URL and params\nSET org_name=\"byzer-org\";\nSET GITHUB_ORGANIZATION_URL=\"https://api.github.com/orgs/${org_name}\";\n\n-- Load Github Organization API as table\nLOAD Rest.`$GITHUB_ORGANIZATION_URL` \n where `config.connect-timeout`=\"10s\"\n and `config.method`=\"GET\"\n and `header.accept`=\"application/vnd.github.v3+json\"\nas github_org;\n\n\n-- decode API response from binary to a json string\nselect string(content) as content from github_org as response_content;\n\n-- expand the json string \nrun response_content as JsonExpandExt.`` where inputCol=\"content\" and structColumn=\"true\" as github_org;\n\n-- retrieve user infomation and process as a table\nselect content.* from github_org as org_info;\n\n-- save the table to delta lake\nsave overwrite org_info as delta.`github_info_db.byzer_org`;\n```\n\n\nFor more details about the Byzer-lang grammer, please refer to the user manaual [Byzer-lang Grammar](https://docs.byzer.org/#/byzer-lang/zh-cn/grammar/outline)\n\n### Development\n\n1. Fork this repository and clone to local machine\n\n```\ngit clone https://github.com/{YOUR_GITHUB}/byzer-lang.git\n```\n\n2. Use Intellj Idea to open the project, choose the scala version `2.12.10`\n\n3. In Intellj Idea Maven Setting, check the profile below\n    - gpg\n    - local\n    - scala-2.12\n    - spark-3.0.0\n    - streamingpro-spark-3.0.0-adaptor\n\n4. Click Maven Refresh and wait for Idea load finished\n\n5. Find the class `tech.mlsql.example.app.LocalSparkServiceApp`, click Debug button then Byzer Engine will be started, then you can access the Byzer Web Console in [http://localhost:9003/](http://localhost:9003/#/)\n\n### Build\n\nYou can refer to the project [byzer-org/byzer-build](https://github.com/byzer-org/byzer-build) to check how to build the Byzer engine binary packages and images\n\n### How to contribute to Byzer-Lang\n\nIf you are planning to contribute to this project, please create an issue at [our Issue page](https://github.com/byzer-org/byzer-lang/issues)\neven if the topic is not related to source code itself (e.g., documentation, new idea and proposal).\n\nThis is an active open source project for everyone,\nand we are always open to people who want to use this system or contribute to it.\n\nFor more details about how to contribute to the Byzer Org, please refer to [How to Contribute](https://docs.byzer.org/#/byzer-lang/zh-cn/appendix/contribute)\n\n\n### Contributors\n\n\u003ca href=\"https://github.com/byzer-org/byzer-lang/graphs/contributors\"\u003e\n  \u003cimg src=\"https://contrib.rocks/image?repo=byzer-org/byzer-lang\" /\u003e\n\u003c/a\u003e\n\nMade with [contrib.rocks](https://contrib.rocks).\n\n###  Community\n\n- **Slack**: [byzer-org.slack.com](https://byzer-org.slack.com)\n- **Wechat Official Account:** Byzer Community\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbyzer-org%2Fbyzer-lang","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbyzer-org%2Fbyzer-lang","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbyzer-org%2Fbyzer-lang/lists"}