{"id":33179062,"url":"https://github.com/51nb/marble","last_synced_at":"2026-01-14T01:55:00.581Z","repository":{"id":51614653,"uuid":"178313880","full_name":"51nb/marble","owner":"51nb","description":"A high performance in-memory hive sql engine based on Apache Calcite","archived":false,"fork":false,"pushed_at":"2022-06-25T03:41:49.000Z","size":1769,"stargazers_count":192,"open_issues_count":8,"forks_count":45,"subscribers_count":8,"default_branch":"master","last_synced_at":"2025-11-20T21:05:20.466Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/51nb.png","metadata":{"files":{"readme":"README.MD","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-03-29T02:01:34.000Z","updated_at":"2025-10-21T08:42:23.000Z","dependencies_parsed_at":"2022-08-22T11:50:43.059Z","dependency_job_id":null,"html_url":"https://github.com/51nb/marble","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/51nb/marble","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/51nb%2Fmarble","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/51nb%2Fmarble/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/51nb%2Fmarble/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/51nb%2Fmarble/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/51nb","download_url":"https://codeload.github.com/51nb/marble/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/51nb%2Fmarble/sbom","scorecard":{"id":4728,"data":{"date":"2025-08-11","repo":{"name":"github.com/51nb/marble","commit":"64c2de9934a2630cd151c9b43f85591855ffecfc"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":1.7,"checks":[{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Code-Review","score":0,"reason":"Found 1/25 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Pinned-Dependencies","score":-1,"reason":"no dependencies found","details":null,"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: Apache License 2.0: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 4 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}},{"name":"Vulnerabilities","score":0,"reason":"82 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: GHSA-22wj-vf5f-wrvj","Warn: Project is vulnerable to: GHSA-45hx-wfhj-473x","Warn: Project is vulnerable to: GHSA-7rpj-hg47-cx62","Warn: Project is vulnerable to: GHSA-h376-j262-vhq6","Warn: Project is vulnerable to: GHSA-h46c-h94j-95f3","Warn: Project is vulnerable to: GHSA-wf8f-6423-gfxg","Warn: Project is vulnerable to: GHSA-27xj-rqx5-2255","Warn: Project is vulnerable to: GHSA-288c-cq4h-88gq","Warn: Project is vulnerable to: GHSA-4w82-r329-3q67","Warn: Project is vulnerable to: GHSA-57j2-w4cx-62h2","Warn: Project is vulnerable to: GHSA-58pp-9c76-5625","Warn: Project is vulnerable to: GHSA-5949-rw7g-wx7w","Warn: Project is vulnerable to: GHSA-5p34-5m6p-p58g","Warn: Project is vulnerable to: GHSA-5r5r-6hpj-8gg9","Warn: Project is vulnerable to: GHSA-5ww9-j83m-q7qx","Warn: Project is vulnerable to: GHSA-6fpp-rgj9-8rwc","Warn: Project is vulnerable to: GHSA-758m-v56v-grj4","Warn: Project is vulnerable to: GHSA-85cw-hj65-qqv9","Warn: Project is vulnerable to: GHSA-89qr-369f-5m5x","Warn: Project is vulnerable to: GHSA-8c4j-34r4-xr8g","Warn: Project is vulnerable to: GHSA-8w26-6f25-cm9x","Warn: Project is vulnerable to: GHSA-95cm-88f5-f2c7","Warn: Project is vulnerable to: GHSA-9gph-22xh-8x98","Warn: Project is vulnerable to: GHSA-9m6f-7xcq-8vf8","Warn: Project is vulnerable to: GHSA-9vvp-fxw6-jcxr","Warn: Project is vulnerable to: GHSA-c265-37vj-cwcc","Warn: Project is vulnerable to: GHSA-c2q3-4qrh-fm48","Warn: Project is vulnerable to: GHSA-cf6r-3wgc-h863","Warn: Project is vulnerable to: GHSA-cmfg-87vq-g5g4","Warn: Project is vulnerable to: GHSA-cvm9-fjm9-3572","Warn: Project is vulnerable to: GHSA-f3j5-rmmp-3fc5","Warn: Project is vulnerable to: GHSA-f9xh-2qgp-cq57","Warn: Project is vulnerable to: GHSA-fmmc-742q-jg75","Warn: Project is vulnerable to: GHSA-fqwf-pjwf-7vqv","Warn: Project is vulnerable to: GHSA-gjmw-vf9h-g25v","Warn: Project is vulnerable to: GHSA-gwp4-hfv6-p7hw","Warn: Project is vulnerable to: GHSA-gww7-p5w4-wrfv","Warn: Project is vulnerable to: GHSA-h3cw-g4mq-c5x2","Warn: Project is vulnerable to: GHSA-h4rc-386g-6m85","Warn: Project is vulnerable to: GHSA-h822-r4r5-v8jg","Warn: Project is vulnerable to: GHSA-j823-4qch-3rgm","Warn: Project is vulnerable to: GHSA-jjjh-jjxp-wpff","Warn: Project is vulnerable to: GHSA-m6x4-97wx-4q27","Warn: Project is vulnerable to: GHSA-mc6h-4qgp-37qh","Warn: Project is vulnerable to: GHSA-mph4-vhrx-mv67","Warn: Project is vulnerable to: GHSA-mx7p-6679-8g3q","Warn: Project is vulnerable to: GHSA-p43x-xfjf-5jhr","Warn: Project is vulnerable to: GHSA-q93h-jc49-78gg","Warn: Project is vulnerable to: GHSA-qjw2-hr98-qgfh","Warn: Project is vulnerable to: GHSA-qmqc-x3r4-6v39","Warn: Project is vulnerable to: GHSA-r3gr-cxrf-hg25","Warn: Project is vulnerable to: GHSA-r695-7vr9-jgc2","Warn: Project is vulnerable to: GHSA-rf6r-2c4q-2vwg","Warn: Project is vulnerable to: GHSA-rgv9-q543-rqg4","Warn: Project is vulnerable to: GHSA-rpr3-cw39-3pxh","Warn: Project is vulnerable to: GHSA-v3xw-c963-f5hc","Warn: Project is vulnerable to: GHSA-v585-23hc-c647","Warn: Project is vulnerable to: GHSA-vfqx-33qm-g869","Warn: Project is vulnerable to: GHSA-wh8g-3j2c-rqj5","Warn: Project is vulnerable to: GHSA-5mg8-w23w-74h3","Warn: Project is vulnerable to: GHSA-7g45-4rm6-3mm3","Warn: Project is vulnerable to: GHSA-mvr2-9pj6-7w5j","Warn: Project is vulnerable to: GHSA-4gg5-vx3j-xwc7","Warn: Project is vulnerable to: GHSA-735f-pc8j-v9w8","Warn: Project is vulnerable to: GHSA-77rm-9x9h-xj3g","Warn: Project is vulnerable to: GHSA-g5ww-5jh7-63cx","Warn: Project is vulnerable to: GHSA-h4h5-3hr4-j3g2","Warn: Project is vulnerable to: GHSA-wrvw-hg22-4m67","Warn: Project is vulnerable to: GHSA-pfh2-hfmq-phg5","Warn: Project is vulnerable to: GHSA-j288-q9x7-2f5v","Warn: Project is vulnerable to: GHSA-493p-pfq6-5258","Warn: Project is vulnerable to: GHSA-v528-7hrm-frqp","Warn: Project is vulnerable to: GHSA-w7f5-jrpr-5c2m","Warn: Project is vulnerable to: GHSA-7r82-7xv7-xcpj","Warn: Project is vulnerable to: GHSA-3mc7-4q67-w48m","Warn: Project is vulnerable to: GHSA-98wm-3w3q-mw94","Warn: Project is vulnerable to: GHSA-9w3m-gqgf-c4p9","Warn: Project is vulnerable to: GHSA-c4r9-r8fh-9vj2","Warn: Project is vulnerable to: GHSA-hhhw-99gj-p3c3","Warn: Project is vulnerable to: GHSA-mjmj-j48q-9wg2","Warn: Project is vulnerable to: GHSA-rvwf-54qp-4r6v","Warn: Project is vulnerable to: GHSA-w37g-rhq8-7m4j"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}}]},"last_synced_at":"2025-08-14T13:18:48.233Z","repository_id":51614653,"created_at":"2025-08-14T13:18:48.234Z","updated_at":"2025-08-14T13:18:48.234Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28408692,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-14T00:40:43.272Z","status":"ssl_error","status_checked_at":"2026-01-14T00:40:42.636Z","response_time":56,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-11-16T03:00:36.818Z","updated_at":"2026-01-14T01:55:00.559Z","avatar_url":"https://github.com/51nb.png","language":"Java","funding_links":[],"categories":["大数据"],"sub_categories":[],"readme":"**Marble is a high performance in-memory hive sql engine based on [Apache Calcite](https://calcite.apache.org/).  \nIt can help you to migrate hive sql scripts to a real-time computing system.  \nIt also provides a convenient Table API to help you to build custom SQL engines.**\n\nYou may want another similar project: [direct-spark-sql](https://github.com/direct-spark-sql/direct-spark-sql)\n\n## Build and run tests\n**Requirements**\n\n* Java 1.8 as a build JDK\n* Maven\n\n\n1.build marble\n```$xslt\ncd marble\nmvn clean install -DskipTests\n```\n**(Optional)**  \nif you need modify the patches of Calcite, build [calcite-patch](https://github.com/51nb/calcite-patch) project first\n```$xslt\ngit clone https://github.com/51nb/calcite-patch.git\ncd calcite-patch\nmvn clean install -DskipTests\n```\nIn the long term,we hope to merge the patches to Calcite finally.\n\n\n2.import `marble` project into IDE, but **please don't import `calcite-patch` as a submodule of marble project**\n\n3.run the test `TableEnvTest` and `HiveTableEnvTest`\n\n## Usage\n**Maven dependency**\n```$xslt\n        \u003cdependency\u003e\n            \u003cgroupId\u003eorg.codehaus.janino\u003c/groupId\u003e\n            \u003cartifactId\u003ejanino\u003c/artifactId\u003e\n            \u003cversion\u003e3.0.11\u003c/version\u003e\n        \u003c/dependency\u003e\n        \u003cdependency\u003e\n            \u003cgroupId\u003eorg.codehaus.janino\u003c/groupId\u003e\n            \u003cartifactId\u003ecommons-compiler\u003c/artifactId\u003e\n            \u003cversion\u003e3.0.11\u003c/version\u003e\n        \u003c/dependency\u003e\n        \u003cdependency\u003e\n            \u003cgroupId\u003ecom.u51.marble\u003c/groupId\u003e\n            \u003cartifactId\u003emarble-table-hive\u003c/artifactId\u003e\n            \u003cversion\u003e1.0.0\u003c/version\u003e\n            \u003cexclusions\u003e\n                \u003cexclusion\u003e\n                    \u003cgroupId\u003eorg.apache.calcite\u003c/groupId\u003e\n                    \u003cartifactId\u003ecalcite-core\u003c/artifactId\u003e\n                \u003c/exclusion\u003e\n                \u003cexclusion\u003e\n                    \u003cgroupId\u003eorg.apache.calcite\u003c/groupId\u003e\n                    \u003cartifactId\u003ecalcite-linq4j\u003c/artifactId\u003e\n                \u003c/exclusion\u003e\n                \u003cexclusion\u003e\n                    \u003cgroupId\u003eorg.codehaus.janino\u003c/groupId\u003e\n                    \u003cartifactId\u003ejanino\u003c/artifactId\u003e\n                \u003c/exclusion\u003e\n                \u003cexclusion\u003e\n                    \u003cgroupId\u003eorg.codehaus.janino\u003c/groupId\u003e\n                    \u003cartifactId\u003ecommons-compiler\u003c/artifactId\u003e\n                \u003c/exclusion\u003e\n            \u003c/exclusions\u003e\n        \u003c/dependency\u003e\n```\n**API Overview**\n\n```$xslt\nTableEnv.enableSqlPlanCacheSize(200);\n\nTableEnv tableEnv = HiveTableEnv.getTableEnv();\n\nDataTable t1 = tableEnv.fromJavaPojoList(pojoList);\nDataTable t2 = tableEnv.fromJdbcResultSet(resultSet);\nDataTable t3=tableEnv.fromRowListWithSqlTypeMap(rowList,sqlTypeMap);\n\ntableEnv.addSubSchema(\"test\");\ntableEnv.registerTable(\"test\",\"t1\",t1);\ntableEnv.registerTable(\"test\",\"t2\", t2);\nDataTable queryResult = tableEnv.sqlQuery(\"select * from test.t1 join test.t2 on t1.id=t2.id\");\nList\u003cMap\u003cString, Object\u003e\u003e rowList=queryResult.toMapList();\n```\nIt's recommended to enable plan cache for the same sql query:\n```\nTableEnv.enableSqlPlanCacheSize(200);\n```\n\n`TableEnv` is the main table api to execute sql queries on a dataSet.  \nIt can be used to:\n* convert a java pojo List or jdbc ResultSet to a `DataTable`\n* register a `DataTable` in TableEnv's catalog\n* add subSchemas and customized functions in TableEnv's catalog\n* execute a sql query to get the result `DataTable`\n\nThe `TableEnv` supports Calcite's sql dialect by default,see it's [sql reference](https://calcite.apache.org/docs/reference.html).  \nThe goal of `HiveTableEnv` is to support hive sql as far as possible，developers can  aslo use\na `TableConfig` to create a new TableEnv to support other sql dialects(MysqlTableEnv,PostgreTableEnv ..etc).\n\n\n**Supported hive sql features**\n* specific keywords and operators\n* all of UDF,UDAF\n* part of UDTF\n* implicit type casting\n* load customized UDF,UDAF by package name\n  ```\n  HiveTableEnv.registerHiveFunctionPackages(\"com.u51.data.hive.udf\"); \n  ```\n  \n## Benchmark\nThere're some benchmark tests in the  `benchmark` module,it compares flink,spark and marble on some simple\nsql queries.\n## Design\nIt shows how marble customized calcite in the sql processing flow:\n![how_marble_customized_calcite](./how_marble_customized_calcite.jpg)  \nYou can find more details from calcite-patch's commit history.Now Marble uses calcite `1.18.0`.\n\nThe main type mapping between calcite and hive is:  \n\n| CalciteSqlType | JavaStorageType | HiveObjectInspector |\n| :---:        |     :---:      |     :---:    |\n| BIGINT   | Long    | LongObjectInspector    |\n| INTEGER     | Int      | IntObjectInspector    |\n| DOUBLE   | Double    | DoubleObjectInspectors    |\n| DECIMAL     | BigDecimal       | HiveDecimalObjectInspector      |\n| VARCHAR   | String     | StringObjectInspector    |\n| DATE     | Int      | DateObjectInspector      |\n| TIMESTAMP     | Long       | TimestampObjectInspector     |\n| ARRAY     | List      | StandardListObjectInspector     |\n| ......     | ......      | ......     |\n\n\n\n\n## Roadmap\n*  improve compatibility with hive sql.(high priority)\n*  submit patches to Calcite,make it easy to upgrade calcite-core,\nsome related issues:[CALCITE-2282](https://issues.apache.org/jira/browse/CALCITE-2282),[CALCITE-2973](https://issues.apache.org/jira/browse/CALCITE-2973),[CALCITE-2992](https://issues.apache.org/jira/browse/CALCITE-2992).(high priority)\n*  implements UDTF in a generic way.(high priority)\n*  constant folded for hive udf.(low priority)\n*  use a customized sql Planner to replace the default PlannerImpl.(low priority)\n*  TPC-DS queries with a customized scale.(low priority)\n*  vectorized udf execution.(experimental)\n*  distributed broadcast join.(experimental)\n*  cost based optimizer.(experimental)  \n\nMore issues see [issues](https://github.com/51nb/marble/milestone/1).\n\n## Contributing\nWelcome contributions.\nPlease use the Calcite-idea-code-style.xml under the marble directory to reformat code,\nand ensure that the validation of maven checker-style plugin is success after source code building.\n\n## License\nThis library is distributed under terms of Apache 2 License\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F51nb%2Fmarble","html_url":"https://awesome.ecosyste.ms/projects/github.com%2F51nb%2Fmarble","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F51nb%2Fmarble/lists"}