{"id":13487683,"url":"https://github.com/Qihoo360/Quicksql","last_synced_at":"2025-03-27T23:31:27.823Z","repository":{"id":37285382,"uuid":"164091262","full_name":"Qihoo360/Quicksql","owner":"Qihoo360","description":"A Flexible, Fast, Federated(3F) SQL Analysis Middleware for Multiple Data Sources","archived":false,"fork":false,"pushed_at":"2022-10-25T23:52:06.000Z","size":12393,"stargazers_count":2056,"open_issues_count":89,"forks_count":583,"subscribers_count":123,"default_branch":"master","last_synced_at":"2024-10-29T17:27:03.765Z","etag":null,"topics":["flink","hive","spark","sql"],"latest_commit_sha":null,"homepage":"https://quicksql.readthedocs.io","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Qihoo360.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-01-04T10:06:25.000Z","updated_at":"2024-10-24T10:16:34.000Z","dependencies_parsed_at":"2023-01-19T11:00:19.887Z","dependency_job_id":null,"html_url":"https://github.com/Qihoo360/Quicksql","commit_stats":null,"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Qihoo360%2FQuicksql","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Qihoo360%2FQuicksql/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Qihoo360%2FQuicksql/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Qihoo360%2FQuicksql/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Qihoo360","download_url":"https://codeload.github.com/Qihoo360/Quicksql/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245944037,"owners_count":20697944,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["flink","hive","spark","sql"],"created_at":"2024-07-31T18:01:02.027Z","updated_at":"2025-03-27T23:31:26.121Z","avatar_url":"https://github.com/Qihoo360.png","language":"Java","funding_links":[],"categories":["Java","Integrations","数据库管理系统","大数据"],"sub_categories":["Unsorted","网络服务_其他"],"readme":"[English](./README_doc.md)|[中文](./docs/zh/index.md)\r\n\r\n![200_200](docs/images/logo.jpeg)\r\n\r\n![Language](https://img.shields.io/github/languages/top/qihoo360/quicksql?style=flat)\r\n![Release Version](https://img.shields.io/github/v/release/Qihoo360/quicksql)\r\n![license](https://img.shields.io/badge/license-MIT-blue.svg?style=flat)\r\n![Documentation Status](https://readthedocs.org/projects/quicksql/badge/?version=latest)\r\n![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg)\r\n\r\nQuicksql is a SQL query product which can be used for specific datastore queries or multiple datastores correlated queries.  It supports relational databases, non-relational databases and even datastore which does not support SQL (such as Elasticsearch, Druid) . In addition, a SQL query can join or union data from multiple datastores in Quicksql. For example, you can perform unified SQL query on one situation that a part of data stored on Elasticsearch, but the other part of data stored on Hive. The most important is that QSQL is not dependent on any intermediate compute engine, users only need to focus on data and unified SQL grammar to finished statistics and analysis.\r\n\r\n[![Star-History](docs/images/star-history.png)](https://star-history.t9t.io/#Qihoo360/Quicksql)\r\n\r\n### Architecture\r\n\r\nAn architecture diagram helps you access Quicksql more easily.\r\n\r\n![1540973404791](docs/images/p1.png)\r\n\r\nQSQL architecture consists of three layers：\r\n\r\n- Parsing  Layer: Used for parsing, validation, optimization of SQL  statements, splitting of mixed SQL and finally generating Query Plan;\r\n\r\n- Computing Layer: For routing query plan to a  specific execution plan, then interpreted to executable code for given  storage or engine(such as Elasticsearch JSON query or Hive HQL);\r\n\r\n- Storage Layer: For data prepared extraction and storage;\r\n\r\n### Basic Features\r\n\r\nIn the vast majority of cases, we expect to use a language for data analysis and don't want to consider things that are not related to data analysis, Quicksql is born for this.\r\n\r\nThe goal of Quicksql is to provide three functions: \r\n\r\n***1. Unify all structured data queries into a SQL grammar***\r\n\r\n- **Only Use SQL**\r\n\r\nIn Quicksql, you can query Elasticsearch like this:\r\n\r\n``````sql\r\nSELECT state, pop FROM geo_mapping WHERE state = 'CA' ORDER BY state\r\n``````\r\n\r\nEven an aggregation query:\r\n\r\n``````sql\r\nSELECT approx_count_distinct(city), state FROM geo_mapping GROUP BY state LIMIT 10\r\n``````\r\n\r\nYou won't be annoyed again because the brackets in the JSON query can't match ;)\r\n\r\n- **Eliminate Dialects**\r\n\r\nIn the past, the same semantic statement needs to be converted to a dialect for different engines, such as:\r\n\r\n```sql\r\nSELECT * FROM geo_mapping                       -- MySQL Dialect\r\nLIMIT 10 OFFSET 10                              \r\n```\r\n\r\n``````sql\r\nSELECT * FROM geo_mapping                       -- Oracle Dialect\r\nOFFSET 10 ROWS FETCH NEXT 10 ROWS ONLY          \r\n``````\r\n\r\nIn Quicksql, relational databases no longer have the concept of dialects. You can use the grammar of Quicksql to query any engine, just like this:\r\n\r\n```sql\r\nSELECT * FROM geo_mapping LIMIT 10 OFFSET 10    -- Run Anywhere\r\n```\r\n\r\n***2. Shield the isolation between different data sources***\r\n\r\nConsider a situation where you want to join tables that are in different engines or are not in the same cluster, you may be in trouble.\r\n\r\nHowever, in Quicksql, you can query like this:\r\n\r\n``````sql\r\nSELECT * FROM \r\n    (SELECT * FROM es_raw.profile AS profile    //index.tpye on Elasticsearch \r\n        WHERE note IS NOT NULL )AS es_profile\r\nINNER JOIN \r\n    (SELECT * FROM hive_db.employee AS emp  //database.table on Hive\r\n    INNER JOIN hive_db.action AS act    //database.table on Hive\r\n    ON emp.name = act.name) AS tmp \r\nON es_profile.prefer = tmp.prefer\r\n``````\r\n\r\n***3. Choose the most appropriate way to execute the query***\r\n\r\nA query involving multiple engines can be executed in a variety of ways. Quicksql wants to combine the advantages of each engine to find the most appropriate one. \r\n\r\n### Getting Started\r\n\r\nFor instructions on building Quicksql from source, see [Getting Started](./docs/reference/getting-started.md).\r\n\r\n### Reporting Issues\r\n\r\nIf you find any bugs or have any better suggestions, please file a [GitHub issue](https://github.com/Qihoo360/Quicksql/issues).\r\n\r\nAnd if the issue is approved, a label [QSQL-ID] will be added before the issue description by committer so that it can correspond to commit. Such as: \r\n\r\n```sql\r\n[QSQL-1002]: Views generated after splitting logical plan are redundant.\r\n```\r\n\r\n### Contributing\r\n\r\nWe welcome contributions.\r\n\r\nIf you are interested in Quicksql, you can download the source code from GitHub and execute the following maven command at the project root directory：\r\n\r\n```shell\r\nmvn -DskipTests clean package\r\n```\r\n\r\nIf you are planning to make a large contribution, talk to us first! It helps to agree on the general approach. Log a Issures on GitHub for your proposed feature.\r\n\r\nFork the GitHub repository, and create a branch for your feature.\r\n\r\nDevelop your feature and test cases, and make sure that `mvn install` succeeds. (Run extra tests if your change warrants it.)\r\n\r\nCommit your change to your branch.\r\n\r\nIf your change had multiple commits, use `git rebase -i master` to squash them into a single commit, and to bring your code up to date with the latest on the main line.\r\n\r\nThen push your commit(s) to GitHub, and create a pull request from your branch to the QSQL master branch. Update the JIRA case to reference your pull request, and a committer will review your changes.\r\n\r\nThe pull request may need to be updated (after its submission) for two main reasons:\r\n\r\n1. you identified a problem after the submission of the pull request;\r\n2. the reviewer requested further changes;\r\n\r\nIn order to update the pull request, you need to commit the changes in your branch and then push the commit(s) to GitHub. You are encouraged to use regular (non-rebased) commits on top of previously existing ones.\r\n\r\n## Join us\r\n\r\n[![Slack](docs/images/slack.png)](https://join.slack.com/t/quicksql/shared_invite/enQtODkwMzM0Njc3NTExLWQxNjRlY2M5YTlkMTk4OTM2YzhjMjUxYTUyN2VlNzJlNzQwM2E4YjkxNzA4MDllODg5NWUxNDY4MTMyMzczMWI)\r\n[![Github](docs/images/github.png)](https://github.com/qihoo360/Quicksql/issues)\r\n[![QQ](docs/images/qq.png)](https://jq.qq.com/?_wv=1027\u0026k=5782R6F)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FQihoo360%2FQuicksql","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FQihoo360%2FQuicksql","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FQihoo360%2FQuicksql/lists"}