{"id":44456005,"url":"https://github.com/databendcloud/flink-connector-databend","last_synced_at":"2026-02-12T17:35:54.514Z","repository":{"id":109889310,"uuid":"603274419","full_name":"databendcloud/flink-connector-databend","owner":"databendcloud","description":"Flink SQL connector for Databend.","archived":false,"fork":false,"pushed_at":"2025-08-22T14:06:53.000Z","size":161,"stargazers_count":4,"open_issues_count":9,"forks_count":1,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-08-22T14:21:10.200Z","etag":null,"topics":["databend","flink","flinkcdc"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/databendcloud.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-02-18T02:19:58.000Z","updated_at":"2025-08-22T12:05:49.000Z","dependencies_parsed_at":"2023-11-28T11:24:26.449Z","dependency_job_id":"8cd9d0c6-012a-46f8-99c4-a57225796167","html_url":"https://github.com/databendcloud/flink-connector-databend","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/databendcloud/flink-connector-databend","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/databendcloud%2Fflink-connector-databend","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/databendcloud%2Fflink-connector-databend/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/databendcloud%2Fflink-connector-databend/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/databendcloud%2Fflink-connector-databend/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/databendcloud","download_url":"https://codeload.github.com/databendcloud/flink-connector-databend/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/databendcloud%2Fflink-connector-databend/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29374148,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-12T08:51:36.827Z","status":"ssl_error","status_checked_at":"2026-02-12T08:51:26.849Z","response_time":55,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["databend","flink","flinkcdc"],"created_at":"2026-02-12T17:35:53.480Z","updated_at":"2026-02-12T17:35:54.507Z","avatar_url":"https://github.com/databendcloud.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Flink Databend Connector\n\n[Flink](https://github.com/apache/flink) SQL connector\nfor [Databend](https://github.com/datafuselabs/databend) database, this project Powered\nby [Databend JDBC](https://github.com/databendcloud/databend-jdbc).\n\nCurrently, the project supports `Sink Table`.  \nPlease create issues if you encounter bugs and any help for the project is greatly appreciated.\n\n## Connector Options\n\n| Option               | Required | Default | Type     | Description                                                                                                              |\n|:---------------------|:---------|:--------|:---------|:-------------------------------------------------------------------------------------------------------------------------|\n| url                  | required | none    | String   | The Databend jdbc url in format `databend://\u003chost\u003e:\u003cport\u003e`.                                                              |\n| username             | optional | none    | String   | The 'username' and 'password' must both be specified if any of them is specified.                                        |\n| password             | optional | none    | String   | The Databend password.                                                                                                   |\n| database-name        | optional | default | String   | The Databend database name.                                                                                              |\n| table-name           | required | none    | String   | The Databend table name.                                                                                                 |\n| sink.batch-size      | optional | 1000    | Integer  | The max flush size, over this will flush data.                                                                           |\n| sink.flush-interval  | optional | 1s      | Duration | Over this flush interval mills, asynchronous threads will flush data.                                                    |\n| sink.max-retries     | optional | 3       | Integer  | The max retry times when writing records to the database failed.                                                         |\n| sink.update-strategy | optional | update  | String   | Convert a record of type UPDATE_AFTER to update/insert statement or just discard it, available: update, insert, discard. |\n| sink.ignore-delete   | optional | true    | String   | handle DELETE event or not                                                                                               |\n| sink.primary-key     | optional | \"id\"    | String   | The primary key used in upsert                                                                                           |\n\n**NOTE**: `sink.ignore-delete` default value is true so this connector doesn't support `DELETE` default. If you make `sink.ignore-delete=false` make sure the source database has an `Integer` or `String` type primary key.\n\n**Upsert Data Considerations:**\n\n## Data Type Mapping\n\n| Flink Type          | Databend Type                                          |\n|:--------------------|:-------------------------------------------------------|\n| CHAR                | String                                                 |\n| VARCHAR             | String                                                 |\n| STRING              | String                                                 |\n| BOOLEAN             | Boolean                                                |\n| BYTES               | String                                                 |\n| DECIMAL             | Decimal / Int128 / Int256 / UInt64 / UInt128 / UInt256 |\n| TINYINT             | Int8                                                   |\n| SMALLINT            | Int16 / UInt8                                          |\n| INTEGER             | Int32 / UInt16 / Interval                              |\n| BIGINT              | Int64 / UInt32                                         |\n| FLOAT               | Float                                                  |\n| DOUBLE              | Double                                                 |\n| DATE                | Date                                                   |\n| TIME                | DateTime                                               |\n| TIMESTAMP           | DateTime                                               |\n| TIMESTAMP_LTZ       | DateTime                                               |\n| INTERVAL_YEAR_MONTH | Int32                                                  |\n| INTERVAL_DAY_TIME   | Int64                                                  |\n| ARRAY               | Array                                                  |\n| MAP                 | Map                                                    |\n| ROW                 | Not supported                                          |\n| MULTISET            | Not supported                                          |\n| RAW                 | Not supported                                          |\n\n## Maven Dependency\n\nThe project isn't published to the maven central repository, we need to deploy/install to our own\nrepository before use it, step as follows:\n\n```bash\n# clone the project\ngit clone https://github.com/databendcloud/flink-connector-databend.git\n\n# enter the project directory\ncd flink-connector-databend/\n\n# display remote branches\ngit branch -r\n\n# checkout the branch you need\ngit checkout $branch_name\n\n# install or deploy the project to our own repository\nmvn clean install -DskipTests\nmvn clean deploy -DskipTests\n```\n\n```xml\n\n\u003cdependency\u003e\n    \u003cgroupId\u003eorg.apache.flink\u003c/groupId\u003e\n    \u003cartifactId\u003eflink-connector-databend\u003c/artifactId\u003e\n    \u003cversion\u003e1.0.0-SNAPSHOT\u003c/version\u003e\n\u003c/dependency\u003e\n```\n\n## How to use\n\n### Create and read/write table\n\n```SQL\n\n-- register a databend table `t_user` in flink sql.\nCREATE TABLE t_user\n(\n    `user_id`   BIGINT,\n    `user_type` INTEGER,\n    `language`  STRING,\n    `country`   STRING,\n    `gender`    STRING,\n    `score`     DOUBLE,\n    `list`      ARRAY\u003cSTRING\u003e,\n    `map`       Map\u003cSTRING,\n    BIGINT\u003e,\n    PRIMARY KEY (user_id)\n) WITH (\n      'connector' = 'databend',\n      'url' = 'databend://{ip}:{port}',\n      'database-name' = 'default',\n      'table-name' = 'users',\n      'sink.batch-size' = '500',\n      'sink.flush-interval' = '1000',\n      'sink.ignore-delete' = 'false',\n      'sink.max-retries' = '3'\n      );\n\n-- read data from databend \nSELECT user_id, user_type\nfrom t_user;\n\n-- write data into the databend table from the table `T`\nINSERT INTO t_user\nSELECT cast(`user_id` as BIGINT),\n       `user_type`,\n       `lang`,\n       `country`,\n       `gender`,\n       `score`,\n       ARRAY['CODER',\n       'SPORTSMAN'], CAST(MAP['BABA', cast(10 as BIGINT), 'NIO', cast(8 as BIGINT)] AS MAP\u003cSTRING, BIGINT\u003e)\nFROM T;\n\n```\n\n### Create and use DatabendCatalog\n\n#### Scala\n\n```scala\nval tEnv = TableEnvironment.create(setting)\n\nval props = new util.HashMap[String, String]()\nprops.put(DatabendConfig.DATABASE_NAME, \"default\")\nprops.put(DatabendConfig.URL, \"databend://127.0.0.1:8000\")\nprops.put(DatabendConfig.USERNAME, \"username\")\nprops.put(DatabendConfig.PASSWORD, \"password\")\nprops.put(DatabendConfig.SINK_FLUSH_INTERVAL, \"30s\")\nval cHcatalog = new DatabendConfig(\"databend\", props)\ntEnv.registerCatalog(\"databend\", datbendcatalog)\ntEnv.useCatalog(\"databend\")\n\ntEnv.executeSql(\"insert into `databend`.`default`.`t_table` select...\");\n```\n\n#### Java\n\n```java\nTableEnvironment tEnv=TableEnvironment.create(setting);\n\n        Map\u003cString, String\u003e props=new HashMap\u003c\u003e();\n        props.put(DatabendConfig.DATABASE_NAME,\"default\")\n        props.put(DatabendConfig.URL,\"databend://127.0.0.1:8000\")\n        props.put(DatabendConfig.USERNAME,\"username\")\n        props.put(DatabendConfig.PASSWORD,\"password\")\n        props.put(DatabendConfig.SINK_FLUSH_INTERVAL,\"30s\");\n        Catalog cHcatalog=new DatabendConfig(\"databend\",props);\n        tEnv.registerCatalog(\"databend\",databendcatalog);\n        tEnv.useCatalog(\"databend\");\n\n        tEnv.executeSql(\"insert into `databend`.`default`.`t_table` select...\");\n```\n\n## Roadmap\n\n- [x] Implement the Flink SQL Sink function.\n- [x] Support DatabendCatalog.\n- [x] Implement the Flink SQL Source function.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatabendcloud%2Fflink-connector-databend","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdatabendcloud%2Fflink-connector-databend","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatabendcloud%2Fflink-connector-databend/lists"}