https://github.com/tidb-incubator/tibigdata
TiDB connectors for Flink/Hive/Presto
https://github.com/tidb-incubator/tibigdata
bigdata cdc flink hive presto tidb tikv
Last synced: 6 months ago
JSON representation
TiDB connectors for Flink/Hive/Presto
- Host: GitHub
- URL: https://github.com/tidb-incubator/tibigdata
- Owner: tidb-incubator
- License: apache-2.0
- Created: 2020-06-08T08:51:05.000Z (about 6 years ago)
- Default Branch: master
- Last Pushed: 2024-04-12T07:13:32.000Z (about 2 years ago)
- Last Synced: 2025-08-02T09:27:29.657Z (11 months ago)
- Topics: bigdata, cdc, flink, hive, presto, tidb, tikv
- Language: Java
- Homepage:
- Size: 3.04 MB
- Stars: 219
- Watchers: 15
- Forks: 57
- Open Issues: 29
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# TiBigData
[License](https://github.com/pingcap-incubator/TiBigData/blob/master/LICENSE)
---
[](README.md)
[](README_zh_CN.md)
---
Misc BigData components for TiDB, Presto, Flink and MapReduce connectors for example.
## License
TiBigData project is under the Apache 2.0 license. See the [LICENSE](./LICENSE) file for details.
## Code style
TiBigData use [Google Code Style](https://github.com/google/styleguide).
Format the code with the following command:
```bash
mvn com.coveo:fmt-maven-plugin:format
```
## Getting Started
**Compiling TiBigData requires git and downloading source code directly is not supported.**
[Flink-TiDB-Connector(Batch)](./flink/README.md)
[Flink-TiDB-Connector(Unified Batch & Streaming)](./flink/README_unified_batch_streaming.md)
[PrestoSQL-TiDB-Connector](./prestosql/README.md)
[Trino-TiDB-Connector](./trino/README.md)
[PrestoDB-TiDB-Connector](./prestodb/README.md)
[MapReduce-TiDB-Connector](./mapreduce/README.md)
[Hive-TiDB-Storage-Handler](./hive/README.md)
## Run Tests
Use the following command to run the integration test.
```bash
export TIDB_HOST="127.0.0.1"
export TIDB_PORT="4000"
export TIDB_USER="root"
export TIDB_PASSWORD=""
mvn clean test-compile failsafe:integration-test -am -pl ${MODULE_NAME}
```
Use the following command to run the unit test.
```
mvn clean test -am -pl ${MODULE_NAME}
```
## Write integration test
First, make sure the module add dependency:
```xml
io.tidb
bigdata-test
${project.version}
test
```
Second, use JUnit Category to clarify integration-test:
```java
import io.tidb.bigdata.test.IntegrationTest;
@Category(IntegrationTest.class)
public class ConnectorsPluginTest {
}
```
## Community
Lark / Feishu Group:
## Supported by
### YourKit
YourKit supports open source projects with innovative and intelligent tools
for monitoring and profiling Java and .NET applications.
YourKit is the creator of YourKit Java Profiler,
YourKit .NET Profiler,
and YourKit YouMonitor.
### IntelliJ IDEA
[IntelliJ IDEA](https://www.jetbrains.com/?from=TiBigData) is a Java integrated development environment (IDE) for developing computer software.
It is developed by JetBrains (formerly known as IntelliJ), and is available as an Apache 2 Licensed community edition,
and in a proprietary commercial edition. Both can be used for commercial development.