https://github.com/treasure-data/td-jdbc
JDBC Driver for Treasure Data
https://github.com/treasure-data/td-jdbc
Last synced: about 1 year ago
JSON representation
JDBC Driver for Treasure Data
- Host: GitHub
- URL: https://github.com/treasure-data/td-jdbc
- Owner: treasure-data
- License: apache-2.0
- Created: 2012-03-13T08:21:47.000Z (over 14 years ago)
- Default Branch: master
- Last Pushed: 2024-05-01T05:22:35.000Z (about 2 years ago)
- Last Synced: 2024-05-01T06:30:24.481Z (about 2 years ago)
- Language: Java
- Homepage: http://docs.treasure-data.com/articles/jdbc-driver
- Size: 2.9 MB
- Stars: 11
- Watchers: 82
- Forks: 12
- Open Issues: 19
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGES.txt
- License: LICENSE.txt
- Codeowners: .github/CODEOWNERS
- Security: SECURITY.md
Awesome Lists containing this project
README
# Treasure Data JDBC Driver
__NOTE__: We no longer maintain td-jdbc driver for Presto now that [we support native trino-jdbc drivers](https://trino.io/docs/current/client/jdbc.html)
JDBC Driver for accessing [Treasure Data](http://www.treasuredata.com). This works with Java 1.6 or higher.
- [Download](https://repo1.maven.org/maven2/com/treasuredata/td-jdbc/) td-jdbc-jar-with-dependencies-(version).jar
- Use driver class name: `com.treasuredata.jdbc.TreasureDataDriver`
- Read [Quick Start](#quick-start) or [General Usage](http://docs.treasure-data.com/articles/jdbc-driver)
td-jdbc internally uses [`td-client-java`](https://github.com/treasure-data/td-client-java), a Java client for Treasure Data.
## For Maven Users
[](https://maven-badges.herokuapp.com/maven-central/com.treasuredata/td-jdbc/)
[](http://www.javadoc.io/doc/com.treasuredata/td-jdbc)
```
com.treasuredata
td-jdbc
(version)
```
# Quick Start
Create a `java.sql.Connection` object using JDBC address `jdbc:td://api.treasuredata.com/(database name)`.
* [JDBC connection example](src/test/java/com/treasuredata/jdbc/Example.java)
```java
Properties props = new Properties();
props.setProperty("user", "(your account e-mail)");
props.setProperty("password", "(your password)");
// Alternatively, you can use API key instead of user and password
// props.setProperty("apikey", "(your API key)")
// Set the other options
// Use SSL (default) or not
// props.setProperty("useSSL", "true");
// Run Hive jobs. The default is "presto"
// props.setProperty("type", "hive");
// proxy configurarion (optional)
// props.setProperty("httpproxyhost", "(proxy host)");
// props.setProperty("httpproxyport", "(proxy port)");
// props.setProperty("httpproxyuser", "(proxy username)");
// props.setProperty("httpproxypassword", "(proxy password)");
Connection conn = DriverManager.getConnection("jdbc:td://api.treasuredata.com/sample_datasets", props);
Statement st = conn.createStatement();
try {
ResultSet rs = st.executeQuery("SELECT count(1) FROM www_access");
// You can see the job ID of the query
TDResultSetMetaData rsmd = (TDResultSetMetaData) rs.getResultSetMetaData();
System.out.println("job id: " + rsmd.getJobId());
// Getting the result rows
while (rs.next()) {
int count = rs.getInt(1);
System.out.println("result = " + count);
}
rs.close();
}
finally {
st.close();
conn.close();
}
```
To configure td-jdbc connection parameters, use URL parameters, Properties object or System properties. The precedence of these properties is:
1. Environment variable (only for TD_API_KEY parameter)
1. System properties
1. Properties object passed by `DriverManager.getConnection(jdbc_url, Properties)`
1. JDBC URL parameters (e.g., `jdbc:td://api.treasuredata.com/mydb;type=hive;useSSL=true`), separated by semi-colon `;`
If your environment defines TD_API_KEY variable, td-jdbc uses it. For the other properties, System properties have the highest priority.
## A list of JDBC Configurations
You must provide `apikey` property or both `user` (your account e-mail) and `password` for the authentication:
|key | default value | description |
|--------|---------------|-------------|
|`apikey` | | API key to access Treasure Data. You can also set this via TD_API_KEY environment variable. |
|`user` | | Account e-mail address (unnecessary if `apikey` is set) |
|`password`| | Account password (unnecessary if `apikey` is set) |
|`type` | presto | Query engine. hive, preto or pig |
|`useSSL` | false | Use SSL encryption for accessing Treasure Data |
|`useapikey` | false | Use API key to access Treasure Data even if `user` or `password` is set |
|`httpproxyhost` | | Proxy host (optional) e.g., "myproxy.com" |
|`httpproxyport`| | Proxy port (optional) e.g., "80" |
|`httpproxyuser` | | Proxy user (optional) |
|`httpproxypassword` | | Proxy password (optional) |
If both `user` and `password` are given, td-jdbc uses this pair instead of `apikey`.
You can also use [td-client-java specific options](https://github.com/treasure-data/td-client-java/blob/master/README.md#configuration).
# Internals
When running a query (e.g. SELECT), the driver submits a job request to
Treasure Data. td-jdbc periodically monitors the job progress and fetches the
result after the job completion.
For INSERT statement, td-jdbc buffers the data into local memory,
then flushes it to Treasure Data every 5 minutes, so there will be a delay
until your data becomes accessible in Treasure Data.
## Implementation Status
Following methods have been implemented.
### java.sql.Connection
* createStatement() and createStatement(..)
* getMetaData()
* prepareStatement(..)
### java.sql.Statement
* execute(..)
* executeQuery(..)
* setResultSet()
### java.sql.PreparedStatement
* addBatch()
* clearParameters()
* execute()
* executeQuery()
* getMetaData()
* setBoolean(..)
* setByte(..)
* setDouble(..)
* setFloat(..)
* setInt(..)
* setLong(..)
* setShort(..)
* setString(..)
### java.sql.ResultSet
* findColumn(String)
* getBoolean(..)
* getByte(..)
* getDate(int) and getDate(String)
* getDouble(int)
* getFloat(int)
* getInt(int)
* getLong(int)
* getMetaData()
* next()
* getObject(int) and getObject(String)
* getShort(..)
### java.sql.ResultSetMetaData
* getColumnCount()
* getColumnDisplaySize(int)
* getColumnLabel(int)
* getColumnName(int)
* getColumnType(int)
* getColumnTypeName(int)
### java.sql.DatabaseMetaData
* getCatalogs()
* getColumns(..)
* getSchemas(..)
* getTableTypes()
* getTables(..)
## License
Apache License, Version 2.0
## For developers
### Building
You can get latest source code using git.
$ git clone git@github.com:treasure-data/td-jdbc.git
$ cd td-jdbc
$ mvn package
You will get the td-jdbc jar file in `td-jdbc/target` folder
The file name will be `td-jdbc-${jdbc.version}-jar-with-dependencies.jar`.
See the [pom.xml file](https://github.com/treasure-data/td-jdbc/blob/master/pom.xml)
for more details.
To run production tests, write your apikey to `$HOME/.td/td.conf`:
```
[account]
user = (e-mail address)
apikey = (apikey)
password = (password)
```
### Buidling td-jdbc with JDK7 or higher
jdbc-api-4.1.jar, which is contained in mvn-local, is necessary to build td-jdbc using an older version (4.1) of JDBC API.
#### Building jdbc-api-4.1.jar on Mac OS X
- Install jdk6 https://support.apple.com/kb/DL1572?locale=en_US
```
$ jar xvf jar xvf /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home/bundle/Classes/classes.jar java/sql javax/sql
$ jar cvf jdbc-api-4.1.jar java javax
$ mvn deploy:deploy-file -Durl=file://(path to td-jdbc folder)/mvn-local -Dfile=jdbc-api-4.1.jar -DgroupId=com.treasuredata.thirdparty -DartifactId=jdbc-api -Dpackaging=jar -Dversion=4.1
``
### Publish to Sonatype
The following operations basically follows [this document](https://central.sonatype.org/pages/apache-maven.html)
- Setup your Sonatype account and GPG key
- Prepare `~/.m2/settings.xml` like this
```
sonatype-nexus-staging
$(your oss.sonatype.org account)
**********
```
- Execute these commands
```
$ mvn release:clean release:prepare
$ mvn release:perform
```