{"id":27627206,"url":"https://github.com/treasure-data/td-client-java","last_synced_at":"2025-04-23T13:53:32.836Z","repository":{"id":2636414,"uuid":"3625270","full_name":"treasure-data/td-client-java","owner":"treasure-data","description":"Java Client Library for Treasure Data","archived":false,"fork":false,"pushed_at":"2025-03-19T09:28:42.000Z","size":2037,"stargazers_count":12,"open_issues_count":16,"forks_count":22,"subscribers_count":76,"default_branch":"master","last_synced_at":"2025-03-19T10:23:52.835Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"http://docs.treasuredata.com/articles/java","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/treasure-data.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGES.txt","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":"AUTHORS.txt","dei":null,"publiccode":null,"codemeta":null}},"created_at":"2012-03-05T08:45:22.000Z","updated_at":"2024-12-18T13:25:01.000Z","dependencies_parsed_at":"2023-07-06T06:03:59.593Z","dependency_job_id":"6603aedd-1ca5-4262-824e-665c0434c9dd","html_url":"https://github.com/treasure-data/td-client-java","commit_stats":null,"previous_names":[],"tags_count":114,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/treasure-data%2Ftd-client-java","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/treasure-data%2Ftd-client-java/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/treasure-data%2Ftd-client-java/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/treasure-data%2Ftd-client-java/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/treasure-data","download_url":"https://codeload.github.com/treasure-data/td-client-java/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250444206,"owners_count":21431604,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-04-23T13:53:31.968Z","updated_at":"2025-04-23T13:53:32.819Z","avatar_url":"https://github.com/treasure-data.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![CircleCI](https://dl.circleci.com/status-badge/img/gh/treasure-data/td-client-java/tree/master.svg?style=svg\u0026circle-token=18039f1bcb525695f8f863d9786592c3dacf4541)](https://dl.circleci.com/status-badge/redirect/gh/treasure-data/td-client-java/tree/master)\n\n# td-client for java\n\nA java client for accessing [Treasure Data API](https://api-docs.treasuredata.com/en/api/td-api/quickstart/).\nWith this client, you can:\n - submit Hive/Presto queries to Treasure Data\n - check the status of jobs (queries)\n - retrieve query results\n - check the information of databases and tables\n\nSince td-client-java 0.8.0, it requires Java 1.8 or higher. For Java7, use td-client-java-0.7.x.\n\ntd-client-java is licensed under Apache License Version 2.0.\n\n## Download\n\nYou can download a jar file (td-client-java-(version)-shade.jar) from here: https://repo1.maven.org/maven2/com/treasuredata/client/td-client/\n\nFor the information of the older versions, see \u003chttps://github.com/treasure-data/td-client-java/tree/0.7.x\u003e.\n\n- [Release Notes](CHANGES.txt)\n\n### For Maven Users\n\n[![Maven Central](https://maven-badges.herokuapp.com/maven-central/com.treasuredata.client/td-client/badge.svg)](https://maven-badges.herokuapp.com/maven-central/com.treasuredata.client/td-client/)\n[![javadoc](https://javadoc.io/badge2/com.treasuredata.client/td-client/javadoc.svg)](https://javadoc.io/doc/com.treasuredata.client/td-client)\n\nUse the following dependency settings:\n\n```\n\u003cdependency\u003e\n  \u003cgroupId\u003ecom.treasuredata.client\u003c/groupId\u003e\n  \u003cartifactId\u003etd-client\u003c/artifactId\u003e\n  \u003cversion\u003e(version)\u003c/version\u003e\n\u003c/dependency\u003e\n\n\u003c!-- If you are not using any slf4 logger binder, add the following dependency, too. --\u003e\n\u003cdependency\u003e\n  \u003cgroupId\u003ech.qos.logback\u003c/groupId\u003e\n  \u003cartifactId\u003elogback-classic\u003c/artifactId\u003e\n  \u003cversion\u003e1.2.3\u003c/version\u003e\n\u003c/dependency\u003e\n```\n\n#### Standalone jar\ntd-client-java provides a standalone jar, which include all of the dependencies into a single jar file:\n\n```\n\u003cdependency\u003e\n  \u003cgroupId\u003ecom.treasuredata.client\u003c/groupId\u003e\n  \u003cartifactId\u003etd-client\u003c/artifactId\u003e\n  \u003cversion\u003e(version)\u003c/version\u003e\n  \u003cclassifier\u003eshade\u003c/classifier\u003e\n\u003c/dependency\u003e\n```\n\n## Usage\n\nTo use td-client-java, you need to set your API key in the following file:\n\n**$HOME/.td/td.conf**\n\n```\n[account]\n  user = (your TD account e-mail address)\n  apikey = (your API key)\n```\n\nYou can retrieve your API key from [My profile](https://console.treasuredata.com/users/current) page.\n\nIt is also possible to use `TD_API_KEY` environment variable. Add the following configuration to your shell configuration `.bash_profile`, `.zprofile`, etc.\n\n```\nexport TD_API_KEY = (your API key)\n```\n\nFor Windows, add `TD_API_KEY` environment variable in the user preference panel.\n\n### Proxy Server\n\nIf you need to access Web through proxy, add the following configuration to `$HOME/.td/td.conf` file:\n\n```\n[account]\n  user = (your TD account e-mail address)\n  apikey = (your API key)\n  td.client.proxy.host = (optional: proxy host name)\n  td.client.proxy.port = (optional: proxy port number)\n  td.client.proxy.user = (optional: proxy user name)\n  td.client.proxy.password = (optional: proxy password)\n```\n\n### Example Code\n\n[Example.java](src/test/java/com/treasuredata/client/Example.java)\n\n```java\nimport com.treasuredata.client.*;\nimport com.google.common.base.Function;\nimport org.msgpack.core.MessagePack;\nimport org.msgpack.core.MessageUnpacker;\nimport org.msgpack.value.ArrayValue;\n...\n\n// Create a new TD client by using configurations in $HOME/.td/td.conf\ntry (TDClient client = TDClient.newClient()){\n\n// Retrieve database and table names\nList\u003cTDDatabase\u003e databaseNames = client.listDatabases();\nfor(TDDatabase db : databaseNames) {\n   System.out.println(\"database: \" + db.getName());\n   for(TDTable table : client.listTables(db.getName())) {\n      System.out.println(\" table: \" + table);\n   }\n}\n\n// Submit a new Presto query (for Hive, use TDJobReqult.newHiveQuery)\nString jobId = client.submit(TDJobRequest.newPrestoQuery(\"sample_datasets\", \"select count(1) from www_access\"));\n\n// Wait until the query finishes\nExponentialBackOff backoff = new ExponentialBackOff();\nTDJobSummary job = client.jobStatus(jobId);\nwhile(!job.getStatus().isFinished()) {\n  Thread.sleep(backoff.nextWaitTimeMillis());\n  job = client.jobStatus(jobId);\n}\n\n// Read the detailed job information\nTDJob jobInfo = client.jobInfo(jobId);\nSystem.out.println(\"log:\\n\" + jobInfo.getCmdOut());\nSystem.out.println(\"error log:\\n\" + jobInfo.getStdErr());\n\n// Read the job results in msgpack.gz format\nclient.jobResult(jobId, TDResultFormat.MESSAGE_PACK_GZ, new Function\u003cInputStream, Object\u003e() {\n  @Override\n  public Object apply(InputStream input) {\n  try {\n    MessageUnpacker unpacker = MessagePack.newDefaultUnpacker(new GZIPInputStream(input));\n    while(unpacker.hasNext()) {\n       // Each row of the query result is array type value (e.g., [1, \"name\", ...])\n       ArrayValue array = unpacker.unpackValue().asArrayValue();\n       int id = array.get(0).asIntegerValue().toInt();\n    }\n  }\n});\n\n...\n\n}\n```\n\n### Bulk upload\n\n```java\n// Create a new TD client by using configurations in $HOME/.td/td.conf\nTDClient client = TDClient.newClient();\n\nFile f = new File(\"./sess/part01.msgpack.gz\");\n\nTDBulkImportSession session = client.createBulkImportSession(\"session_name\", \"database_name\", \"table_name\");\nclient.uploadBulkImportPart(session.getName(), \"session_part01\", f);\n```\n\n### Data Connector Bulk Loading\n\n```java\n// Create a new TD client by using configurations in $HOME/.td/td.conf\nTDClient client = TDClient.newClient();\n\nclient.startBulkLoadSession(\"session_name\");\n```\n\n### Configuring TDClient\n\nTo configure TDClient, use `TDClient.newBuilder()`:\n\n```java\nTDClient client = TDClient\n    .newBuilder()\n    .setApiKey(\"(your api key)\")\n    .setEndpoint(\"api.ybi.idcfcloud.net\")   // For using a non-default endpoint\n    .build()\n```\n\nIt is also possible to set the configuration with a `Properties` object:\n\n```java\nProperties prop = new Properties();\n// Set your own properties\nprop.setProperty(\"td.client.retry.limit\", \"10\");\n...\n\n// This overrides the default configuration parameters with the given Properties\nTDClient client = TDClient.newBuilder().setProperties(prop).build();\n```\n\n## List of Configuration Parameters\n\n|key              | default value | description |\n|-----------------|---------------|-------------|\n|`apikey`  |               | API key to access Treasure Data. You can also set this via TD_API_KEY environment variable.  |\n|`user`    |               | Account e-mail address (unnecessary if `apikey` is set) |\n|`password`|               | Account password (unnecessary if `apikey` is set) |\n|`td.client.proxy.host` |         | (optional) Proxy host e.g., \"myproxy.com\"  |\n|`td.client.proxy.port` |         | (optional) Proxy port e.g., \"80\" |\n|`td.client.proxy.user` |         | (optional) Proxy user |\n|`td.client.proxy.password` |     | (optional) Proxy password  |\n|`td.client.usessl` | true | (optional) Use SSL encryption |\n|`td.client.retry.limit` | 7 | (optinoal) The maximum number of API request retry |\n|`td.client.retry.initial-interval` | 500 | (optional) backoff retry interval = (interval) * (multiplier) ^ (retry count) |\n|`td.client.retry.max-interval` | 60000 | (optional) max retry interval |\n|`td.client.retry.multiplier` | 2.0 | (optional) retry interval multiplier |\n|`td.client.connect-timeout` | 15000 | (optional) connection timeout before reaching the API |\n|`td.client.read-timeout` | 60000 | (optional) timeout when no data is coming from API |\n|`td.client.connection-pool-size` | 64 | (optional) Connection pool size|\n|`td.client.endpoint` | `api.treasuredata.com` | (optional) TD REST API endpoint name |\n|`td.client.port` | 80 for non-SSL, 443 for SSL connection | (optional) TD API port number |\n\n\nThe precedence of the configuration parameters are as follows:\n\n1. Properties object passed to `TDClient.newBuilder().setProperties(Properties p)`\n1. Parameters written in `$HOME/.td/td.conf`\n1. System properties (passed with `-D` option when launching JVM)\n1. Environment variable (only for TD_API_KEY parameter)\n\nYou can override the default configuration parameters given by the environment\nvariables with System properties, and then by `$HOME/.td/td.conf` file or `Properties` object.\n\n## For Developers\n\n### Build from the source code\n\n```\n$ git clone https://github.com/treasure-data/td-client-java.git\n$ cd td-client-java\n$ mvn package\n```\n\nThis creates jar files within `target` folder.\n\n### How to deploy to the Central repository\n\n```\n# update pom.xml, README.md and CHANGES.txt\n$ git commit -am \"Release (version)\"\n$ git tag \"(version)\"\n$ mvn deploy -DperformRelease=true\n$ sbt \"sonatypeReleaseAll com.treasuredata\"\n\n# update pom.xml to use the next SNAPSHOT version\n$ git commit -am \"Next snapshot version\"\n$ git push\n$ git push --tags\n```\nAnd then, create a release note page at https://github.com/treasure-data/td-client-java/releases\n\nSee also https://github.com/xerial/sbt-sonatype#publishing-maven-projects to use `sbt sonatypeReleaseAll` command.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftreasure-data%2Ftd-client-java","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftreasure-data%2Ftd-client-java","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftreasure-data%2Ftd-client-java/lists"}