{"id":23413326,"url":"https://github.com/forgineer/duckdb-kettle-plugin","last_synced_at":"2025-04-12T05:24:53.724Z","repository":{"id":219532582,"uuid":"749202610","full_name":"forgineer/duckdb-kettle-plugin","owner":"forgineer","description":"A plugin for Pentaho Data Integration (Kettle) that adds support for DuckDB in the table input/output step.","archived":false,"fork":false,"pushed_at":"2024-09-18T04:19:13.000Z","size":165,"stargazers_count":5,"open_issues_count":1,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-26T00:51:17.677Z","etag":null,"topics":["duckdb","input","kettle","output","pdi","pentaho-data-integration","pentaho-kettle","plugin","table"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/forgineer.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-01-27T21:38:57.000Z","updated_at":"2025-01-10T12:49:31.000Z","dependencies_parsed_at":null,"dependency_job_id":"de08a29e-2851-42b2-aa0c-398fa68930ca","html_url":"https://github.com/forgineer/duckdb-kettle-plugin","commit_stats":null,"previous_names":["forgineer/duckdb-kettle-plugin"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/forgineer%2Fduckdb-kettle-plugin","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/forgineer%2Fduckdb-kettle-plugin/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/forgineer%2Fduckdb-kettle-plugin/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/forgineer%2Fduckdb-kettle-plugin/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/forgineer","download_url":"https://codeload.github.com/forgineer/duckdb-kettle-plugin/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248520968,"owners_count":21117964,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["duckdb","input","kettle","output","pdi","pentaho-data-integration","pentaho-kettle","plugin","table"],"created_at":"2024-12-22T19:32:06.609Z","updated_at":"2025-04-12T05:24:53.669Z","avatar_url":"https://github.com/forgineer.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# duckdb-kettle-plugin\nA plugin for [Pentaho Data Integration (Kettle)](https://github.com/pentaho/pentaho-kettle) that adds support for [DuckDB](https://duckdb.org/) in the table input/output step.\n\n## Background\nThis project was born out of experimentation with [Apache HOP](https://hop.apache.org/) and the realization that it supported DuckDB out of the box, while recent versions of Kettle did not :disappointed_relieved:\n\nAfter a weekend of tinkering with the Kettle SDK and drawing inspiration from the DuckDB plugin in Apache HOP, this plugin brings the same functionality to Kettle.\n\n## Install Instructions\nCurrent and past releases can be found on the [Releases page](https://github.com/forgineer/duckdb-kettle-plugin/releases). Each release includes pre-compiled packages (zip files) containing the necessary JAR file(s) for installation, compiled for a specific version of PDI and DuckDB (see release notes). Download the most recent release that matches your version of Kettle.\n\n\u003e :memo: Releases may not always keep pace with the latest versions of DuckDB or [Pentaho Community Edition](https://support.pentaho.com/hc/en-us/articles/205789159-Pentaho-Product-Lifecycle-Overview). If you need a more recent version of DuckDB, see the [Build Instructions](#build-instructions) for compiling your own package.\n\n1. Unpack the zip file into the `plugins` directory of your local Kettle install (`\\data-integration\\plugins`). \n2. The zip file should include the plugin and necessary JDBC driver for DuckDB, with the following strucutre:\n```\ndata-integration\\\n    plugins\\\n        duckdb-kettle-plugin-x-x-x\\\n            lib\\\n                duckdb_jdbc-x.x.x.jar\n            duckdb-kettle-plugin-x-x-x.jar\n```\n3. Restart Kettle (Spoon).\n\n## Configure DuckDB Connection\nAfter installing the plugin, DuckDB should be available as a connection type from a table input or output step, similar to a SQLite connection.\n\n![DuckDB Connection](./images/duckdb-kettle-connection.png)\n\n## Build Instructions\n\n### Prerequisites\nBefore building the plugin, ensure you have the following installed and configured on your local machine:\n* Maven, version 3+\n* Java JDK 11 (or OpenJDK)\n* The Pentaho Maven [settings.xml](https://raw.githubusercontent.com/pentaho/maven-parent-poms/master/maven-support-files/settings.xml) file in your home `.m2` directory\n\n### Build the Plugin\n1. Clone this repository locally:\n```\ngit clone https://github.com/forgineer/duckdb-kettle-plugin.git\n```\n2. Review the `pom.xml` file and update the `\u003cversion\u003e` tags and to match the same [Pentaho Data Integration (Kettle)](https://mvnrepository.com/artifact/pentaho-kettle/kettle-core) and [DuckDB](https://mvnrepository.com/artifact/org.duckdb/duckdb_jdbc) versions you intend to use. Verify each version on the [Maven repository](https://mvnrepository.com).\n```xml\n  \u003cproperties\u003e\n    \u003c!-- Pentaho Data Integration (Kettle) Version --\u003e\n    \u003c!-- Ex: 9.3.0.0-428, 9.4.0.0-343, etc. --\u003e\n    \u003ckettle.version\u003e9.4.0.0-343\u003c/kettle.version\u003e\n    \u003c!-- DuckDB Version (JDBC driver) --\u003e\n    \u003c!-- Ex: 0.10.2, 1.0.0, etc. --\u003e\n    \u003cduckdb.version\u003e1.1.0\u003c/duckdb.version\u003e\n    ...\n  \u003c/properties\u003e\n```\n3. Update the jar file name of the JDBC driver in the main source (`DuckDBDatabaseMeta.java`) to match the version found in the `pom.xml` file. Again, verify the file name and version on the [Maven repository](https://mvnrepository.com/artifact/org.duckdb/duckdb_jdbc) or directly from DuckDB.\n```java\n@Override\npublic String[] getUsedLibraries() {\n    // The version should match POM\n    return new String[] {\"duckdb_jdbc-1.1.0.jar\"};\n}\n```\n4. Package the jar file:\n```bash\nmvn package \n```\nThis will create a jar and zip file inside of the `target` directory:\n* duckdb-kettle-plugin-x.x.x.jar\n* duckdb-kettle-plugin-x.x.x.zip\n\n## Contributing\nWe welcome contributions to the duckdb-kettle-plugin project. Before submitting a pull request, please:\n* Raise an [issue](https://github.com/forgineer/duckdb-kettle-plugin/issues) to discuss the proposed changes.\n* Ensure that the issue is clear and concise, and that we've discussed and agreed on the changes.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fforgineer%2Fduckdb-kettle-plugin","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fforgineer%2Fduckdb-kettle-plugin","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fforgineer%2Fduckdb-kettle-plugin/lists"}