https://github.com/xiaoyao9184/kettle-project-toolbox
Kettle Project CI/CD Solution
https://github.com/xiaoyao9184/kettle-project-toolbox
cd ci jenkins kettle kitchen pan
Last synced: 19 days ago
JSON representation
Kettle Project CI/CD Solution
- Host: GitHub
- URL: https://github.com/xiaoyao9184/kettle-project-toolbox
- Owner: xiaoyao9184
- License: apache-2.0
- Created: 2018-10-19T08:24:54.000Z (over 6 years ago)
- Default Branch: master
- Last Pushed: 2023-10-13T02:45:28.000Z (over 1 year ago)
- Last Synced: 2025-03-28T04:41:41.609Z (about 1 month ago)
- Topics: cd, ci, jenkins, kettle, kitchen, pan
- Language: Batchfile
- Homepage:
- Size: 2.75 MB
- Stars: 2
- Watchers: 3
- Forks: 5
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Kettle-Project-Toolbox
[Chinese](README_ZH.md)
The toolbox commonly used in the kettle project are gathered.
This includes running scripts, deployment scripts, and repository templates.
In general, we need to use in the background PDI, this time it will not run Spoon, but run Pan and Kitchen, and for complex ETL, a common set of scheduling script can simplify development.
Here I offer a set of toolbox script has been used in the actual project.
# Use
[wiki](https://github.com/xiaoyao9184/Kettle-Project-Toolbox/wiki)
# Compatibility
| Version | Supported | Why |
|:-----:|:-----:|:-----:|
| pdi-ce-7.0.0.0-25 | Yes | Develop on this |
| pdi-ce-7.1.0.0-12 | Yes | Test default flow directory |
| pdi-ce-8.0.0.0-28 | Yes | Test default flow directory |
| pdi-ce-8.1.0.0-365 | Yes | Test default flow directory |
| pdi-ce-8.2.0.0-342 | Yes | Test default flow directory |
| pdi-ce-8.3.0.0-371 | Yes | Test default flow directory |
| pdi-ce-9.0.0.0-423 | Yes | Test default flow directory |
| pdi-ce-9.1.0.0-324 | Yes | Test default flow directory |
## compatibility test log
### pdi-ce-9.1.0.0-324
not real test
### pdi-ce-9.0.0.0-423
not real test
### pdi-ce-8.3.0.0-371
not real test
### pdi-ce-8.2.0.0-342
not real test
### pdi-ce-8.1.0.0-365
Set variable with **valid in the root job** in steps `Set Variables` and `Modified Java Script Value` fail,
cant read it form other transformation,
i think variable must replace by parameter,
and especially at `Transformation (job entry)`,
maybe it cant use static characters mixed with variables, or for the following reasons.
Need to explicitly declare that the parent variable is passed to the subset job. and it work.
### pdi-ce-8.0.0.0-28
No problem found
### pdi-ce-7.1.0.0-12
Since 7.1 changed the relative repository selection window,
such as job items: Job (Job Entry), Transformation (Job Entry),
transformation steps: Simple Mapping, Mapping, etc.
all omit the original path filling box, merged into one text box to fill in the path + name,
However, this text box is still two attributes and will be split into path + name according to the last ‘/’.
Unlike 7.0, the name attribute does not support include path symbol!
But, now it will support it!
### Karaf Folder
Cant change Karaf folder without delete caches folder in `data-integration/system/karaf/caches`
Alway create caches in this folder, no matter modify of `data-integration/system/karaf/instances/instance.properties`
[Wrong answer](https://forums.pentaho.com/threads/207678-Changing-Pentaho-Folder-name-and-Karaf/)
And i think caches files contain own positioning,
if created via a symbolic link will be accessed via a symbolic link path on windows use **Junction**
### Unix folder link
unix not support folder hard link, and symbolic link for `data-integration/` path
not work with current path(./) in KETTLE_HOME use relative repository path.
So use hard link file and copy folder same time for replace folder link.
# Use with jenkins
All scripts can run on jenkins
The JAVA_HOME environment variable must be set.
Otherwise kitchen and pan will return the error exit code, and jenkins will error.