https://github.com/elgca/streamsets-scala-script

在streamsets中使用scala脚本
https://github.com/elgca/streamsets-scala-script

pipeline scala streamsets streamsets-data-collector

Last synced: about 1 month ago
JSON representation

在streamsets中使用scala脚本

• Host: GitHub
• URL: https://github.com/elgca/streamsets-scala-script
• Owner: elgca
• Created: 2018-06-27T13:54:15.000Z (over 6 years ago)
• Default Branch: master
• Last Pushed: 2018-07-06T11:52:35.000Z (over 6 years ago)
• Last Synced: 2024-10-15T02:05:34.585Z (3 months ago)
• Topics: pipeline, scala, streamsets, streamsets-data-collector
• Language: Java
• Size: 1.25 MB
• Stars: 0
• Watchers: 0
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# streamsets-scala-script

在streamsets中使用scala脚本

```shell

mvn package

cd target/

# 上传streamsets-scala-script-1.0-SNAPSHOT.tar.gz至streamsets-libs下

tar -xzvf streamsets-scala-script-1.0-SNAPSHOT.tar.gz

# 重启sdc

```

**注意：不可以解压到`user-lib`,必须放在`streamsets-libs`目录下，否则会出现classloader无法获取**

`com.streamsets.datacollector.runner.PipelineRuntimeException: CONTAINER_0800 - Pipeline 'scalascript898cc1c7-c56a-48fd-9c10-b44baaecd0ae' validation error : CONTAINER_0701 - Stage 'ScalaScript_01' initialization error: java.security.AccessControlException: access denied ("java.lang.RuntimePermission" "getClassLoader")`

# 示例

`example/scala-script.json`

## pipeline

![pipeline](example/pipeline.png)

## 配置依赖

![](example/maven_config.png)

## 执行脚本

![](example/scala-script.png)

## 预览

![](example/preview1.png)

![](example/preview2.png)