https://github.com/zhangyuan/dataworks-cli

Hope to save data engineers from Aliyun Dataworks.
https://github.com/zhangyuan/dataworks-cli

Last synced: 21 days ago
JSON representation

Hope to save data engineers from Aliyun Dataworks.

• Host: GitHub
• URL: https://github.com/zhangyuan/dataworks-cli
• Owner: zhangyuan
• License: mit
• Created: 2024-03-12T13:38:19.000Z (10 months ago)
• Default Branch: main
• Last Pushed: 2024-07-17T11:25:18.000Z (6 months ago)
• Last Synced: 2024-10-27T14:35:28.665Z (2 months ago)
• Language: Go
• Homepage:
• Size: 6.66 MB
• Stars: 0
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# dataworks-cli

A tool to help data engineers and data architects to know about the stuffs locked in Dataworks more clearly and efficiently.

**FYI: Dataworks is a service provided by Aliyun. This project is not associated with the Dataworks nor Aliyun.**

## Usage

### Prepare the `.env` file

Add the following env variables to `.env` file:

```env

ACCESS_KEY_ID=

ACCESS_KEY_SECRET=

DATAWORKS_PROJECT_ID=

DATAWORKS_ENDPOINT=dataworks.cn-beijing.aliyuncs.com # or other endpoint

```

### Get the file list by file types

```bash

dataworks-cli files list -t 10,23  -o files/manifest.json

```

### Download the files from the file list

```bash

dataworks-cli files list -t 10,23  -o files/manifest.json

```

### Get the table list by data source

```bash

dataworks-cli files download -i files.json -o ./files

```

### Get the DI sync tasks

```bash

dataworks-cli di list-sync-tasks -s data-source-name -o di-sync-tasks.json

```

### Get the node list by project env

```bash

dataworks-cli nodes list -e PROD -o nodes.json

```