https://github.com/civitaspo/embulk-plugin-hadoop-distributed-file-system

Jrubyでembulk-plugin-hdfsを作ろうと頑張った痕跡 使えません
https://github.com/civitaspo/embulk-plugin-hadoop-distributed-file-system

Last synced: 3 months ago
JSON representation

Jrubyでembulk-plugin-hdfsを作ろうと頑張った痕跡 使えません

• Host: GitHub
• URL: https://github.com/civitaspo/embulk-plugin-hadoop-distributed-file-system
• Owner: civitaspo
• License: mit
• Created: 2015-02-22T08:11:47.000Z (almost 10 years ago)
• Default Branch: master
• Last Pushed: 2015-02-22T14:35:04.000Z (almost 10 years ago)
• Last Synced: 2024-05-01T15:28:42.767Z (8 months ago)
• Language: Ruby
• Homepage:
• Size: 137 KB
• Stars: 0
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE.txt

Awesome Lists containing this project

README

        
# Hadoop Distributed File System input plugin for Embulk

TODO: Write short description here

## Overview

* **Plugin type**: input

* **Load all or nothing**: yes

* **Resume supported**: no

## Configuration

- **type**: description (string, required)

- **hdfs_input_path**: description (string, required)

- **hadoop_home**: description (string, default: /usr/lib/hadoop)

- **file_format**: description (string, default: tsv)

- **schema**: description (array, required)

- **threads**: description (integer, default: 1)

- **max_line_size**: description (integer, default: 5242880)

## Example

```yaml

in:

  type: hadoop-distributed-file-system

  hdfs_input_path: /user/civitaspo/testdata/user_id.txt

  hadoop_home: /opt/analytics/hadoop

  file_format: tsv

  schema:

    - { name: user_id, type: string }

```

## Build

```

$ rake

```