Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/joker1007/embulk-formatter-avro
https://github.com/joker1007/embulk-formatter-avro
Last synced: 2 days ago
JSON representation
- Host: GitHub
- URL: https://github.com/joker1007/embulk-formatter-avro
- Owner: joker1007
- License: mit
- Created: 2016-12-28T15:26:09.000Z (almost 8 years ago)
- Default Branch: master
- Last Pushed: 2022-11-29T02:57:45.000Z (almost 2 years ago)
- Last Synced: 2024-10-18T03:08:47.058Z (21 days ago)
- Language: Java
- Size: 78.1 KB
- Stars: 4
- Watchers: 2
- Forks: 2
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE.txt
Awesome Lists containing this project
README
# Avro formatter plugin for Embulk
[Avro](http://avro.apache.org/) formatter plugin for Embulk.
## Overview
* **Plugin type**: formatter
## Support avro types
Support all avro basic types.
- string
- int
- long
- float
- double
- boolean
- enum
- fixed
- array
- map
- record
But typecasting is restricted by embulk column type.
See. [AvroValueConverters](https://github.com/joker1007/embulk-formatter-avro/tree/master/src/main/java/org/embulk/formatter/avro/converter),
## Configuration
- **avsc**: avro schema (avsc) filepath (string, required)
- **codec**: avro codec type (enum: `deflate`, `bzip2`, `xz`, `snappy`, optional)
- **compression\_level**: avro codec compression level (integer, optional, for only `deflate` and `xz` codec)
- **skip\_error\_record**: If you want to skip error record, set true (boolean, default: `false`)
## Example
```yaml
out:
type: file
path_prefix: ./out_
file_ext: avro
formatter:
type: avro
avsc: schema.avsc
skip_error_record: true
```
## Build
```
$ ./gradlew gem # -t to watch change of files and rebuild continuously
```