https://github.com/smdmts/embulk-parser-firebase_avro
https://github.com/smdmts/embulk-parser-firebase_avro
embulk embulk-plugin firebase
Last synced: 2 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/smdmts/embulk-parser-firebase_avro
- Owner: smdmts
- License: mit
- Created: 2017-06-09T03:19:17.000Z (almost 8 years ago)
- Default Branch: master
- Last Pushed: 2018-06-27T05:44:26.000Z (almost 7 years ago)
- Last Synced: 2025-02-28T07:14:13.942Z (3 months ago)
- Topics: embulk, embulk-plugin, firebase
- Language: Scala
- Size: 85 KB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE.txt
Awesome Lists containing this project
README
# Firebase Avro parser plugin for Embulk
This parser plug-in supported to Firebase extracted file of *AVRO* format from Google-BigQuery.
Currently (2017/6), Firebase could extract only GCS(Google Cloud Storage) to JSON or AVRO format.
If you want to export Firebase's data, it has to need data pipelining for in the below steps.
1. Link BigQuery with Firebase.
- https://support.google.com/firebase/answer/6318765
1. Dump Firebase Data to Google Cloud Storage.
- Use data pipelining with job scheduler and extract data from BigQuery.
- https://support.google.com/firebase/answer/7029846
1. Use input-gcs plugin with this parser-plugin.
- https://github.com/embulk/embulk-input-gcs
## Overview
* **Plugin type**: parser
* **Guess supported**: no
## Configuration
- No configuration
## Example
```yaml
in:
type: any file input plugin type
parser:
type: firebase_avro
```
```
$ embulk gem install embulk-parser-firebase_avro
```
## Build
```
$ ./gradlew gem # -t to watch change of files and rebuild continuously
```
## Developing or Testing
This plug-in is written by Scala. You could use sbt.
```
$ ./sbt
$ ./sbt test
```