https://github.com/churrodata/spacex-geo-extension
this is an example of a churro extension that you can write to extend churro's transformation logic to meet your unique needs.
https://github.com/churrodata/spacex-geo-extension
etl etl-pipeline golang spacex-api
Last synced: 5 months ago
JSON representation
this is an example of a churro extension that you can write to extend churro's transformation logic to meet your unique needs.
- Host: GitHub
- URL: https://github.com/churrodata/spacex-geo-extension
- Owner: churrodata
- License: apache-2.0
- Created: 2021-09-16T13:57:30.000Z (over 4 years ago)
- Default Branch: main
- Last Pushed: 2021-09-17T19:52:36.000Z (over 4 years ago)
- Last Synced: 2024-06-20T08:03:12.836Z (almost 2 years ago)
- Topics: etl, etl-pipeline, golang, spacex-api
- Language: Go
- Homepage:
- Size: 46.9 KB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Churro - ETL for Kubernetes - spacex extension example
churro is a cloud-native Extract-Transform-Load (ETL) application designed to build, scale, and manage data pipeline applications. That project can
be found at the [github churro site](https://github.com/churrodata/churro).
This particular project, spacex-geo-extension, is an example of
a custom churro extension you can write to perform any sort of
custom transformation logic that your use case might require.
## How does this work?
A churro extension has a gRPC interface that it will implement. churro
registers your extension within a churro pipeline's extract source.
As data is being processed, churro will call any registered extensions
providing it a copy of the data being processed along with primary
keys for that data.
This allows you as within an extension the ability to know what
data is being processed and know its primary key location with
the pipeline database. With that set of values, you can make ny
transform logic you would want to do.
This particular extension is pretty simple to show you the basics
of how a churro extension works as a starting point to more
complex extension writing.
This extension's container image is found on [DockerHub](https://hub.docker.com/repository/docker/churrodata/spacex-geo-extension).
## Design
* churro extensions run as Pods on a churro Kubernetes cluster
* churro extensions implement a gRPC interface, so you could write extensions in any language that support gRPC (e.g. Java, javascript, golang, etc.)
* You deploy your extension into your cluster as you would any other Pod
For more details on the churro design, checkout out the documentation at the [churro github pages](https://churrodata.github.io/churro/design-guide.html).
## Docs
Detailed documentation is found at the [churro github pages](https://churrodata.github.io/churro/), additional content such as blogs can be found at the [churrodata.com](https://www.churrodata.com) web site.
## Contributing
Since churro is open source, you can view the source code and make contributions such as pull requests on our github.com site. We encourage users to log any issues they find on our github issues [site](https://github.com/churrodata/churro/issues).
## Support
churro enterprise support and services are provided by [churrodata.com](https://churrodata.com).