https://github.com/blinklabs-io/cargo
Blockchain sync for the cult of Cardano
https://github.com/blinklabs-io/cargo
blockchain cardano cardano-db-sync cardano-node data-pipeline etl go golang
Last synced: 6 months ago
JSON representation
Blockchain sync for the cult of Cardano
- Host: GitHub
- URL: https://github.com/blinklabs-io/cargo
- Owner: blinklabs-io
- License: apache-2.0
- Created: 2022-01-26T13:22:39.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2024-09-09T14:31:03.000Z (8 months ago)
- Last Synced: 2024-09-09T17:41:06.315Z (8 months ago)
- Topics: blockchain, cardano, cardano-db-sync, cardano-node, data-pipeline, etl, go, golang
- Language: Go
- Homepage:
- Size: 176 KB
- Stars: 7
- Watchers: 2
- Forks: 0
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
- Codeowners: CODEOWNERS
Awesome Lists containing this project
README
# Cargo
Blockchain sync for the cult of Cardano
Cargo is a scalable event-based multi-input/multi-output transform system with
a focus on syncing the Cardano blockchain and related ecosystem.
Inspiration for this tool comes from [CDAP](https://github.com/cdapio/cdap)
and its simplified data pipelines as code, and by tools like
[Cardano DB sync](https://github.com/input-output-hk/cardano-db-sync) and
[Oura](https://github.com/txpipe/oura) within the Cardano ecosystem.
## Design goals
Our primary goal is to create an application which can be used for simple use
cases on a local development machine, yet be horizontally scalable at several
key points while remaining interoperable with the overall software ecosystem.
This interoperability comes from creating standard data formats for passing
events between the various layers so necessary data structure transformations
can be applied. For example, an Oura sink could be written for Cargo, which
would allow reusing Oura and its filtered data as an input source.
* Event based architecture
* DAG based workflow definitions
- Sharable code
* Horizontally scalable
* Configurable inputs (sources)
- Cardano blockchain (cardano-node)
- External programs
- Webhooks
* Configurable transforms (filters)
- External programs
- Can be chained together
* Built-in transform primitives
- JOIN, SPLIT
- Schema alter (internal)
* Configurable outputs (sinks)
- AMQP
- Kafka
- Database (cardano-db-sync compatible)
- Webhooks
* Configurable message bus
- AMQP
- Kafka
- In-memory (single process only)