https://github.com/blockchain-etl/zilliqa-etl

Python scripts for ETL (extract, transform and load) jobs for Zilliqa blockchain data
https://github.com/blockchain-etl/zilliqa-etl

Last synced: 4 days ago
JSON representation

Python scripts for ETL (extract, transform and load) jobs for Zilliqa blockchain data

• Host: GitHub
• URL: https://github.com/blockchain-etl/zilliqa-etl
• Owner: blockchain-etl
• License: mit
• Created: 2020-07-27T15:15:22.000Z (almost 5 years ago)
• Default Branch: master
• Last Pushed: 2023-03-29T01:52:10.000Z (over 2 years ago)
• Last Synced: 2025-06-03T18:28:19.187Z (25 days ago)
• Language: Python
• Homepage: https://console.cloud.google.com/bigquery?page=dataset&d=crypto_zilliqa&p=public-data-finance
• Size: 275 KB
• Stars: 3
• Watchers: 2
• Forks: 7
• Open Issues: 4
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Zilliqa ETL

[![Build Status](https://travis-ci.org/blockchain-etl/zilliqa-etl.svg?branch=master)](https://travis-ci.org/blockchain-etl/zilliqa-etl)

[![Telegram](https://img.shields.io/badge/telegram-join%20chat-blue.svg)](https://t.me/joinchat/GsMpbA3mv1OJ6YMp3T5ORQ)

## Overview

Zilliqa ETL allows you to setup an ETL pipeline in Google Cloud Platform for ingesting Zilliqa blockchain data 

into BigQuery. It comes with [CLI tools](/cli) for exporting Zilliqa data into JSON newline-delimited files

partitioned by day. 

Data is available for you to query right away in 

[Google BigQuery](https://console.cloud.google.com/bigquery?page=dataset&d=crypto_zilliqa&p=public-data-finance).

## Architecture

![zilliqa_etl_architecture.svg](zilliqa_etl_architecture.png)

[Google Slides version](https://docs.google.com/presentation/d/16h_JVok0dZmHQfnWeGAUJAsEzkwUh2DSHVdhTe5LC-E/edit?usp=sharing)

1. [Airflow DAGs](https://airflow.apache.org/) export and load Zilliqa data to BigQuery daily. 

    Refer to [Zilliqa ETL Airflow](/airflow) for deployment instructions.