https://github.com/subsquid-labs/file-store-parquet-example
Minimalistic squid storing data to Apache Parquet files
https://github.com/subsquid-labs/file-store-parquet-example
blockchain blockchain-indexer csv etl indexing parquet sqd subsquid typescript web3
Last synced: about 5 hours ago
JSON representation
Minimalistic squid storing data to Apache Parquet files
- Host: GitHub
- URL: https://github.com/subsquid-labs/file-store-parquet-example
- Owner: subsquid-labs
- License: mit
- Created: 2023-03-09T14:29:56.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2026-06-16T15:52:52.000Z (15 days ago)
- Last Synced: 2026-06-16T17:27:01.191Z (15 days ago)
- Topics: blockchain, blockchain-indexer, csv, etl, indexing, parquet, sqd, subsquid, typescript, web3
- Language: TypeScript
- Homepage: https://docs.sqd.dev/en/sdk
- Size: 30.3 KB
- Stars: 3
- Watchers: 1
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# A squid that saves USDC Transfers to Parquet files
This tiny blockchain indexer scrapes `Transfer` events emitted by the [USDC contract](https://etherscan.io/address/0xa0b86991c6218b36c1d19d4a2e9eb0ce3606eb48) and saves the data in a file-based dataset in a local folder `./data`. It is built with the [Subsquid framework](https://sqd.dev), hence the term "squid".
The squid uses [`@subsquid/file-store`](https://docs.sqd.dev/en/sdk) and [`@subsquid/file-store-parquet`](https://docs.sqd.dev/en/sdk) packages to store the dataset. The result is a partitioned dataset where the data is stored in [Apache Parquet](https://parquet.apache.org) files.
Dependencies: NodeJS, [Squid CLI](https://docs.sqd.dev/en/sdk).
To see it in action, spin up a *processor*, a process that ingests the data from the Ethereum Archive:
```bash
git clone https://github.com/subsquid-labs/file-store-parquet-example
cd file-store-parquet-example/
npm i
sqd process
```
You should see a `./data` folder populated with indexer data appear in a bit:
```bash
$ tree ./data/
./data/
├── 0000000000-0007258859
│ └── transfers.parquet
├── 0007258860-0007671919
│ └── transfers.parquet
...
└── status.txt
```