https://github.com/analyseether/ether_sql
A python library to push ethereum blockchain data into an sql database.
https://github.com/analyseether/ether_sql
analytics blockchain csv data-analysis ethereum etl export postgresql python sql
Last synced: 8 days ago
JSON representation
A python library to push ethereum blockchain data into an sql database.
- Host: GitHub
- URL: https://github.com/analyseether/ether_sql
- Owner: analyseether
- License: apache-2.0
- Created: 2018-05-03T13:03:45.000Z (over 7 years ago)
- Default Branch: master
- Last Pushed: 2022-09-13T22:04:44.000Z (over 3 years ago)
- Last Synced: 2025-09-17T12:05:43.002Z (4 months ago)
- Topics: analytics, blockchain, csv, data-analysis, ethereum, etl, export, postgresql, python, sql
- Language: Python
- Homepage: http://ether-sql.readthedocs.io
- Size: 2.16 MB
- Stars: 67
- Watchers: 6
- Forks: 22
- Open Issues: 16
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# ether_sql
A Python library to push ethereum blockchain data into an sql database.
ether_sql was built by [Analyse Ether](https://www.analyseether.com/), with the goal of making Ethereum data easily available to everyone. This library can be used as a backbone for creating block explorers or performing data analysis.
It is written in Python 3.5+, uses [web3.py](https://github.com/ethereum/web3.py) for geting data using JSON-RPC calls and uses [SqlAlchemy](http://docs.sqlalchemy.org/en/latest/) to connect to a postgressql database.
## Buidl Status
This is currently in very alpha stage, and not recommended for production use until it has received sufficient testing.
Currently supports Geth, Infura and Parity node, but transaction traces (eg. internal transactions) are currently available only with Parity node.
[](https://travis-ci.org/analyseether/ether_sql)
Documentation available at: http://ether-sql.readthedocs.io
# Installation guide
Please find the detailed installation guide [here](http://ether-sql.readthedocs.io/en/latest/installation.html)
# Command line options
ether_sql has several built in cli commands to facilitate scraping data.
## Syncing the data
To start the sync just type.
`$ ether_sql scrape_block_range `
This will start start pushing the data from an Infura node to the psql database.
## Switching nodes
To switch nodes use the settings flag:
`$ ether_sql --settings='PersonalParitySettings' scrape_block_range `
## Using multiple workers to sync data
To start 4 parallel workers use.
`$ ether_sql --settings=YourSettings celery start -c4`
The above command will start 4 workers using the settings `YourSettings`
Then start the sync, which will automatically use the setting used to start workers.
`$ ether_sql scrape_block_range `
Here is a demo of the process: https://www.youtube.com/watch?v=rnkfyAgGJwI&feature=youtu.be
## Exporting as csv
To export the pushed data as a csv
`$ ether_sql sql export_to_csv`
To access other Command Line Interfaces (CLI) checkout the [cli docs](http://ether-sql.readthedocs.io/en/latest/api/cli.html).