Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/codingchili/ethereum-ingest
JavaFX and commandline application to import events from the Ethereum blockchain into ElasticSearch, MongoDB, Hazelcast, CQEngine and SQLite.
https://github.com/codingchili/ethereum-ingest
chili-core cqengine elasticsearch ether ethereum geth hazelcast infura ingest mongodb sqlite vertx
Last synced: 3 months ago
JSON representation
JavaFX and commandline application to import events from the Ethereum blockchain into ElasticSearch, MongoDB, Hazelcast, CQEngine and SQLite.
- Host: GitHub
- URL: https://github.com/codingchili/ethereum-ingest
- Owner: codingchili
- License: mit
- Created: 2017-10-06T22:54:11.000Z (over 7 years ago)
- Default Branch: master
- Last Pushed: 2019-04-01T15:57:42.000Z (almost 6 years ago)
- Last Synced: 2024-09-27T07:41:22.650Z (4 months ago)
- Topics: chili-core, cqengine, elasticsearch, ether, ethereum, geth, hazelcast, infura, ingest, mongodb, sqlite, vertx
- Language: Java
- Homepage: https://ethereum-ingest.com/
- Size: 2.68 MB
- Stars: 35
- Watchers: 5
- Forks: 9
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# ethereum-ingest [](https://travis-ci.org/codingchili/ethereum-ingest)
Imports blocks and transactions from the Ethereum blockchain into ElasticSearch, MongoDB, Hazelcast, CQEngine and SQLite! Works with Geth IPC, Geth REST API and Infura.
Demo video: [YouTube](https://www.youtube.com/watch?v=FFI9OnW9IuI)
Tested with
- Java 11 (minimim requirement)
- ElasticSeach 7.0.0
- MongoDB 4.0.8
- Hazelcast 3.10.5
- geth 1.8.23
### Building
Build with
```console
# detects the current platform.
./gradlew build
# to build for a specific platform, 'win', 'mac' or 'linux'.
./gradlew build -Pplatform=linux
```
Requires chili-core through jitpack or local repo.
### Importing
The first step is to start your ethereum IPC client, for geth use:
```console
geth --rpcapi personal,db,eth,net,web3 --rpc --testnet
```
Start the importer with:
```console
java -jar .jar --import
java -jar .jar --gui
java -jar .jar --help
```
* --import: starts an import using application.json.
* --gui: starts the application with the graphical user interface.
### Configuring
Set configuration in application.json before running --import. WHen using the graphical application the configuration is saved automatically.
Default configuration
```console
{
"startBlock" : "1964770",
"blockEnd" : "1964900",
"storage" : "ELASTICSEARCH",
"targetNode" : "\\\\.\\pipe\\geth.targetNode",
"os" : "WINDOWS",
"txIndex" : "ether-tx-demo-iix",
"blockIndex" : "etdder-block-demo-iix",
"backpressureBlocks" : 6,
"backPressureTx" : 32,
"txImport" : true,
"blockImport" : true
}
```
Backpressure for blocks and tx should not exceed 200 when multiplied with eachother. If you want to
increase these values further you need to make sure the storage is capable of handling that many connections.
Storage can be any of the following
- MONGODB (default: localhost:27017)
- ELASTICSEARCH (default: localhost:9300)
- HAZELCAST
- SQLITE (CQEngine)
- MEMORY
os can be any of the following, required for targetNode to work correctly
- UNIX
- WINDOWS
Imports can be executed multiple times over the same block range without resulting in duplicates.
To configure a custom host:port for MongoDB or ElasticSearch please add/edit this file in "conf/system/storage.yaml"
```console
---
storage:
com.codingchili.core.storage.MongoDBMap:
host: "localhost"
port: 27017
com.codingchili.core.storage.ElasticMap:
host: "localhost"
port: 9200
```
### Contributing
Submit an issue or a PR ! :blue_heart:
[](https://commerce.coinbase.com/checkout/673e693e-be6d-4583-9791-611da87861e3)