https://github.com/crytic/fluxture
A crawling framework for blockchains and peer-to-peer systems
https://github.com/crytic/fluxture
Last synced: 12 months ago
JSON representation
A crawling framework for blockchains and peer-to-peer systems
- Host: GitHub
- URL: https://github.com/crytic/fluxture
- Owner: crytic
- License: apache-2.0
- Created: 2020-10-24T11:45:46.000Z (over 5 years ago)
- Default Branch: master
- Last Pushed: 2023-09-04T22:06:59.000Z (almost 3 years ago)
- Last Synced: 2025-05-22T23:51:31.358Z (about 1 year ago)
- Language: Python
- Size: 202 KB
- Stars: 46
- Watchers: 17
- Forks: 9
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Fluxture
[](https://badge.fury.io/py/fluxture)
[](https://github.com/trailofbits/fluxture/actions)
[](https://slack.empirehacking.nyc)
Fluxture is a lightweight crawler for peer-to-peer networks like Blockchains. It currently supports the latest version
of the Bitcoin protocol: 70015. It implements the minimum amount of the Bitcoin protocol necessary to collect geographic
and topological information.
## Quickstart
```commandline
pip3 install fluxture
```
Or, to install from source (_e.g._, for development):
```commandline
$ git clone https://github.com/trailofbits/fluxture
$ cd fluxture
$ pip3 install -e '.[dev]'
```
## Usage
To crawl the Bitcoin network, run:
```commandline
fluxture crawl bitcoin --database crawl.db
```
The crawl database is a SQLite database that can be reused between crawls.
## Geolocation
Fluxture uses the MaxMind GeoLite2 City database for geolocating nodes based upon their IP address. Various Fluxture
commands will either require a path to the database, or a MaxMind license key (which will be used to automatically
download the database). You can sign up for a free MaxMind license key,
[here](https://www.maxmind.com/en/geolite2/signup).
A KML file (which can be imported to Google Maps or Google Earth) can be generated from a crawl using:
```commandline
fluxture kml --group-by ip crawl.db output.kml
```
The geolocation database can be updated from MaxMind by running:
```commandline
fluxture update-geo-db
```
An existing crawl database can be re-analyzed for missing or updated geolocations (_e.g._, from an updated MaxMind database) by running:
```commandline
fluxture geolocate crawl.db
```
## Topological Analysis
Fluxture can calculate topological statistics about the centrality of a crawled network by running:
```commandline
fluxture topology crawl.db
```
## Programmatic Interface
```python
from fluxture.crawl_schema import CrawlDatabase
with CrawlDatabase("crawl.db") as db:
for node in db.nodes:
print(f"Node {node.ip}:{node.port} {node.state!s}")
location = node.get_location()
if location is not None:
print(f"\tLocation:\t{location.continent_code}\t{location.country_code}\t{location.city}")
else:
print("\tLocation:\t?")
version = node.get_version()
if version is not None:
print(f"\tVersion:\t{version.version!s}")
else:
print("\tVersion:\t?")
print(f"\tOut-Edges:\t{', '.join(str(neighbor.ip) for neighbor in node.get_latest_edges())}")
```
## License and Acknowledgements
This research was developed by [Trail of Bits](https://www.trailofbits.com/) based upon work supported by DARPA under
Contract No. HR001120C0084. Any opinions, findings and conclusions or recommendations expressed in this material are
those of the authors and do not necessarily reflect the views of the United States Government or DARPA.
It is licensed under the [Apache 2.0 license](LICENSE). © 2020–2021, Trail of Bits.