{"id":20048479,"url":"https://github.com/aggstam/btc-classifier","last_synced_at":"2026-05-19T10:38:25.088Z","repository":{"id":155101724,"uuid":"562091297","full_name":"aggstam/btc-classifier","owner":"aggstam","description":"Python scripts suite for generating Bitcoin transactions graphs and analyzing them using machine learning.","archived":false,"fork":false,"pushed_at":"2023-01-31T17:52:05.000Z","size":374,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-01-12T20:33:34.736Z","etag":null,"topics":["bitcoin","classification","cryptocurrency","graph","machine-learning","parsing","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/aggstam.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-11-05T09:31:32.000Z","updated_at":"2024-12-12T08:57:32.000Z","dependencies_parsed_at":null,"dependency_job_id":"27bb3df1-9511-41b8-b1ba-489c8a451b71","html_url":"https://github.com/aggstam/btc-classifier","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aggstam%2Fbtc-classifier","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aggstam%2Fbtc-classifier/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aggstam%2Fbtc-classifier/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aggstam%2Fbtc-classifier/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/aggstam","download_url":"https://codeload.github.com/aggstam/btc-classifier/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241478045,"owners_count":19969212,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bitcoin","classification","cryptocurrency","graph","machine-learning","parsing","python"],"created_at":"2024-11-13T11:44:14.532Z","updated_at":"2026-05-19T10:38:25.082Z","avatar_url":"https://github.com/aggstam.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# btc-classifier\nPython scripts suite for generating Bitcoin transactions graphs and analyzing them using machine learning.\n\n## Overview\nThe solution architecture high-level view:\n\n![Solution architecture](https://github.com/aggstam/btc-classifier/blob/main/images/Solution%20Architecture.png)\n\n### parser.py \nThis script is a modified version of Blockchain parser by Denis Leonov [1].\n\u003cbr\u003e\nScript parses blk*.dat files of the Bitcoin blockchain and produces files containing simplyfied transactions,\nusing the btcpy library [2].\n\n![Simplyfied BTC transaction](https://github.com/aggstam/btc-classifier/blob/main/images/Simplified_Bitcoin_Transaction_Example_corrected.png)\n\nUsing parser.py script, the Bitcoin blockchain was parsed until file blk02399.dat, sizing 298 GB in total.\n\u003cbr\u003e\nAll blk*.dat files were parsed after 60 hours.\n\u003cbr\u003e\nOutput file size averaged at 180 MB, with a total size of 426 GB.\n\n### reader.py\nScript parses the output files of parser.py script and imports retrieved information to the Database.\n\u003cbr\u003e\nThis streamline method was chosen, in order to simplify parsing and importing steps, and also enabling batch execution.\n\u003cbr\u003e\nAll output files of parser.py script were parsed after 9 days 2 hours 32 minutes and 29 seconds, \nresulting in 652 GB of disk size for the Database using row compression.\n\u003cbr\u003e\nTo further increase Database performance, field indexing can be enabled,\nalong with the modification of MySQL configuration parameter innodb_buffer_pool_size,\nwhich can be set to 16 GB, to increase the Database RAM cache size, for faster query executions.\n\u003cbr\u003e\nYou can use provided database_schema_creation.sql to create the following schema:\n\n![Database schema](https://github.com/aggstam/btc-classifier/blob/main/images/Database_Schema.png)\n\n### transactions_retriever.py\nImported data are processed by transactions_retriever.py script, which generates the execution dataset for the analyzer.py script.\n\u003cbr\u003e\nA random address sample is retrieved from the Entity-address dataset for 2010-2018 Bitcoin transactions [3].\n\u003cbr\u003e\nFor each address in the sample, all their transaction ids are retrieved from the Database, to create the execution dataset output files.\n\n### analyzer.py\nThis script performs an unsupervised Machine Learning task, \nusing Deep Graph Infomax [4] and Graph Convolutional Network (GCN) [5] algorithms for node representation learning.\n\u003cbr\u003e\nAfter node features have been extracted, classification of each node on the temporal network graph\nfor the Bitcoin transactions dataset is executed, using Logistic regression.\n\u003cbr\u003e\nStellarGraph [6] libraries are used for the Machine Learning tasks.\n\u003cbr\u003e\nOn each execution, a dedicated folder is created containing all result files.\n\u003cbr\u003e\nResults files include each fold predictions, the graphs .graphml file and a loss over epoch diagram.\n\n![Generated .graphml file](https://github.com/aggstam/btc-classifier/blob/main/images/analyzer_generate_graph_example.png)\n\n![Generated history file](https://github.com/aggstam/btc-classifier/blob/main/images/analyzer_deep_graph_infomax_plot.png)\n\n## Execution\nBefore executing any script, create a `python` virtual environment\nand source it:\n```shell\n$ python -m venv venv\n$ source venv/bin/activate\n```\nInstall required dependencies:\n```shell\n$ pip install -r requirements.txt\n```\n\n**!IMPORTANT:**\n\nstellargraph requires Python \u003e=3.6.0, \u003c3.8.0 so make sure you have that python versions installed, along with their pip module.\n\u003cbr\u003e\nExample requirements installation for python3.8, while system uses python3.10\n```shell\n$ python3.8 -m ensurepip --upgrade\n$ python3.8 -m pip install --user -r requirements.txt\n```\nMySQL connector for python must also be installed.\n\u003cbr\u003e\nLink: https://dev.mysql.com/doc/connector-python/en/connector-python-installation.html\n\nAll scripts can be executed by using their name:\n```shell\n$ python analyzer.py\n```\n\n## Configuration\nThis section describes all the configuration needed for scripts execution.\n\u003cbr\u003e\nPlease configure all values appropriately before execution.\n\n### parser.py\n| Line | Name | Description                      |\n|------|------|----------------------------------|\n|  94  | dirA | path to Bitcoin blk files folder |\n|  95  | dirB | script output folder             |\n\n### reader.py\n| Line | Name        | Description                    |\n|------|-------------|--------------------------------|\n|  70  | host        | MySQL host                     |\n|  71  | user        | MySQL user                     |\n|  72  | password    | MySQL user password            |\n|  73  | database    | MySQL database name            |\n| 151  | dir         | parser.py script output folder |\n| 152  | start_index | parse from blk number          |\n| 153  | end_index   | parse until blk number         |\n\n### transactions_retrieve.py\n| Line  | Name                  | Description                       |\n|-------|-----------------------|-----------------------------------|\n| 33-38 | *_ADDRESSES_FILE      | path to each address file dataset |\n|  42   | TXIN_QUERY.timestamp  | tx timestamp max value            |\n|  43   | TXOUT_QUERY.timestamp | tx timestamp max value            |\n| 46-52 | *_CSV_FILE            | script output csv files           |\n|  70   | host                  | MySQL host                        |\n|  71   | user                  | MySQL user                        |\n|  72   | password              | MySQL user password               |\n|  73   | database              | MySQL database name               |\n\n### analyzer.py\n| Line  | Name          | Description                                      |\n|-------|---------------|--------------------------------------------------|\n|  55   | OUTPUT_FOLDER | script output folder                             |\n| 56-62 | *_CSV_FILE    | transactions_retrieve.py script output csv files |\n|  69   | FOLDS         | K-Fold validation k parameter                    |\n|  70   | EPOCHS        | ML training epochs                               |\n\n## References\n[1] Blockchain parser: https://github.com/ragestack/blockchain-parser\n\u003cbr\u003e\n[2] btcpy: https://github.com/chainside/btcpy\n\u003cbr\u003e\n[3] Original dataset: https://github.com/Maru92/EntityAddressBitcoin\n\u003cbr\u003e\n[4] Veličković, P., Fedus, W., Hamilton, W. L., Lio, P., Bengio, Y., Hjelm, R. D., (2019). Deep Graph Infomax. ICLR, arXiv:1809.10341.\n\u003cbr\u003e\n[5] Kipf, T. N., Max Welling, (2017). Graph Convolutional Networks (GCN): Semi-Supervised Classification with Graph Convolutional Networks. International Conference on Learning Representations (ICLR).\n\u003cbr\u003e\n[6] StellarGraph: https://github.com/stellargraph/stellargraph\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faggstam%2Fbtc-classifier","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faggstam%2Fbtc-classifier","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faggstam%2Fbtc-classifier/lists"}