Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/mcdee/blockparser

Fast, quick and dirty bitcoin blockchain parser
https://github.com/mcdee/blockparser

Last synced: about 2 months ago
JSON representation

Fast, quick and dirty bitcoin blockchain parser

Awesome Lists containing this project

README 

        
blockparser

===========



    Credits:

    --------



        Written by [email protected]

        If you find this useful: 1ZnortsoStC1zSTXbW6CUtkvqew8czMMG



    What:

    -----



        A fairly fast, quick and dirty bitcoin whole blockchain parser.



    Why:

    ----



        . Few dependencies: openssl-dev, boost



        . Very quickly extract information from the entire blockchain.



        . Code is simple and helps to understand how the data structure underlying bitcoin works.



    Build it:

    ---------



        . Turn your x86-64 Ubuntu box on



        . Make sure you have an up to date satoshi client blockchain in ~/.bitcoin



        . Run this:



            sudo apt-get install libssl-dev build-essential g++-4.4 libboost-all-dev libsparsehash-dev git-core perl

            git clone git://github.com/znort987/blockparser.git

            cd blockparser

            make



    Try it:

    -------



        . Compute simple blockchain stats, full chain parse (< 1 second)



            ./parser simpleStats



        . Extract all transactions for popular address 1dice6wBxymYi3t94heUAG6MpG5eceLG1 (20 seconds)



            ./parser transactions 06f1b66fa14429389cbffa656966993eab656f37



        . Compute the closure of an address, that is the list of addresses that provably belong to the same person (20 seconds):



            ./parser closure 06f1b66fa14429389cbffa656966993eab656f37



        . Compute and print the balance for all keys ever used in a TX since the beginning of time (30 seconds):



            ./parser allBalances >allBalances.txt



        . See how much of the BTC 10K pizza tainted each of the TX in the chain



            ./parser taint >pizzaTaint.txt



        . See all the block rewards and fees:



            ./parser rewards >rewards.txt



        . See a greatly detailed dump of the pizza transaction



            ./parser show



    Caveats:

    --------



        . You need an x86-84 ubuntu box and a recent version of GCC(>=4.4), recent versions of boost

          and openssl-dev. The whole thing is very unlikely to work or even compile on anything else.



        . It needs quite a bit of RAM to work. Never exactly measured how much, but the hash maps will

          grow quite fat. I might switch them to something different that spills over to disk at some

          point. For now: it works fine with 8 Gigs.



        . The code isn't particularly clean or well architected. It was just a quick way for me to learn

          about bitcoin. There isnt much in the way of comments either.



        . OTOH, it is fairly simple, short, and efficient. If you want to understand how the blockchain

          data structure works, the code in parser.cpp is a solid way to start.



    Hacking the code:

    -----------------



        . parser.cpp contains a generic parser that mmaps the blockchain, parses it and calls

          "user-defined" callbacks as it hits interesting bits of information.



        . util.cpp contains a grab-bag of useful bitcoin related routines. Interesting examples include:



            showScript

            getBaseReward

            solveOutputScript

            decompressPublicKey



        . cb/allBalances.cpp    :   code to all balance of all addresses.

        . cb/closure.cpp        :   code to compute the transitive closure of an address

        . cb/csv.cpp            :   code to product a CSV dump of the blockchain

        . cb/dumpTX.cpp         :   code to display a transaction in very great detail

        . cb/help.cpp           :   code to dump detailed help for all other commands

        . cb/pristine.cpp       :   code to show all "pristine" (i.e. unspent) blocks

        . cb/rewards.cpp        :   code to show all block rewards (including fees)

        . cb/simpleStats.cpp    :   code to compute simple stats.

        . cb/sql.cpp            :   code to product an SQL dump of the blockchain

        . cb/taint.cpp          :   code to compute the taint from a given TX to all TXs.

        . cb/transactions.cpp   :   code to extract all transactions pertaining to an address.



        . You can very easily add your own custom command. You can use the existing callbacks in

          directory ./cb/ as a template to build your own:



                cp cb/allBalances.cpp cb/myExtractor.cpp

                Add to Makefile

                Hack away

                Recompile

                Run



        . You can also read the file callback.h (the base class from which you derive to implement your

          own new commands). It has been heavily commented and should provide a good basis to pick what

          to overload to achieve your goal.



        . The code makes heavy use of the google dense hash maps. You can switch it to use sparse hash

          maps (see util.h, search for: DENSE, undef it). Sparse hash maps are slower but save quite a

          bit of RAM.



    License:

    --------



        Code is in the public domain.