Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/bitcoin-core/bitcoin-maintainer-tools

External repository for Bitcoin Core related maintenance tools
https://github.com/bitcoin-core/bitcoin-maintainer-tools

Last synced: over 1 year ago
JSON representation

External repository for Bitcoin Core related maintenance tools

Awesome Lists containing this project

README 

          
External repository for Bitcoin Core related maintenance tools.



github-merge

------------



A small script to automate merging pull-requests securely and sign them with GPG.



For example, if the "to" repo is identical to the "from" repo:



```bash

./github-merge.py 1234

```



(in any git repository) will help you merge pull request #1234 for the configured repository.



Otherwise, for a differing "from" repo:



```bash

./github-merge.py --repo-from=bitcoin-core/gui 1234

```



will fetch the pull request from another monotree repository. Be sure to also set `githubmerge.pushmirrors` (see below).



What it does:

* Fetch master and the pull request.

* Locally construct a merge commit.

* Show the diff that merge results in.

* Ask you to verify the resulting source tree (so you can do a make check or whatever).

* Ask you whether to GPG sign the merge commit.

* Ask you whether to push the result upstream.



This means that there are no potential race conditions (where a

pull request gets updated while you're reviewing it, but before you click

merge), and when using GPG signatures, that even a compromised GitHub

couldn't mess with the sources.



### Setup



Configuring the github-merge tool for the bitcoin repository is done in the following way:



    git config githubmerge.repository bitcoin/bitcoin

    git config githubmerge.pushmirrors "git@github.com:bitcoin-core/gui.git,git@github.com:YourPrivateMirror/bitcoin-core.git"

    git config githubmerge.testcmd "make -j4 check" (adapt to whatever you want to use for testing)

    git config --global user.signingkey mykeyid



If you want to use HTTPS instead of SSH for accessing GitHub, you need set the host additionally:



    git config githubmerge.host "https://github.com"  (default is "git@github.com", which implies SSH)



### Authentication (optional)



The API request limit for unauthenticated requests is quite low, but the

limit for authenticated requests is much higher. If you start running

into rate limiting errors it can be useful to set an authentication token

so that the script can authenticate requests.



- First, go to [Personal access tokens](https://github.com/settings/tokens).

- Click 'Generate new token'.

- Fill in an arbitrary token description. No further privileges are needed.

- Click the `Generate token` button at the bottom of the form.

- Copy the generated token (should be a hexadecimal string)



Then do:



    git config --global user.ghtoken "pasted token"



### Create and verify timestamps of merge commits



To create or verify timestamps on the merge commits, install the OpenTimestamps

client via `pip3 install opentimestamps-client`. Then, download the gpg wrapper

`ots-git-gpg-wrapper.sh` and set it as git's `gpg.program`. See

[the ots git integration documentation](https://github.com/opentimestamps/opentimestamps-client/blob/master/doc/git-integration.md#usage)

for further details.



update-translations

-------------------



Run this script from the root of a repository to update all translations from Transifex.

It will do the following automatically:



- Fetch all translations

- Post-process them into valid and committable format

- Add missing translations to the build system (TODO)



To be able to pull translation files from the Transifex website, it needs

the [Transifex CLI](https://github.com/transifex/cli).



clang-format

------------



A script to format cpp source code according to the .clang-format file in the bitcoin repo.

This should only be applied to new files or files which are currently not actively developed on.

Also, git subtrees are not subject to formatting.



Note: The script is currently untested and unmaintained, but kept for archival reasons, in

case it is planned to be used some day.



build-for-compare

--------------------



Build for binary comparison.



See `build-for-compare.py --help` for more information.



Builds from current directory, which is assumed to be a git clone of the bitcoin repository.



**DO NOT RUN this with the nocopy=1 flag set on working tree if you have any local additions, it will nuke all

non-repository files, multiple times over. By leaving nocopy off (default) the git tree is copied to a temporary

directory and all operations are performed there.**



Example:

```bash

git clone https://github.com/bitcoin/bitcoin.git bitcoin-compare

cd bitcoin-compare

../bitcoin-maintainer-tools/build-for-compare.py 4731cab 2f71490

sha256sum /tmp/compare/bitcoind.*.stripped

git diff -W --word-diff /tmp/compare/4731cab /tmp/compare/2f71490

```



backport

--------



Script to backport pull requests in order of merge, to minimize number of conflicts.

Pull ids are listed in `to_backport.txt` or given on the command line, and they must be prefixed

with the repository name, e.g.:



```bash

../bitcoin-maintainer-tools/backport.py bitcoin/bitcoin#21907 bitcoin-core/gui#277 bitcoin-core/gui#365



```



Requires `pip3 install gitpython` or similar.



unittest-statistics

--------------------------



`unittest-statistics.py` can be used to print a table of the slowest 20 unit tests.



Usage:

```bash

unittest-statistics.py  []

```



For example:

```bash

unittest-statistics.py src/test/test_bitcoin wallet_tests

```



treehash512

--------------



This script will show the SHA512 tree has for a certain commit, or HEAD

by default.



Usage:



```bash

treehash512.py []

```



This should match the Tree-SHA512 commit metadata field added by

github-merge.



signoff

----------



This is an utility to manually add a treehash to the HEAD commit and then

gpg-sign it. This is useful when there is the need to manually add a commit.



Usage:



```bash

signoff.py

```

(no command line arguments)



When there is already a treehash on the HEAD commit, it is compared against

what is computed. If this matches, it continues. If the treehash mismatches an

error is thrown. If there is no treehash it adds the "Tree-SHA512:" header with

the computed hash to the commit message.



After making sure the treehash is correct it verifies whether the commit is

signed. If so it just displays the signature, if not, it is signed.



subtree updates

---------------



Bitcoin Core comes with several subtrees (c.f. https://github.com/bitcoin/bitcoin/tree/master/test/lint#git-subtree-checksh)

To update the subtree, make sure to fetch the remote of the subtree.

Then a simple call should pull in and squash the changes:



```sh

git subtree pull --prefix src/${prefix} ${remote_repo} ${ref} --squash

```



For setting up a subtree, refer to `git help subtree`.



check-dnsseeds

---------------



Sanity-check the DNS seeds used by Bitcoin Core.



Usage:



```bash

check-dnsseeds.py

```



Example output:



```bash

* Mainnet

OK   seed.bitcoin.sipa.be (40 results)

OK   dnsseed.bluematt.me (33 results)

FAIL dnsseed.bitcoin.dashjr.org

OK   seed.bitcoinstats.com (50 results)

OK   seed.bitcoin.jonasschnelli.ch (38 results)

OK   seed.btc.petertodd.org (23 results)

OK   seed.bitcoin.sprovoost.nl (35 results)

OK   dnsseed.emzy.de (41 results)



* Testnet

OK   testnet-seed.bitcoin.jonasschnelli.ch (36 results)

OK   seed.tbtc.petertodd.org (38 results)

OK   testnet-seed.bluematt.me (5 results)

```



delete non-reduced fuzz inputs

------------------------------



Refer to the documentation inside the script.



fastcopy-chaindata

-------------------



Fast local copy of Bitcoin Core blockchain state.



```bash

fastcopy-chaindata.py ~/.bitcoin /path/to/temp/datadir

```



This utility hardlinks all but the last block data file (rev and blk),

and hardlinks all .ldb files to the destination. The last data files as well

as the other leveldb data files (such as the log) are copied.



This relies on the fact that block files (except the last) and ldb files

are read-only once they are written.



Warning: Hardlinking only works within a filesystem, and may not work for all

filesystems.



list-pulls

----------



Script to parse git commit list, extract github issues to create a changelog in

text and json format.



Run this in the root directory of the repository.



This requires an up-to-date checkout of https://github.com/zw/bitcoin-gh-meta.git

in the parent directory, or environment variable `GHMETA`.



It takes a range of commits and a .json file of PRs to exclude, for

example if these are already backported in a minor release. This can be the pulls.json

generated from a previous release.



Example usage:



    ../maintainer-tools/list-pulls.py v0.18.0 0.19 relnot/pulls-exclude.json > relnot/pulls.md



The output of this script is a first draft based on rough heuristics, and

likely needs to be extensively manually edited before ending up in the release

notes.



make-tag

--------



Make a new release tag, performing a few checks.



Usage: `make-tag.py `.



ghwatch

-------



This is a script to watch your github notifications in the terminal. It will show a table that is refreshed every 10 minutes (configurable). It can be exited by pressing ESC or Ctrl-C.



### Dependencies



The `github` python module is a required dependency for github API access. This can be installed for your user using `pip3 install --user PyGithub`, or globally using your distribution's package manager e.g. `apt-get install python3-github`.



### Configuration



To generate a default configuration file in `~/.config/ghwatch/ghwatch.conf` do



```

./ghwatch.py --default-config

```



Then, edit the configuration file. Only thing that is necessary to change is `ghtoken`. You will need to create a github [authentication token](https://github.com/settings/tokens) then insert it here:



```

    "ghtoken": "",

```



Depending on your browser preference you might want to change `browser`, this is the command that will be invoked when clicking on an issue number. It defaults to `null` which indicates to use the system web browser.



If you want to see PR status (and other issue details like labels), point `meta` for the `bitcoin/bitcoin` repository to an up-to-date checkout of [bitcoin-gh-meta](https://github.com/zw/bitcoin-gh-meta).

```

    "meta": {

        "bitcoin/bitcoin": "/path/to/bitcoin-gh-meta"

    },

```



To keep this repository up to date you can set the interval in seconds in 'auto_update', default is 0 (i.e. no automatic update). Be aware that [bitcoin-gh-meta](https://github.com/zw/bitcoin-gh-meta) is being refreshed every two hours (7200 seconds).



Sorting the notifications by {reason, time} can be enabled with the 'sort_notifications' boolean field (default=false).



By editing the `label_prio` structure it is possible to affect what labels will be shown. The first label encountered in this list for an issue in the associated repository will be shown as the label in the table.



### Command-line options



Some other settings can be set through command line options. See `./ghwatch.py --help` for the list of command line options and their descriptions.



### Display



Most of the columns are self-explanatory, except for:



- `r` this [notification reason](https://docs.github.com/en/rest/reference/activity#notification-reasons) from the GH API as a two-letter code. This can be:

  - `as` assign

  - `au` author

  - `co` comment

  - `in` invitation

  - `ma` manual

  - `me` mention

  - `rr` review requested

  - `sa` security alert

  - `sc` state change

  - `su` subscribed

  - `tm` team mention

- `k` the kind of notification as a letter. This can be:

  - `P` pull request

  - `I` issue

  - `C` commit



### Controls



Left-click on a PR number to show details in a web browser.



The program can be exited by pressing ESC or Ctrl-C.