Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/hoprnet/hoprnet

HOPR is an open incentivized mixnet which enables privacy-preserving point-to-point data exchange. HOPR is similar to Tor but actually private, decentralized and economically sustainable.
https://github.com/hoprnet/hoprnet

blockchain ethereum privacy

Last synced: 3 months ago
JSON representation

HOPR is an open incentivized mixnet which enables privacy-preserving point-to-point data exchange. HOPR is similar to Tor but actually private, decentralized and economically sustainable.

Awesome Lists containing this project

README 

          




  

    HOPR Logo

  



  

  
HOPR


  


    A project by the HOPR Association

  


  


    HOPR is a privacy-preserving messaging protocol which enables the creation of a secure communication network via relay nodes powered by economic incentives using digital tokens.

  



## Table of Contents



- [Table of Contents](#table-of-contents)

- [About](#about)

- [Install](#install)

  - [Install via Docker](#install-via-docker)

  - [Install via Nix package manager](#install-via-nix-package-manager)

  - [Install via linux package manager](#install-via-linux-package-manager)

- [Usage](#usage)

  - [Environment variables](#environment-variables)

  - [Example execution](#example-execution)

  - [Using Docker Compose with extended HOPR node monitoring](#using-docker-compose-with-extended-hopr-node-monitoring)

  - [REST API](#rest-api)

- [Testnet accessibility](#testnet-accessibility)

- [Migrating between releases](#migrating-between-releases)

- [Develop](#develop)

  - [Nix environment setup](#nix-environment-setup)

    - [Nix flake outputs](#nix-flake-outputs)

    - [Code Formatting](#code-formatting)

    - [Code Linting](#code-linting)

    - [Generate the Python SDK](#generate-the-python-sdk)

  - [Building a Docker image](#building-a-docker-image)

- [Local cluster](#local-cluster)

- [Test](#test)

  - [Github Actions CI](#github-actions-ci)

  - [End-to-End Testing](#end-to-end-testing)

    - [Running Tests Locally](#running-tests-locally)

      - [Test execution](#test-execution)

- [Profiling \& Instrumentation](#profiling--instrumentation)

  - [`tokio` executor instrumentation](#tokio-executor-instrumentation)

  - [OpenTelemetry export](#opentelemetry-export)

  - [Profiling Criterion benchmarks via `flamegraph`](#profiling-criterion-benchmarks-via-flamegraph)

    - [Prerequisites](#prerequisites)

    - [Profiling the benchmarking binaries](#profiling-the-benchmarking-binaries)

  - [HOPR packet capture](#hopr-packet-capture)

- [Contact](#contact)

- [License](#license)



## About



The HOPR project produces multiple artifacts that allow running, maintaining and modifying the HOPR node.

The most relevant components for production use cases are:



1. [hopr-lib](https://hoprnet.github.io/hoprnet/hopr_lib/index.html)

   - A fully self-contained referential implementation of the HOPR protocol over a libp2p based connection mechanism that can be incorporated into other projects as a transport layer.

2. [hoprd](https://hoprnet.github.io/hoprnet/hoprd/index.html)

   - Daemon application providing a higher level interface for creating a HOPR protocol compliant node that can use a dedicated REST API.

3. [hoprd-api-schema](https://hoprnet.github.io/hoprnet/hoprd_api_schema/index.html)

   - Utility to generate the OpenAPI spec for the `hoprd` served REST API.

4. [hoprd-cfg](https://hoprnet.github.io/hoprnet/hoprd_cfg/index.html)

   - Utility for configuration management of the `hoprd`



Unless stated otherwise, the following sections only apply to `hoprd`.



## Install



For production purposes always run the latest stable release.



Multiple options for installation exist, the preferred choice for any production system should be to use the container image (e.g. using `docker`).



All releases and associated changelogs are located in the [official releases](https://github.com/hoprnet/hoprnet/releases) section of the [`hoprnet`](https://github.com/hoprnet/hoprnet) repository.



### Install via Docker



The following instructions show how any `$RELEASE` may be installed, to select the release, override the `$RELEASE` variable, e.g.:



- `export RELEASE=latest` to track the latest changes on the repository's `master` branch

- `export RELEASE=kaunas` to track the latest changes on the repository's `release/kaunas` branch (3.0.X)

- `export RELEASE=` to get a specific ``



Container image has the format

`europe-west3-docker.pkg.dev/hoprassociation/docker-images/$PROJECT:$RELEASE`.

where:



- `$PROJECT` can be: `hoprd`



Pull the container image with `docker`:



```bash

docker pull europe-west3-docker.pkg.dev/hoprassociation/docker-images/hoprd:kaunas

```



It is recommended to setup an alias `hoprd` for the docker command invocation.



### Install via [Nix package manager][1]



WARNING: This setup should only be used for development or advanced usage without any further support.



Clone and initialize the [`hoprnet`](https://github.com/hoprnet/hoprnet) repository:



```bash

git clone https://github.com/hoprnet/hoprnet

cd hoprnet

```



Build and install the `hoprd` binary, e.g. on a UNIX platform:



```bash

nix build

sudo cp result/bin/* /usr/local/bin/

```



To build and access man pages for `hoprd`:



```bash

# Build man page for hoprd

nix build .#hoprd-man

man ./result/share/man/man1/hoprd.1.gz



# Or install them system-wide

sudo cp -r result/share/man/man1/* /usr/local/share/man/man1/

```



### Install via linux package manager



Linux packages are available at every github release, download the latest package from https://github.com/hoprnet/hoprnet/releases/latest

To install on specific distribution, see [detailed information](./deploy/nfpm/README.md)



## Usage



`hoprd` provides various command-line switches to configure its behaviour. For reference these are documented here as well:



```bash

$ hoprd --help

HOPR node executable



Usage: hoprd [OPTIONS]



Options:

      --identity 

          The path to the identity file [env: HOPRD_IDENTITY=]

      --data 

          Specifies the directory to hold all the data [env: HOPRD_DATA=]

      --host 

          Host to listen on for P2P connections [env: HOPRD_HOST=]

      --announce...

          Announce the node on chain with a public address [env: HOPRD_ANNOUNCE=]

      --api...

          Expose the API on localhost:3001 [env: HOPRD_API=]

      --apiHost 

          Set host IP to which the API server will bind [env: HOPRD_API_HOST=]

      --apiPort 

          Set port to which the API server will bind [env: HOPRD_API_PORT=]

      --apiToken 

          A REST API token and for user authentication [env: HOPRD_API_TOKEN=]

      --password 

          A password to encrypt your keys [env: HOPRD_PASSWORD=]

      --blokliUrl 

          URL for Blokli provider to be used for the node to connect to blockchain [env: HOPRD_BLOKLI_URL=]

      --init...

          initialize a database if it doesn't already exist [env: HOPRD_INIT=]

      --forceInit...

          initialize a database, even if it already exists [env: HOPRD_FORCE_INIT=]

      --probeRecheckThreshold 

          Timeframe in seconds after which it is reasonable to recheck the nearest neighbor [env: HOPRD_PROBE_RECHECK_THRESHOLD=]

      --configurationFilePath 

          Path to a file containing the entire HOPRd configuration [env: HOPRD_CONFIGURATION_FILE_PATH=]

      --safeAddress 

          Address of Safe that safeguards tokens [env: HOPRD_SAFE_ADDRESS=]

      --moduleAddress 

          Address of the node management module [env: HOPRD_MODULE_ADDRESS=]

  -h, --help

          Print help

  -V, --version

          Print version

```



### Environment variables



On top of the default configuration options generated for the command line, the following environment variables can be used in order to tweak the node functionality:



- `ENV_WORKER_THREADS` - the number of environment worker threads for the tokio executor

- `HOPRD_LOG_FORMAT` - override for the default stdout log formatter (follows tracing formatting options)

- `HOPRD_USE_OPENTELEMETRY` - enable the OpenTelemetry output for this node

- `HOPRD_OTEL_SIGNALS` - comma-separated OTLP signals to export when OpenTelemetry is enabled (`traces`, `logs`, `metrics`), defaults to `traces`

- `OTEL_SERVICE_NAME` - the name of this node for the OpenTelemetry service

- `HOPR_INTERNAL_CHAIN_DISCOVERY_CHANNEL_CAPACITY` - the maximum capacity of the channel for chain generated discovery signals for the p2p transport

- `HOPR_INTERNAL_ACKED_TICKET_CHANNEL_CAPACITY` - the maximum capacity of the acknowledged ticket processing queue

- `HOPR_INTERNAL_LIBP2P_MAX_CONCURRENTLY_DIALED_PEER_COUNT` - the maximum number of concurrently dialed peers in libp2p

- `HOPR_INTERNAL_LIBP2P_MAX_NEGOTIATING_INBOUND_STREAM_COUNT` - the maximum number of negotiating inbound streams

- `HOPR_INTERNAL_LIBP2P_SWARM_IDLE_TIMEOUT` - timeout for all idle libp2p swarm connections in seconds

- `HOPR_INTERNAL_DB_PEERS_PERSISTENCE_AFTER_RESTART_IN_SECONDS` - cutoff duration from now to not retain the peers with older records in the peers database (e.g. after a restart)

- `HOPR_INTERNAL_MANUAL_PING_CHANNEL_CAPACITY` - the maximum capacity of awaiting manual ping queue

- `HOPR_INTERNAL_MIXER_CAPACITY` - capacity of the mixer buffer

- `HOPR_INTERNAL_MIXER_MINIMUM_DELAY_IN_MS` - the minimum mixer delay in milliseconds

- `HOPR_INTERNAL_MIXER_DELAY_RANGE_IN_MS` - the maximum range of the mixer delay from the minimum value in milliseconds

- `HOPR_INTERNAL_PROTOCOL_BIDIRECTIONAL_CHANNEL_CAPACITY` - the maximum capacity of HOPR messages processed by the node

- `HOPR_INTERNAL_SESSION_CTL_CHANNEL_CAPACITY` - the maximum capacity of the session control channel

- `HOPR_INTERNAL_SESSION_INCOMING_CAPACITY` - the maximum capacity of the queue storing unprocessed incoming and outgoing messages inside a session

- `HOPR_INTERNAL_SESSION_BALANCER_LEVEL_CAPACITY` - the maximum capacity of the session balancer

- `HOPR_INTERNAL_RAW_SOCKET_LIKE_CHANNEL_CAPACITY` - the maximum capacity of the raw socket-like bidirectional API interface

- `HOPR_INTERNAL_IN_PACKET_PIPELINE_CONCURRENCY` - the maximum number of incoming packets to process concurrently (default: 8 × CPU cores, 0 = no limit)

- `HOPR_INTERNAL_OUT_PACKET_PIPELINE_CONCURRENCY` - the maximum number of outgoing packets to process concurrently (default: 8 × CPU cores, 0 = no limit)

- `HOPR_CPU_TASK_QUEUE_LIMIT` - maximum number of CPU-bound tasks (queued + running) in the Rayon thread pool. If not set, the queue is unbounded. Set this to prevent unbounded queue growth under sustained load (e.g., `1000`). When the limit is reached, new tasks are rejected and packet decode operations may fail with "local CPU queue full" errors.

- `HOPR_BALANCER_PID_P_GAIN` - proportional (P) gain for the PID controller in outgoing SURB balancer (default: `0.6`)

- `HOPR_BALANCER_PID_I_GAIN` - integral (I) gain for the PID controller in outgoing SURB balancer (default: `0.7`)

- `HOPR_BALANCER_PID_D_GAIN` - derivative (D) gain for the PID controller in outgoing SURB balancer (default: `0.2`)

- `HOPR_CAPTURE_PACKETS` - allow capturing customized HOPR packet format to a PCAP file or to a `udpdump` host. Note that `hoprd` must be built with the `capture` feature.

- `HOPR_CAPTURE_PATH_TRIGGER` - path used as trigger to start capturing customized HOPR packets. When there exists a file in that path, it will start capturing data.

- `HOPR_TRANSPORT_MAX_CONCURRENT_PACKETS` - maximum number of concurrently processed incoming packets from all peers (default: 10)

- `HOPR_TRANSPORT_STREAM_OPEN_TIMEOUT_MS` - maximum time (in milliseconds) to wait until a stream connection is established to a peer (default: `2000 ms`)

- `HOPR_PACKET_PLANNER_CONCURRENCY` - maximum number of concurrently planned outgoing packets (default: `10`)

- `HOPR_SESSION_FRAME_SIZE` - The maximum chunk of data that can be written to the Session's input buffer (default: 1500)

- `HOPR_SESSION_FRAME_TIMEOUT_MS` - The maximum time (in milliseconds) for an incomplete frame to stay in the Session's output buffer (default: 800 ms)

- `HOPR_PROTOCOL_SURB_RB_SIZE` - size of the SURB ring buffer (default: 10 000)

- `HOPR_PROTOCOL_SURB_RB_DISTRESS` - threshold since number of SURBs in the ring buffer triggers a distress packet signal (default: 1000)

- `HOPRD_SESSION_PORT_RANGE` - allows restricting the port range (syntax: `start:end` inclusive) of Session listener automatic port selection (when port 0 is specified)

- `HOPRD_SESSION_ENTRY_UDP_RX_PARALLELISM` - sets the number of UDP listening sockets for UDP sessions on Entry node (defaults to number of CPU cores)

- `HOPRD_SESSION_EXIT_UDP_RX_PARALLELISM` - sets the number of UDP listening sockets for UDP sessions on Exit node (defaults to number of CPU cores)

- `HOPRD_NAT` - indicates whether the host is behind a NAT and sets transport-specific settings accordingly (default: `false`)

- `HOPRD_NUM_CPU_THREADS` - sets the number of threads for CPU-bound tasks (default: number of CPU cores / 2)

- `HOPRD_NUM_IO_THREADS` - sets the number of threads for IO-bound tasks (default: number of CPU cores / 2)

- `HOPRD_THREAD_STACK_SIZE` - sets the thread stack size (default: 10 MB)

- `HOPR_METRICS_UNACK_PER_PEER` - enable per-peer unacknowledged ticket cache metrics (disabled by default to reduce Prometheus cardinality). Set to `1` or `true` for debugging specific peer acknowledgement issues. **Warning**: Do not enable in production with many peers.



### Example execution



Running the node without any command-line argument might not work depending on the installation method used. Some command line arguments are required.



Some basic reasonable setup uses a custom identity and enables the REST API of the `hoprd`:



```bash

hoprd --identity /app/hoprd-db/.hopr-identity --password switzerland --init --announce --host "0.0.0.0:9091" --apiToken  --blokliUrl "http://blokli-provider.here"

```



Here is a short breakdown of each argument.



```bash

hoprd

  # store your node identity information in the persisted database folder

  --identity /app/hoprd-db/.hopr-identity

  # set the encryption password for your identity

  --password switzerland

  # initialize the database and identity if not present

  --init

  # announce the node to other nodes in the network and act as relay if publicly reachable

  --announce

  # set IP and port of the P2P API to the container's external IP so it can be reached on your host

  --host "0.0.0.0:9091"

  # specify password for accessing REST API

  --apiToken 

  # blokli provider supplying the HOPR updates

  --blokliUrl "http://blokli-provider.here"

```



### Using Docker Compose with extended HOPR node monitoring



Please follow the documentation for [`docker compose` based deployment](./deploy/compose/README.md).



### REST API



`hoprd` running a REST API exposes an endpoint at `http://
/api-docs/openapi.json` with full OpenApi specification of the used REST API, including the current version of the API.



## Testnet accessibility



To participate in a public network the node must be eligible. See [Network Registry](https://github.com/hoprnet/hoprnet/blob/master/NETWORK_REGISTRY.md) for details.



Node eligibility is not required in a local development cluster (see [Develop section below](#develop)).



## Migrating between releases



There is **NO** backward compatibility between releases.



We attempt to provide instructions on how to migrate your tokens between releases.



1. Set your automatic channel strategy to `passive`.

2. Redeem all unredeemed tickets.

3. Close all open payment channels.

4. Once all payment channels have closed, withdraw your funds to an external

   wallet.

5. Run `info` and take note of the **network name**.

6. Once funds are confirmed to exist in a different wallet, backup `.hopr-identity` folder.

7. Launch new `HOPRd` instance using latest release, observe the account address.

8. Only transfer funds to new `HOPRd` instance if `HOPRd` operates on the **same network** as last release, you can compare the two networks using `info`.



## Develop



Either setup `nix` and `flake` to use the nix environment, or [install Rust toolchain](https://www.rust-lang.org/tools/install) from the `rust-toolchain.toml`, as well as `foundry-rs` binaries (`forge`, `anvil`).



### Nix environment setup



Install `nix` from the official website at [https://nix.dev/install-nix.html](https://nix.dev/install-nix.html).



Create a nix configuration file at `~/.config/nix/nix.conf` with the following content:



```nix

experimental-features = nix-command flakes

```



Install the `nix-direnv` package to introduce the `direnv`:



```bash

nix-env -i nix-direnv

```



Append the following line to the shell rc file (depending on the shell used it can be `~\.zshrc`, `~\.bashrc`, `~\.cshrc`, etc.). Modify the `` variable inside the below command with the currently used (`zsh`, `bash`, `csh`, etc.):



```bash

eval "$(direnv hook )"

```



From within the [`hoprnet`](https://github.com/hoprnet/hoprnet) repository's directory, execute the following command.



```bash

direnv allow .

```



#### Nix flake outputs



We provide a couple of packages, apps and shells to make building and

development easier. You may get the full list like so:



```bash

nix flake show

```



#### Code Formatting



All nix, rust, solidity and python code can be automatically formatted:



```bash

nix fmt

```



These formatters are also automatically run as a Git pre-commit check.



#### Code Linting



All linters can be executed via a Nix flake helper app:



```bash

nix run .#lint

```



This will in particular run `clippy` for the entire Rust codebase.



#### Generate the Python SDK



A Python SDK is not distributed but can be generated to connect to the HOPRd API using the [generate-python-sdk.sh](/scripts/generate-python-sdk.sh) script.



Prerequisites:



- swagger-codegen3

- build the repository to get the `hoprd-api-schema` generated



The generated SDK will be available in the `/tmp/hoprd-sdk-python/` directory. Modify the script to generate SDKs for different programming languages supported by swagger-codegen3.



For usage examples of the generated SDK, refer to the generated README.md file in the SDK directory.



### Building a Docker image



Docker images can be built using the respective nix flake outputs.

The available images can be listed with:



```bash

just list-docker-images

```



The following command builds the `hoprd` image for the host platform:



```bash

nix build .#hoprd-docker

```



If needed images for other platforms can be built by specifying the target

platform. For example, to build the `hoprd` image for the `x86_64-linux`

platform, while being on a Darwin host system, use the following command:



```bash

nix build .#packages.x86_64-linux.hoprd-docker

```



NOTE: Building for different platforms requires nix distributed builds to be

set up properly.

See [Nix documentation](https://nix.dev/manual/nix/2.28/advanced-topics/distributed-builds.html)

for more information.



## Local cluster



The best way to test with multiple HOPR nodes is by using a [local cluster of interconnected nodes](https://github.com/hoprnet/hoprnet/blob/master/SETUP_LOCAL_CLUSTER.md).



## Test



Run all tests: `cargo test`.



Run only unit tests: `cargo test --lib`



### Github Actions CI



We run a fair amount of automation using Github Actions. To ease development

of these workflows one can use [act][2] to run workflows locally in a

Docker environment.



E.g. running the build workflow:



```bash

act -j build

```



For more information please refer to [act][2]'s documentation.



### End-to-End Testing



When using the `nix` environment, the test environment preparation and activation is automatic.



Tests are using the `pytest` infrastructure.



#### Running Tests Locally



##### Test execution



With the environment activated, execute the tests locally:



```bash

just run-smoke-test integration

```



## Profiling & Instrumentation



Multiple layers of profiling and instrumentation can be used to debug the `hoprd`:



### `tokio` executor instrumentation



Requires a special build:



1. Set `RUSTFLAGS="--cfg tokio_unstable"` before building

2. Enable the `prof` feature on the `hoprd` package: `cargo build --feature prof`



Once an instrumented tokio is built into hoprd, the application can be instrumented by `tokio_console` as described in the [official crate documentation](https://docs.rs/tokio-console/latest/tokio_console/#instrumenting-the-application).



### OpenTelemetry export



`hoprd` can stream OpenTelemetry to a compatible endpoint. This behavior is turned off by default. To enable it, configure these environment variables:



- `HOPRD_USE_OPENTELEMETRY` - `true` to enable the OpenTelemetry streaming, `false` to disable it

- `HOPRD_OTEL_SIGNALS` - comma-separated signal list from `traces`, `logs`, `metrics` (default: `traces`)

- `OTEL_SERVICE_NAME` - identifier used as `service.name` for this instance (for example `my_hoprd_instance`)

- `OTEL_EXPORTER_OTLP_ENDPOINT` - base URL of an OTLP endpoint. Transport is inferred from URL scheme (`grpc://...`, `http://...`, or `https://...`)



Examples:



- Traces only (backward-compatible default): `HOPRD_OTEL_SIGNALS=traces`

- Metrics only: `HOPRD_OTEL_SIGNALS=metrics`

- Full export: `HOPRD_OTEL_SIGNALS=traces,logs,metrics`

- With metrics enabled, OTEL exports keep Prometheus family naming (``, `_count`, `_sum`, `_bucket`) and labels (`le` for histogram buckets, `quantile` for summaries).



### Profiling Criterion benchmarks via `flamegraph`



#### Prerequisites



- `perf` installed on the host system

- flamegraph (install via e.g. `cargo install flamegraph`)



#### Profiling the benchmarking binaries



1. Perform a build of your chosen benchmark with `--no-rosegment` linker flag:



   ```

   RUSTFLAGS="-Clink-arg=-fuse-ld=lld -Clink-arg=-Wl,--no-rosegment" cargo bench --no-run -p hopr-crypto-packet

   ```



   Use `mold` instead of `lld` if needed.



2. Find the built benchmarking binary and check if it contains debug symbols:



   ```

   readelf -S target/release/deps/packet_benches-ce70d68371e6d19a | grep debug

   ```



   The output of the above command should contain AT LEAST: `.debug_line`, `.debug_info` and `.debug_loc`



3. Run `flamegraph` on the benchmarking binary of a selected benchmark with a fixed profile time (e.g.: 30 seconds):

   ```

   flamegraph -- ./target/release/deps/packet_benches-ce70d68371e6d19a --bench --exact packet_sending_no_precomputation/0_hop_0_surbs --profile-time 30

   ```

4. The `flamegraph.svg` will be generated in the project root directory and can be opened in a browser.



### HOPR packet capture



Using the environment variable `HOPR_CAPTURE_PACKETS` allows capturing customized HOPR packet format to a PCAP file or to a `udpdump` host. Also define the environment variable `HOPR_CAPTURE_PATH_TRIGGER` with a path that will be periodically inspected, and once a file exists on that path, it will start capturing packets.

However, for that to work the `hoprd` binary has to be built with the feature `capture`.

For ease of use we provide different nix flake outputs that build the `hoprd`

with the `capture` feature enabled:



- `nix build .#hoprd-x86_64-linux-profile`

- `nix build .#hoprd-aarch64-linux-profile`

- `nix build .#hoprd-x86_64-darwin-profile`

- `nix build .#hoprd-aarch64-darwin-profile`



## Contact



- [X](https://x.com/hoprnet)

- [Telegram](https://t.me/hoprnet)

- [Medium](https://medium.com/hoprnet)

- [Reddit](https://www.reddit.com/r/HOPR/)

- [Email](mailto:contact@hoprnet.org)

- [Discord](https://discord.gg/5FWSfq7)

- [Youtube](https://www.youtube.com/channel/UC2DzUtC90LXdW7TfT3igasA)



## License



[GPL v3](https://github.com/hoprnet/hoprnet/blob/master/LICENSE) © HOPR Association



[1]: https://nixos.org/learn.html

[2]: https://github.com/nektos/act