{"id":25768439,"url":"https://github.com/nodefactoryio/vedran","last_synced_at":"2025-02-27T00:51:55.316Z","repository":{"id":53750442,"uuid":"289919989","full_name":"NodeFactoryIo/vedran","owner":"NodeFactoryIo","description":"Auto-funded public p2p infrastructure (APPI) for the Polkadot and, specifically, Kusama ecosystem.","archived":false,"fork":false,"pushed_at":"2021-03-16T08:16:22.000Z","size":968,"stargazers_count":4,"open_issues_count":9,"forks_count":6,"subscribers_count":5,"default_branch":"dev","last_synced_at":"2024-06-21T18:46:44.736Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/NodeFactoryIo.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null}},"created_at":"2020-08-24T12:21:14.000Z","updated_at":"2022-01-17T10:41:27.000Z","dependencies_parsed_at":"2022-09-07T04:00:21.313Z","dependency_job_id":null,"html_url":"https://github.com/NodeFactoryIo/vedran","commit_stats":null,"previous_names":[],"tags_count":14,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NodeFactoryIo%2Fvedran","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NodeFactoryIo%2Fvedran/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NodeFactoryIo%2Fvedran/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NodeFactoryIo%2Fvedran/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/NodeFactoryIo","download_url":"https://codeload.github.com/NodeFactoryIo/vedran/tar.gz/refs/heads/dev","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240959077,"owners_count":19884910,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-02-27T00:51:54.615Z","updated_at":"2025-02-27T00:51:55.306Z","avatar_url":"https://github.com/NodeFactoryIo.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Vedran\n\n\u003e Polkadot chain load balancer.\n\n### Architecture\n\n_Vedran loadbalancer_ is used in conjunction with [Vedran daemon](https://github.com/NodeFactoryIo/vedran-daemon).\nSuppose the node owner wants to register to loadbalancer, than it is required to install and run _Vedran daemon_.\nDaemon executes the registration process and starts providing all relevant information (ping, metrics) to the _Vedran loadbalancer_.\nPlease check [Vedran daemon repo](https://github.com/NodeFactoryIo/vedran-daemon) for more details on the daemon itself.\n\n![Image of vedran architecture](./assets/vedran-arch.png)\n\n## Demo\n\n_This is dockerized demo of entire setup with loadbalancer, node and daemon_\n\n### Requirements\n\n- Install [Docker Engine](https://docs.docker.com/engine/install/)\n- Install [Docker Compose](https://docs.docker.com/compose/install/)\n\n**Run demo with `docker-compose up`**\n\n_After all components have been started and node has sent first valid metrics report (after 30 seconds),\nyou can invoke RPC methods on `localhost:4000` using HTTP requests or on `localhost:4000/ws` using WebSocket request_\nMetrics can be seen on `localhost:3000` hosted grafana under `vedran-dashboard`\n\nYou can check available rpc methods with:\n\n```bash\ncurl -H \"Content-Type: application/json\" -d '{\"id\":1, \"jsonrpc\":\"2.0\", \"method\": \"rpc_methods\"}' http://localhost:4000\n```\nThis demo starts five separate dockerized components:\n- _Polkadot node_ ([repository](https://github.com/paritytech/polkadot))\n- _Vedran daemon_ ([repository](https://github.com/NodeFactoryIo/vedran-daemon))\n- _Vedran loadbalancer_ (port: 4000)\n- _Prometheus server_ (port: 9090) - scrapes metrics from vedran's `/metrics` endpoint\n- _Grafana_ (port: 3000) - Visualizes metrics [Example dashboard](http://localhost:3000/d/7tNmHxAGz/vedran-dashboard?orgId=1\u0026refresh=5s)\n\n\n### Trigger Manual Payout\n\nOur compose setup runs dev chain, and our load balancer uses Allice account to do payout, \nso you don't have to obtain dev DOTs. Polkadot node operator is Bob (he received payout from Allice).\nLoad balancer in this setup runs payout daily, if you don't want to wait,\nyou can run following command which will create an additional container (in compose network) and trigger payout from Allice account:\n\n```\ndocker run --network vedran_default nodefactory/vedran:latest payout --private-key 0xe5be9a5092b81bca64be81d212e7f2f9eba183bb7a90954f7b76361f6edb5c0a --payout-reward 100 --load-balancer-url \"http://vedran:4000/ws\"\n```\n\n## Get **vedran** binary releases\n\nDownload prebuild binary from [releases](https://github.com/NodeFactoryIo/vedran/releases).\nBe careful to chose appropriate binary depending on your OS. For more details on how to run _vedran loadbalancer_ see [Starting loadbalancer](#starting-loadbalancer) part.\n\n## Get **vedran** package\nAlternatively, it is possible to get _vedran_ golang package:\n1. Install [Golang](https://golang.org/doc/install) **1.15 or greater**\n2. Run the command below\n```\ngo get github.com/NodeFactoryIo/vedran\n```\n3. Run vedran from your Go bin directory. For linux systems it will likely be:\n```\n~/go/bin/vedran\n```\nNote that if you need to do this, you probably want to add your Go bin directory to your $PATH to make things easier!\n\n\n## Starting loadbalancer\n\nFirst download latest prebuilt binaries [from releases](https://github.com/NodeFactoryIo/vedran/releases) and unzip it.\n\nLoad balancer is started by invoking **start** command.\n\nFor example `./vedran start --auth-secret=supersecret --private-key=lb-wallet-private-key`.\n\nFor more information you can always run vedran with `--help` flag.\nFor list of all commands run `vedran --help` or for list of all options for specific command run `vedran start --help`.\n\n**Load balancer will expose Polkadot RPC API on port 80 by default (can be changed using flag `--server-port`)**\n\nVedran loadbalancer supports both **HTTP** and **Websockets** protocols for Polkadot RPC API.\n\n- **HTTP - available on root path** `/`\n- **WS - available on separate path** `/ws`\n\n**For production use certificates (e.g. https://certbot.eff.org/) should be generated and passsed via flags: `--key-file`, `--cert-file` and port changed to 443**\n\nStart command will start application on 2 ports that need to be exposed to public:\n 1. RPC entrypoint to nodes and API for nodes to register to load balancer (default: 80)\n 2. HTTP tunnel server for creating tunnels between the node and load balancer so node operators don't to have expose nodes to public network (default: 5223)\n\n### Required flags\n\n`--auth-secret` - authentication secret used for generating tokens\n\n`--private-key` - loadbalancers wallet private key, used for sending founds on payout\n\n### Most important flags\n\n| Flag | Description | Default value |\n|----|-----------|:--------:|\n|`--server-port`|port on which RPC API is exposed|80|\n|`--public-ip`|public IP address of loadbalancer|uses multiple services to find out public IP|\n|`--cert-file`|path to SSL certification file|uses HTTP|\n|`--key-file`|path to SSL private key file|uses HTTP|\n|`--tunnel-port`|port on which tunnel server is listening for connect requests|5223|\n|`--tunnel-port-range`|range of ports that will be used for creating tunnels|20000:30000|\n\n### Other flags\n\n| Flag | Description | Default value |\n|----|-----------|:--------:|\n|`--name`|public name for load balancer|autogenerated name is used|\n|`--capacity`|maximum number of nodes allowed to connect|unlimited capacity|\n|`--whitelist`|comma separated list of node id-s, if provided only these nodes will be allowed to connect. This flag can't be used together with --whitelist-file flag, only one option for setting whitelisted nodes can be used|all nodes are whitelisted|\n|`--whitelist-file`|path to file with node id-s in each line, if provided only these nodes will be allowed to connect. This flag can't be used together with --whitelist flag, only one option for setting whitelisted nodes can be used|all nodes are whitelisted|\n|`--fee`|value between 0-1 representing fixed fee percentage that loadbalancer will take|0.1 (10%)|\n|`--selection`|type of selection that is used for selecting nodes on new request, valid values are `round-robin` and `random`|`round-robin`|\n|`--payout-interval`|automatic payout interval specified as number of days, for more details see [payout instructions](#payouts)|-|\n|`--payout-reward`|defined reward amount that will be distributed on the payout (amount in Planck), for more details see [payout instructions](#payouts)|-|\n|`--lb-payout-address`|address on which load balancer fee will be sent|-|\n|`--log-level`|log level (debug, info, warn, error)|error|\n|`--log-file`|path to file in which logs will be saved|`stdout`|\n|`--root-dir`|root directory for all generated files (e.g. database file, log file)|uses current directory|\n\n### Obtaining DOTs\nIf you want to do anything on Polkadot, Kusama, or Westend, then you'll need to get an account and some DOT, KSM, or WND tokens, respectively.\nWhen initializing payout, you will provide loadbalancer with created account and from this account rewards will be sent to connected nodes on payout.\n\nFor Westend's WND tokens, see the faucet [instructions on the Wiki](https://wiki.polkadot.network/docs/en/learn-DOT#getting-westies).\n\n## Payouts\n\n### Automatic payout\n\nWhen starting _vedran loadbalancer_ it is possible to configure automatic payout by providing these flags:\n\n`--private-key` - loadbalancers wallet private key (string representation of hex value prefixed with 0x), used for sending rewards on the payout\n\n`--payout-interval` - automatic payout interval specified as number of days\n\n`--payout-reward` - defined total reward amount that will be distributed on the payout (amount in Planck). If omitted, the entire balance of lb wallet will be used as a total reward, and in this case `--lb-payout-fee-address` must be set\n\n`--lb-payout-address` - address on which load balancer fee will be sent. If omitted, load balancer fee will be left on load balancer wallet after payout. This flag is **required** if `--payout-reward` is not set (or set to -1)\n\nIf all flags have been provided, then each {_payout-interval_} days automatic payout will be started.\n\n### Manual payout\n\nIt is possible to run payout script at any time by invoking `vedran payout` command through the console.\n\n`--private-key` - loadbalancers wallet private key (string representation of hex value prefixed with 0x), used for sending rewards on the payout\n\n`--payout-reward` - defined total reward amount that will be distributed on the payout (amount in Planck). If omitted, the entire balance of lb wallet will be used as a total reward, and in this case `--lb-payout-fee-address` must be set\n\n`--lb-payout-fee-address` - address on which load balancer fee will be sent. If omitted, load balancer fee will be left on load balancer wallet after payout. This flag is **required** if `--payout-reward` is not set (or set to -1)\n\nAdditionally, it is possible to change URL on which payout script will connect with loadbalancer when executing transactions by setting flag (default value will be _http://localhost:80_)\n\n`--load-balancer-url` - loadbalancer URL\n\n### Get private key\nYou can use [subkey](https://substrate.dev/docs/en/knowledgebase/integrate/subkey) tool to get private key for your wallet.\n\nAfter installing subkey tool call `subkey inspect \"insert your mnemonic here\"`.\nYou can find private key as _Secreet seed_. See example output of subkey command:\n\n```\n  Secret seed:      0x1a84771145cdcee05e49142aaff2e5d669ce4b29344a09b973b751ae661acabf\n  Public key (hex): 0xa4548fa9b3b15dc4d1c59789952f0ccf6138dd63faf802637895c941f0522d35\n  Account ID:       0xa4548fa9b3b15dc4d1c59789952f0ccf6138dd63faf802637895c941f0522d35\n  SS58 Address:     5FnAq6wrMzri5V6jLfKgBkbR2rSAMkVAHVYWa3eU7TAV5rv9\n```\n\n## Monitoring\n\nMonitoring is done via grafana and prometheus which are expected to be installed.\n\n### Installation\n - [Grafana installation](https://grafana.com/docs/grafana/latest/installation/)\n - [Prometheus installation](https://prometheus.io/docs/prometheus/latest/installation/)\n\n### Configuration\n\n-  ### Grafana\n    Should be configured to fetch data from prometheus server as data source [Tutorial](https://prometheus.io/docs/visualization/grafana/).\n\n    Should have a dashboard that visualizes data scraped from prometheus server. Example configuration can be found [here](./infra/grafana/provisioning/dashboards/vedran-dashboard.json) and can be imported like [this](https://grafana.com/docs/grafana/latest/dashboards/export-import/#importing-a-dashboard).\n\n\n-  ### Prometheus\n    Prometheus should be configured to scrape metrics from vedran's `/metrics` endpoint\n    via prometheus .yml configuration. Example of which can be found [here](./infra/prometheus/prometheus.yml)\n   \n### Stats collection intervals\n\nIt is possible to change default calculation intervals for specific statistic categories. These intervals define how\noften will these statistics be recalculated.\n\nValid values are time intervals such as \"5s\", \"1.5h\" or \"2h45m\". Valid time units are \"ms\", \"s\", \"m\", \"h\"\n\n| ENV | Description | Default value |\n|----|-----------|:--------:|\n|`PROM_FEE_STATS_INTERVAL`|payout fees for nodes and load balancer|12 hours|\n|`PROM_NODE_STATS_INTERVAL`|active and penalized nodes|15 seconds|\n|`PROM_REQUEST_STATS_INTERVAL`|successful and failed requests|15 seconds|\n|`PROM_PAYOUT_STATS_INTERVAL`|payout distribution|1 minute|\n\n\n## Vedran loadbalancer API\n\n`POST   api/v1/nodes`\n\nRegister node to loadbalancer. Body should contain details about node:\n\n```json\n{\n  \"id\": \"string\",\n  \"config_hash\": \"string\",\n  \"payout_address\": \"string\"\n}\n```\n\nReturns **token** used for invoking rest of API and **tunnel_server_address** on which daemon can open tunnel toward loadbalancer.\n\n```json\n{\n  \"token\": \"string\",\n  \"tunnel_server_address\": \"string\"\n}\n```\n\n---\n\n`POST   api/v1/nodes/pings`\n\nPing loadbalancer from node. Auth token should be in header as `X-Auth-Header`.\n\n---\n\n`PUT    api/v1/nodes/metrics`\n\nSend metrics for node. Auth token should be in header as `X-Auth-Header`. Body should contain metrics as:\n\n```json\n{\n  \"peer_count\": \"int32\",\n  \"best_block_height\": \"int64\",\n  \"finalized_block_height\": \"int64\",\n  \"target_block_height\": \"int64\",\n  \"ready_transaction_count\": \"int32\"\n}\n```\n\n---\n\n`GET    api/v1/stats`\n\nReturns statistics for all nodes (mapped on node payout address).\n\n```json\n{\n  \"stats\": {\n    \"node_1_payout_address\": {\n      \"total_pings\": \"float64\",\n      \"total_requests\": \"float64\"\n    },\n    \"node_2_payout_address\": {\n      \"total_pings\": \"float64\",\n      \"total_requests\": \"float64\"\n    }\n  }\n}\n```\n\n---\n\n`GET    api/v1/stats/lb`\n\nReturns statistics on reward distribution between load balancer and nodes.\n\n```json\n{\n  \"lb_fee\": \"string\",\n  \"nodes_fee\": \"string\"\n}\n```\n\n## Development\n\n### Clone\n\n```bash\ngit clone git@github.com:NodeFactoryIo/vedran.git\n```\n\n### Lint\n[Golangci-lint](https://golangci-lint.run/usage/install/#local-installation) is expected to be installed.\n\n```bash\nmake lint\n```\n\n### Build\n\n```bash\nmake build\n```\n\n### Test\n\n```bash\nmake test\n```\n\n## License\n\nThis project is licensed under Apache 2.0:\n- Apache License, Version 2.0, ([LICENSE-APACHE](http://www.apache.org/licenses/LICENSE-2.0))\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnodefactoryio%2Fvedran","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnodefactoryio%2Fvedran","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnodefactoryio%2Fvedran/lists"}