{"id":13651699,"url":"https://github.com/paritytech/polkadot-runtime-prom-exporter","last_synced_at":"2025-04-22T22:31:53.851Z","repository":{"id":39595370,"uuid":"353377501","full_name":"paritytech/polkadot-runtime-prom-exporter","owner":"paritytech","description":"Prometheus exporter for polkadot runtime metrics","archived":true,"fork":false,"pushed_at":"2022-10-26T08:50:03.000Z","size":610,"stargazers_count":10,"open_issues_count":9,"forks_count":3,"subscribers_count":7,"default_branch":"master","last_synced_at":"2024-10-30T01:37:04.537Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/paritytech.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-03-31T14:04:51.000Z","updated_at":"2023-04-12T18:10:11.000Z","dependencies_parsed_at":"2023-01-19T17:32:56.383Z","dependency_job_id":null,"html_url":"https://github.com/paritytech/polkadot-runtime-prom-exporter","commit_stats":null,"previous_names":[],"tags_count":27,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paritytech%2Fpolkadot-runtime-prom-exporter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paritytech%2Fpolkadot-runtime-prom-exporter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paritytech%2Fpolkadot-runtime-prom-exporter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paritytech%2Fpolkadot-runtime-prom-exporter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/paritytech","download_url":"https://codeload.github.com/paritytech/polkadot-runtime-prom-exporter/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223906318,"owners_count":17223045,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-02T02:00:51.629Z","updated_at":"2024-11-10T02:31:12.850Z","avatar_url":"https://github.com/paritytech.png","language":"TypeScript","funding_links":[],"categories":["Tools"],"sub_categories":[],"readme":"# Runtime Exporter Installation and Setup guide\n\n## Prometheus exporter for polkadot runtime metrics\n\nThe Runtime Exporter collects in real time onchain current data, and historical data, in a form of data series, with the goal of visualising analytics, for any parachain of the Polkadot and Kusama ecosystems.\n\nThe Runtime Exporter Dashboards are Grafana based, and any user can customise and add his own metrics and dashboards.\n\nThe Runtime Exporter supports agnostically all parachains, and every pallet, when supported by the same parachain.\n\nThe Runtime Exporter data can be stored on prometheus, for the real time data only, and Timescaledb, for the real time data, and also for the historical data.\nBoth databases can be used simultaneously, or only one of them at a time.\n\nThe Runtime Exporter is Open Source, and any one can add its own metrics, while taking advantage of a shared deployment for data analytics visualisation.\n\n## Installation\n\n### Supported OS\n\nThe Runtime exporter is working on any version of Linux Debbian based system and MacOS.\n\n### Pre-requesites\n\nThe Runtime Exporter uses two types of database, Prometheus and Timescalesdb. Timescaledb has the advantage of loading historical data and real time data, while Prometheus stores only real time data.\n\nA minimum of one database is required, and it's up to you to install both Prometheus and Timescaledb.\n\n**Yarn** is required for the installation.\n\n#### Prometheus: \nhttps://prometheus.io/docs/prometheus/latest/installation/\n\nOnce Prometheus installed, go to the root of your runtime exporter folder and run this command :\n\n```sh\n../prometheus/prometheus-\u003cyour version\u003e/prometheus --config.file=\"./prometheus.yml\"\n```\n\n#### Timescaledb\n\nhttps://docs.timescale.com/install/latest/self-hosted/installation-debian/\n\nOnce Timescaledb installed, run the following script to create the metric tables and configuration tables.\n\nhttps://github.com/paritytech/polkadot-runtime-prom-exporter/src/sql/create_tables_timescale.sql\n\n```sh\npsql -U postgres -d \u003cyour database\u003e -a -f sql/create_tables_timescale.sql\n```\n\n#### Grafana\n\nInstall Grafana\n\nhttps://grafana.com/docs/grafana/latest/setup-grafana/installation/debian/\n\n#### .env file\n\n```sh\nPORT=8000\nTSDB_CONN='postgres://\u003cpostgres user\u003e:\u003cpostgres password\u003e@localhost:5432/tsdb'\nCONFIG_FULL_PATH='\u003cthe full path of config.json\u003e'\n```\n\nPORT is the prometheus connection port corresponding to your installation.\n\nTSDB_CONN is the timescaledb connection string .\n\nCONFIG_FULL_PATH is the full path of the config.json file.\n\nIf you are not using TimescaleDB, leave it empty.\n\n#### config.json file\n\nThe config.json file contains a section named \"rpcs\" with the list of parachains you want to monitor, and you need to specify the rpc connection string for each of them.\n\nFor example:\n\n```json\n   \"rpcs\": [\n        \"wss://rpc.polkadot.io\",\n        \"wss://kusama-rpc.polkadot.io\"\n    ],\n```\n\nWill monitor Polkadot and Kusama.\n\n#### Install the Runtime Exporter\n\nAt this stage, you should be able to run the Runtime Exporter in a simple mode, meaning without loading history, which will be explained later.\n\nGo to your Runtime Exporter root directory.\n\nIf you run the Runtime Exporter for the first time:\n\n```sh\nyarn\n```\n\nAnd then run the Runtime Exporter:\n\n```sh\nyarn run run\n```\n\nYou should see the first lines of log:\n\n```sh\nyarn run v1.22.17\n$ yarn run build \u0026\u0026 node build/index.js\n$ ./node_modules/.bin/rimraf ./build \u0026\u0026 ./node_modules/.bin/tsc\n[22-08-12 17:06:56] debug: Threads per exporter 2\n[22-08-12 17:06:56] debug: Server listening on port 8000\n```\n\nIf you run the Runtime Exporter with Prometheus, you can check your metrics right away:\n\nhttp://localhost:9090/graph\nor\nhttp://localhost:9090/metrics\n\nCongratulations, the Runtime Exporter is installed and running!\n\nHowver, this is just the beginning, we will explain below how to configure the Runtime Exporter in order to load history for your chosen metrics and parachains.\n\n#### Create a service for the Runtime Exporter\n\nIn order to guaranty that the Runtime Exporter is always up and running, it is recommended to define it as a service that will launch automatically in case of failure or disconnection.\n\nCopy the following lines in a file named **exporter.bash** in \u003c your script directory \u003e\n\n```sh\n#!/bin/bash\ndate\ndate +\"%FORMAT\"\nvar=$(date)\nvar=`date`\necho \"restart exporter at $var\" \u003e\u003e /tmp/runtime-exporter.log\n\ncd \u003cyour working directory\u003e/polkadot-runtime-prom-exporter\nsudo yarn run run\n```\n\nYou can find running linux service under path:\n\n```sh\n\ncd /lib/systemd/system/\nvi polkexporter.service\n```\n\nPaste and change **\u003c your script directory \u003e**:\n\n```sh\n[Unit]\nDescription=Runtime Exporter Metrics for Polkadot \u0026 Kusama\nStartLimitIntervalSec=500\nStartLimitBurst=5\n\n[Service]\nExecStart=/\u003cyour working directory\u003e/exporter.bash\nRestart=on-failure\nRestartSec=5s\n\n[Install]\nWantedBy=multi-user.target\n```\n\n## Load History\n\n###\n\nIn order to load historical data for specific metrics and parachains, you need to configure the section **history** of the file **config.json** located at _\u003c runtime exporter dir \u003e/src_\n\nNote that this file can be empty, and in this case, the Runtime Exporter will store the current ongoing data for every block, but not the history.\n\nFor example:\n\n```json\n    \"history\": [{\n            \"chain\": \"wss://rpc.polkadot.io\",\n            \"startingBlock\": 11512045,\n            \"endingBlock\": 10512045,\n            \"pallets\": \"balances\",\n            \"distanceBetweenBlocks\": [{\n                    \"pallet\": \"balances\",\n                    \"dist\": 600\n                },\n                {\n                    \"pallet\": \"nominationPools\",\n                    \"dist\": 6000\n                }]\n        },\n        {\n            \"chain\": \"wss://kusama-rpc.polkadot.io\",\n            \"startingBlock\": 13951344,\n            \"endingBlock\": 13950344,\n            \"pallets\": \"system,nominationPools\",\n            \"distanceBetweenBlocks\": [{\n                    \"pallet\": \"nominationPools\",\n                    \"dist\": 6000\n                }]\n        }\n    ]\n\n```\n\nWe can see here 2 sections, each of them containing 5 parameters.\n\n- **chain**: the same rpc connection string that you defined in the parachains.json for the corresponding parachain.\n- **startingBlock**: the block to start loading history. Note that this number must be higher than the following endingBlock, as history is loaded backward.\n- **endingBlock**: the block where history will stop loading.\n- **pallets**: the list of pallets that you want to load. Every exporter that uses this pallet will be loaded. If the string is empty, then all the pallets are requested to load, meaning also all the existing exporters.\n- **distanceBetweenBlocks**: the distance between blocks when querying historical data. You can define per pallet the distance bwtween blocks. If this section is not defined, the distance between blocks will use a default value of 1.\n\nPlease note that the last parameter of the config file, distanceBetweenBlocks , does not fit with all the exporters.\nFor example, the palletsMethodsCalls exporter counts the total number of calls per pallet and per method and per block. So in that case, if you define a distanceBetweenBlocks parameter, you will get the count only for the selected blocks, which represent only a portion of what it should be.\n\nA good example of use could apply to the **balance**s exporter, as it is giving an instant snapshot of the total issuance amount. So whether it is collected for every block or for every 1000 blocks, the result will be the same. The effect of this parameter will be that the loading of historical record will be much faster than without.\n\nYou can modify this file as many times as you want, and restart the Runtime Exporter.\nIf the same configuration is used when restarting the Runtime Exporter, and history was successfully loaded in a previous run, the Runtime Exporter will ignore the record in order avoid to load again and again the same data.\n\n## Versionning\n\nThis is an advanced section, you can skip it if you don't intend to write new exporters on your own.\n\nThe Runtime Exporter is an Open Source, and is subject to future additions and modifications.\nSome exporters may require more metrics to be collected in the future, and when it happens, the versionning mechanism will allow to reload historical data only for Exporters where new metrics were added.\n\nIf you are a developer and add a new metric in a specific Exporter, you need to increase the version of the exporter by one.\n\nFor example, if you add a metric to the Balances Exporter (src/exporters/balances.ts), you have to change **this.exporterVersion** to 2 if it was 1.\n\n```javascript\nclass BalancesExporter extends Balances implements Exporter {\n    palletIdentifier: any;\n    exporterVersion: number;\n\texporterIdentifier: string;\n\n    registry: PromClient.Registry;\n\n    constructor(registry: PromClient.Registry) {\n        //worker needs .js\n        super(BALANCE_WORKER_PATH, registry, true);\n        this.registry = registry;\n        this.palletIdentifier = \"balances\";\n        this.exporterIdentifier = \"balances\";\n        this.exporterVersion = 2;\n\n    }\n```\n\n### Checking that history has been loaded\n\nWhen restarting, the Runtime Exporter, will detect which records that were previously loaded using the parachains_load_history.json file will have to be loaded again due to version changes.\n\nEvery time a historical record has finished loading, it writes a record into the table exporters_versions;\n\nIn order to verify that a record has been loaded from parachains_load_history.json, check the exporters_versions table and see if your record is registered:\n\n```sql\nselect * from exporters_versions;\n\n            time            | startingblock | endingblock |  chain   |          exporter          | version | distancebb\n----------------------------+---------------+-------------+----------+----------------------------+---------+------------\n 2022-08-12 21:18:35.55+02  |      11512045 |    11511045 | Polkadot | timestamp                  |       1 |          1\n 2022-08-12 21:19:46.131+02 |      11512045 |    11511045 | Polkadot | palletMethodsCalls         |       1 |          1\n 2022-08-12 21:20:11.697+02 |      11512045 |    11511045 | Polkadot | balances                   |       1 |          1\n 2022-08-12 21:20:19.367+02 |      11512045 |    11511045 | Polkadot | xcmTransfers               |       1 |          1\n 2022-08-12 21:20:42.498+02 |      11512045 |    11511045 | Polkadot | transactionPayment         |       1 |          1\n\n```\n\nYou can alternatively query the tables that correspond to the metrics loaded by the requested pallets.\n\n## Visualising data in Grafana\n\nThere are today around 40 different metrics, with 10 exporters.\nEvery metric can be visualized in Grafana, either using the Prometheus database as a data source, either timescaledb, or both of them.\n\nThere is a shared list of dashboard provided in this repository, that can be found at src/grafana-dashboards with many examples of database queries and charts.\n\n\n## Adding Metrics and Exporters\n\nThe Runtime Exporter is an Open Source, and anyone can add its own metrics and exporters on top of it.\nWe're going to explain through examples how to add a new metric to an existing exporter, and how to add a new exporter.\n\nThis example will show you how to implement every method for the 2 supported databases (prometheus and timescaledb, but it's up to you to implement only one of the databases, if you do not intend to use the other when running the Runtime Exporter, having in mind that Prometheus would only work for runtime data collection.\n\n### How to add a new metric to an existing exporter\n\nIn this example, we're going to add a new metric to the Balances Exporter, which is a simple exporter with one metric. \nIn terms of code structure, every exporter is composed of 2 main files, in our case, **exporters/balances.ts** and **workers/balancesWorker.ts**.\n\n\n\nLet's have a look at balances.ts\n\n```javascript\nimport * as PromClient from \"prom-client\"\nimport { logger } from '../logger';\nimport { Exporter } from './IExporter';\nimport { ApiPromise } from \"@polkadot/api\";\nimport { Header } from \"@polkadot/types/interfaces\";\nimport { Balances } from '../workers/balancesWorker'\nimport { BALANCE_WORKER_PATH } from '../workers/workersPaths'\n\nclass BalancesExporter extends Balances implements Exporter {\n    palletIdentifier: any;\n    exporterVersion: number;\n\texporterIdenfier: string;\n    \n    registry: PromClient.Registry;\n\n    constructor(registry: PromClient.Registry) {\n        //worker needs .js \n        super(BALANCE_WORKER_PATH, registry, true);\n        this.registry = registry;\n        this.palletIdentifier = \"balances\";\n        this.exporterIdenfier = \"balances\";\n        this.exporterVersion = 1;\n        \n    }\n\n    async perBlock(api: ApiPromise, header: Header, chainName: string): Promise\u003cvoid\u003e {\n\n        const blockNumber = parseInt(header.number.toString());\n        const result = await this.doWork(this, api, blockNumber, chainName)\n\n    }\n\n    async perDay(api: ApiPromise, chainName: string) { }\n\n    async perHour(api: ApiPromise, chainName: string) { }\n\n    async init(chainName: string, startingBlockTime: Date, endingBlockTime: Date) { \n       \n        await this.clean(  chainName.toString(), startingBlockTime, endingBlockTime);\n    }\n\n    async launchWorkers(threadsNumber: number, startingBlock: number, endingBlock: number, chain: string, chainName: string, distanceBB: number) {\n        super.launchWorkers(threadsNumber, startingBlock, endingBlock, chain, this.exporterIdenfier, this.exporterVersion, chainName,distanceBB)\n    }\n\n}\n\nexport { BalancesExporter };\n```\n\nAs we can see, there are 3 parameters:\n\n**palletIdentifier**: the name of the pallet defined by Polkadot.\n\n**exporterVersion**: the version of the exporter, which needs to be iterated by one every time a new metric is added, or when the exporter is modified.\n\n**exporterIdenfier**: the name of the exporter.\n\nThen, we can see 3 methods, **perBlock**, **perHour** and **perDay**.\nThis class allows you to choose at which frequency you are going to monitor your metric(s).\nIn our case we want to collect our new metric per block.The function doWork, that we are going to explain below is responsible for collecting the data, at the frequency of once per block.\n\nThen we can see the **init** function, which is responsible to clean the data before loading a new historical record for the same metric, when requested by configuration.\nAnd finally, the **launchWorkers** function, that calls the workers threads of the same exporter, when historical records are loaded.\n\n\nAt this stage, we modify the **exporterVersion** to 2, because we are going to add a new metric. \n\nIncreasing the version for this exporter will have the effect of loading all the historical records that were previously loaded with this Exporter.\n\nThis is all the change that needs to be done in balances.ts.\n\nNow let's review the **balancesWorker.ts** and see what we need to change here in order to add our new metric.\n\n```javascript\nimport { ApiPromise } from \"@polkadot/api\";\nimport { config } from \"dotenv\";\nimport { decimals, sequelizeParams } from '../utils'\nimport * as PromClient from \"prom-client\"\nimport { CTimeScaleExporter } from './CTimeScaleExporter';\nimport { BALANCE_WORKER_PATH } from './workersPaths'\nimport { launchLoading } from './LoadHistory'\n\nconfig();\nconst connectionString = process.env.TSDB_CONN || \"\";\nconst Sequelize = require('sequelize');\nconst sequelize = (connectionString != \"\") ? new Sequelize(connectionString, { sequelizeParams, logging: false }) : null;\n\nexport class Balances extends CTimeScaleExporter {\n    balancesSql: typeof Sequelize;\n    totalIssuanceMetric: any;\n    withProm: boolean;\n    withTs: boolean;\n    registry: PromClient.Registry;\n    exportersVersionsSql: typeof Sequelize;\n\n    constructor(workerPath: string, registry: PromClient.Registry, withProm: boolean) {\n\n        super(workerPath);\n        this.registry = registry;\n        this.withProm = withProm;\n        this.withTs = (connectionString == \"\" ? false : true);\n\n        if (this.withTs) {\n            this.balancesSql = sequelize.define(\"runtime_total_issuance\", {\n                time: { type: Sequelize.DATE, primaryKey: true },\n                chain: { type: Sequelize.STRING, primaryKey: true },\n                issuance: { type: Sequelize.INTEGER },\n            }, { timestamps: false, freezeTableName: true });\n        }\n        if (this.withProm) {\n            this.totalIssuanceMetric = new PromClient.Gauge({\n                name: \"runtime_total_issuance\",\n                help: \"the total issuance of the runtime, updated per block\",\n                labelNames: [\"type\", \"chain\"]\n            })\n            registry.registerMetric(this.totalIssuanceMetric);\n        }\n    }\n\n    async write(time: number, myChain: string, issuance: number, withProm: boolean) {\n\n        if (this.withTs) {\n            const result = await this.balancesSql.create(\n                {\n                    time: time,\n                    chain: myChain,\n                    issuance: issuance\n                }, { fields: ['time', 'chain', 'issuance'] },\n                { tableName: 'runtime_total_issuance' });\n        }\n\n        if (this.withProm) {\n            this.totalIssuanceMetric.set({ chain: myChain }, issuance);\n        }\n    }\n\n    async doWork(exporter: Balances, api: ApiPromise, indexBlock: number, chainName: string) {\n\n        const blockHash = await api.rpc.chain.getBlockHash(indexBlock);\n        const apiAt = await api.at(blockHash);\n        const timestamp = (await api.query.timestamp.now.at(blockHash)).toNumber();\n\n        const issuance = (await apiAt.query.balances.totalIssuance()).toBn();\n        const issuancesScaled = issuance.div(decimals(api)).toNumber();\n        await exporter.write(timestamp, chainName.toString(), issuancesScaled, exporter.withProm);\n\n    }\n\n    async clean(myChainName: string, startingBlockTime: Date, endingBlockTime: Date) {\n        await super.cleanData(this.balancesSql, myChainName, startingBlockTime, endingBlockTime)\n    }\n}\n\nasync function run() {\n    await launchLoading(exporter);\n}\n\nconst registry = new PromClient.Registry();\nlet exporter = new Balances(BALANCE_WORKER_PATH, registry, false);\nrun();\n\n```\n\nThe Balances class is mainly responsible for storing the data, and is called by one of the functions perBlock, perHour and perDay, and also by the worker threads, when loading history.\n\nCurrently, the Runtime Exporter supports 2 databases, prometheus and Timescaledb.\nPrometheus stores only runtime data, while Timescaledb stores both runtime data and historical data.\nAll the database interaction resides in 3 methods:\n\n* The constructor where the data sctructures of the respective databases are defined. \n* The write function where data is written for all the databases, per metric.\n* The clean function that deletes data of the same required segment by configuration, before loading data from new. This clean function applies only to historical records of course.\n\nIn order to keep the simple format of the Runtime Exporter, we recommend to write a new write_NEW_METRIC function for every single metric, and then use this write_NEW_METRIC function when implementing the code in the doWork function.\n\nThe doWork function is the place where we are adding our new metric code.\nThis function is called by the balanceExporter in runtime, and also by the **loadHistoryFromApi** function, located in LoadHistory.ts.\n\nLet's say that we want to add a new metric in this exporter, that collects the balance of a specific address. Let's call it balanceUserA.\n\nSo all we have to do is:\n\n- declare a new object of type Sequelize for timescaledb\n- declare a new metric of prometheus\n\nThese 2 new variables will be placed just below the existing ones.\n\n```javascript\nexport class Balances extends CTimeScaleExporter {\n    balancesSql: typeof Sequelize;\n    totalIssuanceMetric: any;\n\n    balancesUserASql: typeof Sequelize;\n    BalanceUserAMetric: any;\n\n```\n\nIn the constructor, declare the data structure for the same objects:\n\n```javascript\n        if (this.withTs) {\n            this.balancesSql = sequelize.define(\"runtime_total_issuance\", {\n                time: { type: Sequelize.DATE, primaryKey: true },\n                chain: { type: Sequelize.STRING, primaryKey: true },\n                issuance: { type: Sequelize.INTEGER },\n            }, { timestamps: false, freezeTableName: true });\n\n            this.balancesUserASql = sequelize.define(\"runtime_balance_user_a\", {\n                time: { type: Sequelize.DATE, primaryKey: true },\n                chain: { type: Sequelize.STRING, primaryKey: true },\n                balanceusera: { type: Sequelize.INTEGER },\n            }, { timestamps: false, freezeTableName: true });\n        }\n        if (this.withProm) {\n            this.totalIssuanceMetric = new PromClient.Gauge({\n                name: \"runtime_total_issuance\",\n                help: \"the total issuance of the runtime, updated per block\",\n                labelNames: [\"type\", \"chain\"]\n            })\n            registry.registerMetric(this.totalIssuanceMetric);\n\n            this.totalIssuanceMetric = new PromClient.Gauge({\n                name: \"runtime_balance_user_a\",\n                help: \"the balance of a specific\",\n                labelNames: [\"chain\"]\n            })\n            registry.registerMetric(this.totalIssuanceMetric);\n        }\n\n```\n\nUnder the withTs section, we add the new Timescaledb data structure, and under withProm, we add the prometheus data structure plus the declaration of the registry.\n\nShould we add a new database to the Runtime Exporter, the data sctructures and objects should be declared in this section.\n\nNow we add a new write function that will take care of the storage of our metric, using the same principle, under the withThs section, add the timescaledb code, and under the withProm the prometheus related code.\n\n```javascript\nasync writeBalanceA(time: number, myChain: string, balance: number, withProm: boolean) {\n\n    if (this.withTs) {\n        const resultA = await this.balancesUserASql.create(\n            {\n                time: time,\n                chain: myChain,\n                balance: balance\n            }, { fields: ['time', 'chain', 'balance'] },\n            { tableName: 'runtime_balance_a' });\n    }\n\n    if (this.withProm) {\n        this.balancesUserASql.set({ chain: myChain }, balance);\n    }\n}\n```\n\nAdd also the code for the clean function:\n\n```javascript\n  async clean(myChainName: string, startingBlockTime: Date, endingBlockTime: Date) {\n        await super.cleanData(this.balancesSql, myChainName, startingBlockTime, endingBlockTime)\n        await super.cleanData(this.balancesUserASql, myChainName, startingBlockTime, endingBlockTime)\n    }\n```\n\nNow that we have our data objects properly declared and our write/clean functions, we can add our code at the end of the doWork function:\n\n```javascript\nasync doWork(exporter: Balances, api: ApiPromise, indexBlock: number, chainName: string) {\n\n        const blockHash = await api.rpc.chain.getBlockHash(indexBlock);\n        const apiAt = await api.at(blockHash);\n        const timestamp = (await api.query.timestamp.now.at(blockHash)).toNumber();\n\n        const issuance = (await apiAt.query.balances.totalIssuance()).toBn();\n        const issuancesScaled = issuance.div(decimals(api)).toNumber();\n        await exporter.write(timestamp, chainName.toString(), issuancesScaled, exporter.withProm);\n\n        const BALANCE_A_ADDRESS = 'GtGGqmjQeRt7Q5ggrjmSHsEEfeXUMvPuF8mLun2ApaiotVr';\n        let balanceA = (await apiAt.query.system.account(BALANCE_A_ADDRESS));\n        if (balanceA.data.free != null) {\n            await exporter.writeBalanceA(timestamp, chainName.toString(), (balanceA.data.free.toBn().div(decimals(api))).toNumber(), exporter.withProm);\n        } else {\n            await exporter.writeBalanceA(timestamp, chainName.toString(), 0, exporter.withProm);\n        }\n    }\n```\n\nPlease note that this code can read the state of the blockchain at a specific block, by using the **api.at** request.\nAs we remember that the doWork function is called both for the real time data, and also for the historical data.\n\n\nThat's it for the code.\nWe also want to create the table in the timescaledb database, which needs to be done externally with your preferred database tool:\n\n```sql\nCREATE TABLE IF NOT EXISTS runtime_balance_a (\n  time TIMESTAMPTZ NOT NULL,\n  chain TEXT NOT NULL,\n  balance INTEGER NOT NULL\n);\n```\n\nCongratulations, you've just added a new metric to an existing Exporter, you can now check on Grafana that the new metric collects data properly.\n\n### How to add a new exporter\n\nNow that we have seen how to add a new metric to an existing Exporter, we understand also that the same procedure can be repeated in the same exporter for additional metrics. \n\nBut you might also want to create a new exporter of your own, so this is what we are going to explain below.\n\nWe saw before that an exporter consists of 2 files, for example:\n\n- balances.ts\n- balancesWorker.ts\n\nIn order to create a new exporter, we suggest to copy these 2 files in their respective directory, with a different name.\nWe'll call our new exporter **TheNew**.\n\nSo we have now:\n\n- theNew.ts in exporters\n- theNewWorker.ts in workers\n\nIn the import section, modify the names:\n\nimport { TheNew } from '../workers/theNewWorker.ts'\nimport { THE_NEW_WORKER_PATH } from '../workers/workersPaths'\n\nModify the existing BalanceExporter class name to TheNewExporter\n\nThen, modify the constructor, with the palletIdentifier you will monitor in your exporter, and with the name of the new exporter. Also, set the THE_NEW_WORKER_PATH in the super call, that will initiate the worker's path:\n\n```\n   constructor(registry: PromClient.Registry) {\n        //worker needs .js \n        super(THE_NEW_WORKER_PATH, registry, true);\n        this.registry = registry;\n        this.palletIdentifier = \"balances\";\n        this.exporterIdenfier = \"thenew\";\n        this.exporterVersion = 1;\n        \n    }\n\n```\n\nThat's it for the theNew.ts file.\n\nNow we add in the workersPaths.ts file the new THE_NEW_WORKER_PATH variable:\n\n\n```\nexport const THE_NEW_WORKER_PATH = '/build/workers/theNewWorker.js';\n```\n\nThis is because the nodejs Workers feature works only with .js extention, not the .ts.\n\nAnd we add the new exporter in the index.ts located in src/exporters:\n\n````\nexport * from '../exporters/system';\nexport * from '../exporters/balances';\nexport * from '../exporters/xcmTransfers';\nexport * from '../exporters/stakingMinerAccount';\nexport * from '../exporters/transactionPayment';\nexport * from '../exporters/staking';\nexport * from '../exporters/palletsMethodsCalls';\nexport * from '../exporters/electionProviderMultiPhase';\nexport * from '../exporters/timestamp';\nexport * from '../exporters/nominationPools';\n\nexport * from '../exporters/theNew';\n````\n\nWe are almost at the end, we are going to make the changes in the theNewWorker.ts:\n\nIn the import section, change:\n\nimport { THE_NEW_WORKER_PATH } from './workersPaths'\n\nIn the code, we are replacing the name of the class with the new one, in 3 places:\n\n````\nexport class TheNew extends CTimeScaleExporter {\n\n````\n````\nasync doWork(exporter: TheNew, api: ApiPromise, indexBlock: number, chainName: string) {\n\n````\n````\nlet exporter = new TheNew(THE_NEW_WORKER_PATH, registry, false);\n\n````\n\nThat's it, we now just need to instanciate this new exporter before running it.\n\nIn the src/index.ts, add your new exporter in the import section:\n\n```\nimport { SystemExporter, BalancesExporter, XCMTransfersExporter, StakingMinerAccountExporter, TransactionPaymentExporter, StakingExporter, PalletsMethodsExporter, ElectionProviderMultiPhaseExporter, TimestampExporter, NominationPoolsExporter, NewExporter } from \"./exporters\";\n```\n\nIn the main function, we add the new exporter as well:\n\n```javascript\n\t\t\tconst exporters = [new SystemExporter(registry),\n\t\t\tnew StakingExporter(registry),\n\t\t\tnew BalancesExporter(registry),\n\t\t\tnew TransactionPaymentExporter(registry),\n\t\t\tnew StakingMinerAccountExporter(registry),\n\t\t\tnew ElectionProviderMultiPhaseExporter(registry),\n\t\t\tnew TimestampExporter(registry),\n\t\t\tnew XCMTransfersExporter(registry),\n\t\t\tnew PalletsMethodsExporter(registry),\n\t\t\tnew NominationPoolsExporter(registry),\n\t\t\tnew TheNewExporter(registry),\n\t\t\t]\n```\n\nThat's all, you have now a new exporter, that at this stage, works exactly as the balance exporter, and it's now your turn to transform it for your needs, based on the explanations provided above, on how to add a new metric.\n\n## Multi-threading\n\nIn order to increase performance when loading historical records, the Runtime Exporter runs in a multi-threaded environment. \nThe principle of multi-threading in our case, is to let every thread loading a continuous portion of the required time segment for the same Exporter.\nFor example, if the startingBlock is 1000000 and the ending block is 900000(see config.json), and the number of threads is 5, then thread #1 will take charge of blocks 1000000-980000, thread 2, 980000-960000, etc...\n\nWhen all the workers have finished, a new record in the exporters_versions table will be added, which will testify that a new historical record was added for this specific Exporter.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fparitytech%2Fpolkadot-runtime-prom-exporter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fparitytech%2Fpolkadot-runtime-prom-exporter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fparitytech%2Fpolkadot-runtime-prom-exporter/lists"}