{"id":13652142,"url":"https://github.com/paritytech/substrate-api-sidecar","last_synced_at":"2025-05-14T18:07:28.593Z","repository":{"id":36991825,"uuid":"224672691","full_name":"paritytech/substrate-api-sidecar","owner":"paritytech","description":"REST service that makes it easy to interact with blockchain nodes built using Substrate's FRAME framework.","archived":false,"fork":false,"pushed_at":"2025-05-02T14:19:34.000Z","size":46304,"stargazers_count":257,"open_issues_count":50,"forks_count":159,"subscribers_count":16,"default_branch":"master","last_synced_at":"2025-05-08T00:13:03.542Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://paritytech.github.io/substrate-api-sidecar/dist/","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/paritytech.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":"CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2019-11-28T14:38:47.000Z","updated_at":"2025-04-29T06:49:17.000Z","dependencies_parsed_at":"2023-10-23T14:52:49.547Z","dependency_job_id":"4cceb6dd-2368-4e92-8c23-eb22da4ae321","html_url":"https://github.com/paritytech/substrate-api-sidecar","commit_stats":{"total_commits":786,"total_committers":48,"mean_commits":16.375,"dds":0.5661577608142494,"last_synced_commit":"8fdfef82e5a3ededf597744589c55680b6b6ccf8"},"previous_names":[],"tags_count":196,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paritytech%2Fsubstrate-api-sidecar","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paritytech%2Fsubstrate-api-sidecar/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paritytech%2Fsubstrate-api-sidecar/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paritytech%2Fsubstrate-api-sidecar/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/paritytech","download_url":"https://codeload.github.com/paritytech/substrate-api-sidecar/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253942017,"owners_count":21987960,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-02T02:00:55.957Z","updated_at":"2025-05-14T18:07:23.584Z","avatar_url":"https://github.com/paritytech.png","language":"TypeScript","funding_links":[],"categories":["Tools","Smart Contract Platforms","TypeScript"],"sub_categories":[],"readme":"\n\u003cbr /\u003e\u003cbr /\u003e\n\n\u003cdiv align=\"center\"\u003e\n  \u003ch1 align=\"center\"\u003e@substrate/api-sidecar\u003c/h1\u003e\n  \u003ch4 align=\"center\"\u003e REST service that makes it easy to interact with blockchain nodes built using Substrate's\n    \u003ca href=\"https://substrate.dev/docs/en/knowledgebase/runtime/frame\"\u003eFRAME\u003c/a\u003e\n    framework.\u003c/h4\u003e\n  \u003cp align=\"center\"\u003e\n    \u003ca href=\"https://www.npmjs.com/package/@substrate/api-sidecar\"\u003e\n      \u003cimg alt=\"npm\" src=\"https://img.shields.io/npm/v/@substrate/api-sidecar\" /\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://github.com/paritytech/substrate-api-sidecar/actions\"\u003e\n      \u003cimg alt=\"Github Actions\" src=\"https://github.com/paritytech/substrate-api-sidecar/workflows/pr/badge.svg\" /\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://github.com/paritytech/substrate-api-sidecar/blob/master/LICENSE\"\u003e\n      \u003cimg alt=\"GPL-3.0-or-later\" src=\"https://img.shields.io/npm/l/@substrate/api-sidecar\" /\u003e\n    \u003c/a\u003e\n  \u003c/p\u003e\n\u003c/div\u003e\n\n\u003cbr /\u003e\u003cbr /\u003e\n\n## Prerequisites\n\n### \u003c= v15.0.0\nThis service requires Node versions 14 or higher.\n\nCompatibility:\n| Node Version  | Stablility  |\n|---------------|:-----------:|\n|     v14.x.x   |   Stable    |\n|     v16.x.x   |   Stable    |\n|     v17.x.x   |   Stable    |\n|     v18.x.x   |   Stable    |\n|     v19.x.x   |   stable    |\n\n### \u003e= v16.0.0\nThis service requires Node versions 18.14 or higher.\n\nCompatibility:\n| Node Version  | Stablility  |\n|---------------|:-----------:|\n|     v18.14.x  |   Stable    |\n|     v20.x.x   |   Stable    |\n|     v21.x.x   |   Pending   |\n\nNOTE: Node LTS (`long term support`) versions start with an even number, and odd number versions are subject to a 6 month testing period with active support before they are unsupported. It is recommended to use sidecar with a stable actively maintained version of node.js.\n\n## Table of contents\n\n- [NPM package installation and usage](#npm-package-installation-and-usage)\n- [Source code installation and usage](#source-code-installation-and-usage)\n- [Configuration](#configuration)\n- [Debugging fee and staking payout calculations](#debugging-staking-payout-calculations)\n- [Available endpoints](https://paritytech.github.io/substrate-api-sidecar/dist/)\n- [Chain integration guide](./guides/CHAIN_INTEGRATION.md)\n- [Docker](#docker)\n- [Notes for maintainers](#notes-for-maintainers)\n- [Hardware requirements](#hardware-requirements)\n\n## NPM package installation and usage\n\n### Global installation\n\nInstall the service globally:\n\n```bash\nnpm install -g @substrate/api-sidecar\n# OR\nyarn global add @substrate/api-sidecar\n```\n\nRun the service from any directory on your machine:\n\n```bash\nsubstrate-api-sidecar\n```\n\nTo check your version you may append the `--version` flag to `substrate-api-sidecar`.\n\n### Local installation\n\nInstall the service locally:\n\n```bash\nnpm install @substrate/api-sidecar\n# OR\nyarn add @substrate/api-sidecar\n```\n\nRun the service from within the local directory:\n\n```bash\nnode_modules/.bin/substrate-api-sidecar\n```\n\n### Finishing up\n\n[Jump to the configuration section](#configuration) for more details on connecting to a node.\n\n[Click here for full endpoint docs.](https://paritytech.github.io/substrate-api-sidecar/dist/)\n\nIn the full endpoints doc, you will also find the following `trace` related endpoints :\n- `/experimental/blocks/{blockId}/traces/operations?actions=false`\n- `/experimental/blocks/head/traces/operations?actions=false`\n- `/experimental/blocks/{blockId}/traces`\n- `/experimental/blocks/head/traces`\n\nTo have access to these endpoints you need to :\n1. Run your node with the flag `—unsafe-rpc-external`\n2. Check in sidecar if `BlocksTrace` controller is active for the chain you are running.\n\nCurrently `BlocksTrace` controller is active in [Polkadot](https://github.com/paritytech/substrate-api-sidecar/blob/ff0cef5eaeeef74f9a931a0355d83fc5ebdea645/src/chains-config/polkadotControllers.ts#L17) and [Kusama](https://github.com/paritytech/substrate-api-sidecar/blob/ff0cef5eaeeef74f9a931a0355d83fc5ebdea645/src/chains-config/kusamaControllers.ts#L17).\n\n## Source code installation and usage\n\n### Quick install\n\nSimply run `yarn`.\n\n### Rust development installation\n\nIf you are looking to hack on the `calc` Rust crate make sure your machine has an [up-to-date version of `rustup`](https://www.rust-lang.org/tools/install)\ninstalled to manage Rust dependencies.\n\nInstall `wasm-pack` if your machine does not already have it:\n\n```bash\ncargo install wasm-pack\n```\n\nUse yarn to do the remaining setup:\n\n```bash\nyarn\n```\n\n### Running\n\n```bash\n# For live reload in development\nyarn dev\n\n# To build and run\nyarn build\nyarn start\n```\n\n[Jump to the configuration section](#configuration) for more details on connecting to a node.\n\n## Configuration\n\nTo use a specific env profile (here for instance a profile called 'env.sample'):\n\n```bash\nNODE_ENV=sample yarn start\n```\n\nFor more information on our configuration manager visit its readme [here](https://gitlab.com/chevdor/confmgr/-/raw/master/README.adoc). See `Specs.ts` to view the env configuration spec.\n\n### Express server\n\n- `SAS_EXPRESS_BIND_HOST`: address on which the server will be listening, defaults to `127.0.0.1`.\n- `SAS_EXPRESS_PORT`: port on which the server will be listening, defaults to `8080`.\n- `SAS_EXPRESS_KEEP_ALIVE_TIMEOUT`: Set the `keepAliveTimeout` in express.\n- `SAS_EXPRESS_MAX_BODY`: Set the size of request body payload, defaults to `100kb`\n- `SAS_EXPRESS_INJECTED_CONTROLLERS`: When set (_e.g. SAS_EXPRESS_INJECTED_CONTROLLERS=true_), automatically detects the available pallets in the chain's metadata and injects the appropriate controllers. If set to `false`, it uses the controllers defined in the chains configuration files. Defaults to `false` when the flag is not defined. Note: This flag does not replace completely the need of the chains-config files since we are still retrieving the options from that file. \n\n### Substrate node\n\n- `SAS_SUBSTRATE_URL`: URL to which the RPC proxy will attempt to connect to, defaults to\n    `ws://127.0.0.1:9944`. Accepts both a websocket, and http URL.\n\n### Metrics Server\n\n- `SAS_METRICS_ENABLED`: Boolean to enable the metrics server instance with Prometheus (server metrics) and Loki (logging) connections. Defaults to false.\n- `SAS_METRICS_PROM_HOST`: The host of the prometheus server used to listen to metrics emitted, defaults to `127.0.0.1`.\n- `SAS_METRICS_PROM_PORT`: The port of the prometheus server, defaults to `9100`.\n- `SAS_METRICS_LOKI_HOST`: The host of the loki server used to pull the logs, defaults to `127.0.0.1`.\n- `SAS_METRICS_LOKI_PORT`: The port of the loki server, defaults to `3100`\n\n#### Custom substrate types\n\nSome chains require custom type definitions in order for Sidecar to know how to decode the data\nretrieved from the node. Sidecar affords environment variables which allow the user to specify an absolute path to a JSON file that contains type definitions in the corresponding formats. Consult polkadot-js/api for more info on\nthe type formats (see `RegisteredTypes`). There is a helper CLI tool called [generate-type-bundle](https://github.com/paritytech/generate-type-bundle) that can generate a `typesBundle.json` file for you using chain information from [`@polkadot/apps-config`](https://github.com/polkadot-js/apps/tree/master/packages/apps-config). The generated json file from this tool will work directly with the `SAS_SUBSTRATE_TYPES_BUNDLE` ENV variable. \n\n- `SAS_SUBSTRATE_TYPES_BUNDLE`: a bundle of types with versioning info, type aliases, derives, and\n    rpc definitions. Format: `OverrideBundleType` (see [`typesBundle`](https://github.com/polkadot-js/api/blob/21039dec1fcad36061a96bf5526248c5fab38780/packages/types/src/types/registry.ts#L72)).\n- `SAS_SUBSTRATE_TYPES_CHAIN`: type definitions keyed by `chainName`. Format: `Record\u003cstring, RegistryTypes\u003e` (see [`typesChain`](https://github.com/polkadot-js/api/blob/21039dec1fcad36061a96bf5526248c5fab38780/packages/types/src/types/registry.ts#L76)).\n- `SAS_SUBSTRATE_TYPES_SPEC`: type definitions keyed by `specName`. Format: `Record\u003cstring, RegistryTypes\u003e` (see [`typesSpec`](https://github.com/polkadot-js/api/blob/21039dec1fcad36061a96bf5526248c5fab38780/packages/types/src/types/registry.ts#L80)).\n- `SAS_SUBSTRATE_TYPES`: type definitions and overrides, not keyed. Format: `RegistryTypes` (see [`types`](https://github.com/polkadot-js/api/blob/21039dec1fcad36061a96bf5526248c5fab38780/packages/types/src/types/registry.ts#L64)).\n\nYou can read more about [defining types for polkadot-js here.](https://polkadot.js.org/api/start/types.extend.html)\n\n##### Connecting a modified node template\n\nPolkadot-js can recognize the standard node template and inject the correct types, but if you have\nmodified the name of your chain in the node template you will need to add the types manually in a\nJSON `types` file like so:\n\n```json\n// my-chains-types.json\n{\n  \"Address\": \"AccountId\",\n  \"LookupSource\": \"AccountId\"\n}\n```\n\nand then set the enviroment variable to point to your definitions:\n\n```bash\nexport SAS_SUBSTRATE_TYPES=/path/to/my-chains-types.json\n```\n\n### Logging\n\n- `SAS_LOG_LEVEL`: The lowest priority log level to surface, defaults to `info`. Tip: set to `http`\n    to see all HTTP requests.\n- `SAS_LOG_JSON`:Whether or not to have logs formatted as JSON, defaults to `false`.\n    Useful when using `stdout` to programmatically process Sidecar log data.\n- `SAS_LOG_FILTER_RPC`: Whether or not to filter polkadot-js API-WS RPC logging, defaults to `false`.\n- `SAS_LOG_STRIP_ANSI`: Whether or not to strip ANSI characters from logs, defaults\n    to `false`. Useful when logging RPC calls with JSON written to transports.\n- `SAS_LOG_WRITE`: Whether or not to write logs to a log file. Default is set to `false`. Accepts a boolean value. The log files will be written as `logs.log`. **NOTE**: It will only log what is available depending on what `SAS_LOG_LEVEL` is set to.\n- `SAS_LOG_WRITE_PATH`: Specifies the path to write the log files. Default will be where the package is installed.\n- `SAS_LOG_WRITE_MAX_FILE_SIZE`: Specifies in bytes what the max file size for the written log files should be. Default is `5242880` (5MB). **NOTE** Once the the max amount of files have reached their max size, the logger will start to rewrite over the first log file.\n- `SAS_LOG_WRITE_MAX_FILES`: Specifies how many files can be written. Default is 5.\n\n#### Log levels\n\nLog levels in order of decreasing importance are: `error`, `warn`, `info`, `http`, `verbose`, `debug`, `silly`.\n\n| http status code range | log level |\n|------------------------|-----------|\n| `code` \u003c 400           | `http`    |\n| 400 \u003c= `code` \u003c 500    | `warn`    |\n| 500 \u003c `code`           | `error`   |\n\n#### RPC logging\n\nIf looking to track raw RPC requests/responses, one can use `yarn start:log-rpc` to turn on polkadot-js's\nlogging. It is recommended to also set `SAS_LOG_STRIP_ANSI=true` to increase the readability of the logging stream.\n\n**N.B.** If running `yarn start:log-rpc`, the NODE_ENV will be set to `test`. In order still run your `.env`\nfile you can `symlink` it with `.env.test`. For example you could run\n`ln -s .env.myEnv .env.test \u0026\u0026 yarn start:log-rpc` to use `.env.myEnv` to set ENV variables. (see linux\ncommands `ln` and `unlink` for more info.)\n\n### Prometheus server\n\nPrometheus metrics can be enabled by running sidecar with the following env configuration: `SAS_METRICS_ENABLED`=true\n\nYou can also expand the metrics tracking capabilities to include query params by adding to the env configuration: `SAS_METRICS_INCLUDE_QUERYPARAMS`=true\n\nThe metrics endpoint can then be accessed :\n- on the default port : `http://127.0.0.1:9100/metrics` or\n- on your custom port if you defined one : `http://127.0.0.1:\u003cYOUR_CUSTOM_PORT\u003e/metrics`\n\nA JSON format response is available at `http://127.0.0.1:9100/metrics.json`.\n\nThat way you will have access to the default prometheus node instance metrics and the following metrics will be emitted for each route:\n\n- `sas_http_request_error`: type counter and tracks http errors occuring in sidecar\n- `sas_http_request_success`: type counter and tracks successfull http requests\n- `sas_http_requests`: type counter and tracks all http requests\n- `sas_request_duration_seconds`: type histogram and tracks the latency of the requests\n- `sas_response_size_bytes_seconds`: type histogram and tracks the response size of the requests\n- `sas_response_size_latency_ratio_seconds`: type histogram and tracks the response bytes per second of the requests\n\nThe blocks controller also includes the following route-specific metrics:\n\n- `sas_extrinsics_in_request`: type histogram and tracks the number of extrinsics returned in the request when a range of blocks is queried\n- `sas_extrinsics_per_second`: type histogram and tracks the returned extrinics per second\n- `sas_extrinsics_per_block`: type histogram and tracks the returned extrinsics per block\n- `sas_seconds_per_block`: type histogram and tracks the request time per block\n\nThe metrics registry is injected in the Response object when the `SAS_METRICS_ENABLED` flag is set to `true` in the `.env` file, allowing to extend the controller based metrics to any given controller from within the controller functions.\n\nTo successfully run and access the metrics and logs in Grafana (for example) the following are required:\n\n- prometheus server (info [here](https://prometheus.io/docs/prometheus/latest/getting_started/))\n- loki server and promtail (info [here](https://grafana.com/docs/loki/latest/setup/install/))\n\nFor mac users using homebrew:\n```bash\nbrew install prometheus loki promtail\n```\n\n## Debugging fee and staking payout calculations\n\nIt is possible to get more information about the fee and staking payout calculation process logged to\nthe console. Because these calculations happens in the statically compiled web assembly part,\na re-compile with the proper environment variable set is necessary:\n\n```bash\nCALC_DEBUG=1 sh calc/build.sh\n```\n\n## Available endpoints\n\n[Click here for full endpoint docs.](https://paritytech.github.io/substrate-api-sidecar/dist/)\n\n## Chain integration guide\n\n[Click here for chain integration guide.](./guides/CHAIN_INTEGRATION.md)\n\n## Docker\n\nWith each release, the maintainers publish a docker image to dockerhub at [parity/substrate-api-sidecar](https://hub.docker.com/r/parity/substrate-api-sidecar/tags?page=1\u0026ordering=last_updated)\n\n### Pull the latest release\n\n```bash\ndocker pull docker.io/parity/substrate-api-sidecar:latest\n```\n\nThe specific image tag matches the release version.\n\n### Or build from source\n\n```bash\nyarn build:docker\n```\n\n### Run\n\n```bash\n# For default use run:\ndocker run --rm -it --read-only -p 8080:8080 substrate-api-sidecar\n\n# Or if you want to use environment variables set in `.env.docker`, run:\ndocker run --rm -it --read-only --env-file .env.docker -p 8080:8080 substrate-api-sidecar\n```\n\n**NOTE**: While you could omit the `--read-only` flag, it is **strongly recommended for containers used in production**.\n\nthen you can test with:\n\n```bash\ncurl -s http://0.0.0.0:8080/blocks/head | jq\n```\n\n**N.B.** The docker flow presented here is just a sample to help get started. Modifications may be necessary for secure usage.\n\n## Contribute\n\nNeed help or want to contribute ideas or code? Head over to our [CONTRIBUTING](./guides/CONTRIBUTING.md) doc for more information.\n\n## Notes for maintainers\n\n### Commits\n\nAll the commits in this repo follow the [Conventional Commits spec](https://www.conventionalcommits.org/en/v1.0.0/#summary). When merging a PR, make sure 1) to use squash merge and 2) that the title of the PR follows the Conventional Commits spec.\n\n### Updating polkadot-js dependencies\n\n1. Whenever the polkadot-js ecosystem releases a new version, it's important to keep up with these updates and review the release notes for any breaking changes or high priority updates. In order to update all the dependencies and resolutions, create a new branch, such as `yourname-update-pjs`, and then run `yarn up \"@polkadot/*\"` in that branch.\n\n    - @polkadot/api [release notes](https://github.com/polkadot-js/api/releases)\n    - @polkadot/util-crypto [release notes](https://github.com/polkadot-js/common/releases)\n    - @substrate/calc [npm release page](https://www.npmjs.com/package/@substrate/calc)\n\n1. Ensure everything is up to date and working by running the following:\n   ```\n   yarn\n   yarn dedupe\n   yarn build\n   yarn lint\n   yarn test\n   yarn test:historical-e2e-tests\n   yarn test:latest-e2e-tests\n   ```\n\n1. Commit the dependency updates with a name like `chore(deps): update polkadot-js deps` (adjust the title based on what was updated; refer to the commit history for examples). Then, wait for it to be merged.\n\n1. Follow [RELEASE.md](./RELEASE.md) next if you're working through a full sidecar release. This will involve creating a separate PR where the changelog and versions are bumped.\n\n### Maintenance Guide\nA more complete list of the maintainer's tasks can be found in the [MAINTENANCE.md](./guides/MAINTENANCE.md) guide.\n\n## Hardware requirements\n\n### Disk Space\nSidecar is a stateless program and thus should not use any disk space.\n\n### Memory\nThe requirements follow the default of node.js processes which is an upper bound in HEAP memory of a little less than 2GB thus 4GB of memory should be sufficient.\n\n### Running sidecar and a node\nPlease note that if you run sidecar next to a substrate node in a single machine then your system specifications should improve significantly.\n- Our official specifications related to validator nodes can be found in the polkadot wiki [page](https://wiki.polkadot.network/docs/maintain-guides-how-to-validate-polkadot#standard-hardware).\n- Regarding archive nodes :\n  - again as mentioned in the polkadot wiki [page](https://wiki.polkadot.network/docs/maintain-sync#types-of-nodes), the space needed from an archive node depends on which block we are currently on (of the specific chain we are referring to).\n  - there are no other hardware requirements for an archive node since it is not time critical (archive nodes do not participate in the consensus).\n\n### Benchmarks\nDuring the benchmarks we performed, we concluded that sidecar would use a max of 1.1GB of RSS memory.\n\nThe benchmarks were:\n- using 4 threads over 12 open http connections and\n- were overloading the cache with every runtime possible on polkadot.\n\nHardware specs in which the benchmarks were performed:\n```\nMachine type:\nn2-standard-4 (4 vCPUs, 16 GB memory)\n\nCPU Platform:\nIntel Cascade Lake\n\nHard-Disk:\n500GB\n```\n\nBenchmarks are automatically published in Github pages under the url https://paritytech.github.io/substrate-api-sidecar/dev/bench/. The data in the graphs are updated with every new commit/push in the `master` branch (refer to the [benchmark.yml](https://github.com/paritytech/substrate-api-sidecar/blob/master/.github/workflows/benchmark.yml) for more details).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fparitytech%2Fsubstrate-api-sidecar","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fparitytech%2Fsubstrate-api-sidecar","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fparitytech%2Fsubstrate-api-sidecar/lists"}