{"id":13651912,"url":"https://github.com/paritytech/substrate-archive","last_synced_at":"2025-07-02T06:34:06.640Z","repository":{"id":37182496,"uuid":"205807439","full_name":"paritytech/substrate-archive","owner":"paritytech","description":"Blockchain Indexing Engine","archived":false,"fork":false,"pushed_at":"2023-03-06T23:07:15.000Z","size":32706,"stargazers_count":197,"open_issues_count":62,"forks_count":73,"subscribers_count":9,"default_branch":"master","last_synced_at":"2025-07-01T00:18:12.638Z","etag":null,"topics":["blockchain","indexing","polkadot","postgres","rust","substrate"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/paritytech.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":"CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2019-09-02T08:07:32.000Z","updated_at":"2025-04-06T22:17:23.000Z","dependencies_parsed_at":"2024-01-03T05:25:03.121Z","dependency_job_id":null,"html_url":"https://github.com/paritytech/substrate-archive","commit_stats":null,"previous_names":[],"tags_count":10,"template":false,"template_full_name":null,"purl":"pkg:github/paritytech/substrate-archive","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paritytech%2Fsubstrate-archive","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paritytech%2Fsubstrate-archive/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paritytech%2Fsubstrate-archive/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paritytech%2Fsubstrate-archive/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/paritytech","download_url":"https://codeload.github.com/paritytech/substrate-archive/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paritytech%2Fsubstrate-archive/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263087256,"owners_count":23411863,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["blockchain","indexing","polkadot","postgres","rust","substrate"],"created_at":"2024-08-02T02:00:53.401Z","updated_at":"2025-07-02T06:34:06.608Z","avatar_url":"https://github.com/paritytech.png","language":"Rust","funding_links":[],"categories":["Rust","Tools"],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\n# Maintainance status: currently unmmaintained; please fork.\n\t\nHave a look at https://github.com/paritytech/substrate-archive/issues/464 for more information.\n\t\n# Substrate Archive\n\n### Blockchain Indexing Engine\n\n[Install the CLI](#install-the-cli) • [Documentation](#documentation) • [Contributing](#contributing) • [FAQ](#faq)\n\n![Rust](https://github.com/paritytech/substrate-archive/workflows/Rust/badge.svg)\n\u003ca href=\"https://matrix.to/#/!roCGBGBArdcqwsdeXc:matrix.parity.io?via=matrix.parity.io\u0026via=matrix.org\u0026via=web3.foundation\"\u003e\n![Matrix](https://img.shields.io/badge/Matrix-archive%20chatroom-blue)\n\u003c/a\u003e\n\u003c/div\u003e\n\nRun alongside a substrate-backed chain to index all Blocks, State, and Extrinsic data into PostgreSQL.\n\n![](https://i.imgur.com/1eOkKvo.gif)\n\n## Usage\n\nThe schema for the PostgreSQL database is described in the PDF File at the root of this directory.\n\nExamples for how to use substrate-archive are in the [`examples/`](https://github.com/paritytech/substrate-archive/tree/master/examples) directory\n\n## Prerequisites\n\nExtended requirements list found in the [wiki](https://github.com/paritytech/substrate-archive/wiki/)\n\n- depending on the chain you want to index, ~60GB free space\n- PostgreSQL with a database ready for lots of new data\n- Substrate-based Blockchain running with RocksDB as the backend\n- Substrate-based Blockchain running under `--pruning=archive`\n\n## Install The CLI\n\n### The CLI\n\nThe CLI is an easier way to get started with substrate-archive. It provides a batteries-included binary, so that you don't have to write any rust code. All thats required is setting up a PostgreSQL DB, and modifying a config file. More information in the [wiki](https://github.com/paritytech/substrate-archive/wiki)\n\n#### The Node-Template CLI\n\nThe node-template CLI (in /bin/node-template-archive) is provided as an example of implementing substrate-archive for your chain.\n\n### Quick Start\n\n```bash\ngit clone https://github.com/paritytech/substrate-archive.git\n# Set up the databases\nsource ./substrate-archive/scripts/up.sh # Run ./scripts/down.sh to drop the database\ncd substrate-archive/bin/polkadot-archive/\n# Start the normal polkadot node with `pruning` set to `archive`\npolkadot --chain=polkadot --pruning=archive\n# Start up the substrate-archive node. `chain` can be one of `polkadot`, `kusama`, or `westend`.\ncargo run --release --  -c test_conf.toml --chain=polkadot\n```\n\nYou can access the help dialog via `cargo run --release -- --help`. Note that `up` and `down` scripts are meant for convenience and are not meant to be complete. Look in the [wiki](https://github.com/paritytech/substrate-archive/wiki) for more information about the database setup.\n\n## [FAQ](https://github.com/paritytech/substrate-archive/wiki/0.\\)-FAQ)\n\n## Contributing\n\nContributors are welcome!\n\nRead the [Doc](https://github.com/paritytech/substrate-archive/blob/master/CONTRIBUTING.md)\n\n## Documentation\n\nYou can build the documentation for this crate by running `cargo doc`. \nMore Docs [here](https://github.com/paritytech/substrate-archive/wiki)\n\n## Troubleshooting\n\n### Archive fails to start with a `too many open files` error.\n\nBecause of the way a [RocksDB Secondary Instance](https://github.com/facebook/rocksdb/wiki/Secondary-instance) works, it requires that all the files of the primary instance remain open in the secondary instance. This could trigger the error on linux, but simply requires that you raise the max open files limit (`ulimit`):\n\n- With Docker: `$ docker run --ulimit nofile=90000:90000 \u003cimage-tag\u003e`\n- For Current Shell Session: `ulimit -a 90000`\n- Permanantly Per-User\n  - Edit `/etc/security/limits.conf`\n\n    ```\n    # /etc/security/limits.conf\n    *           hard    nofile      4096\n    *           soft    nofile      1024\n    some_usr    hard    nofile      90000\n    some_usr    soft    nofile      90000\n    insipx      hard    nofile      90000\n    insipx      soft    nofile      90000\n    root        hard    nofile      90000\n    root        soft    nofile      90000\n    ```\n\nFor macOS and Linux, a warning message will be raised on the startup when there is a low fd sources limit in the current system, but Windows won't have such a low fd limit warning.\n\n## Contact\n\nYou can contact us at:\n - matrix: #substrate-archive:matrix.parity.io\n - email: andrew.plaza@parity.io\n\n[contribution]: CONTRIBUTING.md\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fparitytech%2Fsubstrate-archive","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fparitytech%2Fsubstrate-archive","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fparitytech%2Fsubstrate-archive/lists"}