{"id":13652506,"url":"https://github.com/holaplex/indexer","last_synced_at":"2025-06-30T22:06:24.082Z","repository":{"id":37217660,"uuid":"408915203","full_name":"holaplex/indexer","owner":"holaplex","description":"Index Solana data using a Geyser plugin (downstream service cluster)","archived":false,"fork":false,"pushed_at":"2023-03-06T11:51:20.000Z","size":4523,"stargazers_count":182,"open_issues_count":33,"forks_count":38,"subscribers_count":6,"default_branch":"dev","last_synced_at":"2025-04-09T22:18:42.077Z","etag":null,"topics":["graphql","rust","solana"],"latest_commit_sha":null,"homepage":"https://holaplex.com","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/holaplex.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"license.md","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2021-09-21T17:35:11.000Z","updated_at":"2025-03-19T04:37:49.000Z","dependencies_parsed_at":"2024-01-12T19:47:13.775Z","dependency_job_id":"142cb5df-f44a-4929-ae0e-0a433dc0fb43","html_url":"https://github.com/holaplex/indexer","commit_stats":null,"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"purl":"pkg:github/holaplex/indexer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/holaplex%2Findexer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/holaplex%2Findexer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/holaplex%2Findexer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/holaplex%2Findexer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/holaplex","download_url":"https://codeload.github.com/holaplex/indexer/tar.gz/refs/heads/dev","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/holaplex%2Findexer/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262857280,"owners_count":23375492,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["graphql","rust","solana"],"created_at":"2024-08-02T02:00:59.961Z","updated_at":"2025-06-30T22:06:23.967Z","avatar_url":"https://github.com/holaplex.png","language":"Rust","funding_links":[],"categories":["Code"],"sub_categories":["Indexers"],"readme":"# `holaplex-indexer`\n*A Solana indexer providing fast, accurate account information*\n\n## Architecture\n\nAs a message producer, the Holaplex Indexing Service leverages the\n[geyser-plugin-interface](https://github.com/solana-labs/solana/tree/master/geyser-plugin-interface)\nto send accounts data directly to a RabbitMQ instance. As a message consumer,\nthe indexer consumes these account messages, deserializes them and inserts them\ninto a PostgreSQL database. Each account needs its own processor, schema and\nmodel.\n\nThis dataset is derived entirely from the messages produced by a validator. This\nsupports a unidirectional dataflow. All data goes directly to the Solana\nblockchain before it is saved in any off chain storage.\n\n![](https://ipfs.cache.holaplex.com/bafkreiceois7frablbcdhiw4573m53rmhboadd5a2tkiw2mkle2el5udke)\n\n### Components\n\n- Solana Geyser plugin, responsible for sending data to our queue system\n- RabbitMQ consumers, responsible for parsing messages and routing them to the\n  proper processor\n- PostgreSQL database, saves the deserialized data\n- GraphQL Crate - serves the PostgreSQL data\n\n\n### Data Indexed\n\nCurrently, the indexer covers the following Solana programs:\n\n- [x] Holaplex wallet graph program\n- [x] Metaplex program\n- [x] Metaplex auction program\n- [x] Metaplex auction house program\n- [x] Metaplex candy machine program\n- [x] Metaplex metadata program\n- [x] SPL token program\n\nAdditionally, the following off-chain data is also indexed:\n\n- [x] Holaplex storefronts\n- [x] Holaplex marketplaces\n- [x] Metaplex JSON metadata\n\n## Getting started\n\n### Diesel\n\nTo set up a development environment, you will need `rustup`, Cargo, Docker,\n`docker-compose`, and the Diesel CLI. Specifically, you will need `diesel_cli`\ninstalled with the `postgres` feature, which can be done like so:\n\n```sh\n$ cargo install diesel_cli --version 1.4.1 --no-default-features --features postgres\n```\n\nInstalling diesel will require `libpq` to be on your system (`brew install\npostgresql` on Mac).\n\n### Migrating\n\nOnce you have the required dependencies, you can get started by running the\nfollowing script to initialize and migrate a containerized Postgres database in\nthe background:\n\n```sh\n$ ./start-developing.sh\n```\n\n## Database Connections\n\nAll indexer crates attempt to connect to the database by reading a Postgres URI\nfrom one of three environment variables:\n\n - `DATABASE_READ_URL` is used by the GraphQL server to identify a read-only\n   database.\n - `DATABASE_WRITE_URL` is used by all indexer services to identify a writable\n   database.\n - `DATABASE_URL` is used as a fallback by all crates, and is assumed (but not\n   guaranteed) to be writeable.\n\nFor debug builds the `.env*` files provided in the repository will provide a\ndefault connection string pointed at the database defined in\n`docker-compose.yml`.  For production builds the database must be manually\nconfigured according to the environment variables above.\n\n## Running the Indexer Cluster\n\nThe indexer consists of four services run by two binaries and a Geyser plugin.\nAll services are connected via a common RabbitMQ node.\n\n### Geyser plugin setup\n\nTo build the Geyser plugin,clone this repo https://github.com/holaplex/indexer-geyser-plugin.git and use the following build command:\n\n```sh\n$ cargo build -pholaplex-indexer-rabbitmq-geyser\n```\n\nThis will produce a build artifact named `libholaplex-indexer-rabbitmq-geyser`\nwith the appropriate file extension for a dynamic library for the host system\n(i.e. `.dll`, `.dylib`, or `.so`).  This plugin can then be used with a Solana\nvalidator.  A sample Geyser JSON configuration for the plugin can be found in\n`crates/geyser-rabbitmq/sample_config.json`.\n\n### Launching the services\n\nOnce the plugin is up and running, the three indexer consumer services can be\nlaunched to process messages from the validator.  The consumers can be launched\nas follows:\n\n```sh\n$ cargo run --bin holaplex-indexer-geyser --features geyser \u0026\n$ cargo run --bin holaplex-indexer-http --features http -- --entity metadata-json \u0026\n$ cargo run --bin holaplex-indexer-http --features http -- --entity store-config \u0026\n```\n\nAll services will need to be configured to run with the same settings that the\nGeyser plugin was configured with, otherwise they will receive no messages or\nsimply fail to start.\n\ni.e :  if your  geyser config json has `\"network\": \"mainnet\"` and `\"startup\": null`, then the exchange name will be `mainnet.startup-all.accounts` and to connect to it you'll need to pass `--network mainnet` `--startup all` to the geyser-consumer binary (or put `NETWORK=mainnet` and `STARTUP=all` in `.env.local`)\n\n## Running the GraphQL Server\n\n### Configuration\n\nThe server binds to the address `[::]:3000` by default.  To change this, set the\n`-p` argument/`PORT` environment variable or the `--addr` argument/`ADDRESS`\nenvironment variable.  \n\nTo see more options for the server, run the following:\n\n```sh\n$ cargo run --bin holaplex-indexer-graphql -- --help\n```\n\n### Startup\n\nTo launch the GraphQL server, simply run the following:\n\n```sh\n$ cargo run --bin holaplex-indexer-graphql\n```\n\n### Contributing\n\nBefore pushing branch changes, run the following (or add it to your Git\npre-push hook) to check for problems, style errors, and schema issues:\n\n```sh\n$ scripts/pre-push.sh\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fholaplex%2Findexer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fholaplex%2Findexer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fholaplex%2Findexer/lists"}