{"id":19610734,"url":"https://github.com/onflow/flow-public-key-indexer","last_synced_at":"2025-07-08T22:06:29.568Z","repository":{"id":41313786,"uuid":"489110167","full_name":"onflow/flow-public-key-indexer","owner":"onflow","description":"A observer service for indexing flow Accounts and by their associated Keys.","archived":false,"fork":false,"pushed_at":"2024-11-04T21:48:13.000Z","size":18114,"stargazers_count":4,"open_issues_count":8,"forks_count":2,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-04-27T22:32:59.932Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/onflow.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-05-05T20:00:55.000Z","updated_at":"2024-11-04T21:48:17.000Z","dependencies_parsed_at":"2024-01-29T19:19:34.044Z","dependency_job_id":"631ee731-d0dc-4b69-8ef1-5e7cf3905217","html_url":"https://github.com/onflow/flow-public-key-indexer","commit_stats":{"total_commits":61,"total_committers":2,"mean_commits":30.5,"dds":"0.032786885245901676","last_synced_commit":"84b523a3baf97066590a839a55bca18e4f748c38"},"previous_names":[],"tags_count":9,"template":false,"template_full_name":null,"purl":"pkg:github/onflow/flow-public-key-indexer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/onflow%2Fflow-public-key-indexer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/onflow%2Fflow-public-key-indexer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/onflow%2Fflow-public-key-indexer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/onflow%2Fflow-public-key-indexer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/onflow","download_url":"https://codeload.github.com/onflow/flow-public-key-indexer/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/onflow%2Fflow-public-key-indexer/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":264357300,"owners_count":23595576,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-11T10:32:54.497Z","updated_at":"2025-07-08T22:06:29.540Z","avatar_url":"https://github.com/onflow.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Flow Public Key Indexer\nA observer service for indexing flow Accounts public keys and REST service that serves up public key data.\n\n\n## Run Parameters\n`KEYIDX_LOGLEVEL` default: \"info\"\n\u003cbr\u003eLog Level: Takes string log level value for zerolog, \"debug\", \"info\", ...\u003c/br\u003e\n\n`KEYIDX_PORT` default: \"8080\"\n\u003cbr\u003ePort: The port the REST service is hosted on\u003c/br\u003e\n\n`KEYIDX_FLOWURL1` default: \"access.mainnet.nodes.onflow.org:9000\"\n\u003cbr\u003eFlow Url: Access node endpoint blockchain data is pulled from, needs to match up with Chain Id. Up to 4 access nodes can be provided, only one is required. The access nodes are cycled through each request to get public key data.\u003c/br\u003e\n\n`KEYIDX_FLOWURL2` default: none\n\u003cbr\u003eFlow Url: Access node endpoint\u003c/br\u003e\n\n`KEYIDX_FLOWURL3` default: none\n\u003cbr\u003eFlow Url: Access node endpoint\u003c/br\u003e\n\n`KEYIDX_FLOWURL4` default: none\n\u003cbr\u003eFlow Url: Access node endpoint\u003c/br\u003e\n\n`KEYIDX_CHAINID` default: \"flow-mainnet\"\n\u003cbr\u003eChain Id: target blockchain, valid values are \"flow-testnet\" and \"flow-mainnet\". Needs to match up with Flow Url\u003c/br\u003e\n\n`KEYIDX_MAXACCTKEYS` default: 1000\n\u003cbr\u003eMax Acct Keys: maximum number of keys to index per account, accounts that have more public keys that exceed max do not get indexed. Accounts are logged out that exceed the max\u003c/br\u003e\n\n`KEYIDX_BATCHSIZE` default: 50000\n\u003cbr\u003eBatch Size: max number of accounts in a batch sent to cadence script that access node executes. Cadence script can exceed execution if accounts have a lot of keys\u003c/br\u003e\n\n`KEYIDX_IGNOREZEROWEIGHT` default: true\n\u003cbr\u003eIgnore Zero Weight: tells the cadence script to ignore public keys with zero weight. These keys will not be indexed\u003c/br\u003e\n\n`KEYIDX_IGNOREREVOKED` default: false\n\u003cbr\u003eIgnore Revoked: tells the cadence script to ignore public keys that have been revoked. These keys will not be indexed\u003c/br\u003e\n\n`KEYIDX_WAITNUMBLOCKS` default: 200\n\u003cbr\u003eWait Num Blocks: number of blocks to wait before running an incremental data load\u003c/br\u003e\n\n`KEYIDX_BLOCKPOLINTERVALSEC` default: 180\n\u003cbr\u003eBlock Pol Interval Sec: number of seconds to wait before checking current block height to determine to run an incremental data load\u003c/br\u003e\n\n`KEYIDX_SYNCDATAPOLINTERVALMIN` default: 1\n\u003cbr\u003eSync Data Polling Interval: number of minutes to wait between sync data operations\u003c/br\u003e\n\n`KEYIDX_SYNCDATASTARTINDEX` default: 30000000\n\u003cbr\u003eSync Data Start Index: starting block height for sync operations\u003c/br\u003e\n\n`KEYIDX_MAXBLOCKRANGE` default: 600\n\u003cbr\u003eMax Block Range: number of blocks that will trigger a bulk load if services falls behind\u003c/br\u003e\n\n`KEYIDX_FETCHSLOWDOWNMS` default: 500\n\u003cbr\u003eFetch Slowdown: milliseconds to wait between fetch operations\u003c/br\u003e\n\n`KEYIDX_PURGEONSTART` default: false\n\u003cbr\u003ePurge on Start: When changing the data structure or want to clear the database and start from scratch change this variable to true\u003c/br\u003e\n\n`KEYIDX_ENABLESYNCDATA` default: true\n\u003cbr\u003eEnable Sync Data: Run this service as a sync service. It's possible to run this service only as rest service\u003c/br\u003e\n\n`KEYIDX_ENABLEINCREMENTAL` default: true\n\u003cbr\u003eEnable Incremental: Enable incremental updates of the database\u003c/br\u003e\n\n## PostgreSQL configurations\n`KEYIDX_POSTGRESQLHOST` default: \"localhost\"\n`KEYIDX_POSTGRESQLPORT` default: 5432\n`KEYIDX_POSTGRESQLUSERNAME` default: \"postgres\"\n`KEYIDX_POSTGRESQLPASSWORD` not required, no default\n`KEYIDX_POSTGRESQLDATABASE` default: \"keyindexer\"\n`KEYIDX_POSTGRESQLSSL` default: true\n`KEYIDX_POSTGRESQLLOGQUERIES` default: false\n`KEYIDX_POSTGRESQLSETLOGGER` default: false\n`KEYIDX_POSTGRESQLRETRYNUMTIMES` default: 30\n`KEYIDX_POSTGRESQLRETRYSLEEPTIME` default: \"1s\"\n`KEYIDX_POSTGRESQLPOOLSIZE` default: 1\n`KEYIDX_POSTGRESLOGGERPREFIX` default: \"keyindexer\"\n`KEYIDX_POSTGRESPROMETHEUSSUBSYSTEM` default: \"keyindexer\"\n\n## Re-indexing Accounts\n\nThe service supports re-indexing of specific accounts by adding them to the `addressprocessing` table. This feature is useful when you need to:\n- Update account information that might have changed\n- Re-process accounts that may have had errors during initial indexing\n- Force a refresh of specific account data\n\n### How to Re-index Accounts\n\n1. Add addresses to the `addressprocessing` table:\n```sql\nINSERT INTO addressprocessing (account) \nVALUES ('0x1234...'), ('0x5678...')\nON CONFLICT (account) DO NOTHING;\n```\n\n2. The service will automatically:\n   - Pick up these addresses during the next bulk processing cycle\n   - Re-fetch their public key information\n   - Update the database with any changes\n   - Remove the addresses from the `addressprocessing` table once processed\n\n### Processing Behavior\n- Addresses in the `addressprocessing` table are processed in batches (defined by `KEYIDX_BATCHSIZE`)\n- Duplicate addresses are automatically ignored (ON CONFLICT DO NOTHING)\n- Processing occurs during the bulk load cycle (controlled by `KEYIDX_SYNCDATAPOLINTERVALMIN`)\n- After successful processing, addresses are automatically removed from the `addressprocessing` table\n\n### Related Configuration Parameters\n- `KEYIDX_BATCHSIZE` default: 50000\n  \u003cbr\u003eControls how many addresses are processed in each batch\u003c/br\u003e\n- `KEYIDX_SYNCDATAPOLINTERVALMIN` default: 1\n  \u003cbr\u003eDetermines how frequently the service checks for new addresses to process\u003c/br\u003e\n\n## How to Run\nSince this is a golang service there are many ways to run it. Below are two ways to run this service\n### Command line\n```go run .```\n### Docker\nConfiguration: Run docker in default 10 gig memory size. Reducing the running memory size reduces performance, the lowest is 6 gig, bulk sync and public key query responses are reasonable compared to running with more memory.\u003cbr\u003e\nCreate a docker container\u003cbr\u003e\n```docker build -t key-indexer .``` \u003cbr\u003e\n\n``` need to configure to use postgresql ```\nThis service stores public key data and needs persistent storage in postgresql\u003cbr\u003e\nRun the docker and map the rest service port\u003cbr\u003e\nNotice that environmental variables can be passed in. See variables above\u003cbr\u003e\n```docker run -p 8888:8080 --env KEYIDX_POSTGRESQLHOST=localhost``` \u003cbr\u003e\nTo see the logs of the container, get the container id \u003cbr\u003e\n```docker container ls``` \u003cbr\u003e\nView the containers logs \u003cbr\u003e\n```docker logs \u003ccontainer id\u003e``` \u003cbr\u003e\n## REST service\n`Endpoints`\n* `GET /key/{public key}`\n\u003cp\u003enote: public key is in base64 format\nserves up json object\u003c/p\u003e\n\n```json\n{\n    \"publicKey\": string,  // public key string in base64\n    \"accounts\": [\n        {\n            \"address\": string,    // Flow account address\n            \"keyId\": int,        // Key index in the account\n            \"weight\": int,       // Key weight for signing\n            \"sigAlgo\": int,      // Signing algorithm identifier\n            \"hashAlgo\": int,     // Hashing algorithm identifier\n            \"signing\": string,   // Human-readable signing algorithm name\n            \"hashing\": string    // Human-readable hashing algorithm name\n        }\n    ]\n}\n```\n\n\u003cp\u003esigAlgo - signing: 1 - ECDSA_P256, 2 - ECDSA_secp256k1, 3 - ECDSA_P384, 4 - ECDSA_P521, 5 - ECDSA_secp256r1\u003c/p\u003e\n\u003cp\u003ehashAlgo - hashing: 1 - SHA2_256, 2 - SHA3_256, 3 - SHA2_384, 4 - SHA3_384, 5 - SHA2_512, 6 - SHA3_512\u003c/p\u003e\n\n* `GET /status`\n\u003cp\u003enote: this endpoint gives ability to see if the server is active and updating\u003c/p\u003e\n\n```json\n{\n    \"Count\": int,           // Number of unique public keys indexed\n    \"LoadedToBlock\": int,   // Last processed block height\n    \"CurrentBlock\": int     // Current block height on the Flow network\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fonflow%2Fflow-public-key-indexer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fonflow%2Fflow-public-key-indexer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fonflow%2Fflow-public-key-indexer/lists"}