{"id":36878428,"url":"https://github.com/allinbits/eclesia-indexer-core","last_synced_at":"2026-01-12T15:17:08.502Z","repository":{"id":317432139,"uuid":"871090606","full_name":"allinbits/eclesia-indexer-core","owner":"allinbits","description":null,"archived":false,"fork":false,"pushed_at":"2025-12-17T09:20:06.000Z","size":884,"stargazers_count":0,"open_issues_count":4,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-12-20T21:54:12.702Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/allinbits.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2024-10-11T08:52:02.000Z","updated_at":"2025-12-17T09:20:12.000Z","dependencies_parsed_at":"2025-09-30T21:31:14.154Z","dependency_job_id":null,"html_url":"https://github.com/allinbits/eclesia-indexer-core","commit_stats":null,"previous_names":["allinbits/eclesia-indexer-core"],"tags_count":48,"template":false,"template_full_name":null,"purl":"pkg:github/allinbits/eclesia-indexer-core","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allinbits%2Feclesia-indexer-core","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allinbits%2Feclesia-indexer-core/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allinbits%2Feclesia-indexer-core/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allinbits%2Feclesia-indexer-core/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/allinbits","download_url":"https://codeload.github.com/allinbits/eclesia-indexer-core/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allinbits%2Feclesia-indexer-core/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28340439,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-12T12:22:26.515Z","status":"ssl_error","status_checked_at":"2026-01-12T12:22:10.856Z","response_time":98,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-01-12T15:17:07.333Z","updated_at":"2026-01-12T15:17:08.485Z","avatar_url":"https://github.com/allinbits.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Eclesia Indexer Core\n\nA powerful, modular framework for indexing Cosmos SDK blockchain data. Eclesia Indexer provides the tools to efficiently collect, process, and store blockchain data from any Cosmos-based chain.\n\n## Key Features\n\n- Modular Architecture\n- Event-Driven Block Processing\n- PostgreSQL-Optimized Storage Layer\n- Production-Ready Developer CLI\n\n## 🚀 Quick Start\n\n```bash\n# Create a new indexer project\nnpx create-eclesia-indexer@latest\n\n# Configure your chain and follow the instructions to build a postgres-based indexer\n```\n\nMore information in our tutorials: [Simple](TUTORIAL.md) and [Advanced](ADVANCED_TUTORIAL.md)\n\n## 📦 Architecture\n\nEclesia is built as a monorepo with four core packages that work together:\n\n```\n┌─────────────────────┐    ┌──────────────────────┐\n│ create-eclesia-     │    │ @eclesia/            │\n│ indexer             │───▶│ indexer-engine       │\n│ (CLI scaffolding)   │    │ (Core engine)        │\n└─────────────────────┘    └──────────┬───────────┘\n                                      │\n                          ┌───────────▼───────────┐\n                          │ @eclesia/             │\n                          │ basic-pg-indexer      │\n                          │ (PostgreSQL impl)     │\n                          └───────────┬───────────┘\n                                      │\n                          ┌───────────▼───────────┐\n                          │ @eclesia/             │\n                          │ core-modules-pg       │\n                          │ (Pre-built modules)   │\n                          └───────────────────────┘\n```\n\n### Package Responsibilities\n\n| Package | Purpose | Usage |\n|---------|---------|-------|\n| **`@eclesia/indexer-engine`** | Core indexing engine that processes blocks and emits events | Foundation for all indexers |\n| **`@eclesia/basic-pg-indexer`** | PostgreSQL implementation of the indexer engine | Most common use case |\n| **`@eclesia/core-modules-pg`** | Pre-built modules for common Cosmos SDK features | Ready-to-use indexing modules |\n| **`create-eclesia-indexer`** | CLI tool for scaffolding new indexer projects | Getting started quickly |\n\n## 🏗️ Core Concepts\n\n### Event-Driven Architecture\n\nEclesia processes blockchain data by iterating through blocks and emitting events for different types of data:\n\n- **Block Events**: `begin_block`, `block`, `end_block`\n- **Transaction Events**: `tx_events`, `tx_memo`\n- **Validator Events**: Validator set changes and staking data\n- **Custom Events**: Chain-specific events from messages and state changes\n\n### Modular Design\n\nCreate custom modules for the basic PG indexer by implementing the `IndexingModule` interface:\n\n```typescript\ninterface IndexingModule {\n  indexer: EcleciaIndexer           // Reference to main indexer\n  name: string                      // Unique module name\n  depends: string[]                 // Dependencies on other modules\n  provides: string[]                // Capabilities this module provides\n  setup: () =\u003e Promise\u003cvoid\u003e        // Database schema setup\n  init: (...args: any[]) =\u003e void   // Event listener registration\n}\n```\n\n## 📋 Packages\n\n### 🔧 Core Engine (`@eclesia/indexer-engine`)\n\nThe foundational package that provides the core indexing functionality.\n\n**Key Features:**\n- Block processing and event emission\n- WebSocket and polling support for real-time indexing\n- Configurable batch processing\n- Genesis state processing\n- Transaction management\n\n### 🐘 PostgreSQL Indexer (`@eclesia/basic-pg-indexer`)\n\nA PostgreSQL-specific implementation of the indexer engine, perfect for most use cases.\n\n**Key Features:**\n- Built-in PostgreSQL connection management\n- Transaction support with rollback capabilities\n- Database schema management\n- Optimized for high-throughput indexing\n\n### 🧩 Core Modules (`@eclesia/core-modules-pg`)\n\nPre-built indexing modules for common Cosmos SDK functionality.\n\n**Available Modules:**\n- **`Blocks`**: Blocks.Full :Block and transaction indexing or Blocks.Minimal: Height tracking \n- **`AuthModule`**: Account authentication data\n- **`BankModule`**: Token transfers and balances\n- **`StakingModule`**: Validator and delegation data\n\n### 🛠️ Project Generator (`create-eclesia-indexer`)\n\nCLI tool for scaffolding new indexer projects with best practices built-in.\n\n**Features:**\n- Interactive project setup\n- Boilerplate code generation\n- Configuration templates\n- Docker and deployment setup\n\n**Usage:**\n```bash\nnpx create-eclesia-indexer@latest\n```\n\n## ⚙️ Configuration\n\nThe indexer is configured through the `EcleciaIndexerConfig` (or `PgIndexerConfig` for PostgreSQL) interface:\n\n```typescript\nconst config: PgIndexerConfig = {\n  // Block range\n  startHeight: 1,                    // Starting block height\n  endHeight?: number,                // Optional ending height\n\n  // Performance\n  batchSize: 300,                    // Blocks to prefetch and keep ready for processing\n\n  // Chain connection\n  rpcUrl: \"https://rpc.cosmos.network\",\n  usePolling: false,                 // Use WebSocket (false) or polling (true)\n  pollingInterval: 1000,             // Polling interval in ms\n\n  // Database (PostgreSQL specific)\n  dbConnectionString: \"postgres://user:pass@localhost:5432/db\",\n\n  // Features\n  modules: [],                       // Module names to enable\n  minimal: false,                    // Minimal mode (blocks only)\n  processGenesis: false,             // Process genesis state\n  genesisPath: \"./genesis.json\",     // Path to genesis file\n\n  // Logging\n  logLevel: \"info\",                  // Logging verbosity\n\n  // Custom functions\n  init?: () =\u003e Promise\u003cvoid\u003e,        // Custom initialization\n  getNextHeight: () =\u003e Promise\u003cnumber\u003e, // Next block to process\n  shouldProcessGenesis: () =\u003e Promise\u003cboolean\u003e, // Genesis processing check\n}\n```\n\n## 🔄 Typical Workflow\n\n### 1. Standard Workflow (Recommended)\n\n```bash\n# 1. Generate project\nnpx create-eclesia-indexer@latest\n\n# 2. Configure for your chain\ncd my-chain-indexer\n# Edit project as needed\n\n# 3. Choose your modules\n# Use pre-built modules or create custom ones\n\n# 4. Deploy\nnpm run build\nnpm start\n```\n\n### 2. Advanced Workflow (Custom Implementation)\n\nFor advanced users who need full control:\n\n```typescript\nimport { EcleciaIndexer, EcleciaIndexerConfig } from '@eclesia/indexer-engine';\n\nconst config: EcleciaIndexerConfig = {\n  // Custom configuration\n};\n\nconst indexer = new EcleciaIndexer(config);\n// Custom event handlers and modules\nawait indexer.setup();\nawait indexer.run();\n```\n\n## 📊 Example Implementation\n\nHere's how the [AtomOne indexer](https://github.com/allinbits/atomone-indexer) uses Eclesia:\n\n```typescript\nimport {\n  atomoneProtoRegistry,\n} from \"@atomone/atomone-types/atomone/client.js\";\nimport {\n  defaultRegistryTypes,\n} from \"@cosmjs/stargate\";\nimport {\n  PgIndexer, PgIndexerConfig,\n} from \"@eclesia/basic-pg-indexer\";\nimport {\n  AuthModule, BankModule, Blocks, StakingModule,\n} from \"@eclesia/core-modules-pg\";\n\nimport {\n  GovModule,\n} from \"./modules/atomone.gov.v1beta1/index.js\";\n\nconst config: PgIndexerConfig = {\n  startHeight: 1,\n  batchSize: Number(process.env.QUEUE_SIZE) || 300,\n  modules: [],\n  rpcUrl: process.env.RPC_ENDPOINT || \"https://rpc.atomone.network\",\n  logLevel: process.env.LOG_LEVEL as PgIndexerConfig[\"logLevel\"] ?? \"info\",\n  usePolling: false,\n  pollingInterval: 0,\n  processGenesis: process.env.PROCESS_GENESIS === \"true\" || false,\n  minimal: false,\n  genesisPath: \"./genesis.json\",\n  dbConnectionString: process.env.PG_CONNECTION_STRING || \"postgres://postgres:password@localhost:5432/atomone\",\n};\n\nconst registry = defaultRegistryTypes.concat(atomoneProtoRegistry);\nconst blocksModule = new Blocks.FullBlocksModule(registry);\nconst authModule = new AuthModule(registry);\nconst bankModule = new BankModule(registry);\nconst stakingModule = new StakingModule(registry);\nconst govModule = new GovModule(registry);\nconst indexer = new PgIndexer(config, [blocksModule, authModule, bankModule, stakingModule, govModule]);\n\nprocess.on(\"unhandledRejection\", (reason, promise) =\u003e {\n  console.log(\"Unhandled Rejection at:\", promise, \"reason:\", reason);\n  console.trace();\n  process.exit(1);\n});\nconst run = async () =\u003e {\n  try {\n    await indexer.setup();\n    await indexer.run();\n  }\n  catch (error) {\n    console.error(\"Error running indexer:\", error);\n    process.exit(1);\n  }\n};\nrun();\n```\n\n## 🔌 Creating Custom Modules\n\n```typescript\nexport class MyCustomModule implements IndexingModule {\n  name = \"my-custom-module\";\n  depends = [\"blocks\"];              // Depends on blocks module\n  provides = [\"custom-data\"];        // Provides custom data indexing\n\n  constructor(public indexer: EcleciaIndexer) {}\n\n  async setup(): Promise\u003cvoid\u003e {\n    // Create database tables, etc.\n  }\n\n  init(): void {\n    // Register event listeners\n    this.indexer.on(\"tx_events\", async (event) =\u003e {\n      // Process transaction events\n    });\n  }\n}\n```\n\n## 🐳 Deployment\n\nThe generated projects include Docker support with a preconfigured common use case:\n\n```bash\n# Build and run with Docker\ndocker-compose up -d      # Start PostgreSQL, indexer and Hasura instance\n```\n\n## 📈 Performance\n\n- **Batch Processing**: Prefetch multiple blocks in parallel for optimal throughput\n- **WebSocket Support**: Real-time block streaming with automatic reconnection\n- **Transaction Management**: Atomic database operations with automatic rollback\n- **Memory Efficient**: Streaming JSON parsing for large datasets\n- **Optimized Operations**: High-performance insert and serialization operations\n- **Connection Management**: Automatic database connection recycling (every 1500 transactions)\n- **Error Recovery**: Automatic retry logic with exponential backoff\n- **LRU Caching**: Memory-efficient caching for validator data\n\n### Benchmarking\n\nBenchmark the indexer engine without needing an RPC node or database:\n\n```bash\n# Run all benchmarks\npnpm run bench\n\n# Run specific benchmarks\npnpm run bench engine-indexing\n```\n\nThe benchmarking framework provides mock RPC and database clients for isolated performance testing.\n\nFor production performance tuning, see [PERFORMANCE.md](PERFORMANCE.md).\n\n## 🤝 Contributing\n\nThis is a monorepo managed with PNPM workspaces:\n\n```bash\n# Install dependencies\npnpm install\n\n# Build all packages\npnpm build\n\n# Run tests\npnpm test\n\n# Lint code\npnpm lint\n```\n\n## 📜 License\n\nGNO NETWORK GENERAL PUBLIC LICENSE\n\n## 🆘 Support\n\n- **Issues**: Report bugs and feature requests\n- **Documentation**:\n  - [Tutorial](TUTORIAL.md) - Getting started guide\n  - [Advanced Tutorial](ADVANCED_TUTORIAL.md) - Custom module development\n  - [Performance Guide](PERFORMANCE.md) - Production optimization\n  - [Troubleshooting](TROUBLESHOOTING.md) - Common issues and solutions\n- **Examples**: See [AtomOne Indexer](https://github.com/allinbits/atomone-indexer)\n\n---\n\nBuilt with ❤️ for the Cosmos ecosystem","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fallinbits%2Feclesia-indexer-core","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fallinbits%2Feclesia-indexer-core","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fallinbits%2Feclesia-indexer-core/lists"}