{"id":21156930,"url":"https://github.com/resilientecosystem/resilient-node-cache","last_synced_at":"2025-04-13T12:29:36.321Z","repository":{"id":260395170,"uuid":"881187500","full_name":"ResilientEcosystem/resilient-node-cache","owner":"ResilientEcosystem","description":"TypeScript library for syncing ResilientDB data via WebSocket and HTTP with seamless reconnection.","archived":false,"fork":false,"pushed_at":"2024-10-31T04:07:55.000Z","size":49,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-22T08:48:32.338Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://www.npmjs.com/package/resilient-node-cache","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ResilientEcosystem.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-10-31T04:03:54.000Z","updated_at":"2024-11-01T15:41:30.000Z","dependencies_parsed_at":"2024-10-31T07:07:18.036Z","dependency_job_id":null,"html_url":"https://github.com/ResilientEcosystem/resilient-node-cache","commit_stats":null,"previous_names":["resilientecosystem/resilient-node-cache"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ResilientEcosystem%2Fresilient-node-cache","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ResilientEcosystem%2Fresilient-node-cache/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ResilientEcosystem%2Fresilient-node-cache/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ResilientEcosystem%2Fresilient-node-cache/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ResilientEcosystem","download_url":"https://codeload.github.com/ResilientEcosystem/resilient-node-cache/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248713922,"owners_count":21149803,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-20T11:55:06.872Z","updated_at":"2025-04-13T12:29:36.291Z","avatar_url":"https://github.com/ResilientEcosystem.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n# resilient-node-cache\n\n**TypeScript library for syncing ResilientDB data via WebSocket and HTTP with seamless reconnection.**\n\n[![License](https://img.shields.io/badge/license-Apache%202-blue)](https://www.apache.org/licenses/LICENSE-2.0)\n\n`resilient-node-cache` is a library for establishing and managing real-time synchronization between ResilientDB and MongoDB using WebSocket and HTTP. It includes automatic reconnection, batching, and concurrency management for high-performance syncing.\n\n## Features\n\n- Real-time sync between ResilientDB and MongoDB\n- Handles WebSocket and HTTP requests with auto-reconnection\n- Configurable synchronization intervals and batch processing\n- Provides MongoDB querying for ResilientDB transaction data\n\n## Installation\n\nTo use the library, install it via npm:\n\n```bash\nnpm install resilient-node-cache\n```\n\n## Configuration\n\n### MongoDB Configuration\n\nThe library uses MongoDB to store ResilientDB data. Configure it by specifying:\n\n- `uri`: MongoDB connection string\n- `dbName`: Database name in MongoDB\n- `collectionName`: Collection name in MongoDB where ResilientDB data is stored\n\n### ResilientDB Configuration\n\nFor ResilientDB configuration, specify:\n\n- `baseUrl`: The base URL for ResilientDB, e.g., `resilientdb://localhost:18000`\n- `httpSecure`: Use HTTPS if set to `true`\n- `wsSecure`: Use WSS if set to `true`\n- `reconnectInterval`: Reconnection interval in milliseconds (optional)\n- `fetchInterval`: Fetch interval in milliseconds for periodic syncs (optional)\n\n## Usage\n\n### 1. Syncing Data from ResilientDB to MongoDB\n\nCreate a sync script to initialize and start the data synchronization from ResilientDB to MongoDB.\n\n```javascript\n// sync.js\n\nconst { WebSocketMongoSync } = require('resilient-node-cache');\n\nconst mongoConfig = {\n  uri: 'mongodb://localhost:27017',\n  dbName: 'myDatabase',\n  collectionName: 'myCollection',\n};\n\nconst resilientDBConfig = {\n  baseUrl: 'resilientdb://crow.resilientdb.com',\n  httpSecure: true,\n  wsSecure: true,\n};\n\nconst sync = new WebSocketMongoSync(mongoConfig, resilientDBConfig);\n\nsync.on('connected', () =\u003e {\n  console.log('WebSocket connected.');\n});\n\nsync.on('data', (newBlocks) =\u003e {\n  console.log('Received new blocks:', newBlocks);\n});\n\nsync.on('error', (error) =\u003e {\n  console.error('Error:', error);\n});\n\nsync.on('closed', () =\u003e {\n  console.log('Connection closed.');\n});\n\n(async () =\u003e {\n  try {\n    await sync.initialize();\n    console.log('Synchronization initialized.');\n  } catch (error) {\n    console.error('Error during sync initialization:', error);\n  }\n})();\n```\n\nThis script will:\n- Initialize the connection to MongoDB and ResilientDB\n- Continuously sync new blocks received from ResilientDB to MongoDB\n\n### 2. Fetching Transactions by Public Key\n\nAfter syncing, you can retrieve that data from MongoDB depending on your use case. Here’s a sample script to retrieve all transactions associated with a specified public key:\n\n```javascript\n// fetchTransactionsWithPublicKey.js\n\nconst { MongoClient } = require('mongodb');\n\nconst mongoConfig = {\n  uri: 'mongodb://localhost:27017',\n  dbName: 'myDatabase',\n  collectionName: 'myCollection',\n};\n\n// The publicKey for which to fetch transactions\nconst targetPublicKey = \"8LUKr81SmkdDhuBNAHfH9C8G5m6Cye2mpUggVu61USbD\";\n\n(async () =\u003e {\n  const client = new MongoClient(mongoConfig.uri);\n\n  try {\n    await client.connect();\n    const db = client.db(mongoConfig.dbName);\n    const collection = db.collection(mongoConfig.collectionName);\n\n    console.log('Connected to MongoDB for fetching transactions.');\n\n    // Create an index on transactions.value.inputs.owners_before for optimized querying\n    const indexName = await collection.createIndex({ \"transactions.value.inputs.owners_before\": 1 });\n    console.log(`Index created: ${indexName} on transactions.value.inputs.owners_before`);\n\n    // Define aggregation pipeline to fetch all transactions for the specified publicKey in owners_before\n    const pipeline = [\n      { $unwind: \"$transactions\" },\n      { $unwind: \"$transactions.value.inputs\" },\n      { \n        $match: { \n          \"transactions.value.inputs.owners_before\": targetPublicKey \n        }\n      },\n      { $sort: { \"transactions.value.asset.data.timestamp\": -1 } },\n      { $project: { transaction: \"$transactions\", _id: 0 } }\n    ];\n\n    const cursor = collection.aggregate(pipeline);\n    const transactions = await cursor.toArray();\n\n    if (transactions.length \u003e 0) {\n      console.log('Transactions with the specified publicKey in owners_before:', \n                  JSON.stringify(transactions, null, 2));\n    } else {\n      console.log(`No transactions found for publicKey: ${targetPublicKey}`);\n    }\n  } catch (error) {\n    console.error('Error fetching transactions:', error);\n  } finally {\n    await client.close();\n  }\n})();\n```\n\n### API Documentation\n\n#### Class `WebSocketMongoSync`\n\n- **constructor(mongoConfig: MongoConfig, resilientDBConfig: ResilientDBConfig)**:\n  - Initializes the sync object with MongoDB and ResilientDB configurations.\n\n- **initialize()**: Connects to MongoDB, fetches initial blocks, starts periodic fetching, and opens the WebSocket connection to ResilientDB.\n\n- **close()**: Closes the MongoDB and WebSocket connections, stopping the periodic fetching.\n\n### MongoDB Collection Structure\n\nEach document in the MongoDB collection corresponds to a ResilientDB block, containing:\n- **id**: Block identifier\n- **createdAt**: Timestamp for block creation\n- **transactions**: Array of transactions in the block, with details for each transaction’s inputs, outputs, and asset metadata\n\n### License\n\nThis library is licensed under the [Apache License, Version 2.0](https://www.apache.org/licenses/LICENSE-2.0).\n\n---\n\nEnjoy using `resilient-node-cache` to seamlessly synchronize your ResilientDB data to MongoDB with real-time updates and efficient querying!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fresilientecosystem%2Fresilient-node-cache","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fresilientecosystem%2Fresilient-node-cache","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fresilientecosystem%2Fresilient-node-cache/lists"}