{"id":50841047,"url":"https://github.com/tetherto/miningos-wrk-ext-mempool","last_synced_at":"2026-06-14T06:35:26.873Z","repository":{"id":344018339,"uuid":"1129677082","full_name":"tetherto/miningos-wrk-ext-mempool","owner":"tetherto","description":null,"archived":false,"fork":false,"pushed_at":"2026-05-21T12:16:38.000Z","size":160,"stargazers_count":0,"open_issues_count":0,"forks_count":4,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-21T20:53:13.455Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tetherto.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-01-07T12:30:04.000Z","updated_at":"2026-05-21T12:16:44.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/tetherto/miningos-wrk-ext-mempool","commit_stats":null,"previous_names":["tetherto/miningos-wrk-ext-mempool"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/tetherto/miningos-wrk-ext-mempool","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tetherto%2Fminingos-wrk-ext-mempool","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tetherto%2Fminingos-wrk-ext-mempool/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tetherto%2Fminingos-wrk-ext-mempool/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tetherto%2Fminingos-wrk-ext-mempool/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tetherto","download_url":"https://codeload.github.com/tetherto/miningos-wrk-ext-mempool/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tetherto%2Fminingos-wrk-ext-mempool/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34312072,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-14T02:00:07.365Z","response_time":62,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-06-14T06:35:21.733Z","updated_at":"2026-06-14T06:35:26.867Z","avatar_url":"https://github.com/tetherto.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# miningos-wrk-ext-mempool\n\n## Table of Contents\n\n1. [Overview](#overview)\n2. [Object Model](#object-model)\n3. [Installation](#installation)\n4. [Configuration](#configuration)\n5. [Running the Worker](#running-the-worker)\n6. [Data Structure](#data-structure)\n7. [Api Integration](#api-integration)\n\n## Overview\n\nThe Mempool worker is a specialized service within the MiningOS ecosystem that interfaces with the Mempool.space API to collect and store real-time Bitcoin network data. This worker serves as a critical data aggregation layer for mining operations, providing essential metrics such as network hashrate, difficulty adjustments, transaction fees, and Bitcoin prices.\n\n### Key Features\n\n- **Real-time Data Collection**: Fetches Bitcoin network statistics at configurable intervals\n- **Historical Data Storage**: Maintains price history for 24 hours and hashrate history for 180 days\n- **Automatic Calculations**: Computes 24-hour price changes and block reward averages across multiple timeframes\n- **Scheduled Statistics**: Builds aggregated statistics at 30-minute, 3-hour, and daily intervals\n- **Persistent Storage**: Uses Hyperbee for efficient time-series data storage\n- **RPC Interface**: Provides network data to other services through RPC methods\n\n## Object Model\n\nThe worker follows a hierarchical class structure within the MiningOS ecosystem:\n\n\n```mermaid\n---\ntitle: Object Model of MiningOS\n---\nflowchart RL\n    bfx-wrk-base@{ shape: stadium, label: \"*bfx-wrk-base*\" }\n\n\n    tether-wrk-base@{ shape: stadium, label: \"*tether-wrk-base*\" }\n    tether-wrk-base---\u003ebfx-wrk-base\n\n\n\n    miningos-wrk-ext-mempool@{label: \"*miningos-wrk-ext-mempool*\" }\n    miningos-wrk-ext-mempool---\u003etether-wrk-base\n\n    style miningos-wrk-ext-mempool fill:#005,stroke-width:4px,color:white\n```\n\nThe architecture ensures modularity and code reusability while maintaining separation of concerns between generic worker functionality and Mempool-specific operations.\n\n## Installation\n\n### Prerequisites\n\n- Node.js \u003e= 20.0\n\n### Setup Steps\n\n1. **Clone the repository**:\n```bash\ngit clone https://github.com/tetherto/miningos-wrk-ext-mempool.git\ncd miningos-wrk-ext-mempool\n```\n\n2. **Install dependencies**:\n```bash\nnpm install\n```\n\n3. **Configure the worker**:\n```bash\n./setup-config.sh\n```\n\n## Configuration\n\n### Configuration Files\n\nThe worker uses several configuration files located in the `config/` directory:\n\n#### config/mempool.json\n\nControls Mempool API connection and data collection intervals:\n```json\n{\n  \"baseUrl\": \"https://mempool.space/api\",\n  \"dataFetchIntervalMs\": 1800000,           // Current data fetch interval (30 minutes)\n  \"historicalDataFetchIntervalMs\": 43200000  // Historical data fetch interval (12 hours)\n}\n```\n\n#### config/common.json\n\nGeneral worker configuration:\n\n```json\n{\n  \"dir_log\": \"logs\",  // Directory for log files\n  \"debug\": 0          // Debug level (0=off, 1=on)\n}\n```\n\n#### config/facs/ Directory\n\nContains facility configurations for network and storage components:\n- `net.config.json`: Network configuration for RPC communication\n- `store.config.json`: Storage configuration for Hyperbee database\n\n### Environment Variables\n\nEnable comprehensive debug logging:\n```bash\nnode worker.js --wtype wrk-ext-mempool-rack --env development --rack rack-1\n```\n\n### RPC Interface\n\nThe worker exposes three RPC methods:\n\n1. **`ping`** (inherited): Health check endpoint\n2. **`getInstanceId`** (inherited): Returns unique worker instance identifier\n3. **`getWrkExtData`**: Returns mempool data and historical statistics\n\n## Running the Worker\n\n### Starting the Worker\n\nLaunch the Mempool worker with the following command:\n\n```bash\nDEBUG=\"*\" node worker.js --wtype wrk-ext-mempool-rack --env development --rack rack-1\n```\n\nParameters explained:\n- `--wtype`: Worker type identifier (must be `wrk-ext-mempool-rack`)\n- `--env`: Environment setting (development/production)\n- `--rack`: Rack identifier for this worker instance\n\n### Verifying Operation\n\nOnce started, the worker will:\n1. Start the RPC server (from parent class)\n2. Register RPC methods: `ping`, `getInstanceId`, `getWrkExtData`\n3. Initialize database connection and load existing data\n4. Initialize the Mempool.space API client\n5. Fetch initial data from all configured endpoints\n6. Schedule periodic updates:\n   - Live data refresh every 30 minutes (configurable)\n   - Historical data refresh every 12 hours (configurable)\n\nCheck the worker status:\n```bash\ncat status/wrk-ext-mempool-rack-rack-1.json\n```\n\n**Note:** The RPC server becomes available immediately upon startup, before data fetching completes. Early RPC calls may return cached data from the database.\n\n## Data Structure\n\n### Internal Data Model\n\nThe worker maintains the following data structure internally:\n\n```javascript\nmempoolData = {\n  // Price tracking\n  prices: [                           // Array of price history entries\n    { time: timestamp, price: value }\n  ],\n  currentPrice: 0,                   // Latest BTC price in USD\n  priceChange24Hrs: 0,              // Calculated 24-hour change percentage\n  \n  // Network metrics\n  blockHeight: 0,                    // Current blockchain height\n  currentHashrate: 0,                // Latest network hashrate\n  currentDifficulty: 0,              // Current mining difficulty\n  \n  // Difficulty adjustment\n  adjustments: {\n    progressToDifficulty: 0,          // Progress percentage\n    nextAdjustmentTs: 0,              // Unix timestamp\n    nextAdjustmentExp: 0,             // Expected change percentage\n    prevAdjustment: 0,                // Previous adjustment percentage\n    avgBlockTime: 0                   // Minutes\n  },\n  \n  // Historical hashrates\n  hashrates: [                        // Array of hashrate snapshots\n    { tag: string, time: timestamp, hashrate: value }\n  ],\n  \n  // Reward calculations\n  blockRewardAvgs: {                  // Averages in BTC\n    '24h': 0, '3d': 0, '1w': 0,\n    '1m': 0, '3m': 0, '6m': 0,\n    '1y': 0, '2y': 0, '3y': 0\n  },\n  \n  // Fee market\n  transactionFees: {                  // Satoshis per vByte\n    fastest: 0,\n    halfHour: 0,\n    hour: 0\n  }\n}\n```\n\n### Data Retention Policies\n\n- **Price History**: Retained for 24 hours\n- **Hashrate History**: Retained for 180 days\n- **Block Rewards**: Calculated from 3-year historical data\n\n## API Integration\n\n### Mempool.space API Endpoints\n\nThe worker interfaces with the following Mempool.space API endpoints:\n\n- **Prices** (/v1/prices): Current Bitcoin price in multiple currencies\n- **Block Height** (/blocks/tip/height): Latest block number\n- **Block Rewards** (/v1/mining/blocks/rewards/3y): Historical mining rewards\n- **Difficulty Adjustments** (/v1/difficulty-adjustment): Next adjustment predictions\n- **Hashrate** (/v1/mining/hashrate/3d): Network hashrate statistics\n- **Transaction Fees** (/v1/fees/recommended): Recommended fee rates\n- **Historical Prices** (/v1/historical-price): Price data at specific timestamps\n- **Block by Timestamp** (/v1/mining/blocks/timestamp/{ts}): Block at given timestamp\n- **Block Details** (/v1/block/{hash}): Detailed block information\n   \n### Rate Limiting\n\nTo respect API rate limits, the worker implements:\n1. 1-second delay between API calls during batch fetches\n2. Configurable fetch interval (default: 30 minutes)\n3. Error Handling:\n   - Failed API calls return `null` and log errors to console\n   - Worker continues operation even if individual fetches fail\n   - No automatic retry mechanism implemented\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftetherto%2Fminingos-wrk-ext-mempool","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftetherto%2Fminingos-wrk-ext-mempool","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftetherto%2Fminingos-wrk-ext-mempool/lists"}