{"id":31522663,"url":"https://github.com/cmccarthyirl/celestia-node","last_synced_at":"2026-04-13T20:32:10.640Z","repository":{"id":305912326,"uuid":"1024359376","full_name":"cmccarthyIrl/celestia-node","owner":"cmccarthyIrl","description":"A TypeScript-based framework for deploying and managing Celestia nodes across remote infrastructure. This framework provides automated deployment pipelines, testing capabilities, and lifecycle management for Celestia blockchain nodes.","archived":false,"fork":false,"pushed_at":"2025-07-22T15:42:47.000Z","size":121,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-10-06T18:06:10.571Z","etag":null,"topics":["azure-pipelines","celestia","celestia-node","data-availability","data-availability-sampling","nodejs","playwright","typescript"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cmccarthyIrl.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-07-22T15:17:22.000Z","updated_at":"2025-07-22T15:42:50.000Z","dependencies_parsed_at":"2025-07-22T17:33:04.330Z","dependency_job_id":"d481275e-8aa0-41c1-bead-acb9e1884307","html_url":"https://github.com/cmccarthyIrl/celestia-node","commit_stats":null,"previous_names":["cmccarthyirl/celestia-node"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/cmccarthyIrl/celestia-node","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cmccarthyIrl%2Fcelestia-node","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cmccarthyIrl%2Fcelestia-node/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cmccarthyIrl%2Fcelestia-node/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cmccarthyIrl%2Fcelestia-node/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cmccarthyIrl","download_url":"https://codeload.github.com/cmccarthyIrl/celestia-node/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cmccarthyIrl%2Fcelestia-node/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31770718,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-13T20:17:16.280Z","status":"ssl_error","status_checked_at":"2026-04-13T20:17:08.216Z","response_time":93,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["azure-pipelines","celestia","celestia-node","data-availability","data-availability-sampling","nodejs","playwright","typescript"],"created_at":"2025-10-03T15:41:02.444Z","updated_at":"2026-04-13T20:32:10.621Z","avatar_url":"https://github.com/cmccarthyIrl.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Celestia Deployment Framework\n\nA comprehensive TypeScript-based framework for deploying and managing Celestia light nodes across remote infrastructure. This framework provides automated deployment pipelines, testing capabilities, and lifecycle management for Celestia blockchain nodes.\n\n## 📋 Table of Contents\n\n- [Architecture Overview](#-architecture-overview)\n- [Documentation](#-documentation)\n- [Features](#-features)\n- [Quick Start](#-quick-start)\n- [Configuration](#️-configuration)\n- [Development](#-development)\n- [Testing](#-testing)\n- [Monitoring \u0026 Health Checks](#-monitoring--health-checks)\n- [Troubleshooting](#-troubleshooting)\n- [Contributing](#-contributing)\n- [Support](#️-support)\n\n## 🏗️ Architecture Overview\n\n```\n┌─────────────────────────────────────────────────────────────────┐\n│                    Celestia Deployment Framework                │\n├─────────────────────────────────────────────────────────────────┤\n│                                                                 │\n│  ┌─────────────────┐  ┌─────────────────┐  ┌─────────────────┐  │\n│  │   Azure DevOps  │  │   TypeScript    │  │   Playwright    │  │\n│  │   Pipelines     │  │   Services      │  │   Testing       │  │\n│  └─────────────────┘  └─────────────────┘  └─────────────────┘  │\n│           │                       │                       │     │\n│           ▼                       ▼                       ▼     │\n│  ┌─────────────────┐  ┌─────────────────┐  ┌─────────────────┐  │\n│  │  SSH Service    │  │ Node Lifecycle  │  │  Test Executor  │  │\n│  │  Management     │  │   Management    │  │    \u0026 Reports    │  │\n│  └─────────────────┘  └─────────────────┘  └─────────────────┘  │\n│           │                       │                       │     │\n│           └───────────────────────┼───────────────────────┘     │\n│                                   ▼                             │\n│                    ┌─────────────────────────────┐              │\n│                    │     Remote Celestia Node    │              │\n│                    │                             │              │\n│                    │  • Light Node Process       │              │\n│                    │  • Configuration Management │              │\n│                    │  • Network Integration      │              │\n│                    │  • Health Monitoring        │              │\n│                    └─────────────────────────────┘              │\n└─────────────────────────────────────────────────────────────────┘\n```\n\n## 📚 Documentation\n\n### Core Documentation\n- **[API Documentation](docs/api-documentation.md)** - Complete API reference for all services, methods, and interfaces\n- **[Deployment Guide](docs/deployment-guide.md)** - Step-by-step deployment instructions for local and CI/CD environments\n- **[Configuration Options](docs/configuration-options.md)** - Comprehensive configuration guide for all components and environments\n- **[Troubleshooting Guide](docs/troubleshooting.md)** - Common issues, solutions, diagnostic tools, and debugging techniques\n- **[Architecture Diagrams](docs/architecture-diagrams.md)** - Visual system architecture, component relationships, and data flows\n\n### Quick Reference\n- [Environment Variables Guide](docs/configuration-options.md#environment-variables) - All available configuration options\n- [Common Error Solutions](docs/troubleshooting.md#common-error-messages) - Quick fixes for frequent issues\n- [Network Configuration](docs/configuration-options.md#network-configuration) - Port and firewall setup\n- [Security Best Practices](docs/deployment-guide.md#security) - Security recommendations and configurations\n\n## ✨ Features\n\n### 🚀 **Automated Deployment**\n- Complete infrastructure setup and provisioning\n- Automated Celestia node compilation and installation\n- Environment configuration and dependency management\n- Network initialization and configuration\n\n### 🔄 **Lifecycle Management**\n- Node start/stop/restart operations\n- Health monitoring and status checking\n- Process management and cleanup\n- Configuration updates and validation\n\n### 🧪 **Testing \u0026 Validation**\n- Comprehensive Playwright test suites\n- End-to-end deployment workflow testing\n- Node connectivity and functionality validation\n- Automated test reporting and results publishing\n\n### 🔒 **Security \u0026 Reliability**\n- SSH-based secure remote operations\n- Credential management and key handling\n- Error handling and retry mechanisms\n- Comprehensive logging and monitoring\n\n## 🚀 Quick Start\n\n### Prerequisites\n\n- Node.js 24.x or higher\n- SSH key pair for remote server access\n- Remote Ubuntu/Debian server with sudo access\n- Azure DevOps (for CI/CD pipelines)\n\n### Installation\n\n1. **Clone the repository**\n   ```bash\n   git clone \u003crepository-url\u003e\n   cd celestia-node\n   ```\n\n2. **Install dependencies**\n   ```bash\n   npm ci\n   ```\n\n3. **Configure environment**\n   ```bash\n   cp .env.example .env\n   # Edit .env with your configuration\n   ```\n\n4. **Set up SSH keys**\n   ```bash\n   mkdir -p ~/.ssh\n   cp your-key.pem ~/.ssh/id_rsa.pem\n   chmod 600 ~/.ssh/id_rsa.pem\n   ```\n\n### Basic Usage\n\n**Local Development:**\n```bash\n# Run full deployment workflow\nnpm run test:pipeline\n\n# Run specific test patterns\nnpx playwright test --grep=@DeploymentWorkflow --project=pipeline --reporter=line --timeout=600000\n\n# Development tasks\nnpm run lint        # Code linting\nnpm run format      # Code formatting\nnpm run check       # All quality checks\n```\n\n**Production Deployment:**\n```bash\n# Deploy via Azure DevOps pipeline\n# Pipeline automatically triggers on push to main/develop branches\n\n# Manual deployment verification\ncurl http://your-server:26658/health\ncurl http://your-server:26659/\n```\n\n## ⚙️ Configuration\n\n### Environment Variables\n\nCreate a `.env` file in the project root:\n\n```bash\n# Remote Server Configuration\nREMOTE_HOST=your-server-ip\nREMOTE_USER=ubuntu\nSSH_KEY_PATH=./keys/id_rsa.pem\n\n# Celestia Node Configuration\nCELESTIA_USER=celestia\nCELESTIA_NETWORK=mocha-4\nCELESTIA_VERSION=v0.23.3-mocha\nNODE_TYPE=light\n\n# Network Ports\nRPC_PORT=26658\nGATEWAY_PORT=26659\n\n# Deployment Settings\nLOG_LEVEL=info\nCREATE_BACKUP=true\nENABLE_DEBUG=false\nPOOL_NAME=your-pool\n```\n\n### Azure DevOps Variables\n\nConfigure these variables in your Azure DevOps variable group `celestial-lightning-variable-group`:\n\n| Variable | Description | Example |\n|----------|-------------|---------|\n| `VM_IP_ADDRESS` | Remote server IP address | `ip address` |\n| `VM_USER` | Remote server username | `ubuntu` |\n| `SSH_KEY_FILE` | SSH private key filename | `id_rsa.pem` |\n\n## 🔧 Development\n\n### Project Structure\n\n```\nsrc/\n├── services/\n│   ├── celestia/\n│   │   └── lifecycle/\n│   │       └── NodeLifecycleService.ts\n│   └── ssh/\n│       └── SshService.ts\n├── tests/\n│   └── CelestiaDeploymentWorkflow.spec.ts\n└── utils/\n    └── PlaywrightTestListener.ts\n\npipeline/\n├── run.yml                    # Main pipeline definition\n├── stages/\n│   ├── cleanup/\n│   ├── deployment/\n│   ├── security/\n│   └── testing/\n├── variables/\n│   └── celestia.yml\n└── workflows/\n    ├── celestia/\n    └── common/\n\ndocs/\n├── api-documentation.md\n├── deployment-guide.md\n├── configuration-options.md\n├── troubleshooting.md\n└── architecture-diagrams.md\n```\n\n### Development Workflow\n\n1. **Local Development**\n   ```bash\n   # Install dependencies\n   npm ci\n\n   # Run linting and formatting\n   npm run lint\n   npm run format\n\n   # Run tests locally\n   npm run test:pipeline\n\n   # Check all code quality\n   npm run check\n   ```\n\n2. **Testing Workflow**\n   ```bash\n   # Unit tests (when available)\n   npm test\n\n   # Integration tests\n   npm run test:pipeline\n\n   # Specific test patterns\n   npx playwright test --grep=\"@DeploymentWorkflow\" --project=pipeline\n\n   # Debug mode testing\n   ENABLE_DEBUG=true npm run test:pipeline\n   ```\n\n3. **CI/CD Integration**\n   ```bash\n   # Automatic deployment on push to main/develop\n   git push origin main\n\n   # Manual pipeline trigger via Azure DevOps UI\n   # View results in pipeline dashboard\n   ```\n\n## 🧪 Testing\n\n### Test Types\n- **Unit Tests**: Component-level testing (planned)\n- **Integration Tests**: Service interaction testing via Playwright\n- **End-to-End Tests**: Full deployment workflow testing\n- **API Tests**: Celestia node API validation and health checks\n\n### Running Tests\n\n```bash\n# All tests (integration + E2E)\nnpm run test\n\n# Pipeline-specific tests\nnpm run test:pipeline\n\n# With specific timeout and reporting\nnpx playwright test --grep=@DeploymentWorkflow --project=pipeline --reporter=line --timeout=600000\n\n# Debug mode with verbose output\nENABLE_DEBUG=true npm run test:pipeline\n\n# Generate and view HTML reports\nnpx playwright show-report\n```\n\n### Test Configuration\n\nThe framework uses Playwright with custom configuration for deployment testing:\n\n- **Timeout**: 600 seconds for deployment operations\n- **Retries**: Configurable retry logic for flaky operations\n- **Parallel**: Single worker for deployment to avoid conflicts\n- **Reports**: HTML and JUnit XML formats for CI/CD integration\n\n## 📊 Monitoring \u0026 Health Checks\n\n### Built-in Health Monitoring\n\nThe framework provides comprehensive health monitoring capabilities:\n\n```bash\n# Node API health check\ncurl http://your-server:26658/health\n\n# Node status and sync information\ncurl http://your-server:26658/status\n\n# Network information and peers\ncurl http://your-server:26658/net_info\n\n# Gateway service availability\ncurl http://your-server:26659/\n```\n\n### Automated Monitoring\n\n```bash\n# Service status monitoring\nssh -i keys/id_rsa.pem ubuntu@server 'systemctl status celestia-light'\n\n# Real-time log monitoring\nssh -i keys/id_rsa.pem ubuntu@server 'journalctl -u celestia-light -f'\n\n# Resource usage monitoring\nssh -i keys/id_rsa.pem ubuntu@server 'top -b -n1 | head -20'\n```\n\n### Logging\n\nApplication logs are available in multiple locations:\n\n- **Node Logs**: `/home/celestia/celestia-light.log`\n- **System Logs**: `journalctl -u celestia-light`\n- **Framework Logs**: Console output and test reports\n- **Pipeline Logs**: Azure DevOps pipeline interface\n\n### Alerting and Notifications\n\nFor production deployments, consider implementing:\n- Health check monitoring (Prometheus/Grafana)\n- Log aggregation (ELK stack)\n- Alert notification systems (PagerDuty, Slack)\n- Automated recovery procedures\n\n## 🚨 Troubleshooting\n\nFor detailed troubleshooting information, see the [Troubleshooting Guide](docs/troubleshooting.md).\n\n### Quick Diagnostic Tools\n\n```powershell\n# Windows health check script\n.\\scripts\\health-check.ps1\n\n# Manual connection test\nssh -i keys\\id_rsa.pem ubuntu@server-ip 'echo \"Connection successful\"'\n\n# Check framework dependencies\nnpm run check\nnode --version\n```\n\n### Common Issues \u0026 Quick Fixes\n\n| Issue | Quick Fix |\n|-------|-----------|\n| SSH connection refused | `Test-NetConnection server-ip -Port 22` |\n| Permission denied | `icacls keys\\id_rsa.pem /inheritance:r` |\n| Node won't start | `ssh server 'sudo systemctl restart celestia-light'` |\n| Tests timeout | Increase `DEPLOYMENT_TIMEOUT` in `.env` |\n| Port conflicts | `ssh server 'sudo pkill -f celestia'` |\n\n### Emergency Recovery\n\n```bash\n# Complete service restart\nssh -i keys/id_rsa.pem ubuntu@server 'sudo systemctl restart celestia-light'\n\n# Full framework reset (if needed)\nnpm ci \u0026\u0026 npm run test:pipeline\n\n# Check service logs for errors\nssh server 'sudo journalctl -u celestia-light -n 50'\n```\n\n**For comprehensive troubleshooting:** See [Troubleshooting Guide](docs/troubleshooting.md)\n\n## 🤝 Contributing\n\nWe welcome contributions to the Celestia Deployment Framework! Here's how to get started:\n\n### Development Setup\n\n1. **Fork and Clone**\n   ```bash\n   git clone https://github.com/your-username/celestia-node.git\n   cd celestia-node\n   ```\n\n2. **Install Dependencies**\n   ```bash\n   npm ci\n   ```\n\n3. **Set Up Environment**\n   ```bash\n   cp .env.example .env\n   # Edit .env with your configuration\n   ```\n\n4. **Run Tests**\n   ```bash\n   npm run check\n   npm run test:pipeline\n   ```\n\n### Development Workflow\n\n1. **Create a Feature Branch**\n   ```bash\n   git checkout -b feature/your-feature-name\n   ```\n\n2. **Make Your Changes**\n   - Follow TypeScript best practices\n   - Write or update tests for new functionality\n   - Update documentation as needed\n   - Ensure code passes linting and formatting\n\n3. **Validate Changes**\n   ```bash\n   npm run lint        # Check code style\n   npm run format      # Format code\n   npm run check       # Run all quality checks\n   npm run test:pipeline  # Run integration tests\n   ```\n\n4. **Commit and Push**\n   ```bash\n   git add .\n   git commit -m \"feat: add your feature description\"\n   git push origin feature/your-feature-name\n   ```\n\n5. **Create Pull Request**\n   - Provide clear description of changes\n   - Include test results and documentation updates\n   - Reference any related issues\n\n### Code Standards\n\n- **TypeScript**: Use strict TypeScript settings\n- **ESLint**: Follow configured linting rules\n- **Prettier**: Use for consistent code formatting\n- **Testing**: Write comprehensive tests for new features\n- **Documentation**: Update all relevant documentation\n- **Commit Messages**: Use conventional commit format\n\n### Testing Requirements\n\nAll contributions must include:\n- Unit tests for new functions/classes\n- Integration tests for service interactions\n- End-to-end tests for complete workflows\n- Documentation updates for configuration changes\n\n### Review Process\n\n1. Automated checks must pass (linting, formatting, tests)\n2. Code review by maintainers\n3. Documentation review for completeness\n4. Manual testing of deployment scenarios\n5. Security review for production changes\n\n## 📄 License\n\nThis project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.\n\n## 🙋‍♂️ Support\n\n### Getting Help\n\nFor support and questions:\n\n- **📖 Documentation**: Check the comprehensive [docs](docs/) directory\n- **🐛 Issues**: [Create a GitHub issue](https://github.com/your-repo/celestia-node/issues) for bugs or feature requests\n- **💬 Community**: Join the [Celestia Discord community](https://discord.gg/celestiacommunity)\n- **📧 Contact**: Reach out to the development team\n\n### Documentation Resources\n\n- **[API Documentation](docs/api-documentation.md)** - Complete service and method reference\n- **[Deployment Guide](docs/deployment-guide.md)** - Step-by-step setup instructions\n- **[Configuration Guide](docs/configuration-options.md)** - All configuration options\n- **[Troubleshooting Guide](docs/troubleshooting.md)** - Common issues and solutions\n- **[Architecture Diagrams](docs/architecture-diagrams.md)** - System architecture overview\n\n### Reporting Issues\n\nWhen reporting issues, please include:\n\n1. **Environment Information**\n   - Operating system and version\n   - Node.js and npm versions\n   - Framework version\n\n2. **Configuration Details** (sanitized)\n   - Environment variables (remove sensitive data)\n   - Network settings\n   - Server specifications\n\n3. **Error Details**\n   - Complete error messages\n   - Log files (sanitized)\n   - Steps to reproduce\n   - Expected vs actual behavior\n\n4. **Testing Results**\n   ```bash\n   # Run health check and include results\n   npm run check\n\n   # Include test output\n   npm run test:pipeline 2\u003e\u00261 | tee test-output.log\n   ```\n\n### Feature Requests\n\nFor new features or enhancements:\n\n1. Check existing issues to avoid duplicates\n2. Describe the use case and benefits\n3. Provide implementation suggestions if possible\n4. Consider contributing the feature yourself\n\n### Security Issues\n\nFor security-related issues:\n- **DO NOT** create public issues\n- Email security concerns privately\n- Follow responsible disclosure practices\n\n## 🔗 Related Projects\n\n- **Celestia Node**: https://github.com/celestiaorg/celestia-node\n- **Celestia App**: https://github.com/celestiaorg/celestia-app\n- **Celestia Docs**: https://docs.celestia.org/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcmccarthyirl%2Fcelestia-node","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcmccarthyirl%2Fcelestia-node","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcmccarthyirl%2Fcelestia-node/lists"}