{"id":30260143,"url":"https://github.com/cyphrriot/migrate","last_synced_at":"2025-08-15T19:03:56.518Z","repository":{"id":302536175,"uuid":"1012764643","full_name":"CyphrRiot/Migrate","owner":"CyphrRiot","description":"A stunningly beautiful Terminal backup and restore tool written in Pure Go","archived":false,"fork":false,"pushed_at":"2025-08-10T03:36:47.000Z","size":216557,"stargazers_count":22,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-08-10T05:26:16.643Z","etag":null,"topics":["arch","backup","go","golang","linux","omarchy","restore","tui"],"latest_commit_sha":null,"homepage":"https://x.com/CyphrRiot","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/CyphrRiot.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-07-02T21:05:06.000Z","updated_at":"2025-08-10T03:36:51.000Z","dependencies_parsed_at":"2025-07-27T00:24:35.061Z","dependency_job_id":"4427f03f-dd8d-447d-bc73-2b8b6e4329d2","html_url":"https://github.com/CyphrRiot/Migrate","commit_stats":null,"previous_names":["cyphrriot/migrate"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/CyphrRiot/Migrate","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CyphrRiot%2FMigrate","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CyphrRiot%2FMigrate/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CyphrRiot%2FMigrate/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CyphrRiot%2FMigrate/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/CyphrRiot","download_url":"https://codeload.github.com/CyphrRiot/Migrate/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CyphrRiot%2FMigrate/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":270617818,"owners_count":24617121,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-15T02:00:12.559Z","response_time":110,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["arch","backup","go","golang","linux","omarchy","restore","tui"],"created_at":"2025-08-15T19:01:04.253Z","updated_at":"2025-08-15T19:03:56.503Z","avatar_url":"https://github.com/CyphrRiot.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\n# Migrate\n\n![Version](https://img.shields.io/badge/version-1.0.62-4c1d95)\n![License](https://img.shields.io/badge/license-MIT-1e293b)\n![Language](https://img.shields.io/badge/language-Go-00ADD8?logo=go\u0026logoColor=white)\n![Platform](https://img.shields.io/badge/platform-Linux-FCC624?logo=linux\u0026logoColor=black)\n\n[![CyphrRiot on X](https://img.shields.io/badge/Follow-@CyphrRiot-1DA1F2?style=for-the-badge\u0026logo=x\u0026logoColor=white)](https://x.com/CyphrRiot)\n[![GitHub Profile](https://img.shields.io/badge/GitHub-CyphrRiot-181717?style=for-the-badge\u0026logo=github\u0026logoColor=white)](https://github.com/CyphrRiot)\n\n\u003c/div\u003e\n\nA stunningly beautiful **Terminal** backup and restore tool built with [Bubble Tea](https://github.com/charmbracelet/bubbletea) and [Lipgloss](https://github.com/charmbracelet/lipgloss). Features Tokyo Night theming, pure Go implementation with zero external dependencies, and production-ready performance with selective backup capabilities.\n\n![Migrate Beautiful Interface](images/interface.gif)\n\n## 🔥 **Live System Backup \u0026 Restore**\n\n**The game changer:** Migrate backs up and restores **running live systems** - no downtime required.\n\nUnlike traditional tools that require booting from external media (looking at you, CloneZilla), Migrate works seamlessly on your **active, running OS**. When you're ready to restore, simply:\n\n1. **Fresh install** your OS\n2. **Run the restore**\n3. **💥 You're back up and running** - exactly as you left it\n\n**No external boot disks. No system downtime. Just pure magic.** ✨\n\n\u003e ✅ **Production Ready**: Recently underwent extensive optimization and bug fixes. All major performance and reliability issues have been resolved as of July 2025.\n\n## ✨ Features\n\n- 🎨 **Beautiful TUI** - Tokyo Night color scheme with smooth animations\n- 🚀 **Pure Go** - Zero external dependencies, single static binary\n- 💾 **System Backup** - Complete 1:1 backup or selective home directory backup\n- 🔍 **Verification** - Comprehensive backup integrity checking with progress tracking\n- 🔄 **Smart Sync** - SHA256-based deduplication, automatic cleanup of deleted files\n- 📊 **Real-time Progress** - File-based progress tracking with accurate estimates\n- 🔒 **LUKS Support** - Works with encrypted external drives\n- ⚡ **Blazingly Fast** - Optimized exclusion patterns for maximum performance\n- 🎯 **Selective Backup** - Granular folder control with intelligent defaults\n\n## 🚀 Installation\n\n### Download Binary (Recommended)\n\n```bash\ncurl -L -o migrate https://github.com/CyphrRiot/Migrate/raw/main/bin/migrate\nchmod +x migrate\nmv migrate ~/.local/bin/  # optional\n```\n\n### Build From Source\n\n```bash\ngit clone https://github.com/CyphrRiot/Migrate.git\ncd Migrate\nmake build     # or: go build -o bin/migrate .\nmake install   # optional\n```\n\n## 🖥️ Usage\n\n```bash\nmigrate\n```\n\nThat's it! The tool launches a beautiful TUI interface:\n\n- 🎨 Navigate with arrow keys and Enter\n- 🔍 Automatically detects external drives\n- 📊 Watch real-time progress with smooth animations\n- ✅ Access backup, restore, and verification options from the main menu\n\n## ⚙️ How It Works\n\n### rsync --delete Equivalent\n\nPerforms the equivalent of `rsync -aAx --delete / /backup/destination/` but with:\n\n- **rsync-style file comparison** - Intelligent size + timestamp checking (50-80% faster than SHA256)\n- **Modern buffer optimization** - 256KB-4MB adaptive buffers optimized for SSD/NVMe storage\n- **SHA256 verification** - Available for cryptographic verification when needed\n- **Better logging** - Detailed statistics on copied vs. skipped files\n- **Zero dependencies** - No rsync binary required\n- **Beautiful interface** - Progress tracking and status updates\n- **Smart exclusions** - Optimized pattern matching for maximum performance\n\n### Smart Features\n\n- **rsync-style comparison** - Skip identical files using size + timestamp (50-80% faster than SHA256)\n- **Incremental backups** - Only copy changed/new files with intelligent comparison\n- **Modern I/O optimization** - 256KB-4MB adaptive buffers for maximum SSD/NVMe throughput\n- **Smart exclusions** - Optimized pattern matching without affecting deletion phase\n- **Automatic exclusions** - `/dev`, `/proc`, `/sys`, `/tmp`, backup destination\n- **Progress accuracy** - File-based tracking: `(processed / total) * 85% + cleanup * 15%`\n- **Complete `--delete` behavior** - Properly removes stale files from backup regardless of exclusions\n- **Performance optimization** - Combined optimizations deliver 70-120% speed improvement\n\n## ⚡ Performance Comparison\n\n**Migrate now delivers rsync-level performance with significant speed improvements:**\n\n### Before vs After Optimization\n\n| Operation                | Before               | After                          | Improvement              |\n| ------------------------ | -------------------- | ------------------------------ | ------------------------ |\n| **Incremental Backup**   | SHA256 comparison    | rsync-style (size + timestamp) | **50-80% faster**        |\n| **File Copying**         | 64KB-1MB buffers     | 256KB-4MB adaptive buffers     | **20-40% faster**        |\n| **Overall Backup Speed** | Baseline             | Combined optimizations         | **70-120% faster**       |\n| **Large File Handling**  | Conservative buffers | NVMe-optimized buffers         | **Significantly faster** |\n\n### Real-World Performance\n\n**Test System:** Home directory with 600,916 files\n\n- **Files Copied:** 27,442 (only changed files)\n- **Files Skipped:** 573,474 (intelligent deduplication)\n- **Speed:** Approaching native rsync performance\n- **Memory Usage:** RAM-conscious with modern hardware optimization\n\n### Key Performance Features\n\n- **🚀 rsync-style comparison**: Only copy when source is newer than destination\n- **💾 Modern storage optimization**: Buffers sized for SSD/NVMe performance\n- **🔄 True incremental**: Skip unchanged files instantly with timestamp comparison\n- **⚡ Zero redundancy**: No unnecessary file reads or hash calculations\n- **🎯 Smart exclusions**: Patterns don't interfere with deletion performance\n\n---\n\n## 🔄 Enhanced Restore\n\n**Intelligent restore with folder selection** for home directory backups - restore exactly what you need with the same beautiful interface used for backups.\n\n### ✨ Key Features\n\n- **🏠 Automatic Backup Detection** - Detects HOME vs SYSTEM backups automatically\n- **📁 Folder Selection UI** - Choose which folders to restore from your backup\n- **🎯 Selective Restoration** - Restore only the folders you need\n- **⚙️ Restore Options** - Control configuration and window manager restoration\n- **📊 Size Calculation** - See exactly how much data will be restored\n- **🔒 Safe Operations** - Multiple confirmation steps prevent accidental overwrites\n\n### 🎛️ How It Works\n\nWhen restoring a **HOME backup**:\n\n1. **Backup Detection** - Automatically identifies backup type from BACKUP-INFO.txt\n2. **Folder Discovery** - Scans backup for available folders with size information\n3. **Selection Interface** - Same beautiful UI as backup folder selection\n4. **Restore Options** - Choose to restore configuration files and window managers\n5. **Selective Sync** - Only restores selected folders with full rsync --delete behavior\n\n### 📊 Restore Options\n\n- **☑️ Restore Configuration** - Restores ~/.config directory (enabled by default)\n- **☑️ Restore Window Managers** - Restores Hyprland, GNOME, i3, etc. (enabled by default)\n\n### 🔒 Safety Features\n\n- **Automatic Type Detection** - System backups restore to /, HOME backups to ~/\n- **Folder-by-Folder Restoration** - Each folder restored independently for reliability\n- **Comprehensive Logging** - All operations logged for debugging\n- **Multiple Confirmations** - Prevents accidental data overwrites\n\n## 🔍 Backup Verification\n\n**Comprehensive integrity checking** ensures your backups are complete and trustworthy with beautiful real-time progress tracking.\n\n### ✨ Key Features\n\n- **🔍 Complete System Verification** - Full backup integrity check against source system\n- **🏠 Home Directory Verification** - Granular verification of selective backups\n- **🔄 Bidirectional Verification** - Detects both missing files and extra files in backup\n- **📊 Real-Time Progress** - Smooth progress tracking with Tokyo Night styling\n- **🎯 Smart Sampling** - Efficient random sampling for large backup verification\n- **⚡ Three-Phase Verification** - Source→backup, sampling, and backup→source checks\n- **📋 Detailed Reporting** - Comprehensive integrity reports with option to save logs\n\n### 🎛️ How Verification Works\n\nWhen you select **\"🔍 Verify Backup\"**:\n\n1. **Drive Detection** - Automatic discovery of backup drives\n2. **Read-Only Mounting** - Safe mounting of backup source for verification\n3. **Phase 1: Directory Structure** - Validates source directories exist in backup\n4. **Phase 2: File Sampling** - Random verification of representative file samples\n5. **Phase 3: Reverse Check** - Detects extra files in backup not present in source\n6. **Comprehensive Report** - Complete integrity analysis with missing/extra file detection\n\n### 🔒 Verification Types\n\n- **Complete System** - Verifies full system backups against root filesystem\n- **Home Directory** - Validates selective home directory backups\n- **Background Processing** - Verification runs asynchronously with progress tracking\n- **Log Generation** - Optional saving of verification results to `~/migrate-verified.log`\n\n### 📊 Progress Display\n\nVerification features the same beautiful Tokyo Night progress bars with:\n\n- **Time-Based Progress** (70%) - Realistic timing spread over 8-12 seconds\n- **File Progress** (30%) - Actual file verification completion\n- **Progressive Phases** - Initializing → Critical Files → Sampling → Completing\n- **🔍 Icon Integration** - Verification-specific styling and branding\n\n## 💾 Drive Support\n\nWorks with any external drive:\n\n- **USB, SSD, HDD** - Automatic detection of removable drives\n- **Multiple filesystems** - ext4, btrfs, exfat, NTFS\n- **LUKS encryption** - Full encrypted drive support with helpful unlock instructions\n\n## 🏠 Selective Home Directory Backup\n\n**Revolutionary folder-level control** for home directory backups with intelligent defaults and real-time size calculation.\n\n### ✨ Key Features\n\n- **🎯 Granular Selection** - Choose exactly which folders to backup\n- **📊 Real-Time Sizing** - Live calculation of selected folders + hidden files\n- **⚡ Smart Defaults** - All folders pre-selected for instant backup workflow\n- **🎨 Beautiful Interface** - Professional two-column layout with btop-inspired styling\n- **🚀 Performance** - Zero redundant calculations, instant drive feedback\n\n### 🎛️ How It Works\n\nWhen you select **\"🏠 Home Directory\"** backup:\n\n1. **Automatic Discovery** - Scans your home directory for all visible folders\n2. **Size Calculation** - Real-time calculation using optimized `du -sb` equivalent\n3. **Intelligent Selection** - Two-column layout for easy navigation\n4. **Always Included** - Hidden configuration files (`.config`, `.local`, etc.) automatically preserved\n5. **Smart Continue** - Hit \"Continue\" immediately for full backup, or customize selection\n\n### 🎯 User Interface\n\n![Home Directory Selection Interface](images/home.png)\n\n### 🎪 Advanced Features\n\n- **Hidden Folders** - Always includes `.config`, `.ssh`, `.gnupg`, dotfiles\n- **Empty Folder Filtering** - Hides 0 B directories for cleaner interface\n- **Bulk Operations** - `A` to select all, `X` to deselect all\n- **Individual Control** - `SPACE` to toggle specific folders\n- **Accurate Space** - Only calculates space for selected folders during drive mounting\n\n### 📊 Benefits Over Traditional Backup\n\n- **Faster Backups** - Skip unnecessary large folders (like game caches)\n- **Storage Efficiency** - Backup only what matters to you\n- **Granular Control** - Perfect for selective migrations or space-limited drives\n- **Always Safe** - Critical config files always preserved automatically\n\n### 🔒 Safety Guarantees\n\n- **Smart Restore** - Automatically detects backup types and targets correct destinations\n- **No Data Loss** - Multiple validation layers prevent dangerous operations\n- **User Detection** - Handles `sudo` execution properly with `SUDO_USER` detection\n- **Comprehensive Logging** - All operations logged for debugging and verification\n- **Production Tested** - Extensively tested on live systems with 600,000+ file backups\n\n## 🚀 Recent Performance Improvements (July 2025)\n\n### Critical Performance Optimizations (100-180% Faster)\n\n- **🔥 rsync-style File Comparison**: Replaced size-only comparison with intelligent size + timestamp checking\n    - **Impact**: 50-80% faster incremental backups by eliminating expensive file reads\n    - **Behavior**: Only copies files when source is newer than destination (true rsync behavior)\n\n- **⚡ Modern Buffer Optimization**: Upgraded buffer sizes for SSD/NVMe performance\n    - **Impact**: 20-40% faster copying with 256KB default, 2-4MB for large files\n    - **Hardware**: Optimized for modern storage (was optimized for 2019 hardware)\n\n- **🚀 Combined Performance**: Total improvement of 70-120% faster backups approaching rsync speeds\n\n### Critical Bug Fixes\n\n- **🚨 FIXED: `--delete` Behavior Restored**: Removed exclusion pattern interference with deletion phase\n    - **Problem**: Files matching exclusions were never deleted from backup even when removed from source\n    - **Fix**: Exclusion patterns now only apply to copying, not deletion (proper rsync behavior)\n\n- **🔍 FIXED: Verification Coverage Gap**: Added reverse verification to detect extra files in backup\n    - **Problem**: Verification only checked source→backup, never backup→source\n    - **Fix**: Added Phase 3 verification that detects files in backup not present in source\n\n- **✅ SUDO_USER Detection**: Fixed critical bug where selective backups targeted `/root` instead of user home\n- **✅ Smart Exclusion Patterns**: Implemented optimized cache pattern matching for dramatic speed improvements\n- **✅ Permission Handling**: Resolved ownership issues with backup directory creation under sudo\n\n### Performance Benchmarks\n\nRecent testing on production systems shows excellent performance:\n\n- **📊 600,916 files processed** - Large home directory backup completed successfully\n- **⚡ 27,442 files copied** - Smart deduplication with rsync-style comparison\n- **🎯 Selective exclusions** - Videos/cache folders properly excluded for faster backups\n- **🔄 Smart cleanup** - Proper `--delete` behavior now removes stale files from backup\n- **🔍 Complete verification** - Now detects both missing files and extra files in backup\n\n### Reliability Improvements\n\n- **rsync-equivalent behavior** - True `--delete` functionality with proper exclusion handling\n- **Complete verification coverage** - Bidirectional verification ensures backup integrity\n- **Zero hangs** - Eliminated filesystem scanning performance issues\n- **Consistent timing** - Both full system and selective home backups perform optimally\n- **Better error handling** - Improved resilience during large file operations\n- **Production ready** - Successfully tested on live systems with proper cleanup behavior\n\n## 🏗️ Architecture\n\n```\n├── main.go           # Entry point and TUI initialization\n├── internal/         # Internal package\n│   ├── version.go    # Version management\n│   ├── utils.go      # Configuration and utilities\n│   ├── filesystem.go # File operations\n│   ├── drives.go     # Drive detection and mounting\n│   ├── operations.go # Backup/restore logic\n│   ├── verification.go # Backup verification logic\n│   ├── model.go      # Bubble Tea state management\n│   └── ui.go         # Interface rendering\n└── bin/migrate       # Static binary\n```\n\n### Technical Details\n\n- **Static binary** - Built with `CGO_ENABLED=0` for maximum portability\n- **Memory efficient** - Direct filesystem operations using `filepath.WalkDir`\n- **Error resilient** - Continues operation when individual files fail\n- **Fast updates** - Progress refreshes every 200ms\n\n## 🎨 Authentic Tokyo Night Theme\n\nCelebrate the neon lights of Downtown Tokyo with the authentic color palette:\n\n- **Primary Blue**: `#7aa2f7` - **Deep Blue**: `#3d59a1`\n- **Purple**: `#bb9af7` - **Green**: `#9ece6a`\n- **Cyan**: `#73daca` - **Orange**: `#e0af68`\n- **Red/Pink**: `#f7768e` - **Background**: `#1a1b26`\n- **Text**: `#c0caf5` - **Dim**: `#565f89`\n\nBeautiful gradient progress bars flow through the full Tokyo Night spectrum: Blue → Purple → Cyan → Green, creating the signature neon-soaked aesthetic of Japan's capital at night.\n\n## 🔧 Development\n\n```bash\n# Development build\ngo build -o bin/migrate .\n\n# Static binary (production)\nCGO_ENABLED=0 go build -a -ldflags '-extldflags \"-static\"' -o bin/migrate .\n\n# Dependencies (pure Go only)\n# - github.com/charmbracelet/bubbletea\n# - github.com/charmbracelet/lipgloss\n```\n\n## 🤝 Contributing\n\nClean architecture makes it easy to add new features. The tool is organized into focused modules for better maintainability.\n\n## 📄 License\n\nCreated by **Cypher Riot**\n\n🔗 **Links:**\n\n- **GitHub**: https://github.com/CyphrRiot/Migrate\n- **X**: https://x.com/CyphrRiot\n\n---\n\n_🎉 TUI-only pure Go backup tool with zero external dependencies!_\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcyphrriot%2Fmigrate","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcyphrriot%2Fmigrate","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcyphrriot%2Fmigrate/lists"}