{"id":30581528,"url":"https://github.com/randomtask2000/usbcopy","last_synced_at":"2025-08-29T06:37:38.088Z","repository":{"id":310004813,"uuid":"1038339387","full_name":"randomtask2000/usbcopy","owner":"randomtask2000","description":null,"archived":false,"fork":false,"pushed_at":"2025-08-15T03:47:41.000Z","size":12,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-08-15T05:29:10.556Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/randomtask2000.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-08-15T02:46:02.000Z","updated_at":"2025-08-15T03:47:44.000Z","dependencies_parsed_at":"2025-08-15T05:29:11.514Z","dependency_job_id":"e75e86d9-0d1b-49a5-b343-6a08c4cbbd3c","html_url":"https://github.com/randomtask2000/usbcopy","commit_stats":null,"previous_names":["randomtask2000/usbcopy"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/randomtask2000/usbcopy","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/randomtask2000%2Fusbcopy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/randomtask2000%2Fusbcopy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/randomtask2000%2Fusbcopy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/randomtask2000%2Fusbcopy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/randomtask2000","download_url":"https://codeload.github.com/randomtask2000/usbcopy/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/randomtask2000%2Fusbcopy/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":272642142,"owners_count":24968811,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-29T02:00:10.610Z","response_time":87,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-08-29T06:37:37.403Z","updated_at":"2025-08-29T06:37:38.080Z","avatar_url":"https://github.com/randomtask2000.png","language":"Shell","readme":"# Disk Imaging Scripts\n\nThis directory contains scripts for creating disk images from storage devices, particularly for backing up and cloning USB drives and SD cards.\n\n## Quick Start\n\n### Step 1: Identify Your Disk\n```bash\n# List all disks to find your USB/SD card\ndiskutil list\n\n# Look for your device (e.g., /dev/disk4 for SD card, /dev/disk5 for USB)\n# Note the disk identifier for the next steps\n```\n\n### Step 2: Choose and Run a Script\n\n#### Option A: Maximum Compression (Slowest, Smallest File)\n```bash\n# Clone the repository\ngit clone https://github.com/randomtask2000/usbcopy.git\ncd usbcopy\n\n# Run the default compression script\n./create_disk_image.sh\n```\n\n#### Option B: Fast Compression (Balanced Speed/Size)\n```bash\n# Navigate to the repository\ncd usbcopy\n\n# Run the fast compression script\n./create_disk_image_fast.sh\n```\n\n#### Option C: Raw Image (Fastest, Largest File)\n```bash\n# Navigate to the repository\ncd usbcopy\n\n# Run the raw image script (requires full disk size in free space)\n./create_disk_image_raw.sh\n```\n\n### Step 3: Restore to New Disk\n```bash\n# First, identify the target disk\ndiskutil list\n\n# For compressed images:\ngunzip -c disk4_backup.img.gz | sudo dd of=/dev/rdisk5 bs=1m status=progress\n\n# For raw images:\nsudo dd if=disk4_backup.img of=/dev/rdisk5 bs=1m status=progress\n```\n\n## Available Scripts\n\n### Image Creation Scripts\n\n#### 1. `create_disk_image.sh` - Compressed Image (Default Compression)\nCreates a compressed disk image using gzip's default compression level (6).\n- **Pros**: Best compression ratio, saves maximum space\n- **Cons**: Slowest option (2-6 hours for 1TB)\n- **Use when**: You have limited storage space and time is not critical\n\n**How to run:**\n```bash\n./create_disk_image.sh\n```\n\n**What it asks:**\n```\nContinue? (y/n): y\nPassword: [your sudo password]\n```\n\n#### 2. `create_disk_image_fast.sh` - Compressed Image (Fast Compression)\nCreates a compressed disk image using gzip's fastest compression level (1).\n- **Pros**: Faster than default compression, still saves significant space\n- **Cons**: Larger file than maximum compression (1-3 hours for 1TB)\n- **Use when**: You want a balance between speed and space savings\n\n**How to run:**\n```bash\n./create_disk_image_fast.sh\n```\n\n**What it asks:**\n```\nContinue? (y/n): y\nPassword: [your sudo password]\n```\n\n#### 3. `create_disk_image_raw.sh` - Raw Image (No Compression)\nCreates an uncompressed, bit-for-bit copy of the disk.\n- **Pros**: Fastest option, exact sector-by-sector copy\n- **Cons**: Requires full disk size in free space (1TB for 1TB disk)\n- **Use when**: You have sufficient storage space and want maximum speed\n\n**How to run:**\n```bash\n./create_disk_image_raw.sh\n```\n\n**What it asks:**\n```\nContinue anyway? (y/n): y\nPassword: [your sudo password]\n```\n\n### Restore and Clone Scripts\n\n#### 4. `direct_clone_to_2tb.sh` - Direct Clone to 2TB USB\nDirectly clones from 1TB SD card to 2TB USB without creating an intermediate image file.\n- **Pros**: Fastest method (2-3 hours), no storage space needed\n- **Cons**: Both disks must be connected simultaneously\n- **Use when**: You have both source and target disks connected\n\n**How to run:**\n```bash\n./direct_clone_to_2tb.sh\n```\n\n**What it asks:**\n```\nContinue? (y/n): y\nPassword: [your sudo password]\n```\n\n**Sample output:**\n```\n=== Direct Disk Clone: 1TB to 2TB ===\n\nSource: /dev/disk4 (1TB SD Card with Linux filesystem)\nTarget: /dev/disk5 (2TB USB - currently FAT32)\n\n⚠️  WARNING: This operation will:\n   • COMPLETELY ERASE the 2TB USB disk (/dev/disk5)\n   • Copy all 1TB from the SD card to the USB\n   • Take approximately 2-3 hours to complete\n\nContinue? (y/n): y\n\nStep 1: Unmounting target disk...\nStep 2: Starting direct clone...\n953869+0 records in\n953869+0 records out\n1000204886016 bytes transferred in 7234.123456 secs (138234567 bytes/sec)\n\n✅ Clone completed successfully!\nDuration: 120 minutes\n```\n\n#### 5. `restore_or_clone.sh` - Interactive Restore/Clone Tool\nProvides an interactive menu to either restore from an existing image or clone directly between disks.\n- **Pros**: Flexible, detects existing image files, provides multiple options\n- **Cons**: Requires user interaction to choose options\n- **Use when**: You want to choose between different restore methods\n\n**How to run:**\n```bash\n./restore_or_clone.sh\n```\n\n**What it asks:**\n```\nChoose an option:\n  1) Direct clone from /dev/disk4 to /dev/disk5 (fastest, no intermediate storage)\n  2) Restore from existing image file [only shown if image exists]\n  q) Quit\n\nEnter your choice: 1\n\nAre you absolutely sure? Type 'yes' to continue: yes\nPassword: [your sudo password]\n```\n\n**Sample interaction for direct clone:**\n```bash\n$ ./restore_or_clone.sh\n\n=== USB Disk Restore/Clone Tool ===\n\nCurrent disk configuration:\n  Source: /dev/disk4 (1TB SD Card)\n  Target: /dev/disk5 (2TB USB)\n\nFound compressed image: disk4_backup.img.gz (423GB)\n\nChoose an option:\n  1) Direct clone from /dev/disk4 to /dev/disk5 (fastest, no intermediate storage)\n  2) Restore from existing image file\n  q) Quit\n\nEnter your choice: 1\n\nDIRECT DISK CLONE\n=================\nWARNING: This will COMPLETELY ERASE /dev/disk5!\n\nAre you absolutely sure? Type 'yes' to continue: yes\n\nUnmounting target disk...\nStarting direct disk clone...\n[Progress bar shows here]\n\nDisk clone completed successfully!\n```\n\n**Sample interaction for image restore:**\n```bash\n$ ./restore_or_clone.sh\n\nChoose an option:\n  2) Restore from existing image file\n\nEnter your choice: 2\n\nRESTORE FROM IMAGE\n==================\nUsing compressed image: disk4_backup.img.gz\n\nWARNING: This will COMPLETELY ERASE /dev/disk5!\n\nAre you absolutely sure? Type 'yes' to continue: yes\n\nStarting image restoration...\n[Progress bar shows here]\n\nImage restoration completed successfully!\n```\n\n## Command Line Examples\n\n### Complete Workflow Example\n```bash\n# 1. Clone the repository\ngit clone https://github.com/randomtask2000/usbcopy.git\ncd usbcopy\n\n# 2. Check your current disk setup\ndiskutil list\n\n# 3. Verify available space\ndf -h\n\n# 4. Unmount the source disk (if mounted)\ndiskutil unmountDisk /dev/disk4\n\n# 5. Create the image (choose one):\n./create_disk_image_fast.sh    # Recommended for most users\n\n# 6. After swapping the SSD, identify the new disk\ndiskutil list\n\n# 7. Unmount the target disk\ndiskutil unmountDisk /dev/disk5\n\n# 8. Restore the image\ngunzip -c disk4_backup.img.gz | sudo dd of=/dev/rdisk5 bs=1m status=progress\n\n# 9. Verify the restoration\ndiskutil list\n```\n\n### Direct Terminal Commands (Without Scripts)\n\n#### Create Images\n```bash\n# Compressed image (saves space)\nsudo dd if=/dev/rdisk4 bs=1m status=progress | gzip -1 \u003e backup.img.gz\n\n# Maximum compression (smallest file)\nsudo dd if=/dev/rdisk4 bs=1m status=progress | gzip -9 \u003e backup.img.gz\n\n# Raw image (fastest, needs full disk space)\nsudo dd if=/dev/rdisk4 of=backup.img bs=1m status=progress\n```\n\n#### Restore Images\n```bash\n# From compressed image\ngunzip -c backup.img.gz | sudo dd of=/dev/rdisk5 bs=1m status=progress\n\n# From raw image\nsudo dd if=backup.img of=/dev/rdisk5 bs=1m status=progress\n```\n\n#### Monitor Progress (in another terminal)\n```bash\n# Check how much has been written\nsudo killall -INFO dd\n\n# Watch file size grow\nwatch -n 5 'ls -lh *.img*'\n\n# Check disk activity\niostat -w 5\n```\n\n### Advanced Examples\n\n#### Clone Directly Between Two Disks (No Intermediate File)\n```bash\n# When you have both disks connected\nsudo dd if=/dev/rdisk4 of=/dev/rdisk5 bs=1m status=progress\n```\n\n#### Create Image to External Drive\n```bash\n# To save space on main drive\nsudo dd if=/dev/rdisk4 bs=1m status=progress | gzip \u003e /Volumes/ExternalDrive/backup.img.gz\n```\n\n#### Verify Image Integrity\n```bash\n# Create checksums\nsudo dd if=/dev/rdisk4 bs=1m | tee backup.img | md5 \u003e backup.md5\n# Later verify\nmd5 backup.img\ncat backup.md5\n```\n\n#### Resume Interrupted Transfer\n```bash\n# Check how much was copied\nls -la backup.img\n\n# Resume from that point (example: resume at 10GB)\nsudo dd if=/dev/rdisk4 of=backup.img bs=1m seek=10240 skip=10240 status=progress\n```\n\n## How the Code Works\n\n### Core Components\n\n#### 1. The `dd` Command\n```bash\ndd if=/dev/rdisk4 of=output.img bs=1m status=progress\n```\n- `dd`: \"Data duplicator\" - copies data block by block\n- `if=/dev/rdisk4`: Input file (source disk)\n  - `/dev/rdisk4` uses raw disk access (faster than `/dev/disk4` on macOS)\n- `of=output.img`: Output file (destination image)\n- `bs=1m`: Block size of 1 megabyte (optimal for macOS disk operations)\n- `status=progress`: Shows real-time progress updates\n\n#### 2. Compression Pipeline (Compressed Versions)\n```bash\ndd if=/dev/rdisk4 bs=1m status=progress | gzip \u003e output.img.gz\n```\n- The pipe `|` sends dd's output to gzip instead of a file\n- `gzip`: Compresses the data stream\n  - Default level (6): Best compression\n  - `-1` flag: Fastest compression, larger file\n  - `-9` flag: Maximum compression, slowest\n- `\u003e`: Redirects compressed output to a file\n\n#### 3. Why `/dev/rdisk` vs `/dev/disk`\n- `/dev/disk4`: Buffered device (goes through macOS's buffer cache)\n- `/dev/rdisk4`: Raw device (bypasses buffer cache)\n- Raw devices are typically 2-20x faster for sequential operations like imaging\n\n### Script Flow\n\n1. **Display Information**: Shows source disk and output file details\n2. **Check Available Space**: Warns about space requirements\n3. **User Confirmation**: Requires explicit consent before proceeding\n4. **Execute dd Command**: Runs with sudo for disk access permissions\n5. **Progress Monitoring**: Shows bytes copied in real-time\n6. **Error Handling**: Checks exit code and cleans up on failure\n7. **Success Report**: Shows final file size when complete\n\n### Restoring Images\n\nTo restore an image back to a disk:\n\n#### Compressed Image:\n```bash\ngunzip -c disk4_backup.img.gz | sudo dd of=/dev/rdisk5 bs=1m status=progress\n```\n\n#### Raw Image:\n```bash\nsudo dd if=disk4_backup.img of=/dev/rdisk5 bs=1m status=progress\n```\n\n**WARNING**: Be absolutely certain of the destination disk number. This will completely overwrite the target disk!\n\n## Important Notes\n\n1. **Disk Identification**: Always verify disk numbers with `diskutil list` before operations\n2. **Data Loss Risk**: These operations can destroy data if the wrong disk is specified\n3. **Time Requirements**: \n   - 1TB over USB 3.0: ~2-3 hours raw, 3-6 hours compressed\n   - 1TB over USB 2.0: ~8-10 hours raw, 10-15 hours compressed\n4. **Space Requirements**:\n   - Raw: Exactly the size of the source disk\n   - Compressed: Varies based on data content (20-90% of original)\n5. **Sudo Access**: Required for raw disk access on macOS\n\n## Safety Tips\n\n1. Always double-check disk numbers before running\n2. Keep your Mac plugged in and prevent sleep during operations\n3. Don't disconnect devices during operations\n4. Verify the image after creation (optional):\n   ```bash\n   # For raw images\n   sudo dd if=/dev/rdisk4 bs=1m count=1000 | md5\n   dd if=disk4_backup.img bs=1m count=1000 | md5\n   # Compare the checksums\n   ```\n\n## Troubleshooting\n\n- **\"Resource busy\"**: Unmount the disk first: `diskutil unmountDisk /dev/disk4`\n- **\"Operation not permitted\"**: Ensure Terminal has Full Disk Access in System Settings\n- **\"No space left\"**: Check available space with `df -h`\n- **Slow speeds**: Ensure using `/dev/rdisk` not `/dev/disk`, check USB connection type","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frandomtask2000%2Fusbcopy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frandomtask2000%2Fusbcopy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frandomtask2000%2Fusbcopy/lists"}