{"id":22612259,"url":"https://github.com/ruser75/qnap-recovery-kernel","last_synced_at":"2025-04-15T18:42:55.879Z","repository":{"id":202775680,"uuid":"708112640","full_name":"rUser75/qnap-recovery-kernel","owner":"rUser75","description":"A minimal QTS kernel running under qemu with lvm support","archived":false,"fork":false,"pushed_at":"2025-01-29T10:48:02.000Z","size":34740,"stargazers_count":5,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-28T23:46:08.890Z","etag":null,"topics":["kernel","qnap","recovery-tools"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rUser75.png","metadata":{"files":{"readme":"README-en.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-10-21T15:06:32.000Z","updated_at":"2025-01-29T10:48:06.000Z","dependencies_parsed_at":"2025-03-28T23:45:03.485Z","dependency_job_id":"5f742c84-1ee1-4cc2-98eb-32abb8f00585","html_url":"https://github.com/rUser75/qnap-recovery-kernel","commit_stats":null,"previous_names":["ruser75/qnap-recovery-kernel"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rUser75%2Fqnap-recovery-kernel","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rUser75%2Fqnap-recovery-kernel/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rUser75%2Fqnap-recovery-kernel/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rUser75%2Fqnap-recovery-kernel/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rUser75","download_url":"https://codeload.github.com/rUser75/qnap-recovery-kernel/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249131329,"owners_count":21217724,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["kernel","qnap","recovery-tools"],"created_at":"2024-12-08T17:11:40.708Z","updated_at":"2025-04-15T18:42:55.853Z","avatar_url":"https://github.com/rUser75.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# qnap-recovery-kernel\na minimal QTS kernel running under qemu with lvm support\n\n## Indrodution \nsome time ago I've helped a friend of my to recover data from him  QNAP that it wouldn't turn on anymore.\n\nthe data volume was on a LVM volume build on top of a md raid, this is the standard for QNAP devices ( it is at least for the two-disc ones ). \n\nmy first attempt was to connect one of the hdd on a linux computer to mount the volume but I was surprised to find out that I'm not able to do this. Why?\nQNAP doesn't use a standard Linux LVM but a custom compile that it is a little bit different and that's why the data recovery software didn't work.\n\nafter a lot of time we have turned ON the old QNAP with another hdd after that we were able to recover the data on the old hdd.\nbut... if the QNAP is definely broken?\nfor this reason I decided to assemble this kernel starting at a qnap x86 device firmware\n\n## before start\nif the NAS have two disks before start it is important to mark the disks to remember the correct position.\nfor me it is also important don't work directly on the disks but on a copy. \nthe copy of the data partition can be do using some tool as dd or ddrescue.\n\u003e ** If you continue the read I suppose that do you know how to use a linux and that there ara no warranty that all works as expected...**\n\n\n## make copy of the data partition\nattach one of the disks into a linux system (is it possible to use sata to usb adapter, a usb box, directly attached ...)\nusing fdisk to determine to data partition, it is the big one for example (usually the partiotion 3)\n```\n# fdisk -l /dev/sda\n\nDisk /dev/sda: 2000.3 GB, 2000398934016 bytes\n255 heads, 63 sectors/track, 243201 cylinders\nUnits = cylinders of 16065 * 512 = 8225280 bytes\n\n   Device Boot      Start         End      Blocks   Id  System\n/dev/sda1               1          66      530125   83  Linux\n/dev/sda2              67         132      530142   83  Linux\n/dev/sda3             133      243138  1951945693   83  Linux\n/dev/sda4          243139      243200      498012   83  Linux\n```\nafter identified the right partion, using another hdd for save the image file you can use dd or ddrescue\n\n```\ndd if=/dev/sda3 of=/otherstoragepath/sda3.img bs=8k \n```\nor \n```\nddrescue -n /dev/sda3 /otherstoragepath/sda3.img /pathtologfile/logfile\n```\n\u003e **note** using ddrescue you can try to fix some I/O errors if the source hdd was damaged and specifing the logfile (now called domain-mapfile) you can stop end resume the copy\n\n\nBelow the basic commands for install ddrescue on centos\n```\ndnf -y install ddrescue (on centos)\n```\nor on ubuntu\n```\nadd-apt-repository universe\napt-get install gddrescue\n```\n\n## try to recover.\nyou need a linux box (centos, ubuntu live are tested) with sufficient free space for copy the recovered data (with an external usb disk for example)\n\nbefore start check if qemu packages are installed if not install it using \n```\ndnf install -y qemu-kvm (for centos)\n\napt-get  install qemu-kvm (for ubuntu)\n```\n\n\ndownload the kernel images attached here and put them in a directory. \n\nIn the release menu (https://github.com/rUser75/qnap-recovery-kernel/releases/tag/first) you can find a \n* bigger initrdC.boot\n* lvm test file\n  \n  \nrun the follow command to run the custom kernel in a virtual machine on redhat (on ubuntu replace /usr/libexec/qemu-kvm with qemu-system-x86_64)\n\n```\n/usr/libexec/qemu-kvm \\\n-kernel bzImage \\\n-initrd initrdC.boot \\\n-append \"root=/dev/ram0 rw init=/bin/busybox console=ttyS0\" \\\n-nographic -m 1024M \\\n-serial mon:stdio \\\n-drive file=sdb3.img,format=raw \\\n-drive file=/dev/disk_were_copy,format=raw \\\n```\n\u003e sdb3.img is the name of the image file or if you want use the disk the block device of the disk (/dev/sdXXXX)\n\n\u003e /dev/disk_were_copy is the external disk where you can copy data\n\n**note:**  inside the VM the first row -drive become /dev/sda the second /dev/sdb etc\n\nif all works as expected the VM starts.\n```\nWelcome to the recovery kernel for QNAP product\n\nuse admin/admin for login\n\nNAS login:\n```\n\nnow we can try to mount open the volume ( you can use the ddrescue_volume.img for the test)\n\n```\n[~] # mdadm --examine --scan\nARRAY /dev/md/1  metadata=1.0 UUID=e767caf1:30504500:f7da4b48:92bb490e name=1\n\n[~] # mdadm --assemble --run -o /dev/md1 --uuid=e767caf1:30504500:f7da4b48:92b\u003e\n[  185.188681] md: md1 stopped.\n[  185.197586] md/raid1:md1: active with 1 out of 2 mirrors\n[  185.199758] md1: detected capacity change from 0 to 32833536\nmdadm: /dev/md1 has been started with 1 drive (out of 2).\n\n\n[~] # vgs\n  VG   #PV #LV #SN Attr   VSize  VFree\n  vg1    1   3   1 wz--n- 28.00m    0\n\n[~] # lvs\n  LV        VG   Attr       LSize  Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert\n  lv1       vg1  owi---tz-- 12.00m tp1\n  snapShots vg1  swi---s---  4.00m      lv1\n  tp1       vg1  twi---tz-- 20.00m\n\nwe need to active the volume lv1 (or use the proper volume name if it was different)\n\n[~] # lvchange -ay vg1/lv1\n[  312.500140] device-mapper: thin metadata: __create_persistent_data_objects: block manger get correctly\n[  312.556542] device-mapper: thin metadata: dm_pool_set_mapped_threshold: threshold: 0 total_mapped_blocks: 192 new_threshold: 320\n[  312.561102] device-mapper: thin: maybe_resize_data_dev: expand pool origin max threshold to 320\n[  312.569190] device-mapper: snapshots: Snapshot is marked invalid.\n\n\n[~] # lvs\n  LV        VG   Attr       LSize  Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert\n  lv1       vg1  owi-a-tz-- 12.00m tp1         100.00\n  snapShots vg1  swi-I-s---  4.00m      lv1    100.00\n  tp1       vg1  twi---tz-- 20.00m             60.00  36.91\n\nnow the volume lv1 is active and now we can try to mount it\n\n\n[~] # mount -t ext4 -o ro /dev/vg1/lv1 /mnt\n[  409.386651] ext4_init_reserve_inode_table0: dm-4, 2\n[  409.388596] ext4_init_reserve_inode_table2: dm-4, 2, 0, 0, 1024\n[  409.390997] EXT4-fs (dm-4): mounted filesystem (\u003cnone\u003e) with ordered data mode. Opts:\n\n[~] # df -k\nFilesystem           1K-blocks      Used Available Use% Mounted on\nnone                    409600    146788    262812  36% /\ndevtmpfs                481384         0    481384   0% /dev\ntmpfs                   131072        20    131052   0% /tmp\ntmpfs                   500332         0    500332   0% /dev/shm\ntmpfs                    16384         0     16384   0% /share\ndf: /mnt/snapshot/export: No such file or directory\ncgroup_root             500332         0    500332   0% /sys/fs/cgroup\ntmpfs                    24576         0     24576   0% /smb_tmp\ntmpfs                    65536         0     65536   0% /tunnel_agent\n/dev/vg1/lv1             10871     10625         0 100% /mnt\n\nIT'S WORKS\n```\nnow you can mount the other disk (on VM is /dev/sdb to a another mount copy and copy the data.\n\n\n\n## test the qnap test volume on linux box\nyou can try to mount the test volume **ddrescue_volume.img** on a linux box to see what happen-\n\n```\n# losetup /dev/loop10 ddrescue_volume.img\n# mdadm --examine /dev/loop10\n/dev/loop10:\n          Magic : a92b4efc\n        Version : 1.0\n    Feature Map : 0x0\n     Array UUID : e767caf1:30504500:f7da4b48:92bb490e\n           Name : 1\n  Creation Time : Sat Oct 21 09:22:29 2023\n     Raid Level : raid1\n   Raid Devices : 2\n\n Avail Dev Size : 64216 sectors (31.36 MiB 32.88 MB)\n     Array Size : 32064 KiB (31.31 MiB 32.83 MB)\n  Used Dev Size : 64128 sectors (31.31 MiB 32.83 MB)\n   Super Offset : 64232 sectors\n   Unused Space : before=0 sectors, after=104 sectors\n          State : active\n    Device UUID : d2715d67:9bcc2c06:addcbc12:c70281b2\n\n    Update Time : Sat Oct 21 18:21:20 2023\n       Checksum : ef04233c - correct\n         Events : 18\n\n\n   Device Role : Active device 0\n   Array State : AA ('A' == active, '.' == missing, 'R' == replacing)\n# mdadm --assemble --run /dev/md99  /dev/loop10 --uuid=e767caf1:30504500:f7da4b48:92bb490e\nmdadm: /dev/md99 has been started with 1 drive (out of 2).\n\n# vgs\n  WARNING: PV /dev/md99 in VG vg1 is using an old PV header, modify the VG to update.\n  VG     #PV #LV #SN Attr   VSize    VFree\n  cl       1   3   0 wz--n- \u003c110.20g  33.57g\n  datavg   1   1   0 wz--n-    1.36t 596.90g\n  vg1      1   3   1 wz--n-   28.00m      0\n# lvs\n  WARNING: PV /dev/md99 in VG vg1 is using an old PV header, modify the VG to update.\n  LV        VG     Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert\n  home      cl     -wi-a----- \u003c10.63g\n  root      cl     -wi-ao----  50.00g\n  swap      cl     -wi-ao----  16.00g\n  datalv    datavg -wi-ao---- 800.00g\n  lv1       vg1    owi---tz--  12.00m tp1\n  snapShots vg1    swi---s---   4.00m      lv1\n  tp1       vg1    twi---tz--  20.00m\n\n# lvchange -ay vg1/lv1 -v\n  WARNING: PV /dev/md99 in VG vg1 is using an old PV header, modify the VG to update.\n  Activating logical volume vg1/lv1.\n  activation/volume_list configuration setting not defined: Checking only host tags for vg1/lv1.\n  Creating vg1-tp1_tmeta\n  Loading table for vg1-tp1_tmeta (253:4).\n  Resuming vg1-tp1_tmeta (253:4).\n  Creating vg1-tp1_tdata\n  Loading table for vg1-tp1_tdata (253:5).\n  Resuming vg1-tp1_tdata (253:5).\n  Executing: /usr/sbin/thin_check -q --clear-needs-check-flag /dev/mapper/vg1-tp1_tmeta\n  /usr/sbin/thin_check failed: 1\n  Check of pool vg1/tp1 failed (status:1). Manual repair required!\n  Removing vg1-tp1_tmeta (253:4)\n  Removing vg1-tp1_tdata (253:5)\n\n```\nit doesn't work. In other circumstances\n```\n  LV vg1/tp1, segment 1 invalid: does not support flag ERROR_WHEN_FULL. for tier-thin-pool segment.\n  Internal error: LV segments corrupted in tp1.\n  Cannot process volume group vg1\n```\n\n\n\n## notes\nthe tmp space is calculate on the ammount of ram assigned via qemu. if you need more space after the boot of the custom kernel you can do\n```\nmount -t tmpfs tmpfs /tmp -o remount,size=256M\n```\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fruser75%2Fqnap-recovery-kernel","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fruser75%2Fqnap-recovery-kernel","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fruser75%2Fqnap-recovery-kernel/lists"}