Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.
Awesome Lists | Featured Topics | Projects
https://github.com/007revad/synology_enable_deduplication

Enable deduplication with non-Synology SSDs and unsupported NAS models
https://github.com/007revad/synology_enable_deduplication
deduplication diskstation dsm rackstation synology synology-disk-station synology-dsm synology-nas
Last synced: 5 days ago
JSON representation
Enable deduplication with non-Synology SSDs and unsupported NAS models
Host: GitHub
URL: https://github.com/007revad/synology_enable_deduplication
Owner: 007revad
License: mit
Created: 2023-04-02T19:34:52.000Z (almost 2 years ago)
Default Branch: main
Last Pushed: 2024-12-19T00:38:29.000Z (about 2 months ago)
Last Synced: 2025-02-03T14:09:44.066Z (12 days ago)
Topics: deduplication, diskstation, dsm, rackstation, synology, synology-disk-station, synology-dsm, synology-nas
Language: Shell
Homepage:
Size: 780 KB
Stars: 150
Watchers: 11
Forks: 5
Open Issues: 5
Metadata Files:
- Readme: README.md
- Changelog: CHANGES.txt
- Funding: .github/FUNDING.yml
- License: LICENSE
Awesome Lists containing this project

README

        # Synology enable Deduplication





[![Donate](https://img.shields.io/badge/Donate-PayPal-green.svg)](https://www.paypal.com/paypalme/007revad)

[![](https://img.shields.io/static/v1?label=Sponsor&message=%E2%9D%A4&logo=GitHub&color=%23fe8e86)](https://github.com/sponsors/007revad)

[![committers.top badge](https://user-badge.committers.top/australia/007revad.svg)](https://user-badge.committers.top/australia/007revad)

### Description

Enable data deduplication with non-Synology SSDs and unsupported NAS models

- Works for any brand SATA SSD, SAS SSD and M.2 SSD drives in DSM 7.01 and later.

- Now works for HDDs in DSM 7.2.1 and later.

- Now works for M.2 drives in a PCIe adapter card (E10M20-T1, M2D20, M2D18 or M2D17) in DSM 7.2.1 and later.

It works on [Synology models that do offically support data deduplication](https://kb.synology.com/en-global/DSM/tutorial/Which_models_support_data_deduplication).

It also works in DSM 7.0.1 to 7.2.1 on models with one of the following [CPU architectures](https://kb.synology.com/en-global/DSM/tutorial/What_kind_of_CPU_does_my_NAS_have): V1000, R1000, Geminilake and Broadwellnk.

It only works in DSM 7.2.1 on models with a Broadwellnkv2, Broadwell, Purley, Epyc7002 CPU.

It does **NOT** work for models with Apollolake, Avoton or Denverton CPUs.

Please [leave a comment in this discussion](https://github.com/007revad/Synology_enable_Deduplication/discussions/31) if it works, or doesn't work, for you.

### Requirements

- Btrfs Tiny Data Deduplication requires 4GB of memory or more.

- Btrfs Data Deduplication requires 16GB of memory or more.

- The volume needs **Usage detail analysis** enabled. See [Enable and View Usage Details](https://kb.synology.com/en-global/DSM/help/DSM/StorageManager/volume_view_usage?version=7).

### Works in DSM 7.2.1 for the following models

  Click here to see list

| Model      | CPU Arch 
|------------|---------- 
| DS224+     | Geminilake 
| DS1823xs+  | V1000 
| DS923+     | R1000 
| DS723+     | R1000 
| DS423+     | Geminilake 
| DS3622xs+  | Broadwellnk 
| DS2422xs+  | V1000 
| DS1821+    | V1000 
| DS1621+    | V1000 
| DS1621xs+  | Broadwellnk 
| DS1522+    | R1000 
| DS1520+    | Geminilake 
| DS920+     | Geminilake 
| DS720+     | Geminilake 
| DS420+     | Geminilake 
| DS220+     | Geminilake 
| DS3018xs   | Broadwellnk 
| DS3017xsII | Broadwell 
| DS3017xs   | Broadwell 
| | | | | | 
| DVA1622    | Geminilake 
| | | | | | 
| RS2423xs+  | V1000 
| RS822xs+   | V1000 
| RS422xs+   | R1000 
| RS4021xs+  | Broadwellnk 
| RS3621RPxs | Broadwellnk 
| RS3621xs+  | Broadwellnk 
| RS2821RPxs+ | V1000 
| RS2421xs+  | V1000 
| RS1221xs+  | V1000 
| RS1619xs+  | Broadwellnk 
| RS3618xs   | Broadwell 
| RS3617xs+  | Broadwell 
| RS3617RPxs | Broadwell 
| RS18017xs+ | Broadwell 
| RS4017xs+  | Broadwell 
| | | | | | 
| FS6400     | Purley 
| FS3600     | Broadwellnk 
| FS3410     | 
| FS3400     | Broadwell 
| FS2500     | V1000 
| FS2017     | Broadwell 
| FS1018     | Broadwellnk 
| | | | | | 
| HD6500     | Purley 
| | | | | | 
| SA6400     | Epyc7002 
| SA3610     | 
| SA3600     | Broadwellnk 
| SA3410     | 
| SA3400     | Broadwellnk

| DSM version                   | Works  | Notes | -----|-------------------------------|--------|-------| | DSM 7.2.1-69057 Update 1 to 5 | yes    | Use v1.2.14 or later | | DSM 7.2.1-69057 Update 1 to 5 | yes    | Use v1.2.14 or later | | DSM 7.2.1-69057 Update 1 to 5 | yes    | Use v1.2.14 or later | | DSM 7.2.1-69057 Update 1 to 5 | yes    | Use v1.2.14 or later | | DSM 7.2.1-69057 Update 1 to 5 | yes    | Use v1.2.14 or later | | DSM 7.2.1-69057 Update 1 to 5 | yes    | Use v1.2.14 or later | | DSM 7.2.1-69057 Update 1 to 5 | yes    | Use v1.2.14 or later | | DSM 7.2.1-69057 Update 1 to 5 | yes    | Use v1.2.14 or later | | DSM 7.2.1-69057 Update 1 to 5 | yes    | Use v1.2.14 or later | | DSM 7.2.1-69057 Update 1 to 5 | yes    | Use v1.2.14 or later | | DSM 7.2.1-69057 Update 1 to 5 | yes    | Use v1.2.14 or later | | DSM 7.2.1-69057 Update 1 to 5 | yes    | Use v1.2.14 or later | | DSM 7.2.1-69057 Update 1 to 5 | yes    | Use v1.2.14 or later | | DSM 7.2.1-69057 Update 1 to 5 | yes    | Use v1.2.14 or later | | DSM 7.2.1-69057 Update 1 to 5 | yes    | Use v1.2.14 or later | | DSM 7.2.1-69057 Update 1 to 5 | yes    | Use v1.2.14 or later | | DSM 7.2.1-69057 Update 1 to 5 | yes    | Use v1.2.14 or later | | DSM 7.2.1-69057 Update 1 to 5 | yes    | Use v1.2.14 or later | | DSM 7.2.1-69057 Update 1 to 5 | yes    | Use v1.2.14 or later | | DSM 7.2.1-69057 Update 1 to 5 | yes    | Use v1.2.14 or later | | DSM 7.2.1-69057 Update 1 to 5 | yes    | Use v1.2.14 or later | | DSM 7.2.1-69057 Update 1 to 5 | yes    | Use v1.2.14 or later | | DSM 7.2.1-69057 Update 1 to 5 | yes    | Use v1.2.14 or later | | DSM 7.2.1-69057 Update 1 to 5 | yes    | Use v1.2.14 or later | | DSM 7.2.1-69057 Update 1 to 5 | yes    | Use v1.2.14 or later | | DSM 7.2.1-69057 Update 1 to 5 | yes    | Use v1.2.14 or later | | DSM 7.2.1-69057 Update 1 to 5 | yes    | Use v1.2.14 or later | | DSM 7.2.1-69057 Update 1 to 5 | yes    | Use v1.2.14 or later | | DSM 7.2.1-69057 Update 1 to 5 | yes    | Use v1.2.14 or later | | DSM 7.2.1-69057 Update 1 to 5 | yes    | Use v1.2.14 or later | | DSM 7.2.1-69057 Update 1 to 5 | yes    | Use v1.2.14 or later | | DSM 7.2.1-69057 Update 1 to 5 | yes    | Use v1.2.14 or later | | DSM 7.2.1-69057 Update 1 to 5 | yes    | Use v1.2.14 or later | | DSM 7.2.1-69057 Update 1 to 5 | yes    | Use v1.2.14 or later | | DSM 7.2.1-69057 Update 1 to 5 | yes    | Use v1.2.14 or later | | DSM 7.2.1-69057 Update 1 to 5 | yes    | Use v1.2.14 or later | | DSM 7.2.1-69057 Update 1 to 5 | yes    | Use v1.2.14 or later | Broadwellnkv2 | DSM 7.2.1-69057 Update 1 to 5 | yes    | Use v1.2.14 or later | | DSM 7.2.1-69057 Update 1 to 5 | yes    | Use v1.2.14 or later | | DSM 7.2.1-69057 Update 1 to 5 | yes    | Use v1.2.14 or later | | DSM 7.2.1-69057 Update 1 to 5 | yes    | Use v1.2.14 or later | | DSM 7.2.1-69057 Update 1 to 5 | yes    | Use v1.2.14 or later | | DSM 7.2.1-69057 Update 1 to 5 | yes    | Use v1.2.14 or later | | DSM 7.2.1-69057 Update 1 to 5 | yes    | Use v1.2.14 or later | Broadwellnkv2 | DSM 7.2.1-69057 Update 1 to 5 | yes    | Use v1.2.14 or later | | DSM 7.2.1-69057 Update 1 to 5 | yes    | Use v1.2.14 or later | Broadwellnkv2 | DSM 7.2.1-69057 Update 1 to 5 | yes    | Use v1.2.14 or later | | DSM 7.2.1-69057 Update 1 to 5 | yes    | Use v1.2.14 or later |

### Models confirmed working with older DSM versions

  Click here to see list

| Model      | CPU Arch      | DSM version                   | Works  | Notes |

|------------|---------------|-------------------------------|--------|-------|

| DS923+     | R1000         | DSM 7.2-64570 Update 1 to 3   | yes    | |

| DS923+     | R1000         | DSM 7.2-64570                 | yes    | |

| DS3622xs+  | Broadwellnk   | DSM 7.2-64570                 | **No** | Update to DSM 7.2.1 |

| DS3622xs+  | Broadwellnk   | DSM 7.2-64561                 | yes    | |

| DS3622xs+  | Broadwellnk   | DSM 7.1.1-42962 Update 1      | **No** | Update to DSM 7.2.1 |

| RS4021xs+  | Broadwellnk   | DSM 7.2-64570                 | **No** | Update to DSM 7.2.1 |

| RS4021xs+  | Broadwellnk   | DSM 7.1.1-42962 Update 2      | yes    | |

| DS1821+    | V1000         | DSM 7.2-64570 Update 1 to 3   | yes    | |

| DS1821+    | V1000         | DSM 7.2-64570                 | yes    | |

| DS1821+    | V1000         | DSM 7.2-64561                 | yes    | |

| DS1821+    | V1000         | DSM 7.1.1-42962 Update 4      | yes    | |

| DS1621xs+  | Broadwellnk   | DSM 7.2-64570 Update 3        | yes    | |

| DS1621xs+  | Broadwellnk   | DSM 7.2-64570                 | yes    | |

| DS920+     | Geminilake    | DSM 7.2-64570 Update 1 to 3   | yes    | |

| DS920+     | Geminilake    | DSM 7.2-64570                 | yes    | |

| DS720+     | Geminilake    | DSM 7.2-64570 Update 1 to 3   | yes    | |

| DS720+     | Geminilake    | DSM 7.2-64570                 | yes    | |

| DS3617xs   | Broadwell     |                               | **No** | Update to DSM 7.2.1 |

| | | | | |

|            | Apollolake    |                               | **No** | DSM missing many required files |

|            | Avoton        |                               | **No** | DSM missing many required files |

|            | Denverton     |                               | **No** | DSM missing many required files |

## Download the script

1. Download the latest version _Source code (zip)_ from https://github.com/007revad/Synology_enable_Deduplication/releases

2. Save the download zip file to a folder on the Synology.

3. Unzip the zip file.

## How to run the script

### Running the script via SSH

[How to enable SSH and login to DSM via SSH](https://kb.synology.com/en-global/DSM/tutorial/How_to_login_to_DSM_with_root_permission_via_SSH_Telnet)

**Note:** Replace /volume1/scripts/ with the path to where the script is located.

Run the script then reboot the Synology:

```YAML

sudo -s /volume1/scripts/syno_enable_dedupe.sh

```

**Options:**

```YAML

  -c, --check           Check value in file and backup file

  -r, --restore         Undo all changes made by the script

  -t, --tiny            Enable tiny data deduplication (only needs 4GB RAM)

                          DSM 7.2.1 and later only

      --hdd             Enable data deduplication for HDDs.

                          Can cause files to become more fragmented,

                          resulting in decreased access performance.

  -e, --email           Disable colored text in output for scheduler emails

      --autoupdate=AGE  Auto update script (useful when script is scheduled)

                          AGE is how many days old a release must be before

                          auto-updating. AGE must be a number: 0 or greater

  -s, --skip            Skip memory amount check (for testing)

  -h, --help            Show this help message

  -v, --version         Show the script version

```

**Extra Steps:**

To get rid of drive database outdated notifications and unrecognised firmware warnings run Synology_HDD_db which will add your drives to DSM's compatibile drive databases, and prevent the drive compatability databases being updated between DSM updates.

```YAML

sudo -s /path-to-script/syno_hdd_db.sh --noupdate

```

### What about DSM updates?

After any DSM update you will need to run this script, and the Synology_HDD_db script again. 

### Schedule the script to run at shutdown

Or you can schedule both Synology_enable_Deduplication and Synology_HDD_db to run when the Synology shuts down, to avoid having to remember to run both scripts after a DSM update.

See How to schedule a script in Synology Task Scheduler




## Screenshots

Here's the result after running the script and rebooting. Note that the DS1821+ is not officially listed as supporting deduplication, and non-Synology SSDs are being used.

Unsupported NAS model with enough memory



Non-Synology SSDs setup as a volume



Enable Data Deduplication option available after running this script and rebooting



Configure Data Deduplication



Deduplication finished notification



Deduplications works.



Deduplication for HDDs too.