Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/brlin-tw/trim-over-usb-howto

Research the usability of the unused blocks notification operation against flash storage controllers over the USB interface.
https://github.com/brlin-tw/trim-over-usb-howto

deallocate discard flash-memory flash-storage linux trim unmap usb

Last synced: 4 months ago
JSON representation

Research the usability of the unused blocks notification operation against flash storage controllers over the USB interface.

Awesome Lists containing this project

README 

          
# How to notify unused blocks to the flash storage controller over the USB interface



Research the usability of the unused blocks notification operation against flash storage controllers over the USB interface.



  

[![The GitLab CI pipeline status badge of the project's `main` branch](https://gitlab.com/brlin/trim-over-usb-howto/badges/main/pipeline.svg?ignore_skipped=true "Click here to check out the comprehensive status of the GitLab CI pipelines")](https://gitlab.com/brlin/trim-over-usb-howto/-/pipelines) [![GitHub Actions workflow status badge](https://github.com/brlin-tw/trim-over-usb-howto/actions/workflows/check-potential-problems.yml/badge.svg "GitHub Actions workflow status")](https://github.com/brlin-tw/trim-over-usb-howto/actions/workflows/check-potential-problems.yml) [![pre-commit enabled badge](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit&logoColor=white "This project uses pre-commit to check potential problems")](https://pre-commit.com/) [![REUSE Specification compliance badge](https://api.reuse.software/badge/gitlab.com/brlin/trim-over-usb-howto "This project complies to the REUSE specification to decrease software licensing costs")](https://api.reuse.software/info/gitlab.com/brlin/trim-over-usb-howto)



## Table of contents



[TOC]



## Problem



A few important characteristics of flash memory storage devices is that:



* Once a memory cell has data written on it(bit flips from 0→1), it can only be rewritten after it is erased(1→0).

* Memory cells can only be erased in batch(called erase segments), erase segments are usually bigger than memory blocks(e.g. 4MiB).

* When the count of erased memory cells are low the memory controller will need to do the erase on-the-fly, which results in a very poor write performance.



Some flash storage drives supports notifying unused memory blocks to the controller from the Operating System to do garbage collection(GC) using the following protocol commands:



* ATA: TRIM

* NVM Express(NVMe): DEALLOCATE

* SCSI: UNMAP

* SD/MMC: ERASE/DISCARD



in order to let the storage controller do the garbage collection in the background to maintain write performance.  However, the operation may no be available if:



* It is not implemented (common in lower-end flash storage drives)

* There's a translation layer between the storage controller and the operating system, like a USB external hard-drive enclosure/cable that translates the SATA/NVMe/MMC... protocol commands into the counterpart SCSI commands which the USB mass storage/UASP specification uses.



  Another issue is that even if the translation layer support such operation, the Operating System may not utilize it due to the storage drive not complying to the relevant specifications.



## Solution



We'll need to know:



* How to check whether the storage drive supports such operation.

* If the storage drive supports such operation but the Operating System didn't enables it due to uncompliance, how to override it.

* If the storage drive claimed to support such operation, how to check whether it works or not.



The answer of these questions varies based on the following factors:



* The manufacturer and model of the storage drive/enclosure

* The firmware version of the storage drive/enclosure



We'll individually collect related info of each device to avoid bias.



## Research process



This section documents the process of determining the support:



### Determine the kernel name of the storage device



Run the following commands in a text terminal to determine the kernel name of the storage device:



```bash

lsblk_opts=(

    # Don't print sub-device nodes

    --nodeps



    # Exclude loopback devices(e.g. snaps)

    --exclude 7



    # Specify output columns that are useful in determining the kernel

    # name of the storage device block device

    --output NAME,VENDOR,MODEL,SERIAL,SIZE

)

lsblk "${lsblk_opts[@]}"

```



### Determine whether the unused block notification feature is enabled by the operating system by default



Run the following commands in a text terminal to determine the kernel name of the storage device:



```bash

device_kernel_name=_kernel_name_

device="/dev/${device_kernel_name}"

lsblk_opts=(

    # Don't print sub-device nodes

    --nodeps



    # Print information about the unused block notification capabilities

    # for each device.

    --discard

)

lsblk "${lsblk_opts[@]}" "${device}"

```



### Check whether the storage device has declared implementation of the Logical Block Provisioning Management feature specified by SBC-4



Run the following command _as root_ in a text terminal to query the

response of the SCSI READ CAPACITY (16) command of the storage device:



```bash

device_kernel_name=_kernel_name_

device="/dev/${device_kernel_name}"

sg_readcap_opts=(

    # Use the 16 byte cdb variant of the READ CAPACITY command, allow

    # proper results for storage drives capacity over 2TiB

    # (2**32 - 2) * 512 / 1024 / 1024 / 1024

    --long

)

sg_readcap "${sg_readcap_opts[@]}" "${device}"

```



### Check the SCSI vital product data(VPD) pages supported by the SCSI device



Run the following command _as root_ in a text terminal to determine which SCSI vital product data(VPD) pages supported by the SCSI device:



```bash

device_kernel_name=_kernel_name_

device="/dev/${device_kernel_name}"

sg_vpd "${device}"

```



### Check the supported parameters of the SCSI UNMAP command of the storage device



Run the following command _as root_ in a text terminal to query the

Block limits VPD page of the storage device:



```bash

device_kernel_name=_kernel_name_

device="/dev/${device_kernel_name}"

sg_vpd_opts=(

    --page=bl

)

sg_vpd "${sg_vpd_opts[@]}" "${device}"

```



### Enable unused block notification support by force



Although the missing declaration of the SCSI Logical Block Provisioning

Management feature, there's still possibility that the drive actually

implemented the unused block notification functionality in the firmware.

Let's try to find out.



**Warning:** There's a reason why the functionality is not enabled by

default as there's a chance that the controller have a problomatic

reaction when facing the UNMAP SCSI command, which may results in

problems including but not limited to:



* The drive simply fails and no longer functions properly, rendering

  it no longer usable for data storage/access.

* The drive erratically respond to the command and erases memory blocks

  that are not requested, leads to data loss.



**Only continue if you can take the responsibility of device

failure/data recovery.**



In order to force enable the unused block notification feature, we need

to first determine the address of the SCSI device of the storage device

, which can be queried by running the following command in a text

terminal:



```bash

lsscsi

```



We can force enable the unused block notification support feature using

the UNMAP SCSI command by running the following commands _as root_ after

setting the proper `scsi_device_address`:



```bash

scsi_device_address=_address_

echo unmap > "/sys/class/scsi_disk/${scsi_device_address}/provisioning_mode"

```



Now that the UNMAP command is enabled, run the following commands to

check whether the data size of each UNMAP SCSI command set by the kernel

is sane:



```bash

device_kernel_name=_kernel_name_

cat "/sys/block/${device_kernel_name}/queue/discard_max_bytes"

```



### Test whether unused block notification actually work



Now we can test whether the unused block notification really work by

triggering a whole drive/partition block device discard operation by

running the following commands _as root_:



```bash

device_kernel_name=_kernel_name_

device="/dev/${device_kernel_name}"

blkdiscard_opts=(

    # Disable safeguard checks

    --force



    # Print details of the operation

    --verbose

)

blkdiscard "${blkdiscard_opts[@]}" "${device}"

```



## Research results



We currently have research results for the following products:



* [伽利略 M2NVU31 M.2(NVMe) PCI-E SSD to USB3.1 Gen2](<伽利略 M2NVU31 M.2(NVMe) PCI-E SSD to USB3.1 Gen2>)

* [OWC USB-C Travel Dock E(SD card reader)]()

* [Transcend JetFlash 790 Series 64GB]()



## References



The following material are referenced during the development of this project:



* [Discard over USB - Gentoo Wiki](https://wiki.gentoo.org/wiki/Discard_over_USB)  

  Explains how to enable discard/trim operation for block devices via the USB bus.

* [External SSD with TRIM support - Solid state drive - ArchWiki](https://wiki.archlinux.org/title/Solid_state_drive#External_SSD_with_TRIM_support)  

  Explains how to check and enable TRIM operation on solid storage drives over a USB-to-SATA bridge chips.

* [Hardware support - Trim (computing) - Wikipedia](https://en.wikipedia.org/wiki/Trim_(computing)#Hardware_support)  

  Explains the equivalent commands of unused block notification operation for each protocols.

* [sd: disable logical block provisioning if 'lbpme' is not set - Patchwork](https://patchwork.kernel.org/project/linux-scsi/patch/20190214221558.09174756@endymion/)  

  Explains the reasoning that Linux Kernel requires the LBPME bit in order to enable unused memory blocks notification support.

* [SCSI Commands Reference Manual 100293068, Rev. J](https://www.seagate.com/files/staticfiles/support/docs/manual/Interface%20manuals/100293068j.pdf)  

  Documents the Serial Attached SCSI(SAS) commands in the SCSI specifications, sections that may include useful information are:

    + 5.4 Vital product data parameters

* [Enabling TRIM on an external SSD on a Raspberry Pi | Jeff Geerling](https://www.jeffgeerling.com/blog/2020/enabling-trim-on-external-ssd-on-raspberry-pi)

* [Trying to get SSD boot working on pi4 - Page 2 - Raspberry Pi Forums](https://www.raspberrypi.org/forums/viewtopic.php?p=1708655#p1708655)

* The lsblk(8) manual page  

  Explains how to use the `--nodeps` and `--output` command-line options.

* The sg_vpd(8) manual page  

  Explains how to use the `--long` `sg_vpd` command-line option.

* [‎Gemini - LBP in SCSI](https://gemini.google.com/share/e90021aaed7c)  

  Introduces what Logical Block Provisioning Management (LBP) in SCSI do.

* [SCSI device location codes - IBM Documentation](https://www.ibm.com/docs/en/aix/7.3?topic=codes-scsi-device-location)  

  Explains the format of the SCSI device location code.

* [SCSI Addressing](https://tldp.org/HOWTO/SCSI-2.4-HOWTO/scsiaddr.html)  

  Explains the format of the SCSI device addresses.

* [Queue sysfs files | Info on the Block I/O (BIO) layer | Linux kernel plaintext documentation](https://www.kernel.org/doc/Documentation/block/queue-sysfs.txt)  

  Explains the definition of the discard_max_bytes sysfs file.