Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/westerndigitalcorporation/libzbc

ZBC device manipulation library. When submitting a bug report, PLEASE DO NOT SUBMIT CONFIDENTIAL INFORMATION OR INFORMATION SPECIFIC TO DRIVES THAT ARE VENDOR SAMPLES OR NOT PUBLICLY AVAILABLE.
https://github.com/westerndigitalcorporation/libzbc

Last synced: 16 days ago
JSON representation

ZBC device manipulation library. When submitting a bug report, PLEASE DO NOT SUBMIT CONFIDENTIAL INFORMATION OR INFORMATION SPECIFIC TO DRIVES THAT ARE VENDOR SAMPLES OR NOT PUBLICLY AVAILABLE.

Awesome Lists containing this project

README

        

Copyright (C) 2016, Western Digital.

Copyright (C) 2020 Western Digital Corporation or its affiliates.

# libzbc

*libzbc* is a simple library providing functions for manipulating SCSI and ATA
devices supporting the Zoned Block Command (ZBC) and Zoned-device ATA command
set (ZAC) specifications.

*libzbc* implementation is compliant with the latest drafts of the ZBC and ZAC
standards defined by INCITS technical committee T10 and T13 (respectively).

Several example applications using *libzbc* are available under the tools
directory.

## Online Documentation

Information about *libzbc*, including examples, can be found on the
[zonedstorage.io](https://zonedstorage.io/docs/tools/libzbc) site.

Detailed information on the execution of device compliance tests can be found
[here](https://zonedstorage.io/docs/tests/zbc-tests).

## Library version

*libzbc* current major version is 6. Due to interface changes, this version is
not compatible with previous *libzbc* versions (version 5.x). Applications
written for previous *libzbc* versions may require updating to use this new
version.

In version 6, *libzbc* emulation mode allowing to imitate the behavior of a host
managed zoned disk has been removed. Users of that feature are encouraged to
migrate to a more advanced zone device emulation provided by QEMU.

In *libzbc* 6.x, the block zone backend driver has been removed. The
functionality provided by this backend is now available by means of a different
library, [libzbd](https://github.com/westerndigitalcorporation/libzbd).

## ZBC and ZAC Standards Versions Supported

*libzbc* latest version implements INCITS 550 ZBC-2 and ZAC-2 rev. 15
standards. Support for the older draft standards are available with
[previous releases and pre-releases](https://github.com/hgst/libzbc/releases).

## License

*libzbc* source code is distributed under the terms of the BSD 2-clause
license ("Simplified BSD License" or "FreeBSD License", SPDX: *BSD-2-Clause*)
and under the terms of the GNU Lesser General Public License version 3, or any
later version (SPDX: *LGPL-3.0-or-later*).
A copy of these licenses with *libzbc* copyright can be found in the files
[LICENSES/BSD-2-Clause.txt] and [COPYING.BSD] for the BSD 2-clause license and
[LICENSES/LGPL-3.0-or-later.txt] and [COPYING.LESSER] for the LGPL-v3 license.
If not, please see
http://opensource.org/licenses/BSD-2-Clause and http://www.gnu.org/licenses/.

All example applications under the tools directory are distributed under the
terms of the GNU Lesser General Public License version 3, or any later version
(SPDX: *LGPL-3.0-or-later*).

*libzbc* and all its example applications are distributed "as is," without
technical support, and WITHOUT ANY WARRANTY, without even the implied warranty
of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

All source files in *libzbc* contain the BSD 2-clause and LGPL v3 license SPDX
short identifiers in place of the full license text.

```
SPDX-License-Identifier: BSD-2-Clause
SPDX-License-Identifier: LGPL-3.0-or-later
```

Some files such as the `.gitignore` file are public domain specified by the
CC0 1.0 Universal (CC0 1.0) Public Domain Dedication. These files are
identified with the following SPDX header.

```
SPDX-License-Identifier: CC0-1.0
```

See [LICENSES/CC0-1.0.txt] for the full text of this license.

## Contributions and Bug Reports

Contributions are accepted as github pull requests. Any problem may also be
reported through github issue page or by contacting:

* Damien Le Moal ([email protected])
* Dmitry Fomichev ([email protected])

PLEASE DO NOT SUBMIT CONFIDENTIAL INFORMATION OR INFORMATION SPECIFIC TO DRIVES
THAT ARE VENDOR SAMPLES OR NOT PUBLICLY AVAILABLE.

## Compilation and installation

### Requirements

*libzbc* requires the following packages for compilation:

* m4
* autoconf
* automake
* libtool

The GTK3 and GTK3 development packages must be installed to automatically enable
compiling the *gzbc* and *gzviewer* applications.

### Compilation

To compile the library and all example applications under the tools directory,
execute the following commands.

```
$ sh ./autogen.sh
$ ./configure
$ make
```

#### Compilation with GUI tools

The *gzbc* and *gzviewer* tools implement a graphical user interface (GUI) using
the GTK3 toolkit. The configure script will automatically detect the presence of
GTK3 and its development header files and compile these tools if the header
files are found. This behavior can be manually changed and the compilation of
*gzbc* and *gzviewer* disabled using the `--disable-gui` configuration option.

```
$ ./configure --disable-gui
```

#### Compilation for device tests

The test directory contains several test programs and scripts allowing testing
the compatibility of *libzbc* with a particular device. That is, testing if a
device follows the same standard as currently supported by *libzbc*. The
compilation of these test programs is disabled by default.

To compile the test programs, *libzbc* must be configured as follows.

```
$ ./configure --with-test
```

The test programs and scripts are not affected by the execution of "make
install". All defined tests must be executed directly from the test directory
using the *zbc_test.sh* script. To test the device `/dev/`, the
following can be executed.

```
$ cd test
$ sudo ./zbc_test.sh /dev/
```

By default, the script will run through all the test cases. Detailed control
of the test execution is possible using the `-e` (execute) and `-s`(skip)
options. Run `zbc_test.sh --help` for details.

*libzbc* tests check the detailed error output from the device for invalid
commands.

Each test outputs a log file in the `test/log` directory. These files can be
consulted in case of a failed test to identify the reason for the test failure.

### Installation

To install the library and all example applications compiled under the tools
directory, execute the following command.

```
$ sudo make install
```

The library file is by default installed under `/usr/lib` (or `/usr/lib64`).
The library header file is installed in `/usr/include/libzbc`. The executable
files for the example applications are installed under `/usr/bin`.

These default installation locations can be changed using the configure script.
Executing the following command displays the options used to control the
installation paths.

```
$ ./configure --help
```

### Building RPM packages

The *rpm* and *rpmbuild* utilities are necessary to build *libzbc* RPM
packages. Once these utilities are installed, the RPM packages can be built
using the following command.

```
$ sh ./autogen.sh
$ ./configure
$ make rpm
```

Five RPM packages are built:
* A binary package providing *libzbc* library, tools, documentation and
license files.
* A source RPM package
* A *debuginfo* RPM package and a *debugsource* RPM package
* A development package providing the library header files

The source RPM package can be used to build the binary and debug RPM packages
outside of *libzbc* source tree using the following command.

```
$ rpmbuild --rebuild libzbc-.src.rpm
```

## Library Overview

*libzbc* functions operate using a device handle obtained by executing the
*zbc_open()* function. The path specified to identify a device can point to a
regular file, a block device file (*/dev/sdX*)or an SG node device file
(*/dev/sgY*).

As host-aware devices are backward-compatible with regular block device files
(device type/signature 0x00), a host-aware device will always be accessible
either through a block device file as well as its SG node device file.

For host-managed devices, the different device type and device signature require
kernel support for block device files to be enabled. Kernel support for
host-managed devices has been added with Linux kernel 4.10. For older kernels,
host-managed devices will be accessible only through their SG node device files.
Kernel support for the ZAC host-managed device signature (0xabcd) was introduced
in kernel 3.18. Any kernel older than this version will not create an SG node
device file for ZAC host-managed devices connected to an AHCI SATA port on the
target host.

Regular files and block device files for regular devices can be used to operate
*libzbc* in emulation mode. This will enable exposing the target file or block
device as a host-managed zoned device.

### Library Functions

*libzbc* provides functions for discovering the zone configuration of a zoned
device and for accessing the device. Accesses to the device may result in
changes to the device zones condition, attributes and state (such as a
sequential zone write pointer location). These changes are not internally
tracked by *libzbc*. The functions provided to obtain the device zone
information only provide a snapshot of the zone condition and state when
executed. It is the responsibility of an application to implement tracking of
the device zone changes (such as increment to a sequential zone write pointer as
writes to the zone are executed) if necessary.

All *libzbc* functions since version 5.0.0 use 512B sector unit for reporting
zone information and as the addressing unit for device accesses, regardless of
the actual device logical block size. This unification in the unit used by all
API functions can simplify application development by hiding potential
differences in logical block sizes between devices. However, application
programmers must be careful to always implement write accesses to sequential
write required zones of the device in multiple of the physical block size.

The main functions provided by *libzbc* are as follows.

Function | Description
-------------------------|---------------------------------------------
*zbc_open()* | Open a zoned device
*zbc_close()* | Close a zoned device
*zbc_get_device_info()* | Get device information
*zbc_report_nr_zones()* | Get the number of zones of the device
*zbc_report_zones()*
*zbc_list_zones()* | Get zone information
*zbc_zone_operation()* | Execute a zone operation
*zbc_open_zone()* | Explicitly open a zone
*zbc_close_zone()* | Close an open zone
*zbc_finish_zone()* | Finish a zone
*zbc_reset_zone()* | Reset a zone write pointer
*zbc_pread()* | Read data from a zone
*zbc_preadv()* | Read data from a zone using vectored buffer
*zbc_pwrite()* | Write data to a zone
*zbc_pwritev()* | Write data to a zone using vectored buffer
*zbc_flush()* | Flush data to disk

Additionally, the following functions are also provided to facilitate
application development and tests.

Function | Description
---------------------------|---------------------------------------------------
*zbc_map_iov()* | Map a vectored buffer using a single buffer
*zbc_set_log_level()* | Set the logging level of the library functions
*zbc_device_is_zoned()* | Test if a device is a zoned block device
*zbc_print_device_info()* | Print device information to a file (stream)
*zbc_device_type_str()* | Get a string description of a device type
*zbc_device_model_str()* | Get a string description of a device model
*zbc_zone_type_str()* | Get a string description of a zone type
*zbc_zone_condition_str()* | Get a string description of a zone condition
*zbc_errno()* | Get the sense key and code of the last function call
*zbc_sk_str()* | Get a string description of a sense key
*zbc_asc_ascq_str()* | Get a string description of a sense code

*libzbc* does not implement any synchronization mechanism for multiple threads
or processes to safely operate simultaneously on the same zone. In particular,
concurrent write operations by multiple threads to the same zone may result in
write errors without write ordering control by the application. The
*zbc_errno()* function is the only exception to this rule. This function is
thread safe and does not require serialized execution by the application.

### Native Operation Mode

Linux kernels older than version 4.10 do not create a block device file for
host-managed ZBC and ZAC devices. As a result, these devices can only be
accessed through their associated SG node (/dev/sgx device file). For these
older kernels, opening a ZBC or ZAC host managed disk with *libzbc* must thus
be done using the device SG node. For kernel versions 4.10 and beyond compiled
with zoned block device support, the device will be exposed also through a block
device file which can be used with *libzbc* to identify the device.

For host-aware devices, a block device file and an SG node file will exist and
can both be used to open the device.

Once the device is open, accesses to the device are done transparently using the
device handle returned by the *zbc_open()* function. Operations such as report
zones, reset zone write pointer, etc. only need the device handle.

### Functions Documentation

More detailed information on *libzbc* functions and data types is available
through the comments in the file `include/libzbc/zbc.h`. This file has comments
formatted with the doxygen convention. HTML files documenting *libzbc* API can
be generated using the doxygen project file documentation/libzbc.doxygen.

```
$ cd documentation
$ doxygen libzbc.doxygen
```

## Tools

Under the tools directory, several simple applications are available as
examples. These applications are as follows.

* **zbc_info** This application tests if a device file points to a physical
zoned device supporting ZBC or ZAC features. This excludes the emulation mode
implemented by *libzbc*. If the device is identified as a zoned device, some
information about the device are displayed (e.g. the device type, capacity,
sector size, etc).

* **zbc_report_zones** This application illustrates the use of the zone
reporting functions *zbc_report_zones()*, *zbc_report_nr_zones()* and
*zbc_list_zones()*. *zbc_report_zones* obtains the zone information of a
device and displays it in readable form on the standard output.

* **zbc_open_zone** This application illustrates the use of the
*zbc_open_zone()* function allowing opening a zone.

* **zbc_close_zone** This application illustrates the use of the
*zbc_close_zone()* function allowing closing a zone.

* **zbc_finish_zone** This application illustrates the use of the
*zbc_finish_zone()* function allowing finishing a zone.

* **zbc_reset_zone** This application illustrates the use of the
*zbc_reset_zone()* function allowing resetting the write pointer of a zone to
the first sector of the zone.

* **zbc_read_zone** This application reads data from a zone, up to the zone
write pointer location and either sends the read data to the standard output
or copies the data to a regular file. Its implementation illustrates the use
of the functions *zbc_pread()* and *zbc_preadv()*.

* **zbc_write_zone** This application illustrates the use of the functions
*zbc_pwrite()* and *zbc_pwritev()* to write data to a zone at the zone write
pointer location.

* **gzbc** provides a graphical user interface showing zone information of a
zoned device. It also displays the write status (write pointer position) of
zones graphically using color coding (red for written space and green for
unwritten space). Some operations on zones can also be executed directly from
the interface (reset zone write pointer, open zone, close zone, etc).

* **gzviewer** provides a simple graphical user interface showing the write
pointer position and zone state of zones of a zoned device. Similar color
coding as *gzbc* is used.