https://github.com/the8472/platter-walk

Rust library for HDD-aware directory traversal
https://github.com/the8472/platter-walk

filesystem hdd library linux

Last synced: 11 days ago
JSON representation

Rust library for HDD-aware directory traversal

• Host: GitHub
• URL: https://github.com/the8472/platter-walk
• Owner: the8472
• License: mpl-2.0
• Created: 2017-05-10T20:51:43.000Z (over 7 years ago)
• Default Branch: master
• Last Pushed: 2021-03-18T22:05:37.000Z (over 3 years ago)
• Last Synced: 2024-10-11T17:25:55.317Z (28 days ago)
• Topics: filesystem, hdd, library, linux
• Language: Rust
• Size: 13.7 KB
• Stars: 33
• Watchers: 4
• Forks: 4
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
[![Version](https://img.shields.io/crates/v/platter-walk.svg)](https://crates.io/crates/platter-walk)

# platter-walk

A recursive directory entry iterator that optimizes traversal based on physical disk layout.

Takes block offsets (via FIEMAP[[1]](https://github.com/torvalds/linux/commit/abc8746eb91fb01e8d411896f80f7687c0d8372e)), inode tables and disk cache locality into account.

For users (root) who have read access to the underlying block device it also performs readaheads on the directory indicies

spanning several directories. This is somewhat of a hack since `readahead()` and `posix_fadvise()` do not work on directories directly

since they use a separate cache.[[2]](https://www.spinics.net/lists/linux-fsdevel/msg30843.html)[[3]](https://www.spinics.net/lists/linux-fsdevel/msg31321.html)

The largest benefits can be realized on HDDs with ext4 filesystems.

Traversal can be optimized for

* simple directory entry listing (name and `d_type` only)

* detailed entry listing (`stat`)

* reading file contents. Entry batches are sorted by physical offset.

See [ffcnt](https://github.com/the8472/ffcnt#unscientific-benchmark) and [fastar](https://github.com/the8472/fastar#benchmarks) for benchmarks.