Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/abbbi/nullfsvfs
a virtual black hole file system that behaves like /dev/null
https://github.com/abbbi/nullfsvfs
dev filesystem linux linux-kernel null nulldev nullfs performance-testing vfs virtual-file-system
Last synced: 13 days ago
JSON representation
a virtual black hole file system that behaves like /dev/null
- Host: GitHub
- URL: https://github.com/abbbi/nullfsvfs
- Owner: abbbi
- License: gpl-3.0
- Created: 2018-11-08T19:12:22.000Z (about 6 years ago)
- Default Branch: master
- Last Pushed: 2024-06-21T05:58:04.000Z (5 months ago)
- Last Synced: 2024-10-13T07:42:11.267Z (30 days ago)
- Topics: dev, filesystem, linux, linux-kernel, null, nulldev, nullfs, performance-testing, vfs, virtual-file-system
- Language: C
- Homepage:
- Size: 177 KB
- Stars: 289
- Watchers: 8
- Forks: 12
- Open Issues: 8
-
Metadata Files:
- Readme: README.md
- Funding: .github/FUNDING.yml
- License: LICENSE
Awesome Lists containing this project
README
[](https://github.com/abbbi/nullfsvfs/actions/workflows/ci-ubuntu-latest.yml)
- [nullfs](#nullfs)
- [installation](#installation)
- [manual installation](#manual-installation)
- [debian package](#debian-package)
- [Usage](#usage)
- [Keeping file data](#keeping-file-data)
- [ACL](#acl)
- [usecases](#usecases)
- [supported mount options](#supported-mount-options)
- [todos/ideas](#todosideas)
# nullfs
A virtual file system that behaves like /dev/null
It can handle regular file operations like mkdir/rmdir/ln but writing to files
does not store any data. The file size is however saved, reading from the
files behaves like reading from /dev/zero.
Writing and reading is basically an NOOP, so it can be used for performance
testing with applications that require directory structures. Implemented as
kernel module, instead of using FUSE, there is absolutely no overhead for
copying application data from user to kernel space while performing write or
read operations.
```
# sudo make
make -C /lib/modules/4.18.5/build M=/home/abi/lwnfs modules
make[1]: Entering directory '/usr/src/linux-headers-4.18.5'
Building modules, stage 2.
MODPOST 1 modules
make[1]: Leaving directory '/usr/src/linux-headers-4.18.5'
# insmod nullfs.ko
# mkdir /sinkhole
# mount -t nullfs none /sinkhole/
# mkdir /sinkhole/testdir
# touch /sinkhole/testdir/myfile
# echo foobar > /sinkhole/testdir/myfile
# ls -lah /sinkhole/testdir/myfile
-rw-r--r-- 1 root root 0 Nov 8 20:17 /sinkhole/testdir/myfile
# cat /sinkhole/testdir/myfile
# pv < /dev/zero > /sinkhole/testdir/myfile
11.1GiB 0:00:04 [3.85GiB/s] [ <=> ]
# cat /sinkhole/testdir/myfile
#
```
File size is preserved to work around applications that do size checks:
```
# dd if=/dev/zero of=/nullfs/DATA bs=1M count=20
20+0 records in
20+0 records out
20971520 bytes (21 MB, 20 MiB) copied, 0.00392452 s, 5.3 GB/s
# stat -c%s /nullfs/DATA
20971520
```
Reading from the files does not copy anything to userspace and is an NOOP;
makes it behave like reading from /dev/zero:
```
# dd if=/nullfs/DATA of=/tmp/REALFILE
40960+0 records in
40960+0 records out
20971520 bytes (21 MB, 20 MiB) copied, 0.0455288 s, 461 MB/s
# hexdump -C /tmp/REALFILE
00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
```
## installation
### manual installation
To install the module for the running linux kernel use:
```
# make -C /lib/modules/$(uname -r)/build M=$PWD modules_install INSTALL_MOD_DIR=kernel/fs/nullfs
# depmod
```
Running `depmod` is mandatory. Now the module can be loaded via:
```
# modprobe nullfs
```
To automatically load the module during system boot, create a configuration
file suitable for your distribution (usually located in */etc/modules-load.d*):
```
# echo nullfs > /etc/modules-load.d/nullfs.conf
```
Example entry for `/etc/fstab`, mounting the filesystem to `/nullfs`:
```
none /nullfs nullfs auto
```
### debian package
```
# apt-get install debhelper dkms dh-dkms
# dpkg-buildpackage -uc -us -rfakeroot
# dpkg -i ../nullfsvfs__amd64.deb
```
## Usage
### Keeping file data
There is the possibility to exclude certain files from being sent into the
void. If, for example, the file matching "fstab" should be kept in memory,
mount nullfs with the "write=" option.
```
# mount -t nullfs none /sinkhole/ -o write=fstab
# cp /etc/fstab /sinkhole/
# wc -l /sinkhole/fstab
14 /sinkhole/fstab
# cp /etc/passwd /sinkhole/
# wc -l /sinkhole/passwd
0 /sinkhole/passwd
```
Another option is using the sysfs interface to change the exclude string
after the module has been loaded:
```
# echo foo > /sys/fs/nullfs/exclude
```
Keep in mind that file data is kept in memory and no boundary checks are done,
so this might fill up your RAM in case you exclude big files from being
nulled.
### ACL
It is possible to set POSIX ACL attributes via `setfacl` so it appears the
filesystem supports them, they are not saved.
Works with recent linux kernels (5.x), nullfs builds fine with older kernels
(4.x, 3.x) but setting ACL information fails with "Operation not supported".
### usecases
See: [Use Cases ](https://github.com/abbbi/nullfsvfs/labels/Usecase)
The module has been used for performance testing with redis, see:
https://www.usenix.org/system/files/atc20-george.pdf
Please report usecases as regular issues!
### supported mount options
The following mount options are supported:
```
-o mode= set permissions on mount directory ( mount .. -o mode=777 )
-o uid= set uid on mount directory ( mount .. -o uid=1000 )
-o gid= set gid on mount directory ( mount .. -o gid=1000 )
-o write=fn keep data for specific file ( mount .. -o write=fstab )
```
### todos/ideas
* replace simple_statfs call with real one, show free space of a directory that
can be passed during kernel module load
* support multiple parameters for write= option
* allow regex for write= option via trace.h
* simulate xattr support?