https://github.com/jackdoe/sfs
fuse based mysql backed file system
https://github.com/jackdoe/sfs
Last synced: 5 days ago
JSON representation
fuse based mysql backed file system
- Host: GitHub
- URL: https://github.com/jackdoe/sfs
- Owner: jackdoe
- Created: 2011-12-24T13:46:28.000Z (over 14 years ago)
- Default Branch: master
- Last Pushed: 2011-12-27T07:56:36.000Z (over 14 years ago)
- Last Synced: 2023-03-12T13:15:49.994Z (over 3 years ago)
- Language: C
- Homepage:
- Size: 98.6 KB
- Stars: 1
- Watchers: 3
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README
Awesome Lists containing this project
README
*WARNING: experimental stage, work in progress, DOES NOT USE SSL, DO NOT USE IN PRODUCTION*
very basic mysql backed fuse file system.
0: abstract
fuse based, mysql backed file system
1. requirements:
fuse, mysql, pthreads, c compiler
2. install:
download from https://github.com/jackdoe/sfs
$EDOTOR common.h #put proper mysql settings into common.h
make #DEBUG=1 if you want debug output
./sfs /mnt/fuse
3. info:
this is the table that is used:
Q("CREATE TABLE IF NOT EXISTS `entries_%s` ( \
`id` bigint(20) NOT NULL AUTO_INCREMENT, \
`path` varchar(255) NOT NULL, \
`data` longblob, \
`obj_type` int(11) NOT NULL, \
`uid` bigint(20) NOT NULL, \
`gid` bigint(20) NOT NULL, \
`mode` bigint(20) NOT NULL, \
`atime` bigint(20) NOT NULL, \
`mtime` bigint(20) NOT NULL, \
`ctime` bigint(20) NOT NULL, \
`btime` bigint(20) NOT NULL, \
PRIMARY KEY (`id`), \
UNIQUE KEY `path` (`path`))",identifier);
identifier is a copy of the mysql username (for example root)
and the table will be entries_root
table is automaticaly created at start, root directory is auto inserted as well
mysql user needs to select/update/delete/insert and maybe create if you dont have
the table created.
4. idea:
i made this as proof of concept to be used in diskless linux based routers,
where i can edit their config files remotely - for example /etc or /usr/local/etc,
with simple web based application.
5. supported calls:
sfs_oper.getattr = sfs_getattr;
sfs_oper.readdir = sfs_readdir;
sfs_oper.open = sfs_open;
sfs_oper.write = sfs_write;
sfs_oper.truncate = sfs_truncate;
sfs_oper.read = sfs_read;
sfs_oper.create = sfs_create;
sfs_oper.unlink = sfs_delete;
sfs_oper.chmod = sfs_chmod;
sfs_oper.chown = sfs_chown;
sfs_oper.statfs = sfs_statfs;
sfs_oper.mkdir = sfs_mkdir;
sfs_oper.readlink = sfs_readlink;
sfs_oper.symlink = sfs_symlink;
sfs_oper.rename = sfs_rename;
sfs_oper.rmdir = sfs_rmdir;
sfs_oper.utimens = sfs_utimens;
6. limitations:
MAX_PATH_SIZE: 256 chars
max data size: mysql's longblob 4gb
read/write is extreamly slow on big files
currently there is no locking
7. example usage:
$ mkdir ~/mysql_backed_fs
$ ./sfs ~/mysql_backed_fs/
$ cd ~/mysql_backed_fs
mysql_backed_fs $ mkdir test_directory
mysql_backed_fs $ cd test_directory/
test_directory $ ls -la
total 0
drwxr-xr-x 1 jack staff 0 Dec 27 09:24 .
drwxr-xr-x 1 jack staff 0 Dec 24 15:35 ..
test_directory $ touch empty_file
test_directory $ cat empty_file
test_directory $ echo 'not empty anymore' >> empty_file
test_directory $ cat empty_file
not empty anymore
test_directory $ cd ..
mysql_backed_fs $ ln -s test_directory/empty_file link
mysql_backed_fs $ cat link
not empty anymore
mysql_backed_fs $ ln -s test_directory link2
mysql_backed_fs $ cat link2/empty_file
not empty anymore
mysql_backed_fs $ cd ..
$ dd if=/dev/random of=blob.dd bs=1m count=1 && md5 blob.dd
1+0 records in
1+0 records out
1048576 bytes transferred in 0.080215 secs (13072073 bytes/sec)
MD5 (blob.dd) = 7d84c26f72aa5440c778014b1c9d23fe
$ cp blob.dd mysql_backed_fs/ && md5 mysql_backed_fs/blob.dd
MD5 (mysql_backed_fs/blob.dd) = 7d84c26f72aa5440c778014b1c9d23fe
$ cd mysql_backed_fs
as you can see write is very very slow
mysql_backed_fs $ dd if=/dev/zero of=./speed.blob.dd bs=1m count=5
5+0 records in
5+0 records out
5242880 bytes transferred in 14.620257 secs (358604 bytes/sec)
reading is fine on not very large files tho, because it uses SUBSTRING() and
gets only requested offset+size from the mysql
mysql_backed_fs $ time cat speed.blob.dd > /dev/null
real 0m0.005s
user 0m0.001s
sys 0m0.004s
mysql does not have a way for partial updates on blob fields so
there is download/modify (offset+size - usually 65k)/push the whole file and
the speed drops like a rock in water :)
mysql_backed_fs$ dd if=/dev/zero of=./speed.blob.dd bs=65k count=1
1+0 records in
1+0 records out
66560 bytes transferred in 0.004575 secs (14548589 bytes/sec)
mysql_backed_fs$ dd if=/dev/zero of=./speed.blob.dd bs=256k count=1
1+0 records in
1+0 records out
262144 bytes transferred in 0.015075 secs (17389082 bytes/sec)
mysql_backed_fs$ dd if=/dev/zero of=./speed.blob.dd bs=512k count=1
1+0 records in
1+0 records out
524288 bytes transferred in 0.046395 secs (11300513 bytes/sec)
mysql_backed_fs$ dd if=/dev/zero of=./speed.blob.dd bs=1024k count=1
1+0 records in
1+0 records out
1048576 bytes transferred in 0.301245 secs (3480808 bytes/sec)
mysql_backed_fs$ dd if=/dev/zero of=./speed.blob.dd bs=2048k count=1
1+0 records in
1+0 records out
2097152 bytes transferred in 1.876782 secs (1117419 bytes/sec)
but for small files it is fine