Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/bwester/consulfs
ConsulFS is a FUSE distributed filesystem backed by a Consul Key-Value store
https://github.com/bwester/consulfs
Last synced: about 1 month ago
JSON representation
ConsulFS is a FUSE distributed filesystem backed by a Consul Key-Value store
- Host: GitHub
- URL: https://github.com/bwester/consulfs
- Owner: bwester
- License: other
- Created: 2015-07-20T14:20:39.000Z (over 9 years ago)
- Default Branch: master
- Last Pushed: 2021-04-06T16:40:11.000Z (over 3 years ago)
- Last Synced: 2024-08-03T02:05:42.488Z (4 months ago)
- Language: Go
- Homepage:
- Size: 2.02 MB
- Stars: 72
- Watchers: 4
- Forks: 11
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-consul - consulfs: ConsulFS is a FUSE distributed filesystem backed by a Consul Key-Value store
README
ConsulFS
========ConsulFS implements a FUSE filesystem that is backed by a Consul Key-Value
store. Each key in the key store is represented by a file. Read and write the
file to get and put the key's value. "/" characters in a key name are used to
break up the keys into different directories.This project should be considered alpha-quality. It works, but it might not
handle everything you can throw at it.Installation
------------
ConsulFS uses [FUSE](http://fuse.sourceforge.net/) to implement a file system as
a user-space process. Most popular Linux distributions will already include a
FUSE module, so no further packages need to be installed. OS X systems will need
to install a third-party file system package to mount FUSE volumes. [FUSE for OS
X](https://osxfuse.github.io/) is the preferred FUSE package at the time of this
writing.There are currently no binary packages available for ConsulFS, so you will need
to build the package yourself. ConsulFS is written in Go, so install the Go
toolchain available for your system. Look for a "golang" package in your
system's package manager (Linux) or in [Homebrew](http://brew.sh/) (OS X).
Binary packages can also be downloaded from [golang.org](http://golang.org/).Once FUSE and Go are installed, you can download, build, and install ConsulFS by
running the following command:$ go get github.com/bwester/consulfs/cmd/consulfs
That will create the binary `$GOPATH/bin/consulfs`, which you can call directly
or copy to your preferred location for binaries.Command line usage
------------------
The `consulfs` command is used to mount a Consul Key-Value store onto a file
system path. The basic form of the command is:$ consulfs [options] [consul_address] /mount/path
There aren't many options, but you can run `consulfs --help` to see them. The
address of a Consul agent is optional, and if you omit it, the local agent will
be used.ConsulFS runs in the foreground, where it displays all error messages. If you
interrupt the process with ^C or send it a SIGTERM or SIGINT, it will attempt to
unmount the file system before exiting. Or, if the mount point is unmounted
through other means ("umount" or "fusermount"), the process will exit.File system model
-----------------
ConsulFS represents each key in Consul as a file in the file system. The slash
character "/" is a key interpreted as a directory separator, and key names are
broken up in the straightforward way. Reads and writes on a file will cause GETs
and PUTs, respectively, to the key in order to fetch and update the key's value.Consul doesn't itself store directores; those are simply inferred from the keys.
As such, Consul is not updated when creating or removing an empty directory.For example, the key "foo/bar" is exposed to the file system as the file "bar"
in the directory "foo". When "bar" is read, a GET is performed for "foo/bar" and
its value will be read. Writes to "bar" first GET "foo/bar", change the written
bytes, then save them with a PUT to "foo/bar" containing the entire key's
contents. In the directory "foo", if you write to a new file "baz", the key
"foo/baz" will be written.ConsulFS does not currently support durable timestamps, owners, or mode
bits--there isn't enough metadata in Consul to store these! The timestamps are
faked, and owner/mode bits are fixed: attempts to change them will return an
error.Consistency
-----------
ConsulFS doesn't perform any local caching of key values (this was much easier
to write!), so every read will always get the latest value. Writes to a file
_atomically_ PUT the key's value. This might cause an occasional `write()`
syscall to fail in the presence of concurrent writers. As you might imagine,
ConsulFS is currently kinda slow when accessing remote file systems. Future
versions may introduce a data cache.Directory listings are briefly cached, for about 1 second. The access patterns
generated by FUSE repeatedly access this kind of metadata, so a small cache is
required to get any kind of usable performance.