{"id":23473248,"url":"https://github.com/bazil/zipfs","last_synced_at":"2025-04-14T18:42:38.657Z","repository":{"id":24453646,"uuid":"27856354","full_name":"bazil/zipfs","owner":"bazil","description":"Example FUSE filesystem that serves a Zip archive","archived":false,"fork":false,"pushed_at":"2019-12-21T04:39:14.000Z","size":16,"stargazers_count":118,"open_issues_count":3,"forks_count":17,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-03-28T07:11:20.492Z","etag":null,"topics":["fuse","fuse-filesystem"],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bazil.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2014-12-11T05:52:57.000Z","updated_at":"2025-02-02T14:16:10.000Z","dependencies_parsed_at":"2022-09-14T04:41:28.097Z","dependency_job_id":null,"html_url":"https://github.com/bazil/zipfs","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bazil%2Fzipfs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bazil%2Fzipfs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bazil%2Fzipfs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bazil%2Fzipfs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bazil","download_url":"https://codeload.github.com/bazil/zipfs/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248938429,"owners_count":21186408,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["fuse","fuse-filesystem"],"created_at":"2024-12-24T17:17:39.450Z","updated_at":"2025-04-14T18:42:38.614Z","avatar_url":"https://github.com/bazil.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# `zipfs` -- example FUSE filesystem\n\n`zipfs` is an example of a [`bazil.org/fuse`](http://bazil.org/fuse/)\nfilesystem that serves a Zip archive:\n\n``` console\n$ unzip -v archive.zip\nArchive:  archive.zip\n Length   Method    Size  Cmpr    Date    Time   CRC-32   Name\n--------  ------  ------- ---- ---------- ----- --------  ----\n       0  Stored        0   0% 2014-12-11 04:03 00000000  buried/\n       0  Stored        0   0% 2014-12-11 04:03 00000000  buried/deep/\n       5  Stored        5   0% 2014-12-11 04:03 2efcceec  buried/deep/loot\n      13  Stored       13   0% 2014-12-11 04:03 f4247453  greeting\n--------          -------  ---                            -------\n      18               18   0%                            4 files\n$ zipfs archive.zip mnt \u0026\n$ tree mnt\nmnt\n├── buried\n│   └── deep\n│       └── loot\n└── greeting\n\n2 directories, 2 files\n$ cat mnt/greeting\nhello, world\n```\n\n# FUSE\n\n[FUSE](https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/Documentation/filesystems/fuse.txt)\n(Filesystem In Userpace) is a Linux kernel filesystem that sends the\nincoming requests over a file descriptor to userspace. Historically,\nthese have been served with a\n[C library of the same name](http://fuse.sourceforge.net/), but\nultimately FUSE is just a protocol. Since then, the protocol has been\nimplemented for other platforms such as OS X, FreeBSD and OpenBSD.\n\n[bazil.org/fuse](http://bazil.org/fuse) is a reimplementation of that\nprotocol in pure Go.\n\n\n# Structure of Unix filesystems\n\nUnix filesystems consist of *inodes* (\"index nodes\"). These nodes are\nfiles, directories, etc. *Directories* contain *directory entries*\n(*dirent*) that point to child *inodes*. A directory entry is\nidentified by its name, and carries very little metadata. The *inode*\nmanages both the metadata (including things like access control) and\nthe content of the file.\n\nOpen files are identified in userspace with *file descriptors*, which\nare just safe references to kernel objects known as *handles*.\n\n\n# Go API\n\nOur FUSE library is split into two parts. The low-level protocol is in\n[`bazil.org/fuse`](http://godoc.org/bazil.org/fuse) while the\nhigher-level, optional, state machine keeping track of object\nlifetimes is\n[`bazil.org/fuse/fs`](http://godoc.org/bazil.org/fuse/fs).\n\nEach file system has a *root entry*. The interface\n[`fs.FS`](http://godoc.org/bazil.org/fuse/fs#FS) has a method\n[`Root`](http://godoc.org/bazil.org/fuse/fs#FS.Root) that returns an\n[`fs.Node`](http://godoc.org/bazil.org/fuse/fs#Node).\n\nTo access a file (see its metadata, open it, etc), the kernel looks it\nup by name by sending a\n[`fuse.LookupRequest`](http://godoc.org/bazil.org/fuse#LookupRequest)\nto the FUSE server, stating the parent directory and basename. This\nrequest is served by a\n[`Lookup`](http://godoc.org/bazil.org/fuse/fs#NodeRequestLookuper)\nmethod on the parent\n[`fs.Node`](http://godoc.org/bazil.org/fuse/fs#Node). The method\nreturns an [`fs.Node`](http://godoc.org/bazil.org/fuse/fs#Node), and\nthe result is cached in the kernel and reference counted. Dropping a\ncache entry sends a\n[`ForgetRequest`](http://godoc.org/bazil.org/fuse#ForgetRequest), and\nwhen the reference count reaches zero,\n[`Forget`](http://godoc.org/bazil.org/fuse/fs#NodeForgetter) gets\ncalled.\n\nFiles are renamed with\n[`Rename`](http://godoc.org/bazil.org/fuse/fs#NodeRenamer), deleted\nwith [`Remove`](http://godoc.org/bazil.org/fuse/fs#NodeRemover), and\nso on.\n\nKernel file *handles* are created for example by opening a file.\nOpening an existing file sends an\n[`OpenRequest`](http://godoc.org/bazil.org/fuse#OpenRequest), you\nguessed it, served by\n[`Open`](http://godoc.org/bazil.org/fuse/fs#NodeOpener). All methods\ncreating new handles return a\n[`Handle`](http://godoc.org/bazil.org/fuse/fs#Handle). Handles are\nclosed by a combination of\n[`Flush`](http://godoc.org/bazil.org/fuse/fs#HandleFlusher) and\n[`Release`](http://godoc.org/bazil.org/fuse/fs#HandleReleaser).\n\nThe default [`Open`](http://godoc.org/bazil.org/fuse/fs#NodeOpener)\naction, if the method is not implemented, is to use the\n[`fs.Node`](http://godoc.org/bazil.org/fuse/fs#Node) also as a\n[`Handle`](http://godoc.org/bazil.org/fuse/fs#Handle); this tends to\nwork well for stateless read-only files.\n\nReads from a [`Handle`](http://godoc.org/bazil.org/fuse/fs#Handle) are\nserved by [`Read`](http://godoc.org/bazil.org/fuse/fs#HandleReader),\nwrites with\n[`Write`](http://godoc.org/bazil.org/fuse/fs#HandleWriter), and apart\nfrom all the extra data available these look similar to\n[`io.ReaderAt`](http://golang.org/pkg/io/#ReaderAt) and\n[`io.WriterAt`](http://golang.org/pkg/io/#WriterAt). Note that file\nsize changes via\n[`Setattr`](http://godoc.org/bazil.org/fuse/fs#NodeSetattrer), not\nbased on [`Write`](http://godoc.org/bazil.org/fuse/fs#HandleWriter),\nand [`Attr`](http://godoc.org/bazil.org/fuse/fs#Node) needs to return\nthe correct [`Size`](http://godoc.org/bazil.org/fuse#Attr.Size).\n\nListing a directory happens by reading an open file handle that is a\ndirectory. Instead of file contents, the read returns marshaled\ndirectory entries. The\n[`ReadDir`](http://godoc.org/bazil.org/fuse/fs#HandleReadDirer) method\nimplements a slightly higher-level API, where you return a slice of\ndirectory entries.\n\nAnd so on. Learning to write a file system requires a decent\nunderstanding of the kernel data structures and their state changes\non an abstract level, but the actual Go parts of it are quite simple.\nSo let's dive into the code.\n\n\n# `zipfs`\n\nAs our example project, we'll write a filesystem that shows a\nread-only view of the contents of a\n[Zip archive](http://golang.org/pkg/archive/zip/).\n\nThe full source code is available at\nhttps://github.com/bazil/zipfs\n\n## Skeleton\n\nLet's start with a skeleton with argument parsing:\n\n``` go\npackage main\n\nimport (\n\t\"archive/zip\"\n\t\"flag\"\n\t\"fmt\"\n\t\"io\"\n\t\"log\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"strings\"\n\n\t\"bazil.org/fuse\"\n\t\"bazil.org/fuse/fs\"\n)\n\n// We assume the zip file contains entries for directories too.\n\nvar progName = filepath.Base(os.Args[0])\n\nfunc usage() {\n\tfmt.Fprintf(os.Stderr, \"Usage of %s:\\n\", progName)\n\tfmt.Fprintf(os.Stderr, \"  %s ZIP MOUNTPOINT\\n\", progName)\n\tflag.PrintDefaults()\n}\n\nfunc main() {\n\tlog.SetFlags(0)\n\tlog.SetPrefix(progName + \": \")\n\n\tflag.Usage = usage\n\tflag.Parse()\n\n\tif flag.NArg() != 2 {\n\t\tusage()\n\t\tos.Exit(2)\n\t}\n\tpath := flag.Arg(0)\n\tmountpoint := flag.Arg(1)\n\tif err := mount(path, mountpoint); err != nil {\n\t\tlog.Fatal(err)\n\t}\n}\n```\n\nMounting is a bit cumbersome due to OSXFUSE behaving very differently\nfrom Linux; there are several stages where errors may show up.\n\n``` go\nfunc mount(path, mountpoint string) error {\n\tarchive, err := zip.OpenReader(path)\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer archive.Close()\n\n\tc, err := fuse.Mount(mountpoint)\n\tif err != nil {\n\t\treturn err\n\t}\n\tdefer c.Close()\n\n\tfilesys := \u0026FS{\n\t\tarchive: \u0026archive.Reader,\n\t}\n\tif err := fs.Serve(c, filesys); err != nil {\n\t\treturn err\n\t}\n\n\t// check if the mount process has an error to report\n\t\u003c-c.Ready\n\tif err := c.MountError; err != nil {\n\t\treturn err\n\t}\n\n\treturn nil\n}\n```\n\n## Filesystem\n\nOn to the actual file system. We just hold a pointer to the zip\narchive:\n\n``` go\ntype FS struct {\n\tarchive *zip.Reader\n}\n```\n\nAnd we need to provide the `Root` method:\n\n``` go\nvar _ fs.FS = (*FS)(nil)\n\nfunc (f *FS) Root() (fs.Node, fuse.Error) {\n\tn := \u0026Dir{\n\t\tarchive: f.archive,\n\t}\n\treturn n, nil\n}\n```\n\n## Directories\n\nZip files contain a list of files, but typical zip archivers include\nentries for the directories, with a name ending in a slash. We rely on\nthis behavior later.\n\nLet's define our `Dir` type, and implement the mandatory `Attr`\nmethod. We use the `*zip.File` to serve directory metadata.\n\n``` go\ntype Dir struct {\n\tarchive *zip.Reader\n\t// nil for the root directory, which has no entry in the zip\n\tfile *zip.File\n}\n\nvar _ fs.Node = (*Dir)(nil)\n\nfunc zipAttr(f *zip.File) fuse.Attr {\n\treturn fuse.Attr{\n\t\tSize:   f.UncompressedSize64,\n\t\tMode:   f.Mode(),\n\t\tMtime:  f.ModTime(),\n\t\tCtime:  f.ModTime(),\n\t\tCrtime: f.ModTime(),\n\t}\n}\n\nfunc (d *Dir) Attr() fuse.Attr {\n\tif d.file == nil {\n\t\t// root directory\n\t\treturn fuse.Attr{Mode: os.ModeDir | 0755}\n\t}\n\treturn zipAttr(d.file)\n}\n```\n\n## Directory entry lookup\n\nFor our filesystem to contain anything useful, we need to be able to\nfind entries by name. We just iterate over the zip entries, matching\npaths:\n\n``` go\nvar _ = fs.NodeRequestLookuper(\u0026Dir{})\n\nfunc (d *Dir) Lookup(req *fuse.LookupRequest, resp *fuse.LookupResponse, intr fs.Intr) (fs.Node, fuse.Error) {\n\tpath := req.Name\n\tif d.file != nil {\n\t\tpath = d.file.Name + path\n\t}\n\tfor _, f := range d.archive.File {\n\t\tswitch {\n\t\tcase f.Name == path:\n\t\t\tchild := \u0026File{\n\t\t\t\tfile: f,\n\t\t\t}\n\t\t\treturn child, nil\n\t\tcase f.Name[:len(f.Name)-1] == path \u0026\u0026 f.Name[len(f.Name)-1] == '/':\n\t\t\tchild := \u0026Dir{\n\t\t\t\tarchive: d.archive,\n\t\t\t\tfile:    f,\n\t\t\t}\n\t\t\treturn child, nil\n\t\t}\n\t}\n\treturn nil, fuse.ENOENT\n}\n```\n\n## Files\n\nOur `Lookup` above returned `File` types when the matched entry did\nnot end in a slash. Let's define type `File`, using the same `zipAttr`\nhelper as for directories:\n\n``` go\ntype File struct {\n\tfile *zip.File\n}\n\nvar _ fs.Node = (*File)(nil)\n\nfunc (f *File) Attr() fuse.Attr {\n\treturn zipAttr(f.file)\n}\n```\n\nFiles are not very useful unless you can open them:\n\n``` go\nvar _ = fs.NodeOpener(\u0026File{})\n\nfunc (f *File) Open(req *fuse.OpenRequest, resp *fuse.OpenResponse, intr fs.Intr) (fs.Handle, fuse.Error) {\n\tr, err := f.file.Open()\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\t// individual entries inside a zip file are not seekable\n\tresp.Flags |= fuse.OpenNonSeekable\n\treturn \u0026FileHandle{r: r}, nil\n}\n```\n\n## Handles\n\n\n``` go\ntype FileHandle struct {\n\tr io.ReadCloser\n}\n\nvar _ fs.Handle = (*FileHandle)(nil)\n```\n\nWe hold an \"open file\" inside our handle. In this case, it's just a\nhelper type in `archive/zip`, but in another filesystem this might be\na `*os.File`, a network connection, or such. We should be careful to\nclose them:\n\n``` go\nvar _ fs.HandleReleaser = (*FileHandle)(nil)\n\nfunc (fh *FileHandle) Release(req *fuse.ReleaseRequest, intr fs.Intr) fuse.Error {\n\treturn fh.r.Close()\n}\n```\n\nAnd then let's handle actual `Read` operations:\n\n``` go\nvar _ = fs.HandleReader(\u0026FileHandle{})\n\nfunc (fh *FileHandle) Read(req *fuse.ReadRequest, resp *fuse.ReadResponse, intr fs.Intr) fuse.Error {\n\t// We don't actually enforce Offset to match where previous read\n\t// ended. Maybe we should, but that would mean'd we need to track\n\t// it. The kernel *should* do it for us, based on the\n\t// fuse.OpenNonSeekable flag.\n\tbuf := make([]byte, req.Size)\n\tn, err := fh.r.Read(buf)\n\tresp.Data = buf[:n]\n\treturn err\n}\n```\n\n## Readdir\n\nAt this point, our files are accessible by `cat` and such, but you\nneed to know their names. Let's add support for `ReadDir`:\n\n``` go\nvar _ = fs.HandleReadDirer(\u0026Dir{})\n\nfunc (d *Dir) ReadDir(intr fs.Intr) ([]fuse.Dirent, fuse.Error) {\n\tprefix := \"\"\n\tif d.file != nil {\n\t\tprefix = d.file.Name\n\t}\n\n\tvar res []fuse.Dirent\n\tfor _, f := range d.archive.File {\n\t\tif !strings.HasPrefix(f.Name, prefix) {\n\t\t\tcontinue\n\t\t}\n\t\tname := f.Name[len(prefix):]\n\t\tif name == \"\" {\n\t\t\t// the dir itself, not a child\n\t\t\tcontinue\n\t\t}\n\t\tif strings.ContainsRune(name[:len(name)-1], '/') {\n\t\t\t// contains slash in the middle -\u003e is in a deeper subdir\n\t\t\tcontinue\n\t\t}\n\t\tvar de fuse.Dirent\n\t\tif name[len(name)-1] == '/' {\n\t\t\t// directory\n\t\t\tname = name[:len(name)-1]\n\t\t\tde.Type = fuse.DT_Dir\n\t\t}\n\t\tde.Name = name\n\t\tres = append(res, de)\n\t}\n\treturn res, nil\n}\n```\n\n# Testing zipfs\n\nPrepare a zip file:\n\n``` console\n$ mkdir -p data/buried/deep\n$ echo hello, world \u003edata/greeting\n$ echo gold \u003edata/buried/deep/loot\n$ ( cd data \u0026\u0026 zip -r -q ../archive.zip . )\n```\n\nMount it:\n\n``` console\n$ mkdir mnt\n$ zipfs archive.zip mnt \u0026\n```\n\nLookup directory entries:\n\n``` console\n$ ls -ld mnt/greeting\n-rw-r--r-- 1 root root 13 Dec 11  2014 mnt/greeting\n$ ls -ld mnt/buried\ndrwxr-xr-x 1 root root 0 Dec 11  2014 mnt/buried\n```\n\nRead file contents:\n\n``` console\n$ cat mnt/greeting\nhello, world\n$ cat mnt/buried/deep/loot\ngold\n```\n\nReaddir (the \"total 0\" is not correct, but that doesn't matter):\n\n``` console\n$ ls -l mnt\ntotal 0\ndrwxr-xr-x 1 root root  0 Dec 11  2014 buried\n-rw-r--r-- 1 root root 13 Dec 11  2014 greeting\n$ ls -l mnt/buried\ntotal 0\ndrwxr-xr-x 1 root root 0 Dec 11  2014 deep\n```\n\nUnmount (for OS X, use `umount mnt`):\n\n``` console\n$ fusermount -u mnt\n```\n\nThat's it! For a longer and more featureful examples to read, see\nhttps://github.com/bazil/bolt-mount\n([screencast of a code walkthrough](http://eagain.net/talks/bolt-mount/))\nand all of the\n[projects importing fuse](http://godoc.org/bazil.org/fuse?importers).\n\n# Resources\n\n- [Bazil](http://bazil.org/) is a distributed file system designed for\n  single-person disconnected operation. It lets you share your files\n  across all your computers, with or without cloud services.\n\n- [FUSE](https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/Documentation/filesystems/fuse.txt)\n  is a Linux kernel filesystem that makes calls to userspace to serve\n  filesystem content.\n\n- Confusingly also known as [FUSE](http://fuse.sourceforge.net/) is\n  the C library for implementing userspace FUSE filesystems.\n\n- [bazil.org/fuse](http://bazil.org/fuse) is a Go library for writing\n  filesystems. See also GoDoc for\n  [`fuse`](http://godoc.org/bazil.org/fuse) and\n  [`fuse/fs`](http://godoc.org/bazil.org/fuse/fs)\n\n- [OSXFUSE](https://osxfuse.github.io/) is a FUSE kernel\n  implementation for OS X.\n\n- [`bolt-mount`](https://github.com/bazil/bolt-mount) is a more\n  comprehensive example filesystem, including write operations. See\n  also a\n  [screencast of a code walkthrough](http://eagain.net/talks/bolt-mount/).\n\n- [*Writing a file system in Go*](http://bazil.org/talks/2013-06-10-la-gophers/)\n  is an earlier talk that explains FUSE a bit more.\n\n- FUSE questions are welcome on the\n  [bazil-dev Google Group](https://groups.google.com/forum/#!forum/bazil-dev)\n  or on IRC channel\n  [#go-nuts on irc.freenode.net](irc:irc.freenode.net/go-nuts).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbazil%2Fzipfs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbazil%2Fzipfs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbazil%2Fzipfs/lists"}