{"id":38697395,"url":"https://github.com/luzhixing12345/kfs","last_synced_at":"2026-01-17T10:42:30.048Z","repository":{"id":241358624,"uuid":"760423878","full_name":"luzhixing12345/kfs","owner":"luzhixing12345","description":"基于FUSE的用户态EXT4文件系统","archived":false,"fork":false,"pushed_at":"2024-08-15T10:06:14.000Z","size":1099,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-08-15T11:37:45.829Z","etag":null,"topics":["ext4","fuse"],"latest_commit_sha":null,"homepage":"https://luzhixing12345.github.io/kfs/","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/luzhixing12345.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-02-20T11:52:26.000Z","updated_at":"2024-08-15T10:06:18.000Z","dependencies_parsed_at":"2024-08-13T19:25:14.266Z","dependency_job_id":null,"html_url":"https://github.com/luzhixing12345/kfs","commit_stats":null,"previous_names":["luzhixing12345/kfs"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/luzhixing12345/kfs","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/luzhixing12345%2Fkfs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/luzhixing12345%2Fkfs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/luzhixing12345%2Fkfs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/luzhixing12345%2Fkfs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/luzhixing12345","download_url":"https://codeload.github.com/luzhixing12345/kfs/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/luzhixing12345%2Fkfs/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28506593,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-17T10:25:30.148Z","status":"ssl_error","status_checked_at":"2026-01-17T10:25:29.718Z","response_time":85,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ext4","fuse"],"created_at":"2026-01-17T10:42:29.961Z","updated_at":"2026-01-17T10:42:30.028Z","avatar_url":"https://github.com/luzhixing12345.png","language":"C","funding_links":[],"categories":[],"sub_categories":[],"readme":"# kfs\n\n![badge.svg](https://github.com/luzhixing12345/kfs/actions/workflows/main.yaml/badge.svg)\n\nkfs 是一个基于 FUSE 的类 ext4 文件系统, 同时可以利用 Virtio(vhost-user) 协议实现虚拟机(VM)和宿主机之间高效共享文件系统.\n\n![20240731230954](https://raw.githubusercontent.com/learner-lu/picbed/master/20240731230954.png)\n\n本项目共包含四个部分\n\n- `kfs`: 文件系统, 可以挂载磁盘镜像并映射文件系统\n- `mkfs`: 磁盘格式化工具, 可以将磁盘/文件格式化为 kfs 文件系统需要的格式\n- `kfsd`: 通过 Virtio(vhost-user) 协议实现虚拟机(VM)和宿主机之间通信, 将 VM 请求转发至 host 文件系统处理\n- `kfsctl`: 命令行交互工具, 可以查看挂载的文件系统状态, 快照, 恢复等操作\n\n## 编译运行\n\n安装依赖\n\n```bash\nsudo apt-get install fuse3 libfuse3-dev pkg-config libcap-ng-dev libseccomp-dev libz-dev\n```\n\n编译得到文件系统 `src/kfs` 和磁盘格式化程序 `mkfs/mkfs` 和命令行交互工具 `kfsctl/kfsctl`\n\n```bash\nmake\n```\n\n```bash\ncd kfsd\ncargo build --release\n```\n\n创建 `disk.img` 文件(1000 MiB), 并使用 `mkfs/mkfs` 将其格式化为 ext4 文件系统格式\n\n```bash\nmake disk\n```\n\n挂载文件系统到 `tmp/` 下\n\n```bash\nmkdir tmp\nmake run\n```\n\n可以进入 `tmp/` 目录下进行操作, 例如创建/打开/读取/修改文件等等, 所有修改都会保存在 `disk.img` 中, 所有的操作日志保存在 `kfs.log` 中\n\n结束操作后使用取消挂载文件系统即可\n\n```bash\numount tmp\n```\n\n或者以调试模式挂载并执行\n\n```bash\nmake debug_run\n```\n\n测试\n\n```bash\nmake test\n```\n\n## VM 启动\n\nGuest kernel 编译时需要添加 virtiofs 支持\n\n```txt\nCONFIG_VIRTIO_FS\nCONFIG_FUSE\n```\n\n启动 virtfsd\n\n```bash\ncargo run -- --shared-dir \u003cdir\u003e --socket-path /tmp/vhostqemu\n```\n\nqemu 启动时添加 `chardev` 用于创建一个 socket(/tmp/vhostqemu) 实现通信. 创建一个 `vhost-user-fs-pci` 的 device, 添加 tag 为 myfs 用于后续挂载\n\n```bash\nqemu-system-x86_64 \\\n        -kernel \u003cyour-kernel\u003e \\\n        -drive \u003cyour-drive\u003e\n        -chardev socket,id=char0,path=/tmp/vhostqemu \\\n        -device vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=myfs \\\n        -m 4G -object memory-backend-file,id=mem,size=4G,mem-path=/dev/shm,share=on \\\n        -numa node,memdev=mem\n```\n\n```bash\nsudo mount -t virtiofs myfs \u003cmountpoint\u003e\n```\n\n\u003c!-- ## TODO\n\n- [ ] 完善测试\n- [ ] ext4 日志\n- [ ] 支持并发\n- [ ] 大文件读写\n- [ ] 目录 Hash\n- [ ] 高效的磁盘bitmap选择算法\n- [ ] 快照\n- [ ] 校验和\n- [ ] 重复数据删除\n- [ ] 压缩\n- [ ] 延迟分配 --\u003e\n\n## 文档\n\n项目技术文档见 `md-docs/`, 在线阅读: [kfs document](https://luzhixing12345.github.io/kfs/)\n\n关于细节见 impl 部分\n\n## 实现概述\n\n在远程文件系统中,**transport(传输)**和**protocol(协议)**是两个核心概念,它们共同决定了数据如何在客户端和服务器之间传输,以及如何进行通信和交互.\n\n- **Transport** 指数据在网络上传输的方式和路径.在远程文件系统中,传输层负责确保数据包能够在客户端和服务器之间可靠地传递.选择合适的传输协议取决于系统的需求,例如: TCP/IP, USB, RDMA, 通过消息传递或者共享内存等方式\n- **Protocol** 指在客户端和服务器之间进行通信的规则和格式.它定义了消息的结构、命令的含义、错误处理机制等. 例如 NFS, CIFS, SSH 等\n\n![20240731211711](https://raw.githubusercontent.com/learner-lu/picbed/master/20240731211711.png)\n\n那么虚拟化场景下有什么特别的么? 注意到所有虚拟机**共用同一个物理机**, 他们的**物理内存实际上是共享的**, 由 VMM 统一管理和分配, 也就是不需要高开销的网络通信而可以通过一种高效的**内存映射**的方式完成通信\n\n## vhost-user\n\n最知名的虚拟化平台 QEMU 采用 vhost-user 协议. vhost-user 是一种用于用户态进程与内核态虚拟化组件(如KVM/QEMU)之间进行高速数据传输的通信协议.它最常见的用途是在虚拟机和虚拟设备(如虚拟网络接口、虚拟磁盘)之间实现高效的I/O操作\n\n通常由 qemu 的 virtio-net 的实现虚拟网络通信设备作为 vhost-user 协议的 frontend 部分, 本项目实现了 vhost-user 协议的后端(backend) 部分, 与 VM 通信握手\n\n![20240731232130](https://raw.githubusercontent.com/learner-lu/picbed/master/20240731232130.png)\n\n## kfsd\n\n传统 FUSE 的访问流程如下\n\n- FUSE_INIT to creat session\n- FUSE_LOOPUP(FUSE_ROOT_ID, \"foo\") -\u003e nodeid\n- FUSE_OPEN(nodeid, O_RDONLY) -\u003e fh\n- FUSE_READ(fh, offset, \u0026buf, sizeof(buf)) -\u003e nbytes\n\n能否避免/减少与 kfsd 的通信开销? 能否避免与主机频繁的拷贝数据 from/to, `DAX` (Direct Access) 是virtiofs中的一项重要特性. 可以将数据可以直接从存储设备映射到用户空间文件区域\n\n- 将文件区域映射到 guest 的内存区域\n- 允许 guest mmap 访问内存区域\n\n启用 DAX 之后可以充分利用主机的 page cache, pte 等加速访存操作, 避免通信和数据拷贝. \n\n- FUSE_INIT to creat session\n- FUSE_LOOPUP(FUSE_ROOT_ID, \"foo\") -\u003e nodeid\n- FUSE_OPEN(nodeid, O_RDONLY) -\u003e fh\n- FUSE_SETUPMAPPING(fh, offset, len, addr)\n- Memory access to [addr, addr+len]\n\n## 参考\n\n- [Linux 文件系统(一):抽象](https://www.bilibili.com/video/BV1jM411W7jV)\n- [动手实现一个文件系统,提高自己代码能力,加深对底层的理解,探索自己的就业新可能](https://www.bilibili.com/video/BV1eV411A7gw)\n  - [好文分享:EXT文件系统机制原理详解](https://www.51cto.com/article/603104.html)\n  - [Project 06: Simple File System](https://www3.nd.edu/~pbui/teaching/cse.30341.fa19/project06.html)\n- [NTFS 是目前最先进的文件系统吗?](https://www.zhihu.com/question/20619659)\n- [libfuse](https://github.com/libfuse/libfuse)\n- [simplefs](https://github.com/sysprog21/simplefs)\n- [扒一扒 Linux 文件系统的黑历史](https://zhuanlan.zhihu.com/p/28828826)\n- [fuse-backend-rs](https://github.com/cloud-hypervisor/fuse-backend-rs)\n- [300行代码带你实现一个Linux文件系统](https://zhuanlan.zhihu.com/p/579011810)\n- [动手写一个简单的文件系统](https://www.jianshu.com/p/8966d121263b)\n- [Assignment 11: File system in libfuse](https://course.ccs.neu.edu/cs3650sp22/a11.html)\n- [文件系统](https://realwujing.github.io/linux/kernel/%E6%96%87%E4%BB%B6%E7%B3%BB%E7%BB%9F/)\n- [星星说编程](https://space.bilibili.com/50657960/channel/series)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fluzhixing12345%2Fkfs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fluzhixing12345%2Fkfs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fluzhixing12345%2Fkfs/lists"}