{"id":15003682,"url":"https://github.com/sysprog21/simplefs","last_synced_at":"2025-04-04T16:16:10.366Z","repository":{"id":43086430,"uuid":"237059910","full_name":"sysprog21/simplefs","owner":"sysprog21","description":"A simple native file system for Linux kernel","archived":false,"fork":false,"pushed_at":"2024-08-20T13:53:02.000Z","size":98,"stargazers_count":395,"open_issues_count":5,"forks_count":94,"subscribers_count":21,"default_branch":"master","last_synced_at":"2025-03-28T15:06:55.408Z","etag":null,"topics":["filesystem","kernel","linux","linux-kernel","linux-vfs","minimalist","native-file-system"],"latest_commit_sha":null,"homepage":"","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sysprog21.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-01-29T19:03:19.000Z","updated_at":"2025-03-27T23:02:17.000Z","dependencies_parsed_at":"2022-08-12T10:11:41.244Z","dependency_job_id":"2c0d3f79-8088-4ab1-b5aa-1f99085a711d","html_url":"https://github.com/sysprog21/simplefs","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sysprog21%2Fsimplefs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sysprog21%2Fsimplefs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sysprog21%2Fsimplefs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sysprog21%2Fsimplefs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sysprog21","download_url":"https://codeload.github.com/sysprog21/simplefs/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247208181,"owners_count":20901570,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["filesystem","kernel","linux","linux-kernel","linux-vfs","minimalist","native-file-system"],"created_at":"2024-09-24T19:00:17.212Z","updated_at":"2025-04-04T16:16:10.349Z","avatar_url":"https://github.com/sysprog21.png","language":"C","readme":"# simplefs - a simple file system for Linux\n\nThe file system \"simplefs\" serves as a helpful tool for understanding the basics\nof the Linux Virtual File System (VFS) and fs basics. Linux VFS accommodates\nmultiple file systems, with the kernel handling the bulk of the operations while\ndelegating file system-specific tasks to individual file systems via handlers.\nInstead of directly invoking functions, the kernel employs various operation\ntables. These tables are collections of handlers for each operation, essentially\nstructures comprised of function pointers for handlers/callbacks.\n\nSuper block operations are established at the time of mounting. The operation\ntables for inodes and files are set when the inode is accessed. The initial step\nbefore accessing an inode involves a lookup process. The inode for a file is\nidentified by invoking the lookup handler of the parent inode.\n\n## Features\n\n* Directories: create, remove, list, rename;\n* Regular files: create, remove, read/write (through page cache), rename;\n* Hard/Symbolic links (also symlink or soft link): create, remove, rename;\n* No extended attribute support\n\n## Prerequisites\n\nInstall linux kernel header in advance.\n```shell\n$ sudo apt install linux-headers-$(uname -r)\n```\n\n## Build and Run\n\nYou can build the kernel module and tool with `make`.\nGenerate test image via `make test.img`, which creates a zeroed file of 50 MiB.\n\nYou can then mount this image on a system with the simplefs kernel module installed.\nLet's test kernel module:\n```shell\n$ sudo insmod simplefs.ko\n```\n\nCorresponding kernel message:\n```\nsimplefs: module loaded\n```\n\nGenerate test image by creating a zeroed file of 50 MiB. We can then mount\nthis image on a system with the simplefs kernel module installed.\n```shell\n$ mkdir -p test\n$ dd if=/dev/zero of=test.img bs=1M count=50\n$ ./mkfs.simplefs test.img\n$ sudo mount -o loop -t simplefs test.img test\n```\n\nYou shall get the following kernel messages:\n```\nsimplefs: '/dev/loop?' mount success\n```\nHere `/dev/loop?` might be `loop1`, `loop2`, `loop3`, etc.\n\nPerform regular file system operations: (as root)\n```shell\n$ echo \"Hello World\" \u003e test/hello\n$ cat test/hello\n$ ls -lR\n```\n\nRemove kernel mount point and module:\n```shell\n$ sudo umount test\n$ sudo rmmod simplefs\n```\n\n## Design\n\nAt present, simplefs only provides straightforward features.\n\n### Partition layout\n```\n+------------+-------------+-------------------+-------------------+-------------+\n| superblock | inode store | inode free bitmap | block free bitmap | data blocks |\n+------------+-------------+-------------------+-------------------+-------------+\n```\nEach block is 4 KiB large.\n\n### Superblock\nThe superblock, located at the first block of the partition (block 0), stores\nthe partition's metadata. This includes the total number of blocks, the total\nnumber of inodes, and the counts of free inodes and blocks.\n\n### Inode store\nThis section contains all the inodes of the partition, with the maximum number\nof inodes being equal to the number of blocks in the partition. Each inode\noccupies 72 bytes of data, encompassing standard information such as the file\nsize and the number of blocks used, in addition to a simplefs-specific field\nnamed `ei_block`. This field, `ei_block`, serves different purposes depending\non the type of file:\n  - For a directory, it contains the list of files within that directory.\n    A directory can hold a maximum of 40,920 files, with filenames restricted\n    to a maximum of 255 characters to ensure they fit within a single block.\n  ```\n  inode\n  +-----------------------+\n  | i_mode = IFDIR | 0755 |            block 123\n  | ei_block = 123    ----|--------\u003e  +----------------+\n  | i_size = 4 KiB        |         0 | ee_block  = 0  |\n  | i_blocks = 1          |           | ee_len    = 8  |      block 84\n  +-----------------------+           | ee_start  = 84 |---\u003e  +-----------+\n                                      |----------------|    0 | 24 (foo)  |\n                                    1 | ee_block  = 8  |      |-----------|\n                                      | ee_len    = 8  |    1 | 45 (bar)  |\n                                      | ee_start  = 16 |      |-----------|\n                                      |----------------|      | ...       |\n                                      | ...            |      |-----------|\n                                      |----------------|   14 | 0         |\n                                  341 | ee_block  = 0  |      +-----------+\n                                      | ee_len    = 0  |\n                                      | ee_start  = 0  |\n                                      +----------------+\n\n  ```\n  - For a file, it lists the extents that hold the actual data of the file.\n    Given that block IDs are stored as values of `sizeof(struct simplefs_extent)`\n    bytes, a single block can accommodate up to 341 links. This limitation\n    restricts the maximum size of a file to approximately 10.65 MiB (10,912 KiB).\n  ```\n  inode                                                \n  +-----------------------+                           \n  | i_mode = IFDIR | 0644 |          block 93       \n  | ei_block = 93     ----|------\u003e  +----------------+      \n  | i_size = 10 KiB       |       0 | ee_block  = 0  |     \n  | i_blocks = 25         |         | ee_len    = 8  |      extent 94 \n  +-----------------------+         | ee_start  = 94 |---\u003e +--------+\n                                    |----------------|     |        |     \n                                  1 | ee_block  = 8  |     +--------+\n                                    | ee_len    = 8  |      extent 99\n                                    | ee_start  = 99 |---\u003e +--------+ \n                                    |----------------|     |        |\n                                  2 | ee_block  = 16 |     +--------+\n                                    | ee_len    = 8  |      extent 66 \n                                    | ee_start  = 66 |---\u003e +--------+\n                                    |----------------|     |        |\n                                    | ...            |     +--------+\n                                    |----------------|  \n                                341 | ee_block  = 0  | \n                                    | ee_len    = 0  |\n                                    | ee_start  = 0  |\n                                    +----------------+\n  ```\n\n### Extent support\nAn extent spans consecutive blocks; therefore, we allocate consecutive disk blocks\nfor it in a single operation. It is defined by `struct simplefs_extent`, which\ncomprises three members:\n- `ee_block`: the first logical block that the extent covers.\n- `ee_len`: the number of blocks the extent covers.\n- `ee_start`: the first physical block that the extent covers.\"\n\n```\nstruct simplefs_extent\n  +----------------+                           \n  | ee_block =  0  |    \n  | ee_len   =  200|              extent\n  | ee_start =  12 |-----------\u003e +---------+\n  +----------------+    block 12 |         |\n                                 +---------+\n                              13 |         |\n                                 +---------+\n                                 | ...     |\n                                 +---------+\n                             211 |         |\n                                 +---------+\n```\n\n### journalling support\n\nSimplefs now includes support for an external journal device, leveraging the journaling block device (jbd2) subsystem in the Linux kernel. This enhancement improves the file system's resilience by maintaining a log of changes, which helps prevent corruption and facilitates recovery in the event of a crash or power failure.\n\n\nThe journaling support in simplefs is implemented using the jbd2 subsystem, which is a widely-used journaling layer in Linux. Currently, simplefs primarily stores the journal-related information in an external journal device.\n\nFor a detailed introduction to journaling, please refer to these two websites:\n[Journal(jbd2) document](https://www.kernel.org/doc/html/latest/filesystems/ext4/journal.html)\n[Journal(jbd2) api](https://docs.kernel.org/filesystems/journalling.html) \n\nExternal journal device disk layout:\n\n+--------------------+------------------+---------------------------+--------------+\n| Journal Superblock | Descriptor Block | Metadata/Data ( modified ) | Commit Block |\n+--------------------+------------------+---------------------------+--------------+\n\nHint:\nEach transaction starts with a descriptor block, followed by several metadata blocks or data blocks, and ends with a commit block. Every modified metadata (such as inode, bitmap, etc.) occupies its own block. Currently, simplefs primarily records \"extent\" metadata.\n\n\nHow to Enable Journaling in simplefs: \n\nStep 1: Create the Journal Disk Image\nTo create an 8MB disk image for the journal, use the following make command:\n\nNote: \nAssuming an 8 MB size for the external journal device, which is an arbitrary choice for now, I will set the journal block length to a fixed 2048, calculated by dividing the device size by the block size (4096 bytes).\n\n```shell\n$ make journal\n```\n\nStep 2:  Make sure you've loaded the SimpleFS Kernel Module\n\n```shell\n$ insmod simplefs/simplefs.ko\n```\n\nStep 3: Setup the Loop Device for the Journal\nFind an available loop device and associate it with the journal image:\n\n``` shell\n$ loop_device=$(losetup -f)\n$ losetup $loop_device /simplefs/journal.img\n```\n\nYou shall get the following kernel messages:\n```\nloop0: detected capacity change from 0 to 16384\n```\n\nStep 4: Mount the SimpleFS File System with the External Journal\nMount the SimpleFS file system along with the external journal device using the following command:\n\n```shell\nmount -o loop,rw,owner,group,users,journal_path=\"$loop_device\" -t simplefs /simplefs/test.img /test\n```\n\nCorresponding kernel message:\n```\nloop1: detected capacity change from 0 to 409600\nsimplefs: simplefs_parse_options: parsing options 'owner,group,journal_path=/dev/loop0'\nsimplefs: '/dev/loop1' mount success\n```\n\nCurrent Limitations and Known Issues\n\n1. External Journal Device Size:\n\n- The exact size of the external journal device cannot be determined. As a temporary solution, the size is set by dividing the device size by the block size, with the external journal device size fixed at 8 MB.\n\n2. Metadata Recording:\n\n- At present, only \"extent\" metadata is recorded. In the future, additional metadata such as \"super block\" and inode metadata can be included.\n\n3. Implementation of External Journal Device:\n\n- Only the external journal device is implemented. Future improvements can include the use of an internal journal (inode journal). However, this will require the addition of a bmap function and appropriate adjustments to the disk partition during mkfs.\n\n## License\n\n`simplefs` is released under the BSD 2 clause license. Use of this source code\nis governed by a BSD-style license that can be found in the LICENSE file.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsysprog21%2Fsimplefs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsysprog21%2Fsimplefs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsysprog21%2Fsimplefs/lists"}