{"id":29182454,"url":"https://github.com/jxsl13/backupfs","last_synced_at":"2025-07-01T20:05:59.908Z","repository":{"id":39633735,"uuid":"444897293","full_name":"jxsl13/backupfs","owner":"jxsl13","description":"backup-on-write filesystem abstraction layer with rollback mechanism.","archived":false,"fork":false,"pushed_at":"2025-05-19T13:58:09.000Z","size":349,"stargazers_count":3,"open_issues_count":2,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-05-19T14:46:27.499Z","etag":null,"topics":["backup","copy-on-write","filesystem","go","golang","hide-files","path-traversal-protection","rollback"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jxsl13.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-01-05T17:45:24.000Z","updated_at":"2025-05-19T13:57:56.000Z","dependencies_parsed_at":"2023-02-16T09:16:12.613Z","dependency_job_id":"ecdf828f-e36a-42f8-a514-0a888e7711bb","html_url":"https://github.com/jxsl13/backupfs","commit_stats":null,"previous_names":[],"tags_count":43,"template":false,"template_full_name":null,"purl":"pkg:github/jxsl13/backupfs","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jxsl13%2Fbackupfs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jxsl13%2Fbackupfs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jxsl13%2Fbackupfs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jxsl13%2Fbackupfs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jxsl13","download_url":"https://codeload.github.com/jxsl13/backupfs/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jxsl13%2Fbackupfs/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263029215,"owners_count":23402354,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["backup","copy-on-write","filesystem","go","golang","hide-files","path-traversal-protection","rollback"],"created_at":"2025-07-01T20:05:59.265Z","updated_at":"2025-07-01T20:05:59.887Z","avatar_url":"https://github.com/jxsl13.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# BackupFS\n\nMultiple filesystem abstraction layers working together to create a straight forward rollback mechanism for filesystem modifications with OS-dependent file paths.\nThis package provides multiple filesystem abstractions which together create a straight forward backup-on-write abstraction layer.\n\nThey require the filesystem modifications to happen via the provided structs of this package.\n\n## Requirements:\n- for Windows you require either an enabled Developer Mode or the Security Policy that allows you to create symbolic links, see [https://github.com/Schniz/fnm/issues/338](https://github.com/Schniz/fnm/issues/338)\n\n## Example Use Case\n\nMy own use case is the ability to implement the command pattern on top of a filesystem.\nThe pattern consists of a simple interface.\n\n```go\ntype Command interface {\n\tDo() error\n\tUndo() error\n}\n```\n\nA multitude of such commands allows to provision software packages (archives) and configuration files to target systems running some kind of agent software.\nUpon detection of invalid configurations or incorrect software, it is possible to rollback the last transaction.\n\nA transaction is also a command containing a list of non-transaction commands embedding and providing a `BackupFS` to its subcommands requiring to execute filesystem operations.\n\nFor all commands solely operating on the filesystem the `Undo()` mechanism consists of simply calling `BackupFS.Rollback()`\n\nFurther commands might tackle the topics of:\n\n- un/tar\n- creation of files, directories \u0026 symlinks\n- removal of files, directories \u0026 symlinks\n- download of files and writing them to the filesystem\n- rotation of persisted credentials that might not work upon testing\n\nIf you try to tackle the rollback/undo problem yourself you will see pretty fast that the rollback mechanism is a pretty complex implementation with lots of pitfalls where this approach might help you out.\n\nIf you follow the rule that **filesystem modifying commands**\n\n- creation,\n- deletion\n- or modification of files, directories and symlinks\n- creation of systemd unit files (writing service configuration)\n\nare to be strictly separated from **side effects causing commands**\n\n- creation of linux system users and groups\n- start of linux systemd services configured with the above file in the filesystem\n\nthen you will have a much easier time!\n\n## VolumeFS\n\n`VolumeFS` is a filesystem abstraction layer that hides Windows volumes from file system operations.\nIt allows to define a volume of operation like `c:` or `C:` which is then the only volume that can be accessed.\nThis abstraction layer allows to operate on filesystems with operating system independent paths.\n\n## PrefixFS\n\n`PrefixFS` forces a filesystem to have a specific prefix.\nAny attempt to escape the prefix path by directory traversal is prevented, forcing the application to stay within the designated prefix directory.\nThis prefix makes the directory basically the application's root directory.\n\n## BackupFS\n\nThe most important part of this library is `BackupFS`.\nIt is a filesystem abstraction that consists of two parts.\nA base filesystem and a backup filesystem.\nAny attempt to modify a file, directory or symlink in the base filesystem leads to the file being backed up to the backup filesystem.\n\nConsecutive file modifications are ignored, as the initial file state has already been backed up.\n\n## HiddenFS\n\nHiddenFS has a single purpose, that is to hide your backup location and prevent your application from seeing or modifying it.\nIn case you use BackupFS to backup files that are overwritten on your operating system filesystem (OsFS), you want to define multiple filesystem layers that work together to prevent you from creating a non-terminating recursion of file backups.\n\n- The zero'th layer is the underlying real filesystem, be it the OsFS, MemMapFS, etc.\n- The first layer is a VolumeFS filesystem abstraction that removes the need to provide a volume prefix for absolute file paths when accessing files on the underlying filesystem (Windows)\n- The second layer is a PrefixFS that is provided a prefix path (backup directory location) and the above instantiated filesystem (e.g. OsFS)\n- The third layer is HiddenFS which takes the backup location as path that needs hiding and wraps the first layer in itself.\n- The fourth layer is the BackupFS layer which takes the third layer as underlying filesystem to operate on (backup location is not accessible nor viewable) and the second PrefixFS layer to backup your files to.\n\nAt the end you will create something along the lines of:\n\n```go\npackage main\n\nimport (\n\t\"os\"\n\t\"path/filepath\"\n\n\t\"github.com/jxsl13/backupfs\"\n)\n\nfunc main() {\n\n\tvar (\n\t\t// first layer: abstracts away the volume prefix (on Unix the it is an empty string)\n\t\tvolume     = filepath.VolumeName(os.Args[0]) // determined from application path\n\t\tbase       = backupfs.NewVolumeFS(volume, backupfs.NewOSFS())\n\t\tbackupPath = \"/var/opt/app/backups\"\n\n\t\t// second layer: abstracts away a path prefix\n\t\tbackup = backupfs.NewPrefixFS(base, backupPath)\n\n\t\t// third layer: hides the backup location in order to prevent recursion\n\t\tmasked = backupfs.NewHiddenFS(base, backupPath)\n\n\t\t// fourth layer: backup on write filesystem with rollback\n\t\tbackupFS = backupfs.NewBackupFS(masked, backup)\n\t)\n\t// you may use backupFS at this point like the os package\n\t// except for the backupFS.Rollback() machanism which\n\t// allows you to rollback filesystem modifications.\n}\n```\n\n## Example\n\nWe create a base filesystem with an initial file in it.\nThen we define a backup filesystem as subdirectory of the base filesystem.\n\nThen we do wrap the base filesystem and the backup filesystem in the `BackupFS` wrapper and try modifying the file through the `BackupFS` file system layer which has initiall ybeen created in the base filesystem. So `BackupFS` tries to modify an already existing file leading to it being backedup. A call to `BackupFS.Rollback()` allows to rollback the filesystem modifications done with `BackupFS` back to its original state while also deleting the backup.\n\n```go\npackage main\n\nimport (\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\n\t\"github.com/jxsl13/backupfs\"\n)\n\nfunc checkErr(err error) {\n\tif err != nil {\n\t\tfmt.Println(err)\n\t\tos.Exit(1)\n\t}\n}\n\nfunc main() {\n\n\tvar (\n\t\t// base filesystem\n\t\tbaseFS   = backupfs.NewPrefixFS(backupfs.NewOSFS(), os.TempDir())\n\t\tfilePath = \"/var/opt/test.txt\"\n\t)\n\t// create an already existing file in base filesystem\n\tf, err := baseFS.Create(filePath)\n\tcheckErr(err)\n\n\tf.WriteString(\"original text\")\n\tf.Close()\n\n\t// at this point we have the base filesystem ready to be overwritten with new files\n\tvar (\n\t\t// sub directory in base filesystem as backup directory\n\t\t// where the backups should be stored\n\t\tbackup = backupfs.NewPrefixFS(baseFS, \"/var/opt/application/backup\")\n\n\t\t// backup on write filesystem\n\t\tbackupFS = backupfs.NewBackupFS(baseFS, backup)\n\t)\n\n\t// we try to override a file in the base filesystem\n\t// but in this case we use the backup on write filesystem\n\t// on top of the base filesystem.\n\tf, err = backupFS.Create(filePath)\n\tcheckErr(err)\n\tf.WriteString(\"new file content\")\n\tf.Close()\n\n\t// before we overwrite the file a backup was created\n\t// at the same path as the overwritten file was found at.\n\t// due to our backup being on a prefixedfilesystem, we can find\n\t// the backedup file at a prefixed location\n\n\tf, err = backup.Open(filePath)\n\tcheckErr(err)\n\n\tb, err := io.ReadAll(f)\n\tcheckErr(err)\n\t_ = f.Close()\n\n\tbackedupContent := string(b)\n\n\tf, err = baseFS.Open(filePath)\n\tcheckErr(err)\n\tb, err = io.ReadAll(f)\n\tcheckErr(err)\n\n\toverwrittenFileContent := string(b)\n\n\tfmt.Println(\"Overwritten file: \", overwrittenFileContent)\n\tfmt.Println(\"Backed up file  : \", backedupContent)\n\n\tdir, err := backupFS.Open(\"/var/opt/\")\n\tcheckErr(err)\n\tdefer dir.Close()\n\n\tfis, err := dir.Readdir(-1)\n\tcheckErr(err)\n\tfor _, fi := range fis {\n\t\tfmt.Println(\"Found name: \", fi.Name())\n\t}\n}\n```\n\n## TODO\n\n- Add symlink fuzz tests on os filesystem that deletes the symlink after each test.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjxsl13%2Fbackupfs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjxsl13%2Fbackupfs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjxsl13%2Fbackupfs/lists"}