{"id":16887464,"url":"https://github.com/vbatts/go-mtree","last_synced_at":"2025-05-15T10:07:31.392Z","repository":{"id":8237162,"uuid":"54672347","full_name":"vbatts/go-mtree","owner":"vbatts","description":"File systems verification utility and library, in likeness of mtree(8)","archived":false,"fork":false,"pushed_at":"2025-04-14T18:35:54.000Z","size":7741,"stargazers_count":80,"open_issues_count":19,"forks_count":16,"subscribers_count":8,"default_branch":"main","last_synced_at":"2025-05-14T22:01:59.089Z","etag":null,"topics":["filesystem","mtree","tar","verification"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vbatts.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":["vbatts"]}},"created_at":"2016-03-24T20:32:59.000Z","updated_at":"2025-04-06T11:58:50.000Z","dependencies_parsed_at":"2024-03-14T18:53:37.000Z","dependency_job_id":"8054c517-b2ff-45be-88bd-6b870df08c42","html_url":"https://github.com/vbatts/go-mtree","commit_stats":{"total_commits":267,"total_committers":13,"mean_commits":20.53846153846154,"dds":"0.30711610486891383","last_synced_commit":"12e242c268e0dfcd3df0d3aec734bb4a6f792b65"},"previous_names":[],"tags_count":16,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vbatts%2Fgo-mtree","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vbatts%2Fgo-mtree/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vbatts%2Fgo-mtree/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vbatts%2Fgo-mtree/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vbatts","download_url":"https://codeload.github.com/vbatts/go-mtree/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254319720,"owners_count":22051073,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["filesystem","mtree","tar","verification"],"created_at":"2024-10-13T16:44:02.351Z","updated_at":"2025-05-15T10:07:26.380Z","avatar_url":"https://github.com/vbatts.png","language":"Go","funding_links":["https://github.com/sponsors/vbatts"],"categories":[],"sub_categories":[],"readme":"# go-mtree\n\n[![Go](https://github.com/vbatts/go-mtree/actions/workflows/go.yml/badge.svg)](https://github.com/vbatts/go-mtree/actions/workflows/go.yml)\n[![Go Report Card](https://goreportcard.com/badge/github.com/vbatts/go-mtree)](https://goreportcard.com/report/github.com/vbatts/go-mtree)\n\n`mtree` is a filesystem hierarchy validation tooling and format.\nThis is a library and simple cli tool for [mtree(8)][mtree(8)] support.\n\nWhile the traditional `mtree` cli utility is primarily on BSDs (FreeBSD,\nopenBSD, etc), even broader support for the `mtree` specification format is\nprovided with libarchive ([libarchive-formats(5)][libarchive-formats(5)]).\n\nThere is also an [mtree port for Linux][archiecobbs/mtree-port] though it is not widely packaged for Linux distributions.\n\nThere was a [Google Summer of Code project to create a portable library and parser for mtree](https://wiki.freebsd.org/SummerOfCode2015/mtreeParsingLibrary).\nIt is available at [github.com/mratajsky/libmtree](https://github.com/mratajsky/libmtree) and a [talk on it](https://papers.freebsd.org/2016/asiabsdcon/ratajsky-mtree-parsing/).\n\n## Format\n\nThe BSD mtree specification is published in [mtree(5)][mtree(5)].\n\nThe format of hierarchy specification is consistent with the `# mtree v2.0` format.\nBoth the BSD `mtree` and libarchive ought to be interoperable with it with only one definite caveat.\nOn Linux, extended attributes (`xattr`) on files are often a critical aspect of the file, holding ACLs, capabilities, etc.\nWhile FreeBSD filesystem do support `extattr`, this feature has not made its way into their `mtree`.\n\nThis implementation of mtree supports a few non-upstream \"keyword\"s, such as: `xattr` and `tar_time`.\nIf you include these keywords, the FreeBSD `mtree` will fail, as they are unknown keywords to that implementation.\n\nTo have `go-mtree` produce specifications that will be strictly compatible with the BSD `mtree`, use the `-bsd-keywords` flag when creating a manifest.\nThis will make sure that only the keywords supported by BSD `mtree` are used in the program.\n\n### Typical form\n\nWith the standard keywords, plus say `sha256digest`, the hierarchy specification looks like:\n\n```mtree\n# .\n/set type=file nlink=1 mode=0664 uid=1000 gid=100\n. size=4096 type=dir mode=0755 nlink=6 time=1459370393.273231538\n    LICENSE size=1502 mode=0644 time=1458851690.0 sha256digest=ef4e53d83096be56dc38dbf9bc8ba9e3068bec1ec37c179033d1e8f99a1c2a95\n    README.md size=2820 mode=0644 time=1459370256.316148361 sha256digest=d9b955134d99f84b17c0a711ce507515cc93cd7080a9dcd50400e3d993d876ac\n\n[...]\n```\n\nSee the directory presently in, and the files present.\nAlong with each path, is provided the keywords and the unique values for each path.\nAny common keyword and values are established in the `/set` command.\n\n### Extended attributes form\n\n```mtree\n# .\n/set type=file nlink=1 mode=0664 uid=1000 gid=1000\n. size=4096 type=dir mode=0775 nlink=6 time=1459370191.11179595 xattr.security.selinux=dW5jb25maW5lZF91Om9iamVjdF9yOnVzZXJfaG9tZV90OnMwAA==\n    LICENSE size=1502 time=1458851690.583562292 xattr.security.selinux=dW5jb25maW5lZF91Om9iamVjdF9yOnVzZXJfaG9tZV90OnMwAA==\n    README.md size=2366 mode=0644 time=1459369604.0 xattr.security.selinux=dW5jb25maW5lZF91Om9iamVjdF9yOnVzZXJfaG9tZV90OnMwAA==\n\n[...]\n```\n\nSee the keyword prefixed with `xattr.` followed by the extended attribute's namespace and keyword.\nThis setup is consistent for use with Linux extended attributes as well as FreeBSD extended attributes.\n\nSince extended attributes are an unordered hashmap, this approach allows for checking each `\u003cnamespace\u003e.\u003ckey\u003e` individually.\n\nThe value is the [base64 encoded][base64] of the value of the particular extended attribute.\nSince the values themselves could be raw bytes, this approach avoids issues with encoding.\n\n### Tar form\n\n```mtree\n# .\n/set type=file mode=0664 uid=1000 gid=1000\n. type=dir mode=0775 tar_time=1468430408.000000000\n\n# samedir\nsamedir type=dir mode=0775 tar_time=1468000972.000000000\n    file2 size=0 tar_time=1467999782.000000000\n    file1 size=0 tar_time=1467999781.000000000\n    \n[...]\n```\n\nWhile `go-mtree` serves mainly as a library for upstream `mtree` support, `go-mtree` is also compatible with [tar archives][tar] (which is not an upstream feature).\nThis means that we can now create and validate a manifest by specifying a tar file.\nMore interestingly, this also means that we can create a manifest from an archive, and then validate this manifest against a filesystem hierarchy that's on disk, and vice versa.\n\nNotice that for the output of creating a validation manifest from a tar file, the default behavior for evaluating a notion of time is to use the `tar_time` keyword.\nIn the \"filesystem hierarchy\" format of mtree, `time` is being evaluated with nanosecond precision.\nHowever, GNU tar truncates a file's modification time to 1-second precision.\nThat is, if a file's full modification time is 123456789.123456789, the \"tar time\" equivalent would be 123456789.000000000.\nThis way, if you validate a manifest created using a tar file against an actual root directory, there will be no complaints from `go-mtree` so long as the 1-second precision time of a file in the root directory is the same.\n\n## Usage\n\nTo use the Go programming language library, see [the docs][godoc].\n\nTo use the command line tool, first [build it](#Building), then the following.\n\n### Create a manifest\n\nThis will also include the sha512 digest of the files.\n\n```shell\ngomtree validate -c -K sha512digest -p . \u003e /tmp/root.mtree\n```\n\nWith a tar file:\n\n```shell\ngomtree validate -c -K sha512digest -T sometarfile.tar \u003e /tmp/tar.mtree\n```\n\n### Validate a manifest\n\n```shell\ngomtree validate -p . -f /tmp/root.mtree\n```\n\nWith a tar file:\n\n```shell\ngomtree validate -T sometarfile.tar -f /tmp/root.mtree\n```\n\n### See the supported keywords\n\n```shell\ngomtree validate -list-keywords\nAvailable keywords:\n uname\n sha1\n sha1digest\n sha256digest\n xattrs (not upstream)\n link (default)\n nlink (default)\n md5digest\n rmd160digest\n mode (default)\n cksum\n md5\n rmd160\n type (default)\n time (default)\n uid (default)\n gid (default)\n sha256\n sha384\n sha512\n xattr (not upstream)\n tar_time (not upstream)\n size (default)\n ripemd160digest\n sha384digest\n sha512digest\n```\n\n## Building\n\nEither:\n\n```shell\ngo install github.com/vbatts/go-mtree/cmd/gomtree@latest\n```\n\nor\n\n```shell\ngit clone git://github.com/vbatts/go-mtree.git $GOPATH/src/github.com/vbatts/go-mtree\ncd $GOPATH/src/github.com/vbatts/go-mtree\ngo build ./cmd/gomtree\n```\n\n### Build for many OS/Arch\n\n```shell\nmake build.arches\n```\n\n## Testing\n\nOn Linux:\n\n```shell\ncd $GOPATH/src/github.com/vbatts/go-mtree\nmake\n```\n\nOn FreeBSD:\n\n```shell\ncd $GOPATH/src/github.com/vbatts/go-mtree\ngmake\n```\n\n## Related tools\n\n[mtree(5)]: https://man.freebsd.org/cgi/man.cgi?query=mtree\u0026sektion=5\u0026format=html\n[mtree(8)]: https://www.freebsd.org/cgi/man.cgi?mtree(8)\n[libarchive-formats(5)]: https://www.freebsd.org/cgi/man.cgi?query=libarchive-formats\u0026sektion=5\u0026n=1\n[archiecobbs/mtree-port]: https://github.com/archiecobbs/mtree-port\n[godoc]: https://pkg.go.dev/github.com/vbatts/go-mtree\n[tar]: http://man7.org/linux/man-pages/man1/tar.1.html\n[base64]: https://tools.ietf.org/html/rfc4648\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvbatts%2Fgo-mtree","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvbatts%2Fgo-mtree","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvbatts%2Fgo-mtree/lists"}