{"id":17078238,"url":"https://github.com/d3zd3z/gosure","last_synced_at":"2025-04-12T20:31:05.785Z","repository":{"id":66855611,"uuid":"1333834","full_name":"d3zd3z/gosure","owner":"d3zd3z","description":"File integrity implemented in Go","archived":false,"fork":false,"pushed_at":"2019-10-31T18:55:56.000Z","size":344,"stargazers_count":16,"open_issues_count":4,"forks_count":3,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-26T14:50:19.733Z","etag":null,"topics":["backup","go","integrity"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/d3zd3z.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2011-02-06T07:44:01.000Z","updated_at":"2021-02-28T14:17:12.000Z","dependencies_parsed_at":null,"dependency_job_id":"48f9e135-ebca-4f96-aedc-cc3cf794ede3","html_url":"https://github.com/d3zd3z/gosure","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/d3zd3z%2Fgosure","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/d3zd3z%2Fgosure/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/d3zd3z%2Fgosure/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/d3zd3z%2Fgosure/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/d3zd3z","download_url":"https://codeload.github.com/d3zd3z/gosure/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248629307,"owners_count":21136233,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["backup","go","integrity"],"created_at":"2024-10-14T12:20:31.983Z","updated_at":"2025-04-12T20:31:05.749Z","avatar_url":"https://github.com/d3zd3z.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"Gosure file integrity\n*********************\n\nIt has been said that backups aren't useful unless you've tested them.\nBut, how does one know that a test restore actually worked?  Gosure is\ndesigned to help with this.\n\nHistory\n=======\n\nThe md5sum program captures the MD5 hash of a set of files.  It can\nalso read this output and compare the hashes against the files.  By\ncapturing the hashes before the backup, and comparing them after a\ntest restore, you can gain a bit of confidence that the contents of\nfiles is at least correct.\n\nHowever, this doesn't capture the permissions and other attributes of\nthe files.  Sometimes a restore can fail for this kind of reason.\n\nIntrusion detection\n===================\n\nThere have been several similar solutions focused on intrusion\ndetection.  Tripwire and FreeVeracity (or Veracity) come to mind.  The\nidea is that the files are compared in place to verify that nobody has\nmodified them.\n\nUnfortunately, at least tripwire seems to focus so heavily on this\nintrusion detection problem, that the tool doesn't work very well for\nverifying backups.  It really wants a central database, and to use\nfiles by absolute pathname.  FreeVeracity was quite useful for\nverifying backups, however, it appears to have vanished entirely (it\nwas under an unusual license).\n\nIncremental updates\n-------------------\n\nOne thing that none of these solutions addressed was that of\nincremental updates, probably because of the focus on intrusion\ndetection.  In a normal running system, the POSIX *ctime* field can be\nreliably used to determine if a file has been modified.  By making use\nof this, the integrity program can avoid recomputing hashes of files\nthat haven't changed.  This strategy is similar to what most backup\nsoftware does as well.  This is important, because taking the time to\nhash every file can make the integrity update take so long that people\navoid running it.  Full hashing is impractical for the same reasons\nthat regular full backups are usually impractical.\n\nUsing gosure\n************\n\nGetting it\n==========\n\nGosure is written in Go_.\n\n.. _Go: https://golang.org/\n\nThere are two ways to build gosure.  You can just build it,\nstandalone::\n\n    $ git clone https://github.com/d3zd3z/gosure\n    $ cd gosure\n    $ go run build.go\n    $ cp gosure ~/bin\n\nThis will build a version with the release tag information embedded in\nthe executable.\n\nIf you want to do any work on the code, it is generally best to work\nwith Go using its idea of a workspace.  You should create a directory\nsomewhere for go work, and set the environment variable ``GOPATH`` to\npoint to this.  Once this is done, use the go tools to fetch this\nproject::\n\n    $ go get davidb.org/x/gosure/cmd/gosure\n\n.. note::\n   Although this project is hosted at github.com (currently), the go\n   tool should complain if you try to fetch using that path.  This is\n   because the package needs to be able to reference sub-packages by\n   full name, and these will only work if the package is fetched via\n   its canonical name.\n\nOnce the tree is present::\n\n    $ go install davidb.org/x/gosure/cmd/gosure\n\nshould install the gosure program itself in ``$GOPATH/bin``.  Add this\nto the path to make things more convenient.  The execuable is\nstandalone, and has no dependencies on the source tree.\n\nBasic usage\n===========\n\nChange to a directory you wish to keep integrity for, for example, my\nhome directory::\n\n    $ cd\n    $ gosure scan\n\nThis will scan the filesystem (possibly showing progress), and leave a\n``2sure.dat.gz`` (the 2sure is historical, FreeVeracity used a name\nstarting with a 0, and having the digit makes it near the beginning of\na directory listing).  You can view this file if you'd like.  Aside\nfrom being compressed, the format is plain ASCII (even if your\nfilenames are not).\n\nThen you can do::\n\n    $ gosure check\n\nto verify the directory.  This will show any differences.  If you back\nup this file with your data, you can run ``gosure`` after a restore to\ncheck if the backup is correct.\n\nLater, you can run ::\n\n    $ gosure update\n\nwhich will update the sure data, adding another weave delta to the\n``2sure.dat.gz`` file.  The old file will be moved to\n``2sure.back.gz`` for safety (it is not normally needed as each file\nwill have the whole history).  You can then compare the two most\nrecent versions with:\n\n    $ gosure signoff\n\nwill compare the old scan with the current, and report on what has\nchanged between them.\n\nWeave Deltas\n************\n\nGosure uses the weave delta format\\ [#] to store multiple versions in\na single file.\n\n.. [#] The weave format was developed by Marc Rochkind as part of the\n   SCCS revision control system.  Although much of SCCS is dated, the\n   particular way it stores all file revisions in a single “weave”\n   file is particularly useful to the types of changes that happen to\n   surefiles.  Gosure uses the weave data format exactly as SCCS does,\n   but uses its own header.  The headers on SCCS have numerous\n   limitations that would render it less useful, such as file sizes\n   limited to 100,000 lines, and 2 year dates.\n\nEach delta can have arbitrary metadata associated with it.  These\nvalues can be added with ``--tag key=value``.  The ``name`` key will\noverride the default timestamp name.  It may be useful to indicate\nother information about when the scan was taken.  The tags are\narbitrary key/value pairs, although both should be restricted to\nprintable characters.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fd3zd3z%2Fgosure","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fd3zd3z%2Fgosure","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fd3zd3z%2Fgosure/lists"}