{"id":13581525,"url":"https://github.com/monax/hoard","last_synced_at":"2025-08-28T08:06:50.190Z","repository":{"id":22842659,"uuid":"97459421","full_name":"monax/hoard","owner":"monax","description":" Hoard is a stateless, deterministically encrypted, content-addressed object store.","archived":false,"fork":false,"pushed_at":"2023-01-07T05:07:18.000Z","size":23895,"stargazers_count":76,"open_issues_count":10,"forks_count":14,"subscribers_count":11,"default_branch":"main","last_synced_at":"2025-04-13T02:36:56.278Z","etag":null,"topics":["aes-gcm","azure","convergent","cryptography","daemon","encryption","filesystem","gcs","grpc","hoard","ipfs","s3","storage-backend"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/monax.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-07-17T09:32:24.000Z","updated_at":"2025-01-28T01:37:19.000Z","dependencies_parsed_at":"2023-01-14T01:00:13.750Z","dependency_job_id":null,"html_url":"https://github.com/monax/hoard","commit_stats":null,"previous_names":[],"tags_count":37,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/monax%2Fhoard","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/monax%2Fhoard/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/monax%2Fhoard/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/monax%2Fhoard/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/monax","download_url":"https://codeload.github.com/monax/hoard/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248657854,"owners_count":21140843,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aes-gcm","azure","convergent","cryptography","daemon","encryption","filesystem","gcs","grpc","hoard","ipfs","s3","storage-backend"],"created_at":"2024-08-01T15:02:04.409Z","updated_at":"2025-04-13T02:37:51.548Z","avatar_url":"https://github.com/monax.png","language":"Go","funding_links":[],"categories":["Go"],"sub_categories":[],"readme":"# Hoard\n\n![master](https://github.com/monax/hoard/workflows/master/badge.svg)\n[![version](https://img.shields.io/github/tag/monax/hoard.svg)](https://github.com/monax/hoard/releases/latest)\n[![GoDoc](https://godoc.org/github.com/hoard?status.png)](https://pkg.go.dev/github.com/monax/hoard)\n[![license](https://img.shields.io/github/license/monax/hoard.svg)](LICENSE)\n[![LoC](https://tokei.rs/b1/github/monax/hoard?category=lines)](https://github.com/monax/hoard)\n[![codecov](https://codecov.io/gh/monax/hoard/branch/master/graph/badge.svg)](https://codecov.io/gh/monax/hoard)\n\nHoard is a stateless, deterministically encrypted, content-addressed object store.\n\n![hoarding marmot](docs/images/hoard.jpg)\n\n## Introduction\nIt convergently encrypts an object using its (SHA256) hash as the secret key (which can than be shared as a 'grant').\nThe address is then deterministically generated from the encrypted object's (SHA256) digest and allocated to the configured storage back-end:\n\n- In-Memory\n- Filesystem\n- [AWS](https://aws.amazon.com/s3/)\n- [GCP](https://cloud.google.com/storage/)\n- [Azure](https://azure.microsoft.com/en-gb/services/storage/)\n- [IPFS](https://ipfs.io/)\n\n## Installing\nHoard should be go-gettable with:\n\n```shell\n# Install Hoard-Daemon:\ngo get github.com/monax/hoard/cmd/hoard\n\n# Install Hoard-Control:\ngo get github.com/monax/hoard/cmd/hoarctl\n```\n\n## Usage\nHoard runs as a daemon providing a GRPC service to other clients including the command line client `hoarctl`. The purpose of the daemon is to read local secrets (such as PGP or other keys) and to configure itself to use a particular storage backend. You can run the daemon with:\n\n```shell\n# Run the daemon\nhoard\n\n# or with logging\nhoard --logging\n```\n\nYou can initialise a Hoard config by running one of:\n\n```shell\n# Initialise Hoard with memory backend\nhoard config --init memory\n\n# Initialise Hoard with filesystem backend\nhoard config --init filesystem\n\n# Initialise Hoard with AWS (S3) backend\nhoard config --init aws\n\n# Initialise Hoard with Azure backend\nhoard config --init azure\n\n# Initialise Hoard with GCP backend\nhoard config --init gcp\n\n# Initialise Hoard with IPFS backend\nhoard config --init ipfs\n```\n\nThese will provide base configurations you can configure to meet your needs. The config is located by default in `$HOME/.config/hoard.conf` but you can specify a file with `hoard -c /path/to/config`. The XDG base directory specification is used to search for config.\n\nYou can interact with Hoard using the go client `hoarctl`:\n\n```shell\n# Store an object:\nref=$(echo bar | hoarctl put)\n\n# Retrieve 'bar' from its (deterministic) reference\necho $ref | hoarctl get\n\n# Or get information about the object without decrypting\necho $ref | hoarctl stat\n\n# This one-liner exercises the entire API:\necho foo | hoarctl put | hoarctl get | hoarctl putseal | hoarctl unsealget | hoarctl encrypt | hoarctl insert | hoarctl stat | hoarctl cat | hoarctl decrypt -k tbudgBSg+bHWHiHnlteNzN8TUvI80ygS9IULh4rklEw= | hoarctl ref | hoarctl seal | hoarctl reseal | hoarctl unseal | hoarctl get\n```\n\nYou can chop off segments of the final command to see the output of each intermediate command. It is contrived so that the outputs can be used as inputs for the next pipeline step. `hoarctl` either returns JSON references or raw bytes depending on the command. You may find the excellent [jq](https://stedolan.github.io/jq/) useful for working with single-line JSON files on the command line.\n\n## Config\nUsing the filesystem storage backend as an example (generated with `hoard init -o- fs`) you can configure Hoard with a file like:\n\n```toml\n# The listen address, also supported is \"unix:///tmp/hoard.socket\" for a unix domain socket\nListenAddress = \"tcp://localhost:53431\"\n\n[Storage]\n  StorageType = \"filesystem\"\n  # One of: base64, base32, or hex (base 16)\n  AddressEncoding = \"base64\"\n  RootDirectory = \"/home/user/.local/share/hoard\"\n\n[Logging]\n  LoggingType = \"logfmt\"\n  # Removing \"trace\" from this array will reduce log output\n  Channels = [\"info\", \"trace\"]\n```\n\nThe default directory is `$HOME/.config/hoard.toml` or you can pass the file with `hoard -c`.\n\n## Specification\nSee [hoard.proto](protobuf/hoard.proto) for the protobuf3 definition of the API. Hoard uses [GRPC](https://grpc.io/) for its API for which there is a wide range of client libraries available. You should be able to set up a client in any GRPC supported language with relative ease. Also see `hoarctl \u003cCMD\u003e -h` for full help on each sub-command.\n\nFor more information on the design of Hoard please checkout our [documentation](docs/encryption.md).\n\n## Building\nTo build Hoard you will need to have the following installed:\n- Go (Version \u003e= 1.11) (with $GOPATH/bin in $PATH)\n- GNU make\n- [Protocol Buffers 3](https://github.com/google/protobuf/releases/tag/v3.3.0)\n\nThen, from the project root run:\n\n```shell\n# Run checks, tests, and build binaries\nmake build \u0026\u0026 make install\n```\n\n## Javascript Client\nA Javascript client library can be found here: [js](https://github.com/monax/hoard/tree/master/js).\n\nHoard-js is a fairly lightweight wrapper around the Hoard GRPC API. It mainly serves to abstract over the dynamic protobuf library and the static protobuf generation.\n\n### Usage\nFirst we need to have Hoard running. For development purposes this can be accomplished by:\n\n```shell\ngo get github.com/monax/hoard/cmd/hoard \n# Run Hoard with logging\nhoard --logging\n```\n\nHoard will run with an in-memory store by default that will be discarded when it is shutdown, but will expose the same\ninterface as when using remote storage backends.\n\nTo interact with Hoard from Node see [example.js](example.js) for a self-contained example of how to use every method\nof the API. To run use:\n\n```shell\n# Get dependencies\nyarn install\n# Run example\nnode example.js\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmonax%2Fhoard","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmonax%2Fhoard","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmonax%2Fhoard/lists"}