https://github.com/tsileo/blobsfile

BlobStash's storage engine. Mirror of https://git.sr.ht/~tsileo/blobsfile
https://github.com/tsileo/blobsfile

blobstash content-addressed go storage-engine

Last synced: about 1 year ago
JSON representation

BlobStash's storage engine. Mirror of https://git.sr.ht/~tsileo/blobsfile

• Host: GitHub
• URL: https://github.com/tsileo/blobsfile
• Owner: tsileo
• License: mit
• Created: 2017-02-19T11:19:04.000Z (over 9 years ago)
• Default Branch: master
• Last Pushed: 2020-04-25T14:09:44.000Z (about 6 years ago)
• Last Synced: 2025-03-24T08:08:16.662Z (over 1 year ago)
• Topics: blobstash, content-addressed, go, storage-engine
• Language: Go
• Homepage:
• Size: 77.1 KB
• Stars: 17
• Watchers: 4
• Forks: 2
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# BlobsFile

[![builds.sr.ht status](https://builds.sr.ht/~tsileo/blobsfile.svg)](https://builds.sr.ht/~tsileo/blobsfile?)

   [![Godoc Reference](https://godoc.org/a4.io/blobsfile?status.svg)](https://godoc.org/a4.io/blobsfile)

*BlobsFile* is an append-only (i.e. no update and no delete) content-addressed *blob store* (using [BLAKE2b](https://blake2.net/) as hash function).

It draws inspiration from Facebook's [Haystack](http://202.118.11.61/papers/case%20studies/facebook.pdf), blobs are stored in flat files (called _BlobFile_) and indexed by a small [kv](https://github.com/cznic/kv) database for fast lookup.

*BlobsFile* is [BlobStash](https://github.com/tsileo/blobstash)'s storage engine.

## Features

 - Durable (data is fsynced before returning)

 - Immutable (append-only, can't mutate or delete blobs)

 - Optional compression (Snappy or Zstandard)

 - Extra parity data is added to each _BlobFile_ (using Reed-Solomon error correcting code), allowing the database to repair itself in case of corruption.

   - The test suite is literraly punching holes at random places