https://github.com/storacha/pail

🪣 DAG based key value store.
https://github.com/storacha/pail

bucket crdt dag ipfs ipld kv kv-store merkle-crdt merkle-dag shard

Last synced: 3 months ago
JSON representation

🪣 DAG based key value store.

• Host: GitHub
• URL: https://github.com/storacha/pail
• Owner: storacha
• License: other
• Created: 2022-12-21T13:00:12.000Z (about 2 years ago)
• Default Branch: main
• Last Pushed: 2024-04-22T13:20:37.000Z (9 months ago)
• Last Synced: 2024-09-10T13:09:13.820Z (5 months ago)
• Topics: bucket, crdt, dag, ipfs, ipld, kv, kv-store, merkle-crdt, merkle-dag, shard
• Language: JavaScript
• Homepage:
• Size: 175 KB
• Stars: 40
• Watchers: 2
• Forks: 11
• Open Issues: 5
• Metadata Files:
  • Readme: README.md
  • Changelog: CHANGELOG.md
  • License: LICENSE.md

Awesome Lists containing this project

README

        
# pail

[![Test](https://github.com/web3-storage/pail/actions/workflows/ci.yml/badge.svg)](https://github.com/web3-storage/pail/actions/workflows/ci.yml)

[![JavaScript Style Guide](https://img.shields.io/badge/code_style-standard-brightgreen.svg)](https://standardjs.com)

DAG based key value store. Sharded DAG that minimises traversals and work to build shards.

* 📖 [Read the SPEC](https://github.com/web3-storage/specs/blob/4163e28d7e6a7c44cff68db9d9bffb9b37707dc6/pail.md).

* 🎬 [Watch the Presentation](https://youtu.be/f-BrtpYKZfg).

## Install

```

npm install @web3-storage/pail

```

## Usage

```js

import { put, get, del } from '@web3-storage/pail'

import { ShardBlock } from '@web3-storage/pail/shard'

import { MemoryBlockstore } from '@web3-storage/pail/block'

// Initialize a new bucket

const blocks = new MemoryBlockstore()

const init = await ShardBlock.create() // empty root shard

await blocks.put(init.cid, init.bytes)

// Add a key and value to the bucket

const { root, additions, removals } = await put(blocks, init.cid, 'path/to/data0', dataCID0)

console.log(`new root: ${root}`)

// Process the diff

for (const block of additions) {

  await blocks.put(block.cid, block.bytes)

}

for (const block of removals) {

  await blocks.delete(block.cid)

}

```

### Batch operations

If adding many multiple items to the pail together, it is faster to batch them together.

```js

import { put, get, del } from '@web3-storage/pail'

import { ShardBlock } from '@web3-storage/pail/shard'

import { MemoryBlockstore } from '@web3-storage/pail/block'

import * as Batch from '@web3-storage/pail/batch'

// Initialize a new bucket

const blocks = new MemoryBlockstore()

const init = await ShardBlock.create() // empty root shard

await blocks.put(init.cid, init.bytes)

const batch = await Batch.create(blocks, init.cid)

// items is an array of `{ key: string, value: CID }` - the items to add to the pail

for (const item of items) {

  await batch.put(item.key, item.value)

}

const { root, additions, removals } = await batch.commit()

console.log(`new root: ${root}`)

// Process the diff

for (const block of additions) {

  await blocks.put(block.cid, block.bytes)

}

for (const block of removals) {

  await blocks.delete(block.cid)

}

```

## Contributing

Feel free to join in. All welcome. [Open an issue](https://github.com/web3-storage/pail/issues)!

## License

Dual-licensed under [MIT or Apache 2.0](https://github.com/web3-storage/pail/blob/main/LICENSE.md)