https://github.com/blacha/chunkd

Chunked file reading abstraction for S3, HTTP and files
https://github.com/blacha/chunkd

Last synced: about 18 hours ago
JSON representation

Chunked file reading abstraction for S3, HTTP and files

• Host: GitHub
• URL: https://github.com/blacha/chunkd
• Owner: blacha
• License: mit
• Created: 2021-07-21T22:31:16.000Z (over 3 years ago)
• Default Branch: master
• Last Pushed: 2025-01-03T17:42:11.000Z (12 days ago)
• Last Synced: 2025-01-04T01:40:13.287Z (11 days ago)
• Language: TypeScript
• Homepage:
• Size: 1.53 MB
• Stars: 4
• Watchers: 2
• Forks: 1
• Open Issues: 8
• Metadata Files:
  • Readme: README.md
  • Contributing: CONTRIBUTING.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# @chunkd

File system abstraction to work with files from various sources

- [Filesystem](./packages/source-file/)

- [Http](./packages//source-http/)

- [Aws SDK V3](./packages/source-aws/)

- [Memory](./packages/source-memory/)

## Usage

Load a chunks of data from a URL using `fetch`

```typescript

import { fsa } from '@chunkd/fs';

const source = fsa.source(new URL('https://example.com/foo.zip'));

const firstBuffer = await source.fetch(0, 1024); // Load the first 1KB from the source

const lastBuffer = await source.fetch(-1024); // load the last 1KB from the source

const size = source.metadata?.size; // File size if metadata has been fetched

```

# Building

This requires [NodeJs](https://nodejs.org/en/) >= 18 & [Yarn](https://yarnpkg.com/en/)

Use [fnm](https://github.com/Schniz/fnm) to manage nodeJs versions

```bash

# Download the latest nodejs & yarn

fnm use 18

npm install -g yarn

# Install node deps

yarn

# Build everything into /build

yarn run build

# Run the unit tests

yarn run test

```