https://github.com/blacha/chunkd
Chunked file reading abstraction for S3, HTTP and files
https://github.com/blacha/chunkd
Last synced: 24 days ago
JSON representation
Chunked file reading abstraction for S3, HTTP and files
- Host: GitHub
- URL: https://github.com/blacha/chunkd
- Owner: blacha
- License: mit
- Created: 2021-07-21T22:31:16.000Z (over 4 years ago)
- Default Branch: master
- Last Pushed: 2026-02-21T06:01:39.000Z (about 1 month ago)
- Last Synced: 2026-02-21T13:16:09.760Z (about 1 month ago)
- Language: TypeScript
- Homepage:
- Size: 1.88 MB
- Stars: 8
- Watchers: 2
- Forks: 2
- Open Issues: 5
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
Awesome Lists containing this project
README
# @chunkd
File system abstraction to work with files from various sources
- [Filesystem](./packages/source-file/)
- [Http](./packages//source-http/)
- [Aws SDK V3](./packages/source-aws/)
- [Memory](./packages/source-memory/)
## Usage
Load a chunks of data from a URL using `fetch`
```typescript
import { fsa } from '@chunkd/fs';
const source = fsa.source(new URL('https://example.com/foo.zip'));
const firstBuffer = await source.fetch(0, 1024); // Load the first 1KB from the source
const lastBuffer = await source.fetch(-1024); // load the last 1KB from the source
const size = source.metadata?.size; // File size if metadata has been fetched
```
# Building
This requires [NodeJs](https://nodejs.org/en/) >= 18
Use [fnm](https://github.com/Schniz/fnm) to manage nodeJs versions
```bash
# Download the latest nodejs
fnm use 24
# Install node deps
npm
# Build everything into /build
npm run build
# Run the unit tests
npm run test
```