https://github.com/sunny-117/chunkflow
๐ A universal large file upload solution with chunked upload, resumable upload, and instant upload capabilities. | ้็จ็ๅคงๆไปถไธไผ ่งฃๅณๆนๆก๏ผๆฏๆๅ็ไธไผ ใๆญ็น็ปญไผ ๅ็งไผ ๅ่ฝ
https://github.com/sunny-117/chunkflow
chunk fiber file http large-file-upload protocol rust turbo upload upload-client upload-component websocket
Last synced: about 2 hours ago
JSON representation
๐ A universal large file upload solution with chunked upload, resumable upload, and instant upload capabilities. | ้็จ็ๅคงๆไปถไธไผ ่งฃๅณๆนๆก๏ผๆฏๆๅ็ไธไผ ใๆญ็น็ปญไผ ๅ็งไผ ๅ่ฝ
- Host: GitHub
- URL: https://github.com/sunny-117/chunkflow
- Owner: Sunny-117
- License: mit
- Created: 2026-01-29T03:04:02.000Z (5 days ago)
- Default Branch: main
- Last Pushed: 2026-01-31T06:09:14.000Z (3 days ago)
- Last Synced: 2026-01-31T17:38:51.055Z (3 days ago)
- Topics: chunk, fiber, file, http, large-file-upload, protocol, rust, turbo, upload, upload-client, upload-component, websocket
- Language: TypeScript
- Homepage: https://sunny-117.github.io/chunkflow/
- Size: 6.22 MB
- Stars: 4
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
Awesome Lists containing this project
README
ChunkFlow Upload SDK
A Universal Large File Upload Solution
---
## โจ Features
- ๐ **Smart Upload Strategy** - Automatic selection between direct and chunked upload based on file size
- ๐ฆ **Dynamic Chunking** - Adaptive chunk size adjustment based on network conditions (similar to TCP slow start)
- โก **Instant Upload** - Hash-based deduplication for instant uploads (็งไผ )
- ๐ **Resumable Upload** - Continue uploads from where you left off with IndexedDB persistence
- ๐ฏ **Framework Agnostic** - Core layer works with any framework
- โ๏ธ **React Support** - Hooks and components for React applications
- ๐ **Vue Support** - Composables and components for Vue applications
- ๏ฟฝ๏ธ **Highly Extensible** - Plugin system for custom functionality
- ๐ **Type Safe** - Written in TypeScript with full type definitions
- ๐งช **Well Tested** - Comprehensive unit and property-based tests
## ๐ฆ Packages
### Core Packages
- **[@chunkflowjs/protocol](./packages/protocol)** - Protocol layer with type definitions and API interfaces
- **[@chunkflowjs/shared](./packages/shared)** - Shared utilities (event system, concurrency, file utils, storage)
- **[@chunkflowjs/core](./packages/core)** - Core upload engine with state machine and task management
### Client Packages
- **[@chunkflowjs/upload-client-react](./packages/upload-client-react)** - React adapter with hooks
- **[@chunkflowjs/upload-client-vue](./packages/upload-client-vue)** - Vue adapter with composables
- **[@chunkflowjs/upload-component-react](./packages/upload-component-react)** - Ready-to-use React components
- **[@chunkflowjs/upload-component-vue](./packages/upload-component-vue)** - Ready-to-use Vue components
### Server Package
- **[@chunkflowjs/upload-server](./packages/upload-server)** - Server-side SDK with storage adapters
## ๐ Quick Start
### Installation
```bash
# For React projects
pnpm add @chunkflowjs/core @chunkflowjs/upload-client-react
# For Vue projects
pnpm add @chunkflowjs/core @chunkflowjs/upload-client-vue
```
### React Usage
```tsx
import { UploadProvider, useUpload } from "@chunkflowjs/upload-client-react";
import { createFetchAdapter } from "@chunkflowjs/core";
const adapter = createFetchAdapter({
baseURL: "http://localhost:3000/api",
});
function App() {
return (
);
}
function UploadComponent() {
const { upload, status, progress } = useUpload({
onSuccess: (fileUrl) => console.log("Upload complete:", fileUrl),
onError: (error) => console.error("Upload failed:", error),
});
return (
{
const file = e.target.files?.[0];
if (file) upload(file);
}}
/>
Status: {status}
Progress: {progress.percentage}%
);
}
```
### Vue Usage
```html
import { useUpload } from "@chunkflowjs/upload-client-vue";
const { upload, status, progress } = useUpload({
onSuccess: (fileUrl) => console.log("Upload complete:", fileUrl),
onError: (error) => console.error("Upload failed:", error),
});
const handleFileChange = (event) => {
const file = event.target.files?.[0];
if (file) upload(file);
};
Status: {{ status }}
Progress: {{ progress.percentage }}%
```
## ๐๏ธ Architecture
ChunkFlow Upload SDK follows a layered architecture:
```
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ Applications โ
โ Playground (Demo) โ Server (Nest.js) โ Website (Docs) โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ Component Layer โ
โ React Components โ Vue Components โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ Client Adapter Layer โ
โ React Hooks โ Vue Composables โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ Core Layer โ
โ Upload Manager โ Upload Task โ Plugin System โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ Shared Layer โ
โ Event System โ Concurrency โ File Utils โ Storage โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ Protocol Layer โ
โ Type Definitions โ API Interfaces โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
```
## ๐ Documentation
Full documentation is available at: [https://sunny-117.github.io/chunkflow/](https://sunny-117.github.io/chunkflow/)
## ๐ ๏ธ Development
This project uses a monorepo structure managed by pnpm workspaces and Turbo.
### Prerequisites
- Node.js >= 18.0.0
- pnpm >= 9.0.0
### Setup
```bash
# Install dependencies
pnpm install
# Build all packages
pnpm build
# Run tests
pnpm test
# Lint code
pnpm lint
# Format code
pnpm format
```
### Project Structure
```
chunkflow/
โโโ packages/ # SDK packages
โ โโโ protocol/ # Protocol layer
โ โโโ shared/ # Shared utilities
โ โโโ core/ # Core upload engine
โ โโโ upload-client-react/ # React adapter
โ โโโ upload-client-vue/ # Vue adapter
โ โโโ upload-component-react/ # React components
โ โโโ upload-component-vue/ # Vue components
โ โโโ upload-server/ # Server SDK
โโโ apps/ # Applications
โ โโโ server/ # Nest.js server
โ โโโ playground/ # Demo application
โ โโโ website/ # Documentation site
โโโ pnpm-workspace.yaml # Workspace configuration
โโโ turbo.json # Turbo configuration
โโโ package.json # Root package.json
```
## ๐งช Testing
The project uses a dual testing approach:
- **Unit Tests** - Test specific examples and edge cases
- **Property-Based Tests** - Test universal properties across random inputs using fast-check
```bash
# Run all tests
pnpm test
# Run tests in watch mode
pnpm test:watch
# Run tests with coverage
pnpm test -- --coverage
```
## ๐ค Contributing
Contributions are welcome! Please read our [contributing guidelines](./CONTRIBUTING.md) before submitting a PR.
## ๐ License
MIT ยฉ [Sunny-117](https://github.com/Sunny-117)
## ๐ Acknowledgments
- [mitt](https://github.com/developit/mitt) - Event emitter
- [p-limit](https://github.com/sindresorhus/p-limit) - Concurrency control
- [spark-md5](https://github.com/satazor/js-spark-md5) - MD5 hashing
- [fast-check](https://github.com/dubzzz/fast-check) - Property-based testing
---