{"id":30561012,"url":"https://github.com/0necontroller/vellum-stream","last_synced_at":"2025-08-28T11:11:27.871Z","repository":{"id":307292110,"uuid":"1028976760","full_name":"0necontroller/vellum-stream","owner":"0necontroller","description":"A self-hostable server for video processing. Features TUS direct uploads, FFmpeg-powered transcoding to streamable formats, and S3 storage. Offers optional callbacks for processing completion. Integrate video workflows effortlessly.","archived":false,"fork":false,"pushed_at":"2025-08-08T16:46:51.000Z","size":148,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-08-19T14:05:30.970Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/0necontroller.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-07-30T10:38:44.000Z","updated_at":"2025-08-08T16:46:56.000Z","dependencies_parsed_at":"2025-07-31T18:08:02.539Z","dependency_job_id":"1c43624d-4b3f-46b3-a688-7cdd8a493975","html_url":"https://github.com/0necontroller/vellum-stream","commit_stats":null,"previous_names":["0necontroller/vellum","0necontroller/vellum-stream"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/0necontroller/vellum-stream","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/0necontroller%2Fvellum-stream","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/0necontroller%2Fvellum-stream/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/0necontroller%2Fvellum-stream/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/0necontroller%2Fvellum-stream/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/0necontroller","download_url":"https://codeload.github.com/0necontroller/vellum-stream/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/0necontroller%2Fvellum-stream/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":272487821,"owners_count":24942744,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-28T02:00:10.768Z","response_time":74,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-08-28T11:11:24.040Z","updated_at":"2025-08-28T11:11:27.865Z","avatar_url":"https://github.com/0necontroller.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Vellum\n\nVellum is a pluggable video processing server that can be easily integrated into any application. It provides a Docker-based solution for handling video uploads, processing, and streaming with support for resumable uploads using TUS protocol.\n\n## 🚀 Features\n\n- **TUS Protocol Support**: Resumable file uploads for reliable video upload experience\n- **FFmpeg Transcoding**: Automatic video transcoding to HLS streaming format\n- **S3 Compatible Storage**: Works with MinIO, AWS S3, or any S3-compatible storage\n- **Queue-based Processing**: RabbitMQ-powered background video processing\n- **REST API**: Complete API for video management with OpenAPI documentation\n- **Webhook Callbacks**: Optional notifications when video processing completes\n- **Bearer Token Authentication**: Secure API access with configurable API keys\n- **Docker Ready**: Full Docker Compose setup with all dependencies\n\n## 📋 Prerequisites\n\n- Docker and Docker Compose\n- At least 2GB RAM for video processing\n- Sufficient disk space for temporary video storage\n\n## 🛠️ Installation \u0026 Setup\n\n### 1. Clone the Repository\n\n```bash\ngit clone https://github.com/0necontroller/vellum.git\ncd vellum\n```\n\n### 2. Configure Environment\n\nCopy the example environment file and customize it:\n\n```bash\ncp .env.example .env\n```\n\nEdit `.env` with your environments:\n\n```bash\n# Server Configuration\nSERVER_PORT=8001\nNODE_ENV=production\n\n# Upload Configuration\nUPLOAD_PATH=./uploads\nMAX_FILE_SIZE=100mb\nALLOWED_FILE_TYPES=video/mp4,video/avi,video/mov,video/mkv\nMAX_FILES=10\n\n# S3/MinIO Configuration\nS3_ACCESS_KEY=minioadmin\nS3_SECRET_KEY=minioadmin\nS3_ENDPOINT=http://minio:9000\nS3_BUCKET=video-streams\n\n# MinIO Configuration\nMINIO_API_PORT=9000\nMINIO_CONSOLE_PORT=9001\n\n# RabbitMQ Configuration\nRABBITMQ_DEFAULT_USER=guest\nRABBITMQ_DEFAULT_PASS=guest\n\n# API Authentication\nAPI_KEY=your_secure_api_key_here\n```\n\n### 3. Start the Services\n\n```bash\ndocker-compose up -d\n```\n\nThis will start:\n\n- **Vellum API Server** on port 8001\n- **MinIO S3 Storage** on port 9000 (API) and 9001 (Console)\n- **RabbitMQ** on port 5672 (API) and 15672 (Management UI)\n\n### 4. Verify Installation\n\nCheck that all services are running:\n\n```bash\ndocker-compose ps\n```\n\nAccess the API documentation at: `http://localhost:8001/api/v1/docs`\n\n## 📚 API Usage\n\n### Authentication\n\nAll API endpoints require Bearer token authentication. Include your API key in the Authorization header:\n\n```bash\nAuthorization: Bearer your_secure_api_key_here\n```\n\n### Core Endpoints\n\n#### 1. Create Upload Session\n\nCreate a TUS upload session for a video file:\n\n```bash\nPOST /api/v1/video/create\nContent-Type: application/json\nAuthorization: Bearer your_secure_api_key_here\n\n{\n  \"filename\": \"my-video.mp4\",\n  \"filesize\": 104857600,\n  \"callbackUrl\": \"https://myapp.com/webhook\", // optional\n  \"s3Path\": \"/v2/media\" // optional - custom S3 path for storing the video\n}\n```\n\nResponse:\n\n```json\n{\n  \"status\": \"success\",\n  \"message\": \"Upload session created\",\n  \"data\": {\n    \"uploadId\": \"550e8400-e29b-41d4-a716-446655440000\",\n    \"uploadUrl\": \"http://localhost:8001/api/v1/tus/files/550e8400-e29b-41d4-a716-446655440000\",\n    \"videoUrl\": \"http://localhost:9000/video-streams/v2/media/550e8400-e29b-41d4-a716-446655440000/index.m3u8\",\n    \"expiresIn\": 3600\n  }\n}\n```\n\n**Response Fields:**\n\n- `uploadId`: Unique identifier for the upload session\n- `uploadUrl`: TUS endpoint URL for uploading the video file\n- `videoUrl`: Future HLS streaming URL where the processed video will be available (constructed using s3Path if provided)\n- `expiresIn`: Upload session expiration time in seconds\n\n#### 2. Upload Video File\n\nUse the TUS protocol to upload your video file to the `uploadUrl` returned from the create session endpoint. You can use any TUS client library.\n\nJavaScript example using `tus-js-client`:\n\n```javascript\nimport * as tus from \"tus-js-client\";\n\nconst upload = new tus.Upload(file, {\n  endpoint: \"http://localhost:8001/api/v1/tus/files/\",\n  uploadUrl: uploadUrl, // from create session response\n  retryDelays: [0, 3000, 5000, 10000, 20000],\n  metadata: {\n    filename: file.name,\n    filetype: file.type,\n  },\n  onError: (error) =\u003e {\n    console.error(\"Upload failed:\", error);\n  },\n  onProgress: (bytesUploaded, bytesTotal) =\u003e {\n    const percentage = ((bytesUploaded / bytesTotal) * 100).toFixed(2);\n    console.log(`Upload progress: ${percentage}%`);\n  },\n  onSuccess: () =\u003e {\n    console.log(\"Upload completed successfully!\");\n  },\n});\n\nupload.start();\n```\n\n#### 3. Check Video Status\n\nMonitor the processing status of your video:\n\n```bash\nGET /api/v1/video/{uploadId}/status\nAuthorization: Bearer your_secure_api_key_here\n```\n\nResponse:\n\n```json\n{\n  \"status\": \"success\",\n  \"message\": \"Video status retrieved successfully\",\n  \"data\": {\n    \"id\": \"550e8400-e29b-41d4-a716-446655440000\",\n    \"filename\": \"my-video.mp4\",\n    \"status\": \"completed\", // uploading, processing, completed, failed\n    \"progress\": 100,\n    \"streamUrl\": \"http://localhost:9000/video-streams/550e8400.../playlist.m3u8\",\n    \"createdAt\": \"2025-01-15T10:30:00Z\",\n    \"completedAt\": \"2025-01-15T10:35:00Z\"\n  }\n}\n```\n\n#### 4. List All Videos\n\nGet a list of all processed videos:\n\n```bash\nGET /api/v1/videos\nAuthorization: Bearer your_secure_api_key_here\n```\n\n#### 5. Check Callback Status\n\nMonitor webhook delivery status:\n\n```bash\nGET /api/v1/video/{uploadId}/callback-status\nAuthorization: Bearer your_secure_api_key_here\n```\n\n### Custom S3 Storage Paths\n\nYou can specify a custom path within your S3 bucket where videos should be stored using the `s3Path` parameter when creating an upload session.\n\n**Default behavior** (without s3Path):\n\n- Videos are stored directly in the bucket root: `video-streams/{videoId}/`\n- Stream URL: `http://localhost:9000/video-streams/{videoId}/index.m3u8`\n\n**With custom s3Path** (e.g., `\"s3Path\": \"/v2/media\"`):\n\n- Videos are stored under the custom path: `video-streams/v2/media/{videoId}/`\n- Stream URL: `http://localhost:9000/video-streams/v2/media/{videoId}/index.m3u8`\n\n**Path Requirements:**\n\n- Must contain only alphanumeric characters, forward slashes, hyphens, and underscores\n- Leading and trailing slashes are automatically handled\n- Examples: `/v2/media`, `client123/videos`, `year/2025/january`\n\n**Use Cases:**\n\n- **Multi-tenant applications**: `/tenant-{id}/videos`\n- **API versioning**: `/v2/media`, `/v3/content`\n- **Organizational structure**: `/department/marketing/videos`\n- **Date-based organization**: `/2025/january/uploads`\n\n## 🎬 Video Processing Workflow\n\n1. **Upload Session Creation**: Client creates upload session via API\n2. **TUS Upload**: Client uploads video file using TUS protocol\n3. **Queue Processing**: Video is queued for background processing\n4. **FFmpeg Transcoding**: Video is transcoded to HLS format\n5. **S3 Upload**: Processed segments uploaded to S3/MinIO\n6. **Webhook Notification**: Optional callback sent to your URL\n7. **Cleanup**: Original video file deleted from disk\n\n## 🔧 Development\n\n### Local Development Setup\n\n1. Install dependencies:\n\n```bash\nnpm install\n```\n\n2. Start development services (MinIO + RabbitMQ):\n\n```bash\ndocker-compose -f docker-compose.override.yml up -d minio rabbitmq\n```\n\n3. Start the development server:\n\n```bash\nnpm run dev\n```\n\n### Available Scripts\n\n```bash\nnpm run dev    # Start development server with hot reload\nnpm run build  # Build TypeScript to JavaScript\nnpm run start  # Start production server\n```\n\n## 🐳 Docker Configuration\n\n### Environment Variables\n\n| Variable             | Description                | Default                                   |\n| -------------------- | -------------------------- | ----------------------------------------- |\n| `SERVER_PORT`        | API server port            | `8001`                                    |\n| `NODE_ENV`           | Environment mode           | `dev`                                     |\n| `UPLOAD_PATH`        | Temporary upload directory | `./uploads`                               |\n| `MAX_FILE_SIZE`      | Maximum file size          | `100mb`                                   |\n| `ALLOWED_FILE_TYPES` | Allowed MIME types         | `video/mp4,video/avi,video/mov,video/mkv` |\n| `S3_ACCESS_KEY`      | S3/MinIO access key        | `minioadmin`                              |\n| `S3_SECRET_KEY`      | S3/MinIO secret key        | `minioadmin`                              |\n| `S3_ENDPOINT`        | S3/MinIO endpoint URL      | `http://minio:9000`                       |\n| `S3_BUCKET`          | Storage bucket name        | `video-streams`                           |\n| `API_KEY`            | Bearer token for API auth  | `your_api_key_here`                       |\n\n### Custom Docker Compose\n\nFor production, you may want to:\n\n1. Use external S3 service instead of MinIO\n2. Use managed RabbitMQ service\n3. Add reverse proxy (nginx/traefik)\n4. Configure SSL certificates\n5. Set up monitoring and logging\n\n## 🔐 Security Considerations\n\n- Change default API keys in production\n- Use strong, unique API keys\n- Enable HTTPS in production\n- Restrict CORS origins\n- Monitor API usage and rate limiting\n- Regularly update Docker images\n\n## 📊 Monitoring \u0026 Management\n\n### MinIO Console\n\nAccess MinIO management interface at: `http://localhost:9001`\n\n- Username: `minioadmin` (or your `S3_ACCESS_KEY`)\n- Password: `minioadmin` (or your `S3_SECRET_KEY`)\n\n### RabbitMQ Management\n\nAccess RabbitMQ management interface at: `http://localhost:15672`\n\n- Username: `guest` (or your `RABBITMQ_DEFAULT_USER`)\n- Password: `guest` (or your `RABBITMQ_DEFAULT_PASS`)\n\n### API Documentation\n\nInteractive API docs available at: `http://localhost:8001/api/v1/docs`\n\n## 🐛 Troubleshooting\n\n### Common Issues\n\n**Video processing fails:**\n\n- Check Docker container logs: `docker-compose logs app`\n- Ensure sufficient disk space and RAM\n- Verify FFmpeg is available in container\n\n**Upload fails:**\n\n- Check TUS endpoint configuration\n- Verify file size limits\n- Check allowed file types\n\n**Webhook callbacks fail:**\n\n- Ensure callback URL is accessible\n- Check callback endpoint returns 200 status\n- Monitor callback retry attempts\n\n**Storage issues:**\n\n- Verify MinIO/S3 credentials\n- Check bucket permissions\n- Ensure bucket exists\n\n### Logs\n\nView application logs:\n\n```bash\ndocker-compose logs -f app\n```\n\nView all service logs:\n\n```bash\ndocker-compose logs -f\n```\n\n## 🤝 Contributing\n\n1. Fork the repository\n2. Create a feature branch\n3. Make your changes\n4. Add tests if applicable\n5. Submit a pull request\n\n## 📄 License\n\nThis project is licensed under the MIT License.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F0necontroller%2Fvellum-stream","html_url":"https://awesome.ecosyste.ms/projects/github.com%2F0necontroller%2Fvellum-stream","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F0necontroller%2Fvellum-stream/lists"}