{"id":31079616,"url":"https://github.com/mrpurushotam/content-streaming","last_synced_at":"2025-09-16T10:29:35.238Z","repository":{"id":279099842,"uuid":"925211849","full_name":"MrPurushotam/content-streaming","owner":"MrPurushotam","description":"A full stack project based on microservice archieture and a react based frontend to stream video content over the internet using hsl streaming with a seemless ui/ux.","archived":false,"fork":false,"pushed_at":"2025-06-07T16:21:32.000Z","size":2082,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-08-08T19:54:51.029Z","etag":null,"topics":["aws","ci-cd","custompackage","expressjs","ffmpeg","fluent-ffmpeg","github-actions","githubaction-workflow","lambda","microservice","npm","postgressql","prismaorm","reactjs","s3","shadcn-ui","typescript","workflow"],"latest_commit_sha":null,"homepage":"https://stream.purushotamjeswani.in","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MrPurushotam.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-01-31T12:56:46.000Z","updated_at":"2025-08-07T05:52:13.000Z","dependencies_parsed_at":"2025-02-23T19:31:34.670Z","dependency_job_id":"ea91635b-d091-448b-adcf-3c3718cbfe53","html_url":"https://github.com/MrPurushotam/content-streaming","commit_stats":null,"previous_names":["mrpurushotam/content-streaming"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/MrPurushotam/content-streaming","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MrPurushotam%2Fcontent-streaming","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MrPurushotam%2Fcontent-streaming/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MrPurushotam%2Fcontent-streaming/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MrPurushotam%2Fcontent-streaming/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MrPurushotam","download_url":"https://codeload.github.com/MrPurushotam/content-streaming/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MrPurushotam%2Fcontent-streaming/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":275404379,"owners_count":25458923,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-16T02:00:10.229Z","response_time":65,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","ci-cd","custompackage","expressjs","ffmpeg","fluent-ffmpeg","github-actions","githubaction-workflow","lambda","microservice","npm","postgressql","prismaorm","reactjs","s3","shadcn-ui","typescript","workflow"],"created_at":"2025-09-16T10:29:33.645Z","updated_at":"2025-09-16T10:29:35.228Z","avatar_url":"https://github.com/MrPurushotam.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Livestreaming Application\n\nA full-featured video streaming platform built with modern web technologies and cloud infrastructure, enabling users to upload, process, and stream videos with adaptive bitrate streaming.\n\n![Project Banner](./frontend/public/Homepage.png)\n\n## 🌟 Features\n\n### Core Functionality\n- **Video Upload \u0026 Processing**: Secure video upload to AWS S3 with automated processing pipeline\n- **Adaptive Bitrate Streaming**: HLS (HTTP Live Streaming) with multiple quality options (360p, 720p, 1080p)\n- **Real-time Video Player**: Custom HLS player with quality selection and buffering management\n- **Content Management**: Admin dashboard for video management and user approval\n\n### Technical Features\n- **Cloud Infrastructure**: AWS Lambda-based serverless video processing\n- **Queue Management**: Redis-based job queuing for video processing tasks\n- **CDN Integration**: CloudFront distribution for optimized content delivery\n- **File Size Validation**: Client and server-side file size validation\n- **Presigned URLs**: Secure direct-to-S3 uploads with temporary access\n- **Retry Logic**: Automated retry mechanisms for failed video processing\n\n## 🔍 Overview\n\nThis platform enables content creators to upload videos, which are then automatically processed, transcoded into multiple resolutions, and made available for streaming with adaptive bitrate technology. The application ensures high-quality video playback regardless of the viewer's network conditions.\n\nThe system uses a queue-based architecture to handle video processing asynchronously, ensuring scalability and reliability even with large video files and high traffic.\n\n## 🏗️ Architecture\n\nThe application follows a modern microservice-oriented architecture:\n\n```\n┌─────────────┐     ┌─────────────┐     ┌─────────────┐\n│   Frontend  │────▶│    API      │────▶│  Database   │\n│   (React)   │◀────│  (Express)  │◀────│ (PostgreSQL)│\n└─────────────┘     └─────────────┘     └─────────────┘\n                          │  ▲\n                          │  │\n                          ▼  │\n┌─────────────┐     ┌─────────────┐     ┌─────────────┐\n│  S3 Storage │◀───▶│ Video Queue │◀───▶│    Video    │\n│    (AWS)    │     │   (Redis)   │     │ Processing  │\n└─────────────┘     └─────────────┘     └─────────────┘\n```\n\n- **Frontend**: React application with TypeScript, using Zustand for state management\n- **Backend API**: Express.js server handling authentication, content management\n- **Database**: PostgreSQL with Prisma ORM\n- **Queuing System**: Redis-based message queues for video processing jobs\n- **Storage**: AWS S3 buckets for video files and processed content\n- **Processing Pipeline**: FFmpeg-based video transcoding microservice\n\n## 🛠️ Tech Stack\n\n- **Frontend**:\n  - React 18\n  - TypeScript\n  - Zustand (State Management)\n  - TailwindCSS + ShadCN UI\n  - Video.js (Player)\n\n- **Backend**:\n  - Node.js with Express\n  - PostgreSQL with Prisma ORM\n  - Redis for queuing\n  - JWT for authentication\n  - Serverless functions for processing\n\n- **Cloud \u0026 Infrastructure**:\n  - AWS S3 for storage\n  - AWS Lambda for serverless functions\n  - CDN for content delivery\n\n- **Video Processing**:\n  - FFmpeg for transcoding\n  - HLS (HTTP Live Streaming) protocol\n\n## 📦 Installation \u0026 Setup\n\n### Prerequisites\n\n- Node.js (v18+)\n- PostgreSQL\n- Redis\n- AWS Account with S3 access\n- FFmpeg (for local development)\n\n### Backend Setup\n\n1. Clone the repository:\n   ```bash\n   git clone https://github.com/MrPurushotam/content-streaming.git\n   cd livestreaming-application\n   ```\n\n2. Install dependencies for the backend:\n   ```bash\n   cd backend\n   npm install\n   ```\n\n3. Set up environment variables (create `.env` file):\n   ```\n   DATABASE_URL=postgresql://user:password@localhost:5432/videodb\n   AIVEN_REDIS_URL=redis://localhost:6379\n   AIVEN_REDIS_PASSWORD=your_redis_password\n   \n   AWS_REGION=us-east-1\n   AWS_ACCESS_KEY_ID=your_access_key\n   AWS_SECRET_ACCESS_KEY=your_secret_key\n   AWS_S3_PRIMARYBUCKET_NAME=your-primary-bucket\n   AWS_S3_SECONDARYBUCKET_NAME=your-secondary-bucket\n   \n   JWT_SECRET=your_jwt_secret\n   FRONTEND_URL=http://localhost:5173\n   \n   CDN_BASE_URL=https://your-cdn-url.com\n   \n   VIDEO_PROCESSING_FUNCTION_URL=http://localhost:3001\n   S3CLEANUP_FUNCTION_URL=http://localhost:3002\n   ```\n\n4. Start the backend server:\n   ```bash\n   npm run dev\n   ```\n\n### Frontend Setup\n\n1. Install dependencies for the frontend:\n   ```bash\n   cd ../frontend\n   npm install\n   ```\n\n2. Set up environment variables (create `.env` file):\n   ```\n   VITE_API_URL=http://localhost:3000/api/v1\n   ```\n\n3. Start the frontend development server:\n   ```bash\n   npm run dev\n   ```\n\n### Video Processing Service Setup\n\n1. Install dependencies for the video processing service:\n   ```bash\n   cd ../backend/pipeline/VideoProcessing\n   npm install\n   ```\n\n2. Create a `.env` file with the same variables as the backend.\n\n3. Start the video processing service:\n   ```bash\n   npm run dev\n   ```\n\n### S3 Cleanup Service Setup\n\n1. Install dependencies for the S3 cleanup service:\n   ```bash\n   cd ../S3Cleanup\n   npm install\n   ```\n\n2. Create a `.env` file with the same variables as the backend.\n\n3. Start the S3 cleanup service:\n   ```bash\n   npm run dev\n   ```\n\n## 🔧 Configuration\n\n### Database Configuration\n\nThe application uses Prisma ORM with a PostgreSQL database. The schema is defined in:\n- `/Packages/CustomPrismaPackage/prisma/schema.prisma`\n\nKey entities include:\n- `User`: Stores user information and authentication details\n- `Content`: Represents uploaded videos with metadata\n- `BiteRateVideo`: Tracks different resolution versions of videos\n- `VideoSourceInfo`: Manages the original video source information\n\n### AWS S3 Configuration\n\nThe application uses two S3 buckets:\n1. **Primary Bucket**: For initial video uploads (temporary storage)\n2. **Secondary Bucket**: For processed videos and HLS content (permanent storage)\n\n### Redis Queue Configuration\n\nThe application uses Redis for message queues:\n- `ffmpeg_queue`: Video processing jobs\n- `primary_bucket_queue`: S3 cleanup jobs\n\n## 🚀 Usage\n\n### User Workflow\n\n1. **Registration \u0026 Login**:\n   - Create an account with email and password\n   - Admin users require approval before posting content\n\n2. **Uploading Videos**:\n   - Navigate to the upload page\n   - Get a presigned URL for S3 upload\n   - Upload video directly to S3\n   - Add metadata (title, description, thumbnail)\n   - Submit for processing\n\n3. **Processing**:\n   - Video is queued for processing\n   - FFmpeg transcodes video to multiple resolutions\n   - HLS playlists and segments are created\n   - Processed files are stored in S3\n\n4. **Viewing Content**:\n   - Browse videos on the homepage\n   - Click to view a video\n   - Video player automatically selects the appropriate resolution based on bandwidth\n\n### Admin Workflow\n\n1. **User Management**:\n   - Approve or reject new admin users\n   - Manage existing users\n\n2. **Content Management**:\n   - Review uploaded content\n   - Delete inappropriate content\n   - Edit video metadata\n\n## 🧪 Development\n\n### Project Structure\n\n```\nlivestreaming-application/\n├── backend/\n│   ├── libs/                # Shared libraries (AWS, etc.)\n│   ├── middleware/          # Express middleware\n│   ├── pipeline/            # Video processing services\n│   │   ├── VideoProcessing/ # FFmpeg transcoding service\n│   │   └── S3Cleanup/       # S3 cleanup service\n│   ├── prisma/              # Database schema and migrations\n│   ├── routes/              # API routes\n│   └── utils/               # Utility functions\n├── frontend/\n│   ├── public/              # Static assets\n│   └── src/\n│       ├── components/      # Reusable UI components\n│       ├── pages/           # Page components\n│       ├── store/           # Zustand store\n│       └── utils/           # Utility functions\n└── Packages/\n    └── CustomPrismaPackage/ # Shared Prisma client package\n```\n\n### Development Guidelines\n\n- **Frontend**: Follow React best practices, use TypeScript for type safety\n- **Backend**: Use Express middleware for authentication, validation\n- **Database**: Use Prisma migrations for schema changes\n- **Testing**: Write unit tests for critical components\n\n## 📤 Deployment\n\n### Backend Deployment\n\nThe backend is designed to be deployed as serverless functions:\n\n1. **API Server**:\n   - Deploy as an AWS Lambda function\n   - Configure with API Gateway\n\n2. **Video Processing**:\n   - Deploy as a separate AWS Lambda function\n   - Configure with higher memory and longer timeout\n\n3. **S3 Cleanup**:\n   - Deploy as a scheduled AWS Lambda function\n   - Configure to run periodically\n\n### Frontend Deployment\n\n1. Build the frontend:\n   ```bash\n   cd frontend\n   npm run build\n   ```\n\n2. Deploy the built assets to a static hosting service (AWS S3, Vercel, Netlify)\n\n## 📋 API Endpoints\n\n### Authentication\n- `POST /auth/register` - User registration\n- `POST /auth/login` - User login\n- `GET /auth/verify` - Token verification\n\n### Content Management\n- `GET /video/` - Get all public videos\n- `GET /video/:id` - Get specific video\n- `PUT /video/view/:id` - Increment view count\n- `POST /content/metadata` - Upload video metadata\n- `POST /content/confirmsource` - Confirm video upload\n- `GET /content/status/:id` - Get processing status\n\n### Admin\n- `GET /user/uploads` - Get user's uploaded videos\n- `PUT /video/:id` - Update video details\n- `DELETE /video/:id` - Delete video\n- `GET /user/list/admin` - Get pending admin approvals\n\n## 🎥 Video Processing Pipeline\n\n1. **Upload Request**: Client requests presigned URL\n2. **Direct Upload**: Video uploaded directly to S3 primary bucket\n3. **Queue Processing**: Video processing job added to Redis queue\n4. **Lambda Processing**: FFmpeg processes video into multiple bitrates\n5. **HLS Generation**: Creates master playlist and segment files\n6. **S3 Storage**: Processed files stored in secondary bucket\n7. **Database Update**: Video status updated to \"published\"\n8. **CDN Distribution**: Content available via CloudFront\n\n## 🔧 File Size Limits\n\n- **Video Files**: 100MB maximum\n- **Thumbnail Images**: 5MB maximum\n- **Supported Formats**: MP4, MOV, AVI (video) / JPG, PNG, WEBP (thumbnails)\n\n## 🎯 Quality Options\n\nThe application automatically generates three quality levels:\n- **360p**: 640x360, 800k bitrate\n- **720p**: 1280x720, 2.5M bitrate  \n- **1080p**: 1920x1080, 5M bitrate\n\n## 🛡️ Security Features\n\n- JWT-based authentication\n- Admin role-based access control\n- Rate limiting on API endpoints\n- File type and size validation\n- CORS protection\n- Presigned URL expiration (3 hours)\n\n## 📱 Responsive Design\n\nThe application is fully responsive and optimized for:\n- Desktop browsers\n- Tablet devices\n- Mobile phones\n- Touch interactions\n\n## 🤝 Contributing\n\n1. Fork the repository\n2. Create a feature branch\n3. Commit your changes\n4. Push to the branch\n5. Create a Pull Request\n\n## 📄 License\n\nThis project is licensed under the ISC License.\n\n## 👨‍💻 Developer\n\nBuilt by **Purushotam Jeswani**\n- GitHub: [@MrPurushotam](https://github.com/MrPurushotam)\n- LinkedIn: [purushotamjeswani](https://linkedin.com/in/purushotamjeswani)\n- Email: work.purushotam@gmail.com\n\n---\n\n## Acknowledgements\n\n- FFmpeg for video transcoding\n- React and Express teams for the excellent frameworks\n- Video.js for the player implementation\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmrpurushotam%2Fcontent-streaming","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmrpurushotam%2Fcontent-streaming","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmrpurushotam%2Fcontent-streaming/lists"}