{"id":13616951,"url":"https://github.com/xyk2/cloud-transcoder","last_synced_at":"2025-04-14T03:32:14.964Z","repository":{"id":146728864,"uuid":"161835211","full_name":"xyk2/cloud-transcoder","owner":"xyk2","description":"Transcoding long (\u003e1 hour) videos quickly and cost-effectively on GCP to adaptive HLS and MP4 mezzanine files. Up to 40x $ savings.","archived":false,"fork":false,"pushed_at":"2018-12-15T10:35:41.000Z","size":3715,"stargazers_count":12,"open_issues_count":0,"forks_count":3,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-04-21T06:33:31.224Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/xyk2.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2018-12-14T20:08:09.000Z","updated_at":"2023-11-09T19:45:02.000Z","dependencies_parsed_at":"2024-01-07T20:12:08.766Z","dependency_job_id":null,"html_url":"https://github.com/xyk2/cloud-transcoder","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xyk2%2Fcloud-transcoder","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xyk2%2Fcloud-transcoder/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xyk2%2Fcloud-transcoder/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xyk2%2Fcloud-transcoder/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/xyk2","download_url":"https://codeload.github.com/xyk2/cloud-transcoder/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":213578932,"owners_count":15608061,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T20:01:35.123Z","updated_at":"2024-11-08T01:32:30.670Z","avatar_url":"https://github.com/xyk2.png","language":"JavaScript","readme":"# Cloud transcoding adaptive bitrate MP4 and HLS\n\nTranscodes a source video file (MP4, AVI, MPG, M4A, WMV, MOV) to adaptive bitrate MP4 and HLS to be stored in Google Cloud Storage. Cheaper than 3rd party transcoding services by more than 40-50x when running on preemptible instances (compared to AWS Elastic Transcoder, Zencoder, Bitmovin, Cloudinary), and saves time/bandwidth massively by transcoding in the same region as where the files are stored.\n\n\n### Configuring boot disk template for GCE\n```bash\nsudo add-apt-repository ppa:jonathonf/ffmpeg-3 # Enter to confirm\nsudo apt-get update\nsudo apt-get -y upgrade\nsudo apt install -y ffmpeg libav-tools x264 x265\n\nsudo curl -sL https://deb.nodesource.com/setup_6.x -o nodesource_setup.sh\nsudo bash nodesource_setup.sh\nsudo apt-get install nodejs -y\nsudo npm install pm2 -g\n\n# Route 8080 to port 80 to allow node to access port 80 without root privileges\nsudo iptables -t nat -I PREROUTING -p tcp --dport 80 -j REDIRECT --to-port 8080\n\n# Create ROOT RSA private key then copy to Github repo (pull access only)\nsudo ssh-keygen -t rsa\nsudo cat /root/.ssh/id_rsa.pub\n```\n\n\n### Startup script for each new instance\n```bash\n#! /bin/bash\n\nssh-keyscan github.com \u003e\u003e ~/.ssh/known_hosts\ngit clone git@github.com:xyk2/cloud-transcoder.git\n\ncd /cloud-transcoder\nnpm install\nPM2_HOME=/root/.pm2 pm2 kill\nPM2_HOME=/root/.pm2 NODE_ENV=production pm2 start app.js\n```\n\n### Shutdown script before preemption\n* Call localhost endpoint to reset running job in DB queue\n* Terminate PM2\n*\n\n### Workflow to upload \u0026 transcode \u0026 tag game videos from YouTube\n* `youtube-dl` to download and merge original files\n* `gsutil -m cp` to `broadcast-cx-raw-recordings` bucket\n* Insert filename, size, and format into `assets_game_footage` table\n* Insert type, id, status into `assets_transcode_queue`\n* Find game ID of game film, insert into `assets_game_footage` which will be populated\n* Find start_record_time of game film, insert into `assets_game_footage`\n\n\n### Useful snippets\n`youtube-dl`\nyoutube-dl --external-downloader aria2c --external-downloader-args '-x 8' https://www.youtube.com/playlist?list=PLUz3zvlwsLgWJRmJo8fLOTMiXkFC25S61 --playlist-start 46 --playlist-end 150\n\n\n### ffmpeg errors and possible errors\n* `ffmpeg exited with code 1: Conversion failed!` -max_muxing_queue_size 1024\n\n\n\n\n\n","funding_links":[],"categories":["HarmonyOS","Encoding"],"sub_categories":["Windows Manager","Talks Presentations Podcasts"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxyk2%2Fcloud-transcoder","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fxyk2%2Fcloud-transcoder","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxyk2%2Fcloud-transcoder/lists"}