{"id":13616946,"url":"https://github.com/ultravideo/cloud-encoder","last_synced_at":"2025-10-30T00:59:56.812Z","repository":{"id":76822845,"uuid":"172061533","full_name":"ultravideo/cloud-encoder","owner":"ultravideo","description":"Kvazaar Cloud Encoder","archived":false,"fork":false,"pushed_at":"2019-03-11T10:53:58.000Z","size":704,"stargazers_count":4,"open_issues_count":0,"forks_count":0,"subscribers_count":10,"default_branch":"master","last_synced_at":"2024-07-11T15:48:41.796Z","etag":null,"topics":["cloud","encoding","ffmpeg","hevc","transcoding"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-2-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ultravideo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"COPYING","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2019-02-22T12:19:14.000Z","updated_at":"2024-03-28T20:09:56.000Z","dependencies_parsed_at":"2023-04-27T15:35:10.010Z","dependency_job_id":null,"html_url":"https://github.com/ultravideo/cloud-encoder","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ultravideo%2Fcloud-encoder","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ultravideo%2Fcloud-encoder/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ultravideo%2Fcloud-encoder/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ultravideo%2Fcloud-encoder/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ultravideo","download_url":"https://codeload.github.com/ultravideo/cloud-encoder/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":213578929,"owners_count":15608061,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cloud","encoding","ffmpeg","hevc","transcoding"],"created_at":"2024-08-01T20:01:35.061Z","updated_at":"2025-10-30T00:59:51.783Z","avatar_url":"https://github.com/ultravideo.png","language":"JavaScript","readme":"# Kvazaar Cloud Encoder\n\nKvazaar in cloud with 2-clause BSD license\n\nIndent by 4 spaces. (no tabs)\n\n# Building and Running\n\n`\ndocker build -t cloud .\n`\n\n```\ndocker run --rm --name cloud-postgres -v ~/kvazaar-cloud-db:/var/lib/postgresql/data -d --env POSTGRES_PASSWORD=postgres postgres:latest\ndocker run --rm --name cloud-encoder -d --link cloud-postgres:pg -p 80:8080 -p 443:8443 -v /tmp:/tmp cloud\n```\n\nIf you want to use the Kvazaar Cloud Encoder in localhost, you need to change the variable *httpsEnabled* in socket.js to false and run:\n\n```\ndocker run --rm --name cloud-postgres -v ~/kvazaar-cloud-db:/var/lib/postgresql/data -d --env POSTGRES_PASSWORD=postgres postgres:latest\ndocker run --rm --name cloud-encoder -d --link cloud-postgres:pg -p 80:8081 -v /tmp:/tmp cloud\n```\n\n# How everything works\n\n1) User (re)initializes the connection to server using WebSocket\n2) User sends options and file info to server\n3) Server checks that given options are all valid\n   * If theyare -\u003e file upload is rejected\n4) Server check if this file already exists\n   * If it does -\u003e file upload is rejected\n5) File upload is started\n6) When file upload is completed, server checks the uploaded file is video and \u003c FILE_TIME_LIMIT_IN_SECONDS (30min)\n7) Task is added to work queue\n8) One of the workers takes the task under work and user is sent status update about the task\n9) When the task is ready, user is able to download the output file or delete the task\n\nAt any point user is able to cancel the request and make another one. User is also able make multiple concurrent request (some of them may be queued).\n\nDownload limit for each requets is FILE_DOWNLOAD_LIMIT (2) and after that the task is deleted.\n\n# Architecture\n\n## server.js\nServer is responsible for handling the file upload and validation.\n\nFile is considered valid if it's a video file and its duration is \u003c FILE_TIME_LIMIT_IN_SECONDS (30min). For raw videos the limit is FILE_SIZE_LIMIT_IN_BYTES (50GB)\n\nServer.js communicates with the user through socket.js. See the message specification below for more details\n\nIt also spawns NUMBER_OF_WORKER_THREADS (5) worker threads and 1 message queue thread.\n\n## worker.js\nCloud uses NUMBER_OF_WORKER_THREADS worker threads for processing user requests. Workers listen to Kue's message queue for encoding requests.\n\nHow a worker thread works is very straightforward:\n\n1) Check if given video file is raw in yuv420p format\n   * If it is, no need to preprocess\n   * If it isn't:\n      * If input is raw -\u003e convert the given format to yuv420p\n\t  * If the input is not raw -\u003e extract raw video (yuv420p) and audio (if necessary) using FFMPEG\n2) Add Kvazaar logo at the end of video\n3) Encode the raw video using Kvazaar\n4) If user has so specified, add the encoded video to container\n5) Remove intermediate files (extracted audio track and raw video file)\n6) Inform user that the encoding process is ready and file can be downloaded\n\nUser is sent status messages about the encoding process through socket.js.\n\n## socket.js\nSocket.js provides a way for server and workers to interact with the user. Workers and the server send messages through Redis to socket.js which then sends these messages to user.\n\n## parser.js\nParser.js is responsible for parsing and validating most user input. It handles both file related info (f.ex. FPS and resolution) but also Kvazaar options.\n\n## db.js\ndb.js provides a database API for all other files.\n\n# Messages\n\nClient and server communicate with each other using request/reply pairs. For example, when user clicks the \"Encode\" button the front-end code sends an *uploadRequest* to socket.js. Socket then validates the request (section \"How everything works\") and either rejects or approves the upload in *uploadReply* message.\n\nSome other message types used by the system are:\n* *taskQuery* sent by the front-end to fetch all tasks made by the user\n* *cancelInfo* sent by the front-end code when user cancels an ongoing upload\n* Internal *cancelRequest* sent from socket.js to worker.js to cancel an ongoing task\n","funding_links":[],"categories":["HarmonyOS"],"sub_categories":["Windows Manager"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fultravideo%2Fcloud-encoder","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fultravideo%2Fcloud-encoder","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fultravideo%2Fcloud-encoder/lists"}