{"id":13908599,"url":"https://github.com/kahache/video_packaging_platform","last_synced_at":"2025-07-18T07:31:41.718Z","repository":{"id":163418424,"uuid":"282461371","full_name":"kahache/video_packaging_platform","owner":"kahache","description":"Video packaging platform - this will build a Docker with a web API that will let you upload, encrypt and serve videos as MPEG DASH files","archived":false,"fork":false,"pushed_at":"2020-09-06T12:30:15.000Z","size":12231,"stargazers_count":11,"open_issues_count":0,"forks_count":4,"subscribers_count":4,"default_branch":"master","last_synced_at":"2024-11-25T17:46:22.773Z","etag":null,"topics":["api","bento4","dash","docker","encryption","ffmpeg","flask","flask-sqlalchemy","mysql","python3","video","video-process"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kahache.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2020-07-25T14:34:43.000Z","updated_at":"2024-01-19T10:27:32.000Z","dependencies_parsed_at":null,"dependency_job_id":"e132afe4-1aa0-492a-a326-28adc20da9f7","html_url":"https://github.com/kahache/video_packaging_platform","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/kahache/video_packaging_platform","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kahache%2Fvideo_packaging_platform","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kahache%2Fvideo_packaging_platform/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kahache%2Fvideo_packaging_platform/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kahache%2Fvideo_packaging_platform/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kahache","download_url":"https://codeload.github.com/kahache/video_packaging_platform/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kahache%2Fvideo_packaging_platform/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265720599,"owners_count":23817269,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api","bento4","dash","docker","encryption","ffmpeg","flask","flask-sqlalchemy","mysql","python3","video","video-process"],"created_at":"2024-08-06T23:02:51.310Z","updated_at":"2025-07-18T07:31:41.101Z","avatar_url":"https://github.com/kahache.png","language":"Python","readme":"# video_packaging_platform\n\nThis is a very basic API with a persistance layer that will process video files as a background task. It will receive them, encrypt them and process them into MPEG-DASH ready to be streamed.\n\n## Getting Started\n\nObviously, you first need to clone this repo\n```\ngit clone https://github.com/kahache/video_packaging_platform.git\n```\nAnd then, you can launch the main App \n```\ncd video_packaging_platform\ndocker build . -t video_packaging_platform --no-cache\ndocker-compose up\n```\nIt will start running the system on http://0.0.0.0:5000 on your system. \n\nOnce you open that link on your browser, you'll find a small index that will detail you the 4 options you can launch. \n\nIn case you'd like to open locally, please notice you should have the Bento 4 SDK for your current OS and even replace the files at /bin/ and /utils just in case. Once done, just run python3 app/app.py\n\n### Prerequisites\n\nYou'll need some software installed in your system:\n```\nPython 3.6 or higher\nMySQL Server\nDocker\n(in case you run local outside Docker, pip3 install -r app/requirements.txt)\n```\n\n### How does it work\n\nThis platform opens up a simple web file loader. You can insert files and they will be processed. \nOnce you have the files packaged, you can open the files with VLC Media Player or FFPlay. You should be able to hear the audio but you shouldn't be able to see the video.\n\n* The Video logic\n\nFor this exercise, first of all we have considerated about the encryption. The point is that the industry recommends to encrypt all the tracks contained in the MP4 file.\n\nHowever, as in the example given we are only going to have 1 Key and 1 KID, I have decided to encrypt only the video track. Why?\n\nFirst reason is to be able to track all the video process. When it's success, we can open the file and hear the ideo but receive glitches/bad image as video input. I suspect this is the same way BeIn Sports is encrypted in Spain with Movistar Imagenio: even you know the UDP address and try to record it with FFMpeg (theorically they aren't encrypted), you're able to record a video file with a lot of glitches but with clear sound.\n\nThis will also be a way to ensure nobody has done this exercise as this before :-)\n\nSo the first point will be to receive a file and determine wether it can be processed or not, and extract metadata from the container like this\n\n![](images/video_ops_ingest.png)\n\nAs we can see, only files able to be processed will continue the operation.\n\nFor the rest of the process, we are going to work in a chain with the required steps to have an MPEG-DASH output.\n\nThat means, if one step fails, we can't continue the process. This will be the logic contained inside the main class in video_ops.py\n\n![](images/VideoOperationsLogic.png)\n\nAll this is processed in **app/video_ops.py**, which will cal to /bin/ and /utils/ when needed\n\n* The Database\n\nAfter consider the simplicity of the exercise, I've decided to go only with one table only. This is because the API will only do a few stuff. For further professional environments, we should consider to have several tables and add more data into the tables, such as datetimes (creation, processing, etc), users, etc.\n\nI've considered only 9 rows for this exercise:\n```\n+----------------------+--------------+------+-----+---------+----------------+\n| Field                | Type         | Null | Key | Default | Extra          |\n+----------------------+--------------+------+-----+---------+----------------+\n| input_content_id     | int(11)      | NO   | PRI | NULL    | auto_increment |\n| input_content_origin | varchar(255) | YES  |     | NULL    |                |\n| video_track_number   | int(11)      | YES  |     | NULL    |                |\n| status               | varchar(255) | YES  |     | NULL    |                |\n| output_file_path     | varchar(255) | YES  |     | NULL    |                |\n| video_key            | varchar(255) | YES  |     | NULL    |                |\n| kid                  | varchar(255) | YES  |     | NULL    |                |\n| packaged_content_id  | int(11)      | YES  | UNI | NULL    |                |\n| url                  | varchar(255) | YES  |     | NULL    |                |\n+----------------------+--------------+------+-----+---------+----------------+\n9 rows in set (0,00 sec)\n```\n**_input_content_id:_** : the ID generated for each file that is ingested in the platform.  \n\n**_input_content_origin_** : The path of the original file. This will be used to check if we have already uploaded a file with that name or not. \n\n**_video_track_number_** : Video track number ID from metadata extracted with Bento4 utils, notice for other softwares as FFMpeg should be different wether they count zero as first or not.\n\n**_status_** : This is very important. This will be a text cell that will explain the last operation done with that video file.\n    It can be:\n    \n    \"Ingested\" - file moved to storage and video track number saved\n    \"Fragmented\" - file has been fragmentated\n    \"Encrypted\" - file's video track has been encrypted with the KEY and KID values given in the JSON\n    \"Ready\" - file has been converted into MPEG-DASH and has a URL output\n\n**_output_file_path_** : This will be the path of the last processed file. For each video process, we generate a new file. For this exercise we aren't deleting the non-usable files, so we can track the results. Notice that for further production environments, this should be erased with some cronjob \n\n**_video_key_** : video KEY for AES CBCS encryption given by user\n\n**_kid_** : video KID for AES CBCS encryption given by user \n\n**_packaged_content_id_** : ID for a file that is in the process of being packaged. This is a random number generated by our main App.\n\n**_url_** : Last cell, only filled if a video has been packaged and contains its output link. For this exercise it will be inside the filepath inside the server.       \n\nFor the docker, the database is declared in **/db/db.init.sql** \n\n* The operations logic, mixing Video and Database \n\nThis is quite simple. The first endpoint is what happens when we click on \"upload\" and we sent the file into the system:\n\n![](images/video_received.png)\n\nThe second endpoint is called with a JSON, for example:\n```\nPOST​ /packaged_content {\n“input_content_id”: 1,\n“key”: “hyN9IKGfWKdAwFaE5pm0qg”,\n“kid”: “oW5AK5BW43HzbTSKpiu3SQ” }\n```\nSo everything starts to run in the background with this logic:\n\n![](images/VideoOperationsLogic.png)\n\nQuite simple.\n\nThe third endpoint doesn't need explanation, it will just do a Database query and return the output as a JSON.\n\nFor example, we call it like:\n```\nGET​ /packaged_content/55\n```\n\nAnd will receive an output like:\n```\n{\n\"packaged_content_id\": 73\n\"url\": http://localhost:8080/5F8LNI/dash/stream.mpd\n\n}\n```\n\n* The Flask API\n\nThis simply redirects the endpoints of the API with the templates and the operations. It's the main file, which gets executed automatically and starts the service at http://0.0.0.0:5000\n\nIMPORTANT: For this exercise, we aren't erasing the files in the process. This means we can track and check the video outputs before it's full done in the output/ folder. This also means that one video can have 'several' content_package_id. I mean, it's not checking wether the file has been already packaged or not. This has been done like this in order to do several testings with only 1 uploaded file. If we want to avoid this, the code should be changed.\n## Guide to validate playback correctness of the content packaged. \n\nWhen we have the output, I've added an extra endpoint to start a file browser service with different port (this should be user access, not API access).\n```\nGET http://0.0.0.0:5000 \n```\nSo we can call the last method of the API to run this service. Once ran, we should be able to browse the file from the browser opening http://0.0.0.0:8080/videos (as we see, now the output is 8080 instead of the original 5000, be careful so you don't confuse!)\n\nOnce it's open, we can open the video directly with [VLC Media Player](https://www.videolan.org/vlc/index.es.html) or with [FFPlay](https://ffmpeg.org/ffplay.html).\n\nIf we open the video, we shouldn't be able to see the video track as it's encrypted. This has been tested with both softwares.\n\nTo check the decrypted video, you should have Google Chrome installed with a higher version than 69.\n\nFirst you should edit the file \n```\n/tests/test_player/myapp.js\n```\nHere you need to replace the output URL you should have from the packaged content, e.g. for http://localhost:8080/NZ7KLC/dash/stream.mpd\n```\nconst manifestUri =\n    'http://localhost:8080/NZ7KLC/dash/stream.mpd';\n```\n\nThen in the same file, you should convert your KEY and KID into hexadecimal and put it inside. For example, let's assume we have the values:\n```\n\"key\":\"hyN9IKGfWKdAwFaE5pm0qg\",\"kid\":\"oW5AK5BW43HzbTSKpiu3SQ\"\n```\nIf we go to this [Base64 converter website](https://cryptii.com/pipes/binary-to-base64) or any other website, we can convert the previous values into hexadecimal:\n```\n\"key\":\"87237d20a19f58a740c05684e699b4aa\",\"kid\":\"a16e402b9056e371f36d348aa62bb749\"\n```\nNow that you have the values in hexadecimal, you should put them inside the code of myapp.js. You'll find just a few lines down. Keep in mind the KID goes before KEY!:\n```\n  // IMPORTANT!\n  // Here we add the concrete KID and KEY we need to decrypt the video.\n  player.configure({\n  drm: {\n    clearKeys: {\n      // 'key-id-in-hex': 'key-in-hex',\n      'a16e402b9056e371f36d348aa62bb749': '87237d20a19f58a740c05684e699b4aa'\n      }\n    }\n  });\n```\n\nAnd save the file, obviously :-)\n\nSecond we need to open the Chrome browser without [cors](https://en.wikipedia.org/wiki/Cross-origin_resource_sharing), you can follow [this very simple guide and open from terminal](https://alfilatov.com/posts/run-chrome-without-cors/).\nThen open this path as the URL:\n```\nYOUR/PATH/TO/REPO/video_packaging_platform/tests/test_player/player.html\n```\n\nAnd enjoy the video!\n\nIn order to do further testing, we should consider using another specialized players and test with different browsers, such us:\n·Test Player from [DASH IF](https://github.com/Dash-Industry-Forum/dash.js/blob/development/samples/drm/clearkey.html)\n·Test Player from private Video services such as \n[Bitmovin](https://bitmovin.com/demos/drm), info [here](https://bitmovin.com/docs/player/tutorials/how-to-play-mpeg-cenc-clearkey-content)\n[Wowza](https://www.wowza.com/testplayers)\nAnd so on. You can check almost any company you'll find in Pavilion number 14 @ IBC each year. Or for example it seems there are some players online:\n[JW Player](https://www.jwplayer.com/developers/stream-tester/)\n\nIMPORTANT: Please notice none of this players have been tested.\n\n# How would I modify/extend the service so we achieve multi bitrate resolution? \n\nThere are some solutions. \n\nThe first one is to process the video with [FFMpeg](https://ffmpeg.org), the infamous video tool that lets you do (almost) everything. One option can be to transcode the video file and use a multibitrate output. However, FFMpeg doesn't allow us to encrypt the video with ClearKey. This has been tested and I've already did in the past video with different bitrates. So if we avoid the encryption process, this can be a real solution.\n\nThe second one, (this hasn't been tested) and seems to be the real how-to, is to work with [MP4Box](https://gpac.wp.imt.fr). It seems this software allows us to combine with FFMpeg and it lets us to encrypt the files.\nMore info about this method on these 2 links from the guys from Streamroot, first [here](https://blog.streamroot.io/encode-multi-bitrate-videos-mpeg-dash-mse-based-media-players/) and [here](https://blog.streamroot.io/encode-multi-bitrate-videos-mpeg-dash-mse-based-media-players-22/)\n\nAnother important point is this API hasn't been intended to be multithread as it wasn't required. So it would be really interesting in order to extend the service, to work with task queue and be able to manage all the processes.\n## Running the tests\n\nTo run the tests, get with Linux/UNIX terminal into the /tests/ forlder and run:\n```\nbash run_tests.sh\n```\nThere you'll find a dialog box where you can choose the different tests to run. It's highly recommended to first download the test videos otherwise it can give errors. Feel free to check out the code and change variables or situations you'd like to test.\n\nAlso, please notice this aren't the typical unit tests. They can run background operations too. This has been intended to be able to test MP4 corrupted video files, in order to check wether they can be processed or not.\n\n## Files included in this package\n\n```\n.\n└── video_packaging_platform\n    ├── Dockerfile\n    ├── README.md\n    ├── app\n    │   ├── app.py\n    │   ├── database.py\n    │   ├── main_ops.py\n    │   ├── models.py\n    │   ├── requirements.txt\n    │   ├── templates\n    │   │   ├── file_upload_form.html\n    │   │   ├── success.html\n    │   │   └── success_packaged.html\n    │   └── video_ops.py\n    ├── bin\n    │   ├── mp4-dash.py\n    │   ├── mp4dash\n    │   ├── mp4decrypt\n    │   ├── mp4dump\n    │   ├── mp4encrypt\n    │   ├── mp4fragment\n    │   ├── mp4info\n    │   └── mp4split\n    ├── db\n    │   └── init.sql\n    ├── docker-compose.yml\n    ├── images\n    │   ├── Video\\ Operations\\ logic.jpg\n    │   ├── VideoOperationsLogic.png\n    │   ├── video_ops_ingest.jpg\n    │   └── video_ops_ingest.png\n    ├── tests\n    │   ├── TEST_VIDEOS\n    │   │   ├── download_videos.sh\n    │   │   └── public_test_video_list.txt\n    │   ├── __init__.py\n    │   ├── run_tests.sh\n    │   ├── test_app.py\n    │   ├── test_database.py\n    │   ├── test_models.py\n    │   └── test_video_ops.py\n    └── utils\n        ├── aes.py\n        ├── check-indexes.py\n        ├── mp4-dash-clone.py\n        ├── mp4-dash-encode.py\n        ├── mp4-dash.py\n        ├── mp4-hls.py\n        ├── mp4utils.py\n        ├── pr-derive-key.py\n        ├── skm.py\n        ├── subtitles.py\n        └── wv-request.py\n```\n\n## Built With\n\n* [Python](https://www.python.org/downloads/release/python-360/) - Most programming language used\n* [Flask](https://flask.palletsprojects.com/en/1.1.x/) - Framework used to generate the API\n* [Flask-SQLAlchemy](https://flask-sqlalchemy.palletsprojects.com/en/2.x/) - Library used as ORM\n* [MySQL](https://www.mysql.com) - Relational database system used\n* [Bento4](https://www.bento4.com) - MP4 \u0026 Dash library\n* [Docker](https://www.docker.com) - PaaS that uses OS-level virtualization to deliver software in packages called containers\n\n## Authors\n\n* **Javier Brines Garcia** - *Initial work* - [LinkedIn](https://www.linkedin.com/in/javi-brines-cto)\n\n## License\n\nThis project is licensed under the MIT License - see the [LICENSE.md](LICENSE.md) file for details\n\n## Acknowledgments\n\n* Guillem C. \u0026 David V.\n* Roc a.k.a. Roc Rocks, for the Docker comments\n* Ben M., for the tips with the player\n","funding_links":[],"categories":["HarmonyOS"],"sub_categories":["Windows Manager"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkahache%2Fvideo_packaging_platform","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkahache%2Fvideo_packaging_platform","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkahache%2Fvideo_packaging_platform/lists"}