{"id":18377287,"url":"https://github.com/bbc/cnn-rate-distortion","last_synced_at":"2025-04-06T21:31:30.144Z","repository":{"id":136960097,"uuid":"263589866","full_name":"bbc/cnn-rate-distortion","owner":"bbc","description":"CNN for rate-distortion estimation","archived":false,"fork":false,"pushed_at":"2020-06-18T15:35:41.000Z","size":2077,"stargazers_count":4,"open_issues_count":1,"forks_count":2,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-22T06:51:19.234Z","etag":null,"topics":["cnn","distortion","python3","rate","tensorflow","video-coding"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bbc.png","metadata":{"files":{"readme":"Readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-05-13T09:51:45.000Z","updated_at":"2023-07-21T09:18:40.000Z","dependencies_parsed_at":null,"dependency_job_id":"8b9af3cd-d21f-4690-8d03-262a2869bc88","html_url":"https://github.com/bbc/cnn-rate-distortion","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bbc%2Fcnn-rate-distortion","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bbc%2Fcnn-rate-distortion/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bbc%2Fcnn-rate-distortion/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bbc%2Fcnn-rate-distortion/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bbc","download_url":"https://codeload.github.com/bbc/cnn-rate-distortion/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247555107,"owners_count":20957705,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cnn","distortion","python3","rate","tensorflow","video-coding"],"created_at":"2024-11-06T00:27:36.661Z","updated_at":"2025-04-06T21:31:30.137Z","avatar_url":"https://github.com/bbc.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Estimation of Rate Control Parameters for Video Coding Using CNN\nRate-control is essential to ensure efficient video delivery. \nTypical rate-control algorithms rely on bit allocation strategies, \nto appropriately distribute bits among frames. As reference frames \nare essential for exploiting temporal redundancies, intra frames \nare usually assigned a larger portion of the available bits. \nWe proposed an accurate method to estimate number of bits and \nquality of intra frames, which can be used for bit \nallocation in a rate-control scheme. The algorithm is based on \ndeep learning, where networks are trained using the original \nframes as inputs, while distortions and sizes of compressed frames \nafter encoding are used as ground truths.\n\n![approach](resources/imgs/approach.png)\n\n## Check our blog post\n[Machine learning for video coding optimisation](https://www.bbc.co.uk/rd/blog/2020-04-machine-learning-video-coding-optimisation)\n\n## Publication\n2018 IEEE Visual Communications and Image Processing (VCIP).\nThe paper is available at \n[IEEE Xplore](https://dx.doi.org/10.1109/VCIP.2018.8698721) \nand [arXiv](https://arxiv.org/abs/2003.06315).\n\nPlease cite this work as:\n\n```\n@inproceedings{Santamaria2018,\n  author = {M. Santamaria and E. Izquierdo and S. Blasi and M. Mrak},\n  booktitle = {2018 IEEE Visual Communications and Image Processing (VCIP)},\n  title = {Estimation of Rate Control Parameters for Video Coding Using CNN},\n  year = {2018},\n  doi = {10.1109/VCIP.2018.8698721},\n  ISSN = {1018-8770},\n  month = dec,\n}\n```\n\n## How to use\n\n### Dependencies\nThe code is compatible with Python 3.5.2 and TensorFlow 1.14.0. \n\nInstall all dependencies with:\n\n```bash\npip install -r requirements.txt\n```\n\nYUV files are accessed with [VMAF reader](https://github.com/Netflix/vmaf/blob/7c00e97088deab0f2de4c05a2c3fd85f6ed9c205/python/src/vmaf/tools/reader.py).\n\n\n### Data preparation\nTraining data is generated from [MSCOCO dataset](http://cocodataset.org). \n\n#### Encoding data\n1. Convert the images to YUV 420p 8bit and make sure the dimensions width x height are in the file name, i.e.\n  `ffmpeg -i Lenna.png -pix_fmt yuv420p Lenna_512x512.yuv`\n1. Encode the images with a video codec, such as HEVC, using different QPs and:\n  1. Collect the bits per CTU, i.e. `x y bits` (`x` and `y` indicate the starting position of the CTU). \n  1. Keep the reconstructed YUV files. Add both the dimension width x height and QP to the file name, i.e `Lenna_512x512_QP_22.yuv`.\n1. For each image, reformat the bits in a single text file like:\n  `x y bits_qp_1 bits_qp_2 ... bits_qp_n`\n1. Make sure you re-organise the data as follows\n\n```bash\ndata\n├── original\n│   └── img1_512x512.yuv\n├── reconstruction\n│   └── img1\n│       ├── img1_512x512_QP_22.yuv\n│       ├── img1_512x512_QP_27.yuv\n│       ├── img1_512x512_QP_32.yuv\n│       └── img1_512x512_QP_37.yuv\n└── bits\n    └── img1.txt\n```\n\nCheck the example in [resources](./resources).\n\n#### Creating datasets\nYou need two create two different datasets, one for each model.\n\nFor the distortion model dataset, complete and run the following command:\n\n```bash\npython3 cnn-rate-distortion --task prepare_data --data_name [dataset name] --model_name distortion --input_dir [dir original yuv] --label_dir [dir reconstructed yuv] --width [CTU width] --height [CTU height] --levels [num QPs] --h5_dir [output dir]\n```\n\nExample:\n```bash\npython3 cnn-rate-distortion --task prepare_data --data_name Lenna --model_name distortion --input_dir ./resources/data/original --label_dir ./resources/data/reconstruction --width 128 --height 128 --levels 4 --h5_dir ./resources\n```\n\nFor the rate model dataset, complete and run the following command:\n\n```bash\npython3 cnn-rate-distortion --task prepare_data --data_name [dataset name] --model_name rate --input_dir [dir original yuv] --label_dir [dir bits] --width [CTU width] --height [CTU height] --levels [num QPs] --h5_dir [output dir]\n```\n\nExample:\n```bash\npython3 cnn-rate-distortion --task prepare_data --data_name Lenna --model_name rate --input_dir ./resources/data/original --label_dir ./resources/data/bits --width 128 --height 128 --levels 4 --h5_dir ./resources\n```\n\n### Training\nOnce the datasets are created, train the models by completing and running the following command:\n\n```bash\npython3 cnn-rate-distortion --task train --model_name [model name] --data_name [dataset name] --epochs [max epochs] --batch_size [imgs per batch] --loss_type [function] --width [CTU width] --height [CTU height] --levels [num QPs] --archive_dir [output dir] --h5_dir [dir to dataset]\n```\n\n### Testing\nComplete and run the following command to test the trained models:\n\n```bash\npython3 cnn-rate-distortion --task test --model_name [model name] --data_name [dataset name] --batch_size [imgs per batch] --loss_type [function] --width [CTU width] --height [CTU height] --levels [num QPs] --archive_dir [output dir] --test_data [path to test dataset]\n```\n\n## Acknowledgements\nThis work was co-supported by the  [Engineering and \nPhysical Sciences Research Council](https://epsrc.ukri.org/) of the UK, through an [iCASE](https://epsrc.ukri.org/skills/students/industrial-case/intro/) studentship in collaboration with the [School of Electronic Engineering and Computer Science](http://eecs.qmul.ac.uk/), [Queen Mary University of London]([qmul.ac.uk/](https://www.qmul.ac.uk/)).\n\n## Contact\nIf you have any general doubt about our work or code which may be of \ninterest for other researchers, please use the [issues section](https://github.com/bbc/cnn-rate-distortion/issues).\nAlternatively, drop us an e-mail at [maria.santamaria@bbc.co.uk](mailto:maria.santamaria@bbc.co.uk).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbbc%2Fcnn-rate-distortion","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbbc%2Fcnn-rate-distortion","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbbc%2Fcnn-rate-distortion/lists"}