{"id":17983146,"url":"https://github.com/kaiyangzhou/vsumm-reinforce","last_synced_at":"2025-06-24T21:05:34.608Z","repository":{"id":50732317,"uuid":"105387194","full_name":"KaiyangZhou/vsumm-reinforce","owner":"KaiyangZhou","description":"AAAI 2018 - Unsupervised video summarization with deep reinforcement learning (Theano)","archived":false,"fork":false,"pushed_at":"2021-11-30T01:29:57.000Z","size":645,"stargazers_count":137,"open_issues_count":20,"forks_count":35,"subscribers_count":9,"default_branch":"master","last_synced_at":"2025-04-06T09:51:42.132Z","etag":null,"topics":["reinforcement-learning","unsupervised-learning-algorithms","video-summarization"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/KaiyangZhou.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-09-30T17:24:41.000Z","updated_at":"2025-03-21T16:06:47.000Z","dependencies_parsed_at":"2022-09-03T08:21:16.410Z","dependency_job_id":null,"html_url":"https://github.com/KaiyangZhou/vsumm-reinforce","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/KaiyangZhou/vsumm-reinforce","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KaiyangZhou%2Fvsumm-reinforce","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KaiyangZhou%2Fvsumm-reinforce/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KaiyangZhou%2Fvsumm-reinforce/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KaiyangZhou%2Fvsumm-reinforce/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/KaiyangZhou","download_url":"https://codeload.github.com/KaiyangZhou/vsumm-reinforce/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KaiyangZhou%2Fvsumm-reinforce/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":261756590,"owners_count":23205152,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["reinforcement-learning","unsupervised-learning-algorithms","video-summarization"],"created_at":"2024-10-29T18:16:16.170Z","updated_at":"2025-06-24T21:05:34.564Z","avatar_url":"https://github.com/KaiyangZhou.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# vsumm-reinforce\nThis is the official implementation of the AAAI'18 paper [Deep Reinforcement Learning for Unsupervised Video Summarization with Diversity-Representativeness Reward](https://arxiv.org/abs/1801.00054). The code is based on Theano (version `0.9.0`).\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"imgs/pipeline.jpg\" alt=\"train\" width=\"80%\"\u003e\n\u003c/div\u003e\n\nPytorch implementation can be found [here](https://github.com/KaiyangZhou/pytorch-vsumm-reinforce).\n\n## Preparation\nTo get the datasets and models, you will need `wget`.\n\nRun the following commands in order\n```bash\ngit clone https://github.com/KaiyangZhou/vsumm-reinforce\ncd vsumm-reinforce\n# download datasets.tar.gz\nwget http://www.eecs.qmul.ac.uk/~kz303/vsumm-reinforce/datasets.tar.gz\ntar -xvzf datasets.tar.gz\n# download models.tar.gz\nwget http://www.eecs.qmul.ac.uk/~kz303/vsumm-reinforce/models.tar.gz\ntar -xvzf models.tar.gz\n```\n**Updates**: The QMUL server is inaccessible. Download the datasets from this [google drive link](https://drive.google.com/open?id=1Bf0beMN_ieiM3JpprghaoOwQe9QJIyAN).\n\n## How to train\nTraining code is implemented in `vsum_train.py`. To train a RNN, run\n```bash\npython vsum_train.py --dataset datasets/eccv16_dataset_tvsum_google_pool5.h5 --max-epochs 60 --hidden-dim 256\n```\n\n## How to test\nTest code is implemented in `vsum_test.py`. For example, to test with our models, simply run\n```bash\npython vsum_test.py -model models/model_tvsum_reinforceRNN.h5 -d tvsum\npython vsum_test.py -model models/model_tvsum_reinforceRNN_sup.h5 -d tvsum\npython vsum_test.py -model models/model_summe_reinforceRNN.h5 -d summe\npython vsum_test.py -model models/model_summe_reinforceRNN_sup.h5 -d summe\n```\n\nOutput results are saved to `log-test/results.h5`. To visualize score-vs-gtscore, you can use `visualize_results.py` by\n```bash\npython visualize_results.py -p log-test/result.h5\n```\n\n## Visualize summary\nYou can use `summary2video.py` to transform the binary `machine_summary` to real summary video. You need to have a directory containing video frames. The code will automatically write summary frames to a video where the frame rate can be controlled. Use the following command to generate a `.mp4` video\n```bash\npython summary2video.py -p path_to/result.h5 -d path_to/video_frames -i 0 --fps 30 --save-dir log --save-name summary.mp4\n```\nPlease remember to specify the naming format of your video frames on this [line](https://github.com/KaiyangZhou/vsumm-reinforce/blob/master/summary2video.py#L22).\n\n## Citation\n```\n@article{zhou2017reinforcevsumm, \n   title={Deep Reinforcement Learning for Unsupervised Video Summarization with Diversity-Representativeness Reward},\n   author={Zhou, Kaiyang and Qiao, Yu and Xiang, Tao}, \n   journal={arXiv:1801.00054}, \n   year={2017} \n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkaiyangzhou%2Fvsumm-reinforce","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkaiyangzhou%2Fvsumm-reinforce","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkaiyangzhou%2Fvsumm-reinforce/lists"}