{"id":15634549,"url":"https://github.com/m-bain/condensedmovies","last_synced_at":"2025-08-20T04:32:33.921Z","repository":{"id":44679455,"uuid":"257536690","full_name":"m-bain/CondensedMovies","owner":"m-bain","description":"Story-Based Retrieval with Contextual Embeddings. Largest freely available movie video dataset. [ACCV'20]","archived":false,"fork":false,"pushed_at":"2022-09-21T18:39:40.000Z","size":23050,"stargazers_count":163,"open_issues_count":5,"forks_count":28,"subscribers_count":10,"default_branch":"master","last_synced_at":"2024-11-26T01:34:02.009Z","etag":null,"topics":["dataset","precomputed-features","retrieval","source-videos","video-text-retrieval"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/m-bain.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-04-21T08:55:01.000Z","updated_at":"2024-11-11T14:17:55.000Z","dependencies_parsed_at":"2023-01-18T17:32:11.867Z","dependency_job_id":null,"html_url":"https://github.com/m-bain/CondensedMovies","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/m-bain%2FCondensedMovies","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/m-bain%2FCondensedMovies/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/m-bain%2FCondensedMovies/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/m-bain%2FCondensedMovies/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/m-bain","download_url":"https://codeload.github.com/m-bain/CondensedMovies/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":230394228,"owners_count":18218707,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dataset","precomputed-features","retrieval","source-videos","video-text-retrieval"],"created_at":"2024-10-03T10:53:57.849Z","updated_at":"2024-12-19T07:06:42.902Z","avatar_url":"https://github.com/m-bain.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"## CondensedMovies\n\n**** ___N.B: Please use the condensed movies challenge https://github.com/m-bain/CondensedMovies-chall with updated splits since some videos in the original paper are unavailable with missing features ****____\n\n___Contact me directly for the additional dataset queries, details in the challenge repo for feature download.___\n\n\n###############################################\n\nThis repository contains the video dataset, implementation and baselines from \u003cstrong\u003eCondensed Movies: Story Based Retrieval with Contextual Embeddings\u003c/strong\u003e.\n\n[Project page](https://www.robots.ox.ac.uk/~vgg/research/condensed-movies) |\n[arXiv preprint](https://arxiv.org/abs/2005.04208) |\n[Read the paper](https://arxiv.org/pdf/2005.04208.pdf) |\n[Preview the data](https://www.robots.ox.ac.uk/~vgg/research/condensed-movies/#preview)\n\n----\n### CondensedMovies Dataset\n\n![videocaptions](figs/example_captions.png)\n\n\nThe dataset consists of 3K+ movies, 30K+ professionally captioned clips, 1K+ video hours, 400K+ facetracks \u0026 precomputed features from 6 different modalities.\n\n#### Installation\n\nRequirements:\n- Storage\n    - 20GB for features (required for baseline experiments)\n    - 10GB for facetracks\n    - 250GB for videos\n- Libraries\n    - ffmpeg (for video download)\n    - youtube-dl (for video download)\n    - pandas, numpy\n    - python 3.6+\n\n#### Prepare Data\n\n1. Navigate to directory `cd CondensedMovies/data_prep/`\n2. Edit configuration file `config.json` to download desired subsets of the dataset and their destination.\n3. If downloading the source videos (`src: true`), you can edit `youtube-dl.conf` for desired resolution, subtitles etc.\nPlease see [youtube-dl](https://github.com/ytdl-org/youtube-dl) for more info\n4. Run `python download.py`\n\nIf you have trouble downloading the source videos or features (due to geographical restrictions or otherwise), please contact me.\n\n### Video-Text Retrieval\n\n#### Baseline (Mixture of Expert Embeddings)\nEdit `data_dir` and `save_dir` in `configs/moe.json` for the experiments.\n1. `python train.py configs/moe.json`\n2. `python test.py --resume $SAVED_EXP_DIR/model_best.pth`\n\n### Visualisation\n\nRun `python visualise_face_tracks.py` with the appropriate arguments to visualise face tracks for a given videoID (requires facetracks and source videos downloaded).\n\n#### TODO:\n- [x] youtube download script\n- [x] missing videos check\n- [x] precomputed features download script\n- [x] facetrack visualisation\n- [x] dataloader\n- [x] video-text retrieval baselines\n- [ ] intra-movie baselines + char module\n- [ ] release fixed_seg features\n\n\n#### FAQ\n\nWhy did some of the source videos fail to download?\n\u003eThis is most likely due to geographical restrictions on the videos, email me at maxbain@robots.ox.ac.uk and I can help.\n\nThe precomputed features are averaged over the temporal dimension, will you release the original features?\n\u003eThis is to save space, original features in total are ~1TB, contact me to arrange download of this.\n\nI think clip X is incorrectly identified as being from movie Y, what do do?\n\u003ePlease let me know any movie identification mistakes and I'll correct it ASAP.\n\n\n#### Acknowledgements\n\nWe would like to thank Samuel Albanie for his help with feature extraction.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fm-bain%2Fcondensedmovies","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fm-bain%2Fcondensedmovies","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fm-bain%2Fcondensedmovies/lists"}