{"id":13958440,"url":"https://github.com/HuiGuanLab/ms-sl","last_synced_at":"2025-07-21T00:30:47.014Z","repository":{"id":135583730,"uuid":"486849566","full_name":"HuiGuanLab/ms-sl","owner":"HuiGuanLab","description":"Source code of our MM'22 paper Partially Relevant Video Retrieval","archived":false,"fork":false,"pushed_at":"2024-11-04T02:11:18.000Z","size":4753,"stargazers_count":51,"open_issues_count":2,"forks_count":7,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-11-28T02:34:43.427Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/HuiGuanLab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-04-29T05:33:38.000Z","updated_at":"2024-11-04T02:11:22.000Z","dependencies_parsed_at":"2024-11-28T02:32:17.565Z","dependency_job_id":"5e417f4f-c6fd-4bb1-8db0-7692fe1e2e59","html_url":"https://github.com/HuiGuanLab/ms-sl","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/HuiGuanLab/ms-sl","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HuiGuanLab%2Fms-sl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HuiGuanLab%2Fms-sl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HuiGuanLab%2Fms-sl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HuiGuanLab%2Fms-sl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/HuiGuanLab","download_url":"https://codeload.github.com/HuiGuanLab/ms-sl/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HuiGuanLab%2Fms-sl/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266221246,"owners_count":23894964,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-08T13:01:35.474Z","updated_at":"2025-07-21T00:30:46.604Z","avatar_url":"https://github.com/HuiGuanLab.png","language":"Python","funding_links":[],"categories":["其他_机器视觉"],"sub_categories":["网络服务_其他"],"readme":"# Partially Relevant Video Retrieval\nSource code of our ACM MM'2022 paper [Partially Relevant Video Retrieval](https://arxiv.org/abs/2208.12510).\n\nHomepage of our paper [http://danieljf24.github.io/prvr/](http://danieljf24.github.io/prvr/).\n\n\u003cimg src=\"https://github.com/HuiGuanLab/ms-sl/blob/main/figures/pvr_model.png\" width=\"1100px\"\u003e\n\n## Table of Contents\n\n* [Environments](#environments)\n* [MS-SL on TVR](#MS-SL-on-TVR)\n  * [Required Data](#Required-Data)\n  * [Model Training](#Training)\n  * [Model Evaluation](#Evaluation)\n  * [Expected Performance](#Expected-Performance)\n* [MS-SL on Activitynet](#MS-SL-on-activitynet)\n  * [Required Data](#Required-Data-1)\n  * [Model Training](#Training-1)\n  * [Model Evaluation](#Evaluation-1)\n  * [Expected Performance](#Expected-Performance-1)\n* [MS-SL on Charades-STA](#MS-SL-on-Charades-STA)\n  * [Required Data](#Required-Data-2)\n  * [Model Training](#Training-2)\n  * [Model Evaluation](#Evaluation-2)\n  * [Expected Performance](#Expected-Performance-2)\n* [Reference](#Reference)\n* [Acknowledgement](#Acknowledgement)\n\n## Environments \n* **python 3.8**\n* **pytorch 1.9.0**\n* **torchvision 0.10.0**\n* **tensorboard 2.6.0**\n* **tqdm 4.62.0**\n* **easydict 1.9**\n* **h5py 2.10.0**\n* **cuda 11.1**\n\nWe used Anaconda to setup a deep learning workspace that supports PyTorch. Run the following script to install the required packages.\n```\nconda create --name ms_sl python=3.8\nconda activate ms_sl\ngit clone https://github.com/HuiGuanLab/ms-sl.git\ncd ms-sl\npip install -r requirements.txt\nconda deactivate\n```\n\n## MS-SL on TVR\n\n### Required Data\nThe data can be downloaded from [Baidu pan](https://pan.baidu.com/s/1UNu67hXCbA6ZRnFVPVyJOA?pwd=8bh4) or [Google drive](https://drive.google.com/drive/folders/11dRUeXmsWU25VMVmeuHc9nffzmZhPJEj?usp=sharing). Please refer to [here](https://github.com/HuiGuanLab/ms-sl/tree/main/dataset) for more description of the dataset. Run the following script to place the data in the specified path.\n\n```\n# download the data of TVR\nROOTPATH=$HOME/VisualSearch\nmkdir -p $ROOTPATH \u0026\u0026 cd $ROOTPATH\nunzip tvr.zip -d $ROOTPATH\n```\n\n### Training\nRun the following script to train `MS-SL` network on TVR. It will save the chechpoint that performs best on the validation set as the final model.\n\n\n```\n#Add project root to PYTHONPATH (Note that you need to do this each time you start a new session.)\nsource setup.sh\n\nconda activate ms-sl\n\nROOTPATH=$HOME/VisualSearch\nRUN_ID=runs_0\nGPU_DEVICE_ID=0\n\n./do_tvr.sh $RUN_ID $ROOTPATH $GPU_DEVICE_ID\n```\n`$RUN_ID` is the name of the folder where the model is saved in.\n\n`$GPU_DEVICE_ID` is the index of the GPU where we train on.\n### Evaluation\nThe model is placed in the directory $ROOTPATH/$DATASET/results/$MODELDIR after training. To evaluate it, please run the following script:\n```\nDATASET=tvr\nFEATURE=i3d_resnet\nROOTPATH=$HOME/VisualSearch\nMODELDIR=tvr-runs_0-2022_07_11_20_27_02 \n\n./do_test.sh $DATASET $FEATURE $ROOTPATH $MODELDIR\n```\n\nWe also provide the trained checkpoint on TVR, run the following script to evaluate it. The model can also be downloaded from [Here](https://pan.baidu.com/s/1d70cecBvwVqYwmvobJpbGw?pwd=zxzk) or [Google drive](https://drive.google.com/drive/folders/11dRUeXmsWU25VMVmeuHc9nffzmZhPJEj?usp=sharing). \n```\nDATASET=tvr\nFEATURE=i3d_resnet\nROOTPATH=$HOME/VisualSearch\nMODELDIR=checkpoint_tvr\n\ntar -xvf checkpoint_tvr.tar -C $ROOTPATH/$DATASET/results\n\n./do_test.sh $DATASET $FEATURE $ROOTPATH $MODELDIR\n```\n`$DATASET` is the dataset that the model trained and evaluate on.\n\n`$FEATURE` is the video feature corresponding to the dataset.\n\n`$MODELDIR` is the path of checkpoints saved.\n### Expected performance \n\n|             | R@1  | R@5  | R@10 | R@100 | SumR  |\n| :---------: | :--: | :--: | :--: | :---: | :---: |\n| Text-to-Video | 13.5 | 32.1 | 43.4 | 83.4  | 172.3 |\n\n## MS-SL on Activitynet\n### Required Data\nThe data can be downloaded from [Baidu pan](https://pan.baidu.com/s/1UNu67hXCbA6ZRnFVPVyJOA?pwd=8bh4) or [Google drive](https://drive.google.com/drive/folders/11dRUeXmsWU25VMVmeuHc9nffzmZhPJEj?usp=sharing). Please refer to [here](https://github.com/HuiGuanLab/ms-sl/tree/main/dataset) for more description of the dataset. Run the following script to place the data in the specified path.\n\n```\nROOTPATH=$HOME/VisualSearch\nmkdir -p $ROOTPATH \u0026\u0026 cd $ROOTPATH\nunzip activitynet.zip -d $ROOTPATH\n```\n\n### Training\nRun the following script to train `MS-SL` network on Activitynet.\n```\n#Add project root to PYTHONPATH (Note that you need to do this each time you start a new session.)\nsource setup.sh\n\nconda activate ms-sl\n\nROOTPATH=$HOME/VisualSearch\nRUN_ID=runs_0\nGPU_DEVICE_ID=0\n\n./do_activitynet.sh $RUN_ID $ROOTPATH $GPU_DEVICE_ID\n```\n\n### Evaluation\nThe model is placed in the directory $ROOTPATH/$DATASET/results/$MODELDIR after training. To evaluate it, please run the following script:\n```\nDATASET=activitynet\nFEATURE=i3d\nROOTPATH=$HOME/VisualSearch\nMODELDIR=activitynet-runs_0-2022_07_11_20_27_02\n\n./do_test.sh $DATASET $FEATURE $ROOTPATH $MODELDIR\n```\n\nWe also provide the trained checkpoint on Activitynet, run the following script to evaluate it. The model can also be downloaded from [Here](https://pan.baidu.com/s/10zMvaSGRyJWxGUgSPm2ySg?pwd=omgg) or [Google drive](https://drive.google.com/drive/folders/11dRUeXmsWU25VMVmeuHc9nffzmZhPJEj?usp=sharing).\n```\nDATASET=activitynet\nFEATURE=i3d\nROOTPATH=$HOME/VisualSearch\nMODELDIR=checkpoint_activitynet\n\ntar -xvf checkpoint_activitynet.tar -C $ROOTPATH/$DATASET/results\n\n./do_test.sh $DATASET $FEATURE $ROOTPATH $MODELDIR\n```\n\n### Expected performance \n\n|             | R@1  | R@5  | R@10 | R@100 | SumR  |\n| :---------: | :--: | :--: | :--: | :---: | :---: |\n| Text-to-Video | 7.1 | 22.5 | 34.7 | 75.8  | 140.1 |\n\n## MS-SL on Charades-STA\n\n### Required Data\nThe data can be downloaded from [Baidu pan](https://pan.baidu.com/s/1UNu67hXCbA6ZRnFVPVyJOA?pwd=8bh4) or [Google drive](https://drive.google.com/drive/folders/11dRUeXmsWU25VMVmeuHc9nffzmZhPJEj?usp=sharing). Please refer to [here](https://github.com/HuiGuanLab/ms-sl/tree/main/dataset) for more description of the dataset. Run the following script to place the data in the specified path.\n\n```\nROOTPATH=$HOME/VisualSearch\nmkdir -p $ROOTPATH \u0026\u0026 cd $ROOTPATH\nunzip charades.zip -d $ROOTPATH\n```\n\n### Training\nRun the following script to train `MS-SL` network on Charades-STA.\n\n```\n#Add project root to PYTHONPATH (Note that you need to do this each time you start a new session.)\nsource setup.sh\n\nconda activate ms-sl\n\nROOTPATH=$HOME/VisualSearch\nRUN_ID=runs_0\nGPU_DEVICE_ID=0\n\n./do_charades.sh $RUN_ID $ROOTPATH $GPU_DEVICE_ID\n```\n\n### Evaluation\nThe model is placed in the directory $ROOTPATH/$DATASET/results/$MODELDIR after training. To evaluate it, please run the following script:\n```\nDATASET=charades\nFEATURE=i3d_rgb_lgi\nROOTPATH=$HOME/VisualSearch\nMODELDIR=charades-runs_0-2022_07_11_20_27_02\n\n./do_test.sh $DATASET $FEATURE $ROOTPATH $MODELDIR\n```\nWe also provide the trained checkpoint on Charades-STA, run the following script to evaluate it. The model can also be downloaded from [Here](https://pan.baidu.com/s/1IuUI1D04gSSmfiHQwedbgg?pwd=w6mk) or [Google drive](https://drive.google.com/drive/folders/11dRUeXmsWU25VMVmeuHc9nffzmZhPJEj?usp=sharing).\n```\nDATASET=charades\nFEATURE=i3d_rgb_lgi\nROOTPATH=$HOME/VisualSearch\nMODELDIR=checkpoint_charades\n\ntar -xvf checkpoint_charades.tar -C $ROOTPATH/$DATASET/results\n\n./do_test.sh $DATASET $FEATURE $ROOTPATH $MODELDIR\n```\n\n### Expected performance \n\n|             | R@1  | R@5  | R@10 | R@100 | SumR  |\n| :---------: | :--: | :--: | :--: | :---: | :---: |\n| Text-to-Video | 1.8 | 7.1 | 11.8 | 47.7  | 68.4 |\n\n## Reference\n```\n@inproceedings{dong2022prvr,\ntitle = {Partially Relevant Video Retrieval},\nauthor = {Jianfeng Dong and Xianke Chen and Minsong Zhang and Xun Yang and Shujie Chen and Xirong Li and Xun Wang},\nbooktitle = {Proceedings of the 30th ACM International Conference on Multimedia},\nyear = {2022},\n}\n```\n## Acknowledgement\nThe codes are modified from [TVRetrieval](https://github.com/jayleicn/TVRetrieval) and [ReLoCLNet](https://github.com/IsaacChanghau/ReLoCLNet).\n\nThis work was supported by the National Key R\u0026D Program of China (2018YFB1404102), NSFC (62172420,61902347, 61976188, 62002323), the Public Welfare Technology Research Project of Zhejiang Province (LGF21F020010), the Open Projects Program of the National Laboratory of Pattern Recognition, the Fundamental Research Funds for the Provincial Universities of Zhejiang, and Public Computing Cloud of RUC.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FHuiGuanLab%2Fms-sl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FHuiGuanLab%2Fms-sl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FHuiGuanLab%2Fms-sl/lists"}