{"id":18015988,"url":"https://github.com/linusling/mq-ms-sl","last_synced_at":"2025-07-12T22:36:02.958Z","repository":{"id":213786226,"uuid":"700114092","full_name":"LinusLing/mq-ms-sl","owner":"LinusLing","description":"Context-weighted Multi-Query Partially Relevant Video Retrieval","archived":false,"fork":false,"pushed_at":"2024-07-17T08:24:11.000Z","size":2311,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-04T15:25:53.544Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/LinusLing.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-10-04T01:09:04.000Z","updated_at":"2024-07-17T08:24:08.000Z","dependencies_parsed_at":"2024-07-16T16:05:48.142Z","dependency_job_id":"d93bcff0-6390-44c9-82a3-10593c180575","html_url":"https://github.com/LinusLing/mq-ms-sl","commit_stats":null,"previous_names":["linusling/mq-ms-sl"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/LinusLing/mq-ms-sl","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LinusLing%2Fmq-ms-sl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LinusLing%2Fmq-ms-sl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LinusLing%2Fmq-ms-sl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LinusLing%2Fmq-ms-sl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/LinusLing","download_url":"https://codeload.github.com/LinusLing/mq-ms-sl/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LinusLing%2Fmq-ms-sl/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265066200,"owners_count":23706066,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-30T04:15:47.047Z","updated_at":"2025-07-12T22:36:02.916Z","avatar_url":"https://github.com/LinusLing.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Context-weighted Multi-Query Partially Relevant Video Retrieval\n\n\u003cimg src=\"https://github.com/LinusLing/mq-ms-sl/blob/main/figures/MQPRVR.png\" width=\"1100px\"\u003e\n\n## Table of Contents\n\n* [Environments](#environments)\n* [MQ-MS-SL on Activitynet](#MQ-MS-SL-on-activitynet)\n  * [Required Data](#Required-Data-1)\n  * [Model Training](#Training-1)\n  * [Model Evaluation](#Evaluation-1)\n  * [Expected Performance](#Expected-Performance-1)\n* [MQ-MS-SL on Charades-STA](#MQ-MS-SL-on-Charades-STA)\n  * [Required Data](#Required-Data-2)\n  * [Model Training](#Training-2)\n  * [Model Evaluation](#Evaluation-2)\n  * [Expected Performance](#Expected-Performance-2)\n* [Reference](#Reference)\n* [Acknowledgement](#Acknowledgement)\n\n## Environments \n* **python 3.9**\n* **pytorch 2.1.0**\n* **torchvision 0.16.0**\n* **tensorboard 2.6.0**\n* **tqdm 4.62.0**\n* **easydict 1.9**\n* **h5py 2.10.0**\n* **cuda 12.2**\n\nWe used Anaconda to setup a deep learning workspace that supports PyTorch. Run the following script to install the required packages.\n```\nconda create --name mq_ms_sl python=3.9\nconda activate mq_ms_sl\ngit clone https://github.com/LinusLing/mq-ms-sl.git\ncd mq-ms-sl\npip install -r requirements.txt\nconda deactivate\n```\n\n## MQ-MS-SL on Activitynet\n### Required Data\nRun the following script to download the video feature and text feature of the Activitynet dataset and place them in the specified path. The data can also be downloaded from [Baidu pan](https://pan.baidu.com/s/1UNu67hXCbA6ZRnFVPVyJOA?pwd=8bh4). Please refer to [here](https://github.com/HuiGuanLab/ms-sl/tree/main/dataset) for more description of the dataset.\n\n```\nROOTPATH=$HOME/VisualSearch\nmkdir -p $ROOTPATH \u0026\u0026 cd $ROOTPATH\nwget http://8.210.46.84:8787/prvr/data/activitynet.tar\ntar -xvf activitynet.tar\n```\n\n### Training\nRun the following script to train `MQ-MS-SL` network on Activitynet.\n```\n#Add project root to PYTHONPATH (Note that you need to do this each time you start a new session.)\nsource setup.sh\n\nconda activate mq-ms-sl\n\nROOTPATH=$HOME/VisualSearch\nRUN_ID=runs_0\nGPU_DEVICE_ID=0\n\n./do_activitynet.sh $RUN_ID $ROOTPATH $GPU_DEVICE_ID\n```\n\n### Evaluation\nThe model is placed in the directory $ROOTPATH/$DATASET/results/$MODELDIR after training. To evaluate it, please run the following script:\n```\nDATASET=activitynet\nFEATURE=i3d\nROOTPATH=$HOME/VisualSearch\nMODELDIR=activitynet-runs_0-2022_07_11_20_27_02\n\n./do_test.sh $DATASET $FEATURE $ROOTPATH $MODELDIR\n```\n\nWe also provide the trained checkpoint on Activitynet, run the following script to evaluate it. The model can also be downloaded from [Here](https://pan.baidu.com/s/1Owodx7F55CPsOz8vsEU5sg?pwd=6aax).\n```\nDATASET=activitynet\nFEATURE=i3d\nROOTPATH=$HOME/VisualSearch\nMODELDIR=checkpoint_activitynet\n\nwget http://8.210.46.84:8787/prvr/checkpoints/checkpoint_activitynet.tar\ntar -xvf checkpoint_activitynet.tar -C $ROOTPATH/$DATASET/results\n\n./do_test.sh $DATASET $FEATURE $ROOTPATH $MODELDIR\n```\n\n### Expected performance \n\n|             | R@1 | R@5  |  R@10  | R@100 | SumR  |\n| :---------: |:---:|:----:|:------:|:-----:|:-----:|\n| Text-to-Video | 7.6 | 24.0 | 35.8 | 74.7  | 142.1 |\n\n## MQ-MS-SL on Charades-STA\n\n### Required Data\nRun the following script to download the video feature and text feature of the Charades-STA dataset and place them in the specified path. The data can also be downloaded from [Baidu pan](https://pan.baidu.com/s/1UNu67hXCbA6ZRnFVPVyJOA?pwd=8bh4). Please refer to [here](https://github.com/HuiGuanLab/ms-sl/tree/main/dataset) for more description of the dataset.\n\n```\nROOTPATH=$HOME/VisualSearch\nmkdir -p $ROOTPATH \u0026\u0026 cd $ROOTPATH\nwget http://8.210.46.84:8787/prvr/data/charades.tar\ntar -xvf charades.tar\n```\n\n### Training\nRun the following script to train `MQ-MS-SL` network on Charades-STA.\n\n```\n#Add project root to PYTHONPATH (Note that you need to do this each time you start a new session.)\nsource setup.sh\n\nconda activate mq-ms-sl\n\nROOTPATH=$HOME/VisualSearch\nRUN_ID=runs_0\nGPU_DEVICE_ID=0\n\n./do_charades.sh $RUN_ID $ROOTPATH $GPU_DEVICE_ID\n```\n\n### Evaluation\nThe model is placed in the directory $ROOTPATH/$DATASET/results/$MODELDIR after training. To evaluate it, please run the following script:\n```\nDATASET=charades\nFEATURE=i3d_rgb_lgi\nROOTPATH=$HOME/VisualSearch\nMODELDIR=charades-runs_0-2022_07_11_20_27_02\n\n./do_test.sh $DATASET $FEATURE $ROOTPATH $MODELDIR\n```\nWe also provide the trained checkpoint on Charades-STA, run the following script to evaluate it. The model can also be downloaded from [Here](https://pan.baidu.com/s/1WPtRm5Lg52bwg3Ya9S3sSA?pwd=6ccw).\n```\nDATASET=charades\nFEATURE=i3d_rgb_lgi\nROOTPATH=$HOME/VisualSearch\nMODELDIR=checkpoint_charades\n\nwget http://8.210.46.84:8787/prvr/checkpoints/checkpoint_charades.tar\ntar -xvf checkpoint_charades.tar -C $ROOTPATH/$DATASET/results\n\n./do_test.sh $DATASET $FEATURE $ROOTPATH $MODELDIR\n```\n\n### Expected performance \n\n|             | R@1 | R@5 | R@10 | R@100 | SumR |\n| :---------: |:---:|:---:|:----:|:-----:|:----:|\n| Text-to-Video | 1.6 | 6.7 | 11.9 | 49.2  | 69.5 |\n\n## Reference\n\nTo be added.\n\n## Acknowledgement\nThe codes are modified from [TVRetrieval](https://github.com/jayleicn/TVRetrieval), [ReLoCLNet](https://github.com/IsaacChanghau/ReLoCLNet) and [MS_SL](https://github.com/HuiGuanLab/ms-sl).\n\nThis work was supported by the National Natural Science Foundation of China(No. 61902347) and Scientific Research Fund of Zhejiang Provincial Education Department (No. Y202351804).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flinusling%2Fmq-ms-sl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flinusling%2Fmq-ms-sl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flinusling%2Fmq-ms-sl/lists"}