{"id":13644964,"url":"https://github.com/bupt-ai-cz/LLVIP","last_synced_at":"2025-04-21T10:34:59.762Z","repository":{"id":38309945,"uuid":"377085976","full_name":"bupt-ai-cz/LLVIP","owner":"bupt-ai-cz","description":"LLVIP: A Visible-infrared Paired Dataset for Low-light Vision","archived":false,"fork":false,"pushed_at":"2024-01-30T18:02:01.000Z","size":70720,"stargazers_count":699,"open_issues_count":28,"forks_count":65,"subscribers_count":25,"default_branch":"main","last_synced_at":"2025-04-02T03:34:38.880Z","etag":null,"topics":["cnn","computer-vision","deep-learning","gan","iccv2021","image-fusion","image-to-image-translation","low-light-image","low-light-vision","object-detection","visible-infrared"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bupt-ai-cz.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-06-15T08:10:42.000Z","updated_at":"2025-04-01T07:56:14.000Z","dependencies_parsed_at":"2024-11-03T01:31:48.564Z","dependency_job_id":"78df5957-c03e-4a4c-8811-a485ac5f52f4","html_url":"https://github.com/bupt-ai-cz/LLVIP","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bupt-ai-cz%2FLLVIP","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bupt-ai-cz%2FLLVIP/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bupt-ai-cz%2FLLVIP/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bupt-ai-cz%2FLLVIP/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bupt-ai-cz","download_url":"https://codeload.github.com/bupt-ai-cz/LLVIP/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250041022,"owners_count":21365201,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cnn","computer-vision","deep-learning","gan","iccv2021","image-fusion","image-to-image-translation","low-light-image","low-light-vision","object-detection","visible-infrared"],"created_at":"2024-08-02T01:02:21.622Z","updated_at":"2025-04-21T10:34:54.752Z","avatar_url":"https://github.com/bupt-ai-cz.png","language":"Jupyter Notebook","funding_links":[],"categories":["Jupyter Notebook","Applications","Multispectral Image Datasets","Researchers"],"sub_categories":["Datasets"],"readme":"# LLVIP: A Visible-infrared Paired Dataset for Low-light Vision ![visitors](https://visitor-badge.glitch.me/badge?page_id=bupt-ai-cz.LLVIP)\n[Project](https://bupt-ai-cz.github.io/LLVIP/) | [Arxiv](https://arxiv.org/abs/2108.10831) | [Kaggle](https://www.kaggle.com/c/find-person-in-the-dark) | [![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/llvip-a-visible-infrared-paired-dataset-for/pedestrian-detection-on-llvip)](https://paperswithcode.com/sota/pedestrian-detection-on-llvip?p=llvip-a-visible-infrared-paired-dataset-for) | [![Tweet](https://img.shields.io/twitter/url/http/shields.io.svg?style=social)](https://twitter.com/intent/tweet?text=Codes%20and%20Data%20for%20Our%20Paper:%20\"LLVIP:%20A%20Visible-infrared%20Paired%20Dataset%20for%20Low-light%20Vision\"%20\u0026url=https://github.com/bupt-ai-cz/LLVIP)  \n\n## News\n- ⚡(2024-1-8): The pre-trained pix2pixGAN on LLVIP is released [Here.](https://github.com/bupt-ai-cz/LLVIP/releases)\n- ⚡(2023-2-21): The annotations of a small part of images have been corrected and updated, including the annotation of some missing pedestrians, and the optimization of some imprecise annotations. The updated dataset is now available from the [homepage](https://bupt-ai-cz.github.io/LLVIP/) or [here](https://github.com/bupt-ai-cz/LLVIP/blob/main/download_dataset.md). If you need the previous version of the annotations, please refer to [here](https://github.com/bupt-ai-cz/LLVIP/blob/main/previous%20annotations.md).\n- ⚡(2022-5-24): We provide a [toolbox](https://github.com/bupt-ai-cz/LLVIP/blob/main/toolbox/toolbox_readme.md) for various format conversions (xml to yolov5, xml to yolov3, xml to coco)\n- ⚡(2022-3-27): We released some raw data (unregistered image pairs and videos) for further research including image registration. Please visit [homepage](https://bupt-ai-cz.github.io/LLVIP/) to get the update. (2022-3-28 We have updated the link of Baidu Yun of LLVIP raw data, the data downloaded from the new link supports decompression under `windows` and `macos`. The original link only support `windows`.)\n- ⚡(2021-12-25): We released a Kaggle Community Competition \"[Find Person in the Dark!](https://www.kaggle.com/c/find-person-in-the-dark)\" based on part of LLVIP dataset. Welcome playing and having fun! Attention: only the visible-image data we uploaded in Kaggle platform is allowed to use (the infrared images in LLVIP or other external data are forbidden)\n- ⚡(2021-11-24): Pedestrian detection models were released\n- ⚡(2021-09-01): We have released the dataset, please visit [homepage](https://bupt-ai-cz.github.io/LLVIP/) or [here](https://github.com/bupt-ai-cz/LLVIP/blob/main/download_dataset.md) to get the dataset. (Note that we removed some low-quality images from the original dataset, and for this version there are 30976 images.)\n\n---\n\n![figure1-LR](imgs/figure1-LR.png)\n\n---\n\n## Dataset Downloading:\n\n### [Homepage](https://bupt-ai-cz.github.io/LLVIP/) or [DatasetDownload](https://github.com/bupt-ai-cz/LLVIP/blob/main/download_dataset.md)\n\n---\n\n## Citation \nIf you use this data for your research, please cite our paper [LLVIP: A Visible-infrared Paired Dataset for Low-light Vision](https://arxiv.org/abs/2108.10831):\n\n\n```\n@inproceedings{jia2021llvip,\n  title={LLVIP: A visible-infrared paired dataset for low-light vision},\n  author={Jia, Xinyu and Zhu, Chuang and Li, Minzhen and Tang, Wenqi and Zhou, Wenli},\n  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},\n  pages={3496--3504},\n  year={2021}\n}\n```\nor\n\n```\n@misc{https://doi.org/10.48550/arxiv.2108.10831,\n  doi = {10.48550/ARXIV.2108.10831}, \n  url = {https://arxiv.org/abs/2108.10831},\n  author = {Jia, Xinyu and Zhu, Chuang and Li, Minzhen and Tang, Wenqi and Liu, Shengjie and Zhou, Wenli}, \n  keywords = {Computer Vision and Pattern Recognition (cs.CV), Artificial Intelligence (cs.AI), FOS: Computer and information sciences, FOS: Computer and information sciences},\n  title = {LLVIP: A Visible-infrared Paired Dataset for Low-light Vision},\n  publisher = {arXiv},\n  year = {2021},\n  copyright = {arXiv.org perpetual, non-exclusive license}\n}\n```\n\n\u003ch2\u003e \u003cp align=\"center\"\u003e Image Fusion \u003c/p\u003e \u003c/h2\u003e  \n\nBaselines\n   - [GTF](https://github.com/jiayi-ma/GTF)\n   - [FusionGAN](https://github.com/jiayi-ma/FusionGAN)\n   - [Densefuse](https://github.com/hli1221/imagefusion_densefuse)\n   - [IFCNN](https://github.com/uzeful/IFCNN)\n\n## FusionGAN\n### Preparation\n- Install requirements\n  ```bash\n  git clone https://github.com/bupt-ai-cz/LLVIP.git\n  cd LLVIP/FusionGAN\n  # Create your virtual environment using anaconda\n  conda create -n FusionGAN python=3.7\n  conda activate FusionGAN\n  \n  conda install matplotlib scipy==1.2.1 tensorflow-gpu==1.14.0 \n  pip install opencv-python\n  sudo apt install libgl1-mesa-glx\n  ```\n- File structure\n  ```\n  FusionGAN\n  ├── ...\n  ├── Test_LLVIP_ir\n  |   ├── 190001.jpg\n  |   ├── 190002.jpg\n  |   └── ...\n  ├── Test_LLVIP_vi\n  |   ├── 190001.jpg\n  |   ├── 190002.jpg\n  |   └── ...\n  ├── Train_LLVIP_ir\n  |   ├── 010001.jpg\n  |   ├── 010002.jpg\n  |   └── ...\n  └── Train_LLVIP_vi\n      ├── 010001.jpg\n      ├── 010002.jpg\n      └── ...\n  ```\n### Train\n  ```bash\n  python main.py --epoch 10 --batch_size 32\n  ```\nSee more training options in `main.py`.\n### Test\n  ```bash\n  python test_one_image.py\n  ```\nRemember to put pretrained model in your `checkpoint` folder and change corresponding model name in `test_one_image.py`.\nTo acquire complete LLVIP dataset, please visit https://bupt-ai-cz.github.io/LLVIP/.\n\n## Densefuse\n### Preparation\n- Install requirements\n  ```bash\n  git clone https://github.com/bupt-ai-cz/LLVIP\n  cd LLVIP/imagefusion_densefuse\n  \n  # Create your virtual environment using anaconda\n  conda create -n Densefuse python=3.7\n  conda activate Densefuse\n  \n  conda install scikit-image scipy==1.2.1 tensorflow-gpu==1.14.0\n  ```\n- File structure\n  ```\n  imagefusion_densefuse\n  ├── ...\n  ├──datasets\n  |  ├──010001_ir.jpg\n  |  ├──010001_vi.jpg\n  |  └── ...\n  ├──test\n  |  ├──190001_ir.jpg\n  |  ├──190001_vi.jpg\n  |  └── ...\n  └──LLVIP\n     ├── infrared\n     |   ├──train\n     |   |  ├── 010001.jpg\n     |   |  ├── 010002.jpg\n     |   |  └── ...\n     |   └──test\n     |      ├── 190001.jpg\n     |      ├── 190002.jpg\n     |      └── ...\n     └── visible\n         ├──train\n         |   ├── 010001.jpg\n         |   ├── 010002.jpg\n         |   └── ...\n         └── test\n             ├── 190001.jpg\n             ├── 190002.jpg\n             └── ...\n  ```\n  \n### Train \u0026 Test\n  ```bash\n  python main.py \n  ```\nCheck and modify training/testing options in `main.py`. Before training/testing, you need to rename the images in LLVIP dataset and put them in the designated folder. We have provided a script named `rename.py` to rename the images and save them in the `datasets` or `test` folder. Checkpoints are saved in `./models/densefuse_gray/`. To acquire complete LLVIP dataset, please visit https://bupt-ai-cz.github.io/LLVIP/.\n\n## IFCNN\nPlease visit https://github.com/uzeful/IFCNN.\n\n\u003ch2\u003e \u003cp align=\"center\"\u003e Pedestrian Detection \u003c/p\u003e \u003c/h2\u003e \n\nBaselines\n   - [Yolov5](https://github.com/ultralytics/yolov5)\n   - [Yolov3](https://github.com/ultralytics/yolov3)\n## Yolov5\n### Preparation\n#### Linux and Python\u003e=3.6.0\n- Install requirements\n  ```bash\n  git clone https://github.com/bupt-ai-cz/LLVIP.git\n  cd LLVIP/yolov5\n  pip install -r requirements.txt\n  ```\n- File structure\n\n  The training set of LLVIP is used for training the yolov5 model and the testing set of LLVIP is used for the validation of the yolov5 model.\n  ```\n  yolov5\n  ├── ...\n  └──LLVIP\n     ├── labels\n     |   ├──train\n     |   |  ├── 010001.txt\n     |   |  ├── 010002.txt\n     |   |  └── ...\n     |   └──val\n     |      ├── 190001.txt\n     |      ├── 190002.txt\n     |      └── ...\n     └── images\n         ├──train\n         |   ├── 010001.jpg\n         |   ├── 010002.jpg\n         |   └── ...\n         └── val\n             ├── 190001.jpg\n             ├── 190002.jpg\n             └── ...\n  ```\n  We provide a [toolbox](https://github.com/bupt-ai-cz/LLVIP/blob/main/toolbox/toolbox_readme.md) for converting annotation files to txt files in yolov5 format.\n### Train\n  ```bash\n  python train.py --img 1280 --batch 8 --epochs 200 --data LLVIP.yaml --weights yolov5l.pt --name LLVIP_export\n  ```\nSee more training options in `train.py`. The pretrained model `yolov5l.pt` can be downloaded from [here](https://github.com/ultralytics/yolov5/releases). The trained model will be saved in `./runs/train/LLVIP_export/weights` folder.\n### Test\n  ```bash\n  python val.py --data --img 1280 --weights last.pt --data LLVIP.yaml\n  ```\n  Remember to put the trained model in the same folder as `val.py`.\n  \n  Our trained model can be downloaded from here: [Google-Drive-Yolov5-model](https://drive.google.com/file/d/1SPbr0PDiItape602-g-bstkX0P7NZo0q/view?usp=sharing) or [BaiduYun-Yolov5-model](https://pan.baidu.com/s/1q3mGhQzT_D3uiqdfAHVqCA) (code: qepr)\n- Click [Here](yolov3/README.md) for the tutorial of Yolov3 （Our trained Yolov3 model can be downloaded from here: [Google-Drive-Yolov3-model](https://drive.google.com/file/d/1-BYauAZGXhw7PjKp8M4CHlnMNYqm8n7J/view?usp=sharing) or [BaiduYun-Yolov3-model](https://pan.baidu.com/s/1eZcyugmpo_3VZjd3wpwcow) (code: ine5)）.\n### Results\nWe retrained and tested Yolov5l and Yolov3 on the updated dataset (30976 images).\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"https://user-images.githubusercontent.com/33684330/138012320-3340bf17-481a-4d69-a8a9-fc7427055cf4.jpg\" height=\"130\" width=\"700\"\u003e\n\u003c/div\u003e\n\nWhere AP means the average of AP at IoU threshold of 0.5 to 0.95, with an interval of 0.05.\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"https://user-images.githubusercontent.com/33684330/138388453-9953c403-4a5f-4b45-a488-e44fcec7b955.png\" height=\"510\" width=\"700\"\u003e\n\u003c/div\u003e\nThe figure above shows the change of AP under different IoU thresholds. When the IoU threshold is higher than 0.7, the AP value drops rapidly. Besides, the infrared image highlights pedestrains and achieves a better effect than the visible image in the detection task, which not only proves the necessity of infrared images but also indicates that the performance of visible-image pedestrian detection algorithm is not good enough under low-light conditions.\n\nWe also calculated log average miss rate based on the test results and drew the miss rate-FPPI curve.\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"https://user-images.githubusercontent.com/33684330/138281822-ea7dd310-bb9d-4197-8dfe-f17bb4534986.jpeg\" height=\"110\" width=\"700\"\u003e\n\u003c/div\u003e\n\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"https://user-images.githubusercontent.com/33684330/138312218-274810d5-0191-4fbc-abfe-01736dc285bf.png\" height=\"510\" width=\"700\"\u003e\n\u003c/div\u003e\n\n\u003ch2\u003e \u003cp align=\"center\"\u003e Image-to-Image Translation \u003c/p\u003e \u003c/h2\u003e \n\nBaseline\n   - [pix2pixGAN](https://github.com/phillipi/pix2pix)\n## pix2pixGAN\n### Preparation\n- Install requirements\n  ```bash\n  cd pix2pixGAN\n  pip install -r requirements.txt\n  ```\n- [Prepare dataset](https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix/blob/master/docs/datasets.md)\n- File structure\n  ```\n  pix2pixGAN\n  ├── ...\n  └──datasets\n     ├── ...\n     └──LLVIP\n        ├── train\n        |   ├── 010001.jpg\n        |   ├── 010002.jpg\n        |   ├── 010003.jpg\n        |   └── ...\n        └── test\n            ├── 190001.jpg\n            ├── 190002.jpg\n            ├── 190003.jpg\n            └── ...\n  ```\n\n### Train\n  ```bash\n  python train.py --dataroot ./datasets/LLVIP --name LLVIP --model pix2pix --direction AtoB --batch_size 8 --preprocess scale_width_and_crop --load_size 320 --crop_size 256 --gpu_ids 0 --n_epochs 100 --n_epochs_decay 100\n  ```\n### Test\n  ```bash\n  python test.py --dataroot ./datasets/LLVIP --name LLVIP --model pix2pix --direction AtoB --gpu_ids 0 --preprocess scale_width_and_crop --load_size 320 --crop_size 256\n  ```\n  See `./pix2pixGAN/options` for more train and test options.\n\n\n\u003cimg src='imgs/LLVIP.gif' align=\"right\"  height=192 width=448\u003e\n\u003cbr\u003e\n\n### Results\nWe retrained and tested pix2pixGAN  on the updated dataset(30976 images). The structure of generator is unet256, and the structure of discriminator is the basic PatchGAN as default. \n\n\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"https://user-images.githubusercontent.com/33684330/138233570-1440a5bf-7a05-4e96-b8ab-fc32a9c59748.jpeg\" height=\"100\" width=\"700\"\u003e\n\u003c/div\u003e\n\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"https://user-images.githubusercontent.com/33684330/135420925-72b9722a-3838-437b-b1a7-5f9e81c91d85.png\" height=\"610\" width=\"700\"\u003e\n\u003c/div\u003e\n\n## License\nThis LLVIP Dataset is made freely available to academic and non-academic entities for non-commercial purposes such as academic research, teaching, scientific publications, or personal experimentation. Permission is granted to use the data given that you agree to our [license terms](Term%20of%20Use%20and%20License.md).\n\n## Call For Contributions\n\nWelcome to point out errors in data annotation. If you want to modify the label, please refer to the [annotation tutorial](https://github.com/SantJay/LLVIP-1/blob/main/annotation%20tutorial.md), and email us the corrected label file.\n\nMore annotation forms are also welcome (such as segmentation), please contact us.\n\n## Acknowledgments\nThanks [XueZ-phd](https://github.com/XueZ-phd) for his contribution to LLVIP dataset. He corrected the imperfect annotations in the dataset.\n\n## Contact\n\nemail: shengjie.Liu@bupt.edu.cn, czhu@bupt.edu.cn, jiaxinyujxy@qq.com, tangwenqi@bupt.edu.cn\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbupt-ai-cz%2FLLVIP","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbupt-ai-cz%2FLLVIP","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbupt-ai-cz%2FLLVIP/lists"}