{"id":26981753,"url":"https://github.com/visionxlab/castdet","last_synced_at":"2025-04-03T15:38:45.524Z","repository":{"id":234852342,"uuid":"788501108","full_name":"VisionXLab/CastDet","owner":"VisionXLab","description":"Code repo for \"Toward Open Vocabulary Aerial Object Detection with CLIP-Activated Student-Teacher Learning\" (ECCV2024)","archived":false,"fork":false,"pushed_at":"2025-02-08T07:23:44.000Z","size":1771,"stargazers_count":36,"open_issues_count":7,"forks_count":1,"subscribers_count":1,"default_branch":"main_obb","last_synced_at":"2025-02-08T07:25:42.849Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/VisionXLab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-04-18T14:37:47.000Z","updated_at":"2025-02-08T07:23:50.000Z","dependencies_parsed_at":"2025-02-08T07:23:17.575Z","dependency_job_id":"f41a9fd0-623a-4342-b3fa-7becee390709","html_url":"https://github.com/VisionXLab/CastDet","commit_stats":null,"previous_names":["lizzy8587/castdet"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VisionXLab%2FCastDet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VisionXLab%2FCastDet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VisionXLab%2FCastDet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VisionXLab%2FCastDet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/VisionXLab","download_url":"https://codeload.github.com/VisionXLab/CastDet/tar.gz/refs/heads/main_obb","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247029408,"owners_count":20871883,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-04-03T15:38:44.854Z","updated_at":"2025-04-03T15:38:45.506Z","avatar_url":"https://github.com/VisionXLab.png","language":"Python","readme":"# [ECCV 2024] Toward Open Vocabulary Aerial Object Detection with CLIP-Activated Student-Teacher Learning\n\n\n\u003e **Note**: This branch is for oriented open-vocabulary aerial object detection. **Please switch to another branch for horizontal detection.**\n\n## ✨ Latest Updates\n\n\n📆 [**2025-02-08**] : The code for **Oriented CastDet** is now available! 🎉 CastDet now supports Open-vocabulary Oriented Aerial Object Detection. Stay tuned—**Oriented GLIP**, **Oriented GroundingDINO**, and **Oriented ViLD** are coming soon! 🚀\n\n📆 [**2024-11-04**] : Our paper [\"Exploiting Unlabeled Data with Multiple Expert Teachers for Open Vocabulary Aerial Object Detection and Its Orientation Adaptation\"](https://arxiv.org/abs/2411.02057) is available open on arxiv!\n\n📆 [**2024-07-01**] : Our paper [\"Toward Open Vocabulary Aerial Object Detection with CLIP-Activated Student-Teacher Learning\"](https://arxiv.org/abs/2311.11646) has been accepted by ECCV 2024!\n\n📆 [**2024-04-21**] : We've released the code of **CastDet** now.\n\n\n## Introduction\n\nIn recent years, aerial object detection has been increasingly pivotal in various earth observation applications. However, current algorithms are limited to detecting pre-annotated categories. In this paper, we put forth a novel formulation of the aerial object detection problem, namely open-vocabulary aerial object detection (OVAD), which can detect objects beyond training categories without costly collecting new labeled data. We propose CastDet, a CLIP-activated student-teacher detection framework that serves as the first OVAD detector specifically designed for the challenging aerial scenario, where objects exhibit weak appearance features and arbitrary orientations. Our framework integrates a robust localization teacher along with several box selection strategies to generate high-quality proposals for novel objects. Additionally, the RemoteCLIP model is adopted as an omniscient teacher, which provides rich knowledge to enhance classification capabilities for novel categories. A dynamic label queue is devised to maintain high-quality pseudo-labels during training. By doing so, the proposed CastDet boosts not only novel object proposals but also classification. Furthermore, we extend our approach from horizontal to oriented OVAD with tailored algorithm designs for bounding box representation and pseudo-label generation. \n\nMoreover, we integrate multiple open-vocabulary oriented detection algorithms into this repo, including Oriented CastDet, Oriented ViLD, Oriented GLIP and Oriented GroundingDINO; and build a more comprehensive benchmark for the open-vocabulary aerial detection task, including both horizontal object detection and oriented object detection.\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"./projects/CastDetv2/resources/toolbox.png\" width=\"80%\"\u003e\n\u003c/div\u003e\n\n\n## Installation\n\n```shell\nconda create --name castdet python=3.8 -y\nconda activate castdet\nconda install pytorch torchvision -c pytorch\n\n# Install MMEngine and MMCV using MIM.\npip install -U openmim\nmim install mmengine\nmim install mmcv-full\nmim install mmdet==3.3.0\n\ncd mmrotate\npip install -r requirements/build.txt\npip install -v -e .\n\n# Install other packages\npip install imagesize\npip install open_clip_torch\npip install future tensorboard\n```\n\nTo use the Oriented GLIP or Oriented GroundingDINO, you need to install additional dependency packages:\n```shell\n# source installation\npip install -r requirements/multimodal.txt\n\n# or mim installation\nmim install mmdet[multimodal]\n```\n\n## Supported algorithms\n\nPlease refer to [castdet_readme](./projects/CastDetv2/README.md) to prepare the data, and other tutorials to train the model:\n\n1. Horizontal Open-vocabulary Aerial Object Detection\n- [x] CastDet (ECCV'24)\n\n2. Oriented Open-vocabulary Aerial Object Detection\n- [x] [Oriented CastDet](./projects/CastDetv2/README.md)\n- [x] [Oriented GroundingDINO](./projects/GroundingDINO/README.md)\n- [x] [Oriented GLIP](./projects/GLIP/README.md)\n- [x] [Oriented ViLD](./projects/ViLD/README.md)\n\n\n\n## Main results\n\n![](./projects/CastDetv2/resources/vis_result.png)\n\n![](./projects/CastDetv2/resources/results_hbb.png)\n\n![](./projects/CastDetv2/resources/results_obb.png)\n\n\n\n## Acknowledgement\n\nThanks the wonderful open source projects [MMDetection](https://github.com/open-mmlab/mmdetection), [MMRotate](https://github.com/open-mmlab/mmrotate), [RemoteCLIP](https://github.com/ChenDelong1999/RemoteCLIP), [RHINO](https://github.com/SIAnalytics/RHINO), [ViLD](https://github.com/tensorflow/tpu/tree/master/models/official/detection/projects/vild), [GroundingDINO](https://github.com/IDEA-Research/GroundingDINO), and [GLIP](https://github.com/microsoft/GLIP)!\n\n## Citation\n\nIf you find this repo useful for your research, please use the following BibTeX entry.\n\n```\n@misc{li2024open,\n      title={Toward Open Vocabulary Aerial Object Detection with CLIP-Activated Student-Teacher Learning}, \n      author={Yan Li and Weiwei Guo and Xue Yang and Ning Liao and Dunyun He and Jiaqi Zhou and Wenxian Yu},\n      year={2024},\n      eprint={2311.11646},\n      archivePrefix={arXiv},\n      primaryClass={cs.CV}\n}\n\n@misc{li2024exploitingunlabeleddatamultiple,\n      title={Exploiting Unlabeled Data with Multiple Expert Teachers for Open Vocabulary Aerial Object Detection and Its Orientation Adaptation}, \n      author={Yan Li and Weiwei Guo and Xue Yang and Ning Liao and Shaofeng Zhang and Yi Yu and Wenxian Yu and Junchi Yan},\n      year={2024},\n      eprint={2411.02057},\n      archivePrefix={arXiv},\n      primaryClass={cs.CV},\n      url={https://arxiv.org/abs/2411.02057}, \n}\n```","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvisionxlab%2Fcastdet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvisionxlab%2Fcastdet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvisionxlab%2Fcastdet/lists"}