https://github.com/wivizhang/EarthMarker
https://github.com/wivizhang/EarthMarker
Last synced: 8 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/wivizhang/EarthMarker
- Owner: wivizhang
- Created: 2024-07-18T06:23:53.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2025-01-02T10:17:26.000Z (10 months ago)
- Last Synced: 2025-01-02T11:22:59.368Z (10 months ago)
- Size: 10.3 MB
- Stars: 20
- Watchers: 5
- Forks: 0
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
- awesome-vlgfm - EarthMarker: A Visual Prompt Learning Framework for Region-level and Point-level Remote Sensing Imagery Comprehension
- awesome-vlgfm - EarthMarker: A Visual Prompt Learning Framework for Region-level and Point-level Remote Sensing Imagery Comprehension
README
# EarthMarker: A Visual Prompting Multi-modal Large Language Model for Remote Sensing
Official repository for [EarthMarker](https://ieeexplore.ieee.org/document/10817639).
Authors: Wei Zhang*, Miaoxin Cai*, Tong Zhang, Yin Zhuang, and Xuerui Mao
* The authors contributed equally to this work.
## :mega: News
- [2024.01.06]: We have released the dataset RSVP! 🔥 :fire::fire:
- [2024.12.22]: EarthMarker has been accepted to [IEEE TGRS](https://ieeexplore.ieee.org/document/10817639). 🎉
* [2024.07.19]: The paper for EarthMarker is released [arxiv](https://arxiv.org/abs/2407.13596). 🚀
## :sparkles: Introduction
A visual prompting MLLM called EarthMarker is proposed in the remote sensing (RS) domain for the first time. EarthMarker can comprehend RS imagery under visual and text joint prompts, and flexibly switch interpretation levels, including image, region, and point levels. More importantly, the proposed EarthMarker fills the gap in visual prompting MLLMs for RS, significantly catering to the fine-grained interpretation needs of RS imagery in real-world applications. EarthMarker is capable of various RS visual tasks including scene classification, referring object classification, captioning, and relationship analyses, which are beneficial to making informed decisions in real-world applications.
## :sparkles: The first RS Visual Prompting instruction Dataset RSVP
The entire data of RSVP is released! 🚀 RSVP contains roughly 3.65 M image-point-text and image-region-text pairings.
link1: https://pan.baidu.com/s/1_kMO5bBje7JXTNpxDiCvqg?pwd=gqdb
pwd: gqdb
link2: OneDrive version is uploading.
## :bookmark: Citation
```bash
@article{zhang2024earthmarker,
title={EarthMarker: A Visual Prompting Multi-modal Large Language Model for Remote Sensing},
author={Zhang, Wei and Cai, Miaoxin and Zhang, Tong and Zhuang, Yin and Li, Jun and Mao, Xuerui},
journal={IEEE Transactions on Geoscience and Remote Sensing},
year={2024},
publisher={IEEE}
}
```
## :memo: Acknowledgment
This paper benefits from [llama](https://github.com/facebookresearch/llama). Thanks for their wonderful work.