Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/likaiucas/OBM
https://github.com/likaiucas/OBM
Last synced: about 2 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/likaiucas/OBM
- Owner: likaiucas
- Created: 2024-06-28T17:13:32.000Z (7 months ago)
- Default Branch: main
- Last Pushed: 2024-06-28T17:21:12.000Z (7 months ago)
- Last Synced: 2024-06-28T18:37:57.758Z (7 months ago)
- Size: 2.97 MB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
- Awesome-Segment-Anything - [code
README
# TGRS preprint paper
We have two contributions:
1) we first introduced the "Offset Token" concept and its decode-encode methods. This is a very big contribution, which brings the former method in BFE into the era of the Transformer and Foundation Models.
2) We newly discovered a common pattern of predicting offsets.
[![MIT License][license-shield]][license-url]
Extract Building Footprint from Aerial Images with Offset-Building Model (OBM)
Extract building footprint like a human
Explore the docs »
View Demo
·
Report Bug
·
Request Feature
Table of Contents
## What is OBM?
| ![Product Name Screen Shot][product-screenshot] | ![Product Name Screen Shot][ROAM-screenshot] |
| :---: | :---: |
| (a) OBM Model | (b) ROAM module |
We first proposed a novel interactive model for footprint extraction and designed a series of Distance NMS algorithms tailored for the Building Footprint Extraction (BFE) problem. We also designed new metrics to measure the interactive BFE model.
Based on our accurate extraction of the roof and root-to-footprint offset. We can rebuild buildings' relative height maps.
**(a)** illustrates our structure which inherits from the Segment Anything Model (SAM).
**(b)** gives out the structure of our core module, Reference Offset Adaptive Module (ROAM). During inference, based on the results of the base offset head, each building will classified by its length and then roam to the adaptive head. The final result is determined by the Basehead and adaptive head.
Our model reaches new SOTA:
* A clear drop of offset error by 16.99%, increase roof boundary iou by 13.15%, on open dataset BONAI.
* Without extra training, we tested all models on a newly annotated dataset for generalization, improving 20.29% in vector and 76.36% in offset angle.
* DNMS series can bring extra gain in all situations.
### Built With
Our work is based on the following:
- [pytorch]
- [MMDetection]
- [BONAI]
- [SAM]
Our weight of OBM is available at OBM weight.
## Getting Started
This is a quick start of OBM.
### Prerequisites
The code is built on the old version of mmdetection, and trained with a server that has 6x RTX3090.
* ensure you have the correct CUDA and compatible torch version
```sh
nvidia-smi
```
Our runnable environments:
`pytorch 1.7.0, CUDA 11.1`
### Dataset
Downloads training images and test images from [BONAI]. We newly added building segementation label in our newly launched dataset which are avaliable with [Huizhou] together.
### Installation
NOTE: Please follow the installation of [BONAI] and the early edition of [MMdetection].
## Usage
* Train on your own dataset:
```bash
bash tools/dist_train.sh configs/obm_seg_fintune/smlcdr_obm_pretrain.py 6 # train with ROAM
bash tools/dist_train.sh configs/obm_seg/obm_seg_b.py 6 # train without ROAM
```
* Inference your dataset:
```bash
python tools/test_offset.py --config configs/obm_seg/obm_seg_b.py
```
_``WARNING: OUR TRAINING DOES NOT SUPPORT FINETUNE LIKE LORA, BACKBONE IS INCLUDED, PLEASE SET `samples_per_gpu = 1` !!!``_
_``WARNING: PLEASE SET `samples_per_gpu = 1` WHILE INFERENCING !!!``_
* Improve offset quality:
```bash
# using function fixangle()
## parameter: model = 'max' represents DNMS
## parameter: model = 'guassia_std' represents soft-DNMS
python tools/postprocess_offset.py
```
* Visualize your results:
```bash
# we provide two kinds of visualizing functions:
## 1: vis_3d() for relative height maps.
## 2: vis_boundary_offset: for roof and footprint boundary.
python tools/visual_offset.py
# if you want to visualize the results of LOFT in BONAI
python tools/visual_instance_seg.py
```
## Workflow and Results
Our model simulates the process of how a human annotates a footprint.
### 1. At the **`first`** stage, we input an image with some prompts to imply buildings' rough location, using box prompts as an example:
### 2. **`Then`** Our OBM will give out roof segmentation and a roof-to-footprint offset for each prompt.
### 3. **`Finally`**, we drag the roof to its footprint via the offset.
* We provide two kinds of operation: one is to directly get **footprints**, and the other is to get the **relative height maps**.
## License
Distributed under the MIT License. See `LICENSE.txt` for more information.
## Contact
Offset tokens are hard to train; they are very sensitive to the settings.
If you have any problems training the offset tokens, please contact me by likai211#mails.ucas.ac.cn or kaili37-c#my.cityu.edu.hk.
I think my experience of failure in training will help you train your model.
You can also contact me for any building-related problem or collaboration.
## Citing
```
@ARTICLE{obm,
author={Li, Kai and Deng, Yupeng and Kong, Yunlong and Liu, Diyou and Chen, Jingbo and Meng, Yu and Ma, Junxian and Wang, Chenhao},
journal={IEEE Transactions on Geoscience and Remote Sensing},
title={Prompt-Driven Building Footprint Extraction in Aerial Images with Offset-Building Model},
year={2024},
volume={},
number={},
pages={1-1},
keywords={Buildings;Prediction algorithms;Production;Data models;Data mining;Remote sensing;Instance segmentation;Feature extraction;Training;Three-dimensional displays;Building footprint extraction;Roof segmentation;Roof-to-footprint offset extraction;Segment Anything Model (SAM);Non-Maximum Suppression(NMS)},
doi={10.1109/TGRS.2024.3487652}}
```
[contributors-shield]: https://img.shields.io/github/contributors/othneildrew/Best-README-Template.svg?style=for-the-badge
[contributors-url]: https://github.com/
[forks-shield]: https://img.shields.io/github/forks/othneildrew/Best-README-Template.svg?style=for-the-badge
[forks-url]: https://github.com/network/members
[stars-shield]: https://img.shields.io/github/stars/othneildrew/Best-README-Template.svg?style=for-the-badge
[stars-url]: https://github.com/stargazers
[issues-shield]: https://img.shields.io/github/issues/othneildrew/Best-README-Template.svg?style=for-the-badge
[issues-url]: https://github.com/issues
[license-shield]: https://img.shields.io/github/license/othneildrew/Best-README-Template.svg?style=for-the-badge
[license-url]: https://github.com/blob/master/LICENSE.txt
[product-screenshot]: images/model2.png
[ROAM-screenshot]: images/ROAM.png
[Next.js]: https://img.shields.io/badge/next.js-000000?style=for-the-badge&logo=nextdotjs&logoColor=white
[Next-url]: https://nextjs.org/
[React.js]: https://img.shields.io/badge/React-20232A?style=for-the-badge&logo=react&logoColor=61DAFB
[React-url]: https://reactjs.org/
[Vue.js]: https://img.shields.io/badge/Vue.js-35495E?style=for-the-badge&logo=vuedotjs&logoColor=4FC08D
[Vue-url]: https://vuejs.org/
[Angular.io]: https://img.shields.io/badge/Angular-DD0031?style=for-the-badge&logo=angular&logoColor=white
[Angular-url]: https://angular.io/
[Svelte.dev]: https://img.shields.io/badge/Svelte-4A4A55?style=for-the-badge&logo=svelte&logoColor=FF3E00
[Svelte-url]: https://svelte.dev/
[Laravel.com]: https://img.shields.io/badge/Laravel-FF2D20?style=for-the-badge&logo=laravel&logoColor=white
[Laravel-url]: https://laravel.com
[Bootstrap.com]: https://img.shields.io/badge/Bootstrap-563D7C?style=for-the-badge&logo=bootstrap&logoColor=white
[Bootstrap-url]: https://getbootstrap.com
[JQuery.com]: https://img.shields.io/badge/jQuery-0769AD?style=for-the-badge&logo=jquery&logoColor=white
[JQuery-url]: https://jquery.com
[Huizhou]: https://portland-my.sharepoint.com/:f:/g/personal/kaili37-c_my_cityu_edu_hk/Ep2EnO01ZghPuebKASl5h60BUjbBvrbiSHKoSiwfIBYBNg?e=7hdlzQ
[OBMweight]: https://1drv.ms/u/c/4714078bb90d0216/EbVLDOlRwlZHr5h60nC3HuoBk1QrRFc6QqY_oXPCz-_3nA?e=Xbe1Qc
[pytorch.org]: https://pytorch.org
[MMDetection.git]: https://github.com/open-mmlab/mmdetection
[pytorch]: https://pytorch.org
[MMDetection]: https://github.com/open-mmlab/mmdetection
[BONAI.git]: https://github.com/jwwangchn/BONAI
[BONAI]: https://github.com/jwwangchn/BONAI
[SAM]: https://github.com/facebookresearch/segment-anything