{"id":20339273,"url":"https://github.com/encounter1997/sfa","last_synced_at":"2025-07-08T18:06:05.680Z","repository":{"id":43536633,"uuid":"389637868","full_name":"encounter1997/SFA","owner":"encounter1997","description":"Official Implementation of \"Exploring Sequence Feature Alignment for Domain Adaptive Detection Transformers\"","archived":false,"fork":false,"pushed_at":"2022-01-05T04:54:34.000Z","size":304,"stargazers_count":99,"open_issues_count":4,"forks_count":7,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-06-04T11:33:01.183Z","etag":null,"topics":["detection-transformer","domain-adaptation","pytorch"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/encounter1997.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-07-26T13:13:22.000Z","updated_at":"2025-04-07T08:42:52.000Z","dependencies_parsed_at":"2022-07-15T09:17:03.085Z","dependency_job_id":null,"html_url":"https://github.com/encounter1997/SFA","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/encounter1997/SFA","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/encounter1997%2FSFA","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/encounter1997%2FSFA/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/encounter1997%2FSFA/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/encounter1997%2FSFA/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/encounter1997","download_url":"https://codeload.github.com/encounter1997/SFA/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/encounter1997%2FSFA/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":264320949,"owners_count":23590561,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["detection-transformer","domain-adaptation","pytorch"],"created_at":"2024-11-14T21:16:08.222Z","updated_at":"2025-07-08T18:06:05.652Z","avatar_url":"https://github.com/encounter1997.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Sequence Feature Alignment (SFA)\n\nBy Wen Wang,  Yang Cao,  Jing Zhang, Fengxiang He, Zheng-jun Zha, Yonggang Wen, and Dacheng Tao\n\nThis repository is an official implementation of the paper [Exploring Sequence Feature Alignment for Domain Adaptive Detection Transformers](https://arxiv.org/abs/2107.12636), which is accepted to ACM MultiMedia 2021.\n\n\n## Introduction\n\n**TL; DR.**  We develop a domain adaptive object detection method **SFA** that is specialized for adaptive detection transformers. It contains a domain query-based feature alignment model and a token-wise feature alignment module for global and local feature alignment respectively, and a bipartite matching consistency loss for improving robustness. \n\n![SFA](./figs/SFA.png)\n\n**Abstract.**  Detection transformers have recently shown promising object detection results and attracted increasing attention. However, how to develop effective domain adaptation techniques to improve its cross-domain performance remains unexplored and unclear. In this paper, we delve into this topic and empirically find that direct feature distribution alignment on the CNN backbone only brings limited improvements, as it does not guarantee domain-invariant sequence features in the transformer for prediction. To address this issue, we propose a novel Sequence Feature Alignment (SFA) method that is specially designed for the adaptation of detection transformers. Technically, SFA consists of a domain query-based feature alignment (DQFA) module and a token-wise feature alignment (TDA) module. In DQFA, a novel domain query is used to aggregate and align global context from the token sequence of both domains. DQFA reduces the domain discrepancy in global feature representations and object relations when deploying in the transformer encoder and decoder, respectively. Meanwhile, TDA aligns token features in the sequence from both domains, which reduces the domain gaps in local and instance-level feature representations in the transformer encoder and decoder, respectively. Besides, a novel bipartite matching consistency loss is proposed to enhance the feature discriminability for robust object detection. Experiments on three challenging benchmarks show that SFA outperforms state-of-the-art domain adaptive object detection methods.\n\n## Main Results\n\nThe experimental results and model weights for Cityscapes to Foggy Cityscapes are shown below.\n\n|       Model       | mAP  | mAP@50 | mAP@75 | mAP@S | mAP@M | mAP@L | Log \u0026 Model |\n| :----------------: | :------: | :------: | :------: | :------: | :------: | :------: | :------: |\n| SFA-DefDETR | 21.5 |  41.1  |  20.0  |  3.9  | 20.9  | 43.0  | [Google Drive](https://drive.google.com/drive/folders/1onPNgNaJkKKfYCizisZKmm1-xGZ9hLeB?usp=sharing) |\n| SFA-DefDETR-BoxRefine | 23.9 | 42.6 | 22.5 | 3.8 | 21.6 | 46.7 | [Google Drive](https://drive.google.com/drive/folders/1CHAj09z1hkDSZ2X_4U28I-RLlA03lBqV?usp=sharing) |\n| SFA-DefDETR-TwoStage | 24.1 | 42.5 | 22.8 | 3.8 | 22.0 | 48.1 | [Google Drive](https://drive.google.com/drive/folders/1eODtm93v21V523rGvnoHX8kDhz375Fqr?usp=sharing) |\n\n*Note:*\n\n1. All models of SFA are trained with total batch size of 4. \n2. \"DefDETR\" means Deformable DETR (with R50 backbone). \n3. \"BoxRefine\" means Deformable DETR with iterative box refinement.\n4. \"TwoStage\" indicates the two-stage Deformable DETR variant.\n5. The original implementation is based on our internal codebase. There are slight differences in the released code are slight differences. For example, we only use the middle features output by the first encoder and decoder layers for hierarchical feature alignment, to reduce computational costs during training. \n\n\n## Installation\n\n### Requirements\n\n* Linux, CUDA\u003e=9.2, GCC\u003e=5.4\n  \n* Python\u003e=3.7\n\n    We recommend you to use Anaconda to create a conda environment:\n    ```bash\n    conda create -n sfa python=3.7 pip\n    ```\n    Then, activate the environment:\n    ```bash\n    conda activate sfa\n    ```\n  \n* PyTorch\u003e=1.5.1, torchvision\u003e=0.6.1 (following instructions [here](https://pytorch.org/))\n\n    For example, if your CUDA version is 9.2, you could install pytorch and torchvision as following:\n    ```bash\n    conda install pytorch=1.5.1 torchvision=0.6.1 cudatoolkit=9.2 -c pytorch\n    ```\n  \n* Other requirements\n    ```bash\n    pip install -r requirements/requirements.txt\n    ```\n\n* Logging using [wandb](https://wandb.ai/site) (optional)\n    ```bash\n    pip install -r requirements/optional.txt\n    ```\n\n### Compiling CUDA operators\n```bash\ncd ./models/ops\nsh ./make.sh\n# unit test (should see all checking is True)\npython test.py\n```\n\n## Usage\n\n### Dataset preparation\n\nWe use the preparation of Cityscapes to Foggy Cityscapes adaptation as demonstration. Other domain adaptation benchmarks can be prepared in analog.\nCityscapes and Foggy Cityscapes datasets can be downloaded from [here](https://www.cityscapes-dataset.com/login/). The annotations in COCO format can be obtained from [here](https://drive.google.com/drive/folders/1x95Qjun_AIi6BvtqGwWF6y9kPrVFIcI9?usp=sharing). Afterward, please organize the datasets and annotations as following:\n\n```\n[coco_path]\n└─ cityscapes\n   └─ leftImg8bit\n      └─ train\n      └─ val\n└─ foggy_cityscapes\n   └─ leftImg8bit_foggy\n      └─ train\n      └─ val\n└─ CocoFormatAnnos\n   └─ cityscapes_train_cocostyle.json\n   └─ cityscapes_foggy_train_cocostyle.json\n   └─ cityscapes_foggy_val_cocostyle.json\n```\n\n### Training\n\nAs an example, we provide commands for training our SFA on a single node with 4 GPUs for weather adaptation.\n\n#### Training SFA-DeformableDETR\n\n```bash\nGPUS_PER_NODE=4 ./tools/run_dist_launch.sh 4 ./configs_da/sfa_r50_deformable_detr.sh --wandb\n```\n\n#### Training SFA-DeformableDETR-BoxRefine\n\n```bash\nGPUS_PER_NODE=4 ./tools/run_dist_launch.sh 4 ./configs_da/sfa_r50_deformable_detr_plus_iterative_bbox_refinement.sh --wandb\n```\n\n#### Training SFA-DeformableDETR-TwoStage\n\n```bash\nGPUS_PER_NODE=4 ./tools/run_dist_launch.sh 4 ./configs_da/sfa_r50_deformable_detr_plus_iterative_bbox_refinement_plus_plus_two_stage.sh --wandb\n```\n\n#### Training Source-only DeformableDETR\nPlease refer to the ```source``` branch.\n\n### Evaluation\n\nYou can get the config file and pretrained model of SFA (the link is in \"Main Results\" session), then run following command to evaluate it on Foggy Cityscapes validation set:\n\n```bash\n\u003cpath to config file\u003e --resume \u003cpath to pre-trained model\u003e --eval\n```\n\nYou can also run distributed evaluation by using ```./tools/run_dist_launch.sh``` or ```./tools/run_dist_slurm.sh```.\n\n## Acknowledgement \n\nThis project is based on [DETR](https://github.com/facebookresearch/detr) and  [Deformable DETR](https://github.com/fundamentalvision/Deformable-DETR). Thanks for their wonderful works. See [LICENSE](./LICENSE) for more details. \n\n\n## Citing SFA\nIf you find SFA useful in your research, please consider citing:\n```bibtex\n@inproceedings{wang2021exploring ,\n  title={Exploring Sequence Feature Alignment for Domain Adaptive Detection Transformers},\n  author={Wen, Wang and Yang, Cao and Jing, Zhang and Fengxiang, He and Zheng-Jun, Zha and Yonggang, Wen and Dacheng, Tao},\n  booktitle={Proceedings of the 29th ACM International Conference on Multimedia},\n  year={2021}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fencounter1997%2Fsfa","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fencounter1997%2Fsfa","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fencounter1997%2Fsfa/lists"}