Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/ignacio-rocco/weakalign
End-to-end weakly-supervised semantic alignment
https://github.com/ignacio-rocco/weakalign
Last synced: 22 days ago
JSON representation
End-to-end weakly-supervised semantic alignment
- Host: GitHub
- URL: https://github.com/ignacio-rocco/weakalign
- Owner: ignacio-rocco
- License: mit
- Created: 2017-12-20T07:14:37.000Z (over 6 years ago)
- Default Branch: master
- Last Pushed: 2018-11-09T16:35:35.000Z (over 5 years ago)
- Last Synced: 2024-02-12T23:48:58.343Z (4 months ago)
- Language: Python
- Size: 118 KB
- Stars: 207
- Watchers: 11
- Forks: 47
- Open Issues: 10
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Lists
- awesome-cbir-papers - End-to-end weakly-supervised semantic alignment
README
# End-to-end weakly-supervised semantic alignment
## About
This is the implementation of the paper "End-to-end weakly-supervised semantic alignment" by I. Rocco, R. Arandjelović and J. Sivic.
For more information check out the project [[website](http://www.di.ens.fr/willow/research/weakalign/)] and the paper on [[arXiv](https://arxiv.org/abs/1712.06861)].
## Getting started
### Dependencies
The code is implemented using Python 3 and PyTorch 0.2. All dependencies are included in the standard Anaconda distribution.
### Training
The code includes scripts for pre-training the models with strong supervision (`train_strong.py`) as proposed in [our previous work](http://www.di.ens.fr/willow/research/cnngeometric/), as well as to fine-tune the model using weak supervision (`train_weak.py`) as proposed in this work.
Training scripts can be found in the `scripts/` folder.
### Evaluation
Evaluation is implemented in the `eval.py` file. It can evaluate a single affine or TPS model (with the `--model-aff` and `--model-tps` parameters respectively), or a combined affine+TPS model (with the `--model`) parameter.
The evaluation dataset is passed with the `--eval-dataset` parameter.
### Trained models
Trained models for the baseline method using only strong supervision and the proposed method using additional weak supervision are provided below. You can store them in the `trained_models/` folder.
With the provided code below you should obtain the results from Table 2 of the paper.
**CNNGeometric with VGG-16 baseline:** [[affine model](http://www.di.ens.fr/willow/research/weakalign/trained_models/cnngeo_vgg16_affine.pth.tar)],[[TPS model](http://www.di.ens.fr/willow/research/weakalign/trained_models/cnngeo_vgg16_tps.pth.tar)]
```
python eval.py --feature-extraction-cnn vgg --model-aff trained_models/cnngeo_vgg16_affine.pth.tar --model-tps trained_models/cnngeo_vgg16_tps.pth.tar --eval-dataset pf-pascal
```
**CNNGeometric with ResNet-101 baseline:** [[affine model](http://www.di.ens.fr/willow/research/weakalign/trained_models/cnngeo_resnet101_affine.pth.tar)],[[TPS model](http://www.di.ens.fr/willow/research/weakalign/trained_models/cnngeo_resnet101_tps.pth.tar)]
```
python eval.py --feature-extraction-cnn resnet101 --model-aff trained_models/cnngeo_resnet101_affine.pth.tar --model-tps trained_models/cnngeo_resnet101_tps.pth.tar --eval-dataset pf-pascal
```
**Proposed method:** [[combined aff+TPS model](http://www.di.ens.fr/willow/research/weakalign/trained_models/weakalign_resnet101_affine_tps.pth.tar)]
```
python eval.py --feature-extraction-cnn resnet101 --model trained_models/weakalign_resnet101_affine_tps.pth.tar --eval-dataset pf-pascal
```
## BibTeX
If you use this code in your project, please cite our paper:
````
@article{Rocco18,
author = "Rocco, I. and Arandjelovi\'c, R. and Sivic, J.",
title = "End-to-end weakly-supervised semantic alignment",
journal={arXiv preprint arXiv:1712.06861},
}
````