https://github.com/qzed/raft-meets-dicl
Master Thesis
https://github.com/qzed/raft-meets-dicl
Last synced: 9 months ago
JSON representation
Master Thesis
- Host: GitHub
- URL: https://github.com/qzed/raft-meets-dicl
- Owner: qzed
- Created: 2021-07-10T23:26:28.000Z (almost 5 years ago)
- Default Branch: main
- Last Pushed: 2024-04-17T09:03:36.000Z (about 2 years ago)
- Last Synced: 2025-09-02T10:40:55.739Z (10 months ago)
- Language: Python
- Size: 2.11 MB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Citation: CITATIONS.bib
Awesome Lists containing this project
README
# RAFT meets DICL
This repository contains the code for
> **[RAFT meets DICL: A Recurrent All-Pair Transform for Optical Flow Estimation based on Displacement Invariant Cost Volume Learning](http://dx.doi.org/10.18419/opus-13298)**
> _Master Thesis, University of Stuttgart, 2022_
> Maximilian Luz
If you find this work useful please [cite via BibTeX](CITATIONS.bib).
## Setup
This project uses `pipenv`, i.e. in terms of code, setup is as easy installing `pipenv` (e.g. via `python -m pip install --user pipenv`) and running `pipenv install` (or `pipenv install -d` if you want to also install development dependencies).
Commands and scripts of this project can then be run either in a shell opened with `pipenv shell` or directly via `pipenv run `.
The `pipenv` command can also be used to generate a `requirements.txt` file for integration into other python environments.
### Datasets
Datasets are expected to be placed in `../datasets`, i.e. a `datasets` folder next to the folder containing this code.
The dataset folder should follow the specification below:
```
../datasets
├──
│ └── data # the dataset root directory
```
For all supported datasets see `cfg/data/dataset/`.
The `path` attribute of these config files specifies the path local to the config file and may be adapted if needed.
A subset of supported datasets would look like this:
```
../datasets
├── hci-hd1k # HCI HD1K
│ └── data
│ ├── hd1k_challenge
│ ├── hd1k_flow_gt
│ ├── hd1k_flow_uncertainty
│ └── hd1k_input
├── kitti-flow-2012 # KITTI 2012
│ └── data
│ ├── testing
│ └── training
├── mpi-sintel-flow # MPI Sintel
│ └── data
│ ├── bundler
│ ├── flow_code
│ ├── test
│ ├── training
│ └── README.txt
├── ufreiburg-flyingchairs # Uni Freiburg FlyingChairs
│ ├── data
│ │ ├── data
│ │ └── README.txt
│ └── train_val.txt
└── ufreiburg-flyingthings3d # Uni Freiburg FlyingThings3D
└── data
├── frames_cleanpass
├── frames_finalpass
└── optical_flow
```
## Training
Training is done via
```
./main.py train -d -m
```
where `` is a strategy config specifying training stages, i.e. data and optimizer parameters, and `` is a model specification file, describing the model parameters.
For strategy config files see `cfg/strategy/`, for model config files see `cfg/model/`.
Config files are documented in `cfg/_doc/`.
By default, training will store tensorboard and other log files in the `runs//` directory.
If specified in the training strategy and stages, checkpoints will be stored in `runs//checkpoints/`.
Metrics etc. can be customized via an inspection config
See `./main.py train -h` for more information.
## Evaluation
Saved checkpoints can be evaluated via
```
./main.py evaluate -d -m -c
```
where `` is a data source specification (`cfg/data/`), `` is a model specification (`cfg/model/`), and `` is the checkpoint to evaluate.
Note that the model specification should match the one used for training the model (i.e. generating the checkpoint).
By default, this prints a set of metrics per sample and a summary collecting means.
This can e.g. be adapted via a custom evaluation config file.
The `eval` sub-command can also be used to visualize flow, visualize error metrics, and generate flow files for submission.
See `./main.py eval -h` for more information.
### Using original RAFT/DICL checkpoints
Original RAFT and DICL checkpoints can be used for evaluation with this framework.
To do this, the checkpoints have to be converted via
```
./scripts/chkpt_convert.py -i -o -f
```
where `` is the original checkpoint file, `` is the output file (to be generated) and `` is the format of the original checkpoint, i.e. one of `dicl` or `raft`.