https://github.com/interpause/dinosavi
Self-supervised learning of Video Object Segmentation using DINOSAUR and SAVi
https://github.com/interpause/dinosavi
computer-vision object-centric-learning object-centric-video-prediction object-segmentation video-object-segmentation
Last synced: 11 months ago
JSON representation
Self-supervised learning of Video Object Segmentation using DINOSAUR and SAVi
- Host: GitHub
- URL: https://github.com/interpause/dinosavi
- Owner: Interpause
- Created: 2023-02-15T01:57:17.000Z (almost 3 years ago)
- Default Branch: main
- Last Pushed: 2025-01-05T11:23:16.000Z (about 1 year ago)
- Last Synced: 2025-01-23T01:08:51.427Z (about 1 year ago)
- Topics: computer-vision, object-centric-learning, object-centric-video-prediction, object-segmentation, video-object-segmentation
- Language: Jupyter Notebook
- Homepage:
- Size: 1.06 MB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# DINOSAVi
> Self-supervised learning of Video Object Segmentation using DINOSAUR and SAVi
**Presentation Slides**:
> VOS shown above is from `slot-s256d-line-t01-tline-ini5` model
Models:
**Note**: is used to manage configuration, CLI and experiment running. See for CLI override grammar. Jump to [Codebase Notes](#codebase-notes) for more info.
## Installation
```sh
# (Optional) Use conda for virtual environment instead. Poetry creates venv by default.
conda create -n dinosavi python=3.10
poetry install
```
## Training
```sh
python -m dinosavi mode=train hparam={hparam_file_name} exp_name={experiment_name}
```
## Evaluation
```sh
python -m dinosavi mode=eval resume={.ckpt_to_load} exp_name={results_name} device={cpu_or_cuda}
```
## Codebase Notes
- [Hydra](https://hydra.cc/) (built on [OmegaConf](https://omegaconf.readthedocs.io/)) is used for configuration management, CLI and experiment running.
- For CLI options, you can refer to [`dinosavi/cfg/main.yaml`](dinosavi/cfg/main.yaml) and [`dinosavi/cfg/mode`](dinosavi/cfg/mode/).
- Hyperparameter config files are stored in [`dinosavi/cfg/hparam`](dinosavi/cfg/hparam/).
- [OmegaConf Variable Interpolation](https://omegaconf.readthedocs.io/en/2.3_branch/usage.html#variable-interpolation) is heavily used to link hyperparameter values into relevant config files.
- Many parts of the code (i.e., models, datasets, trainers) are implemented as config files using [Hydra's Instantiate API](https://hydra.cc/docs/advanced/instantiate_objects/overview/).
- We use [Black](https://github.com/psf/black) for formatting, [isort](https://github.com/PyCQA/isort) for import sorting, and Google-style docstrings.
- Anything to do with [Contrastive Random Walk](https://ajabri.github.io/videowalk/) (CRW) is leftover legacy code from earlier experiments.