https://github.com/weigertlab/trackastra
https://github.com/weigertlab/trackastra
Last synced: 22 days ago
JSON representation
- Host: GitHub
- URL: https://github.com/weigertlab/trackastra
- Owner: weigertlab
- License: bsd-3-clause
- Created: 2024-03-08T12:47:06.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2025-01-31T11:02:21.000Z (10 months ago)
- Last Synced: 2025-01-31T12:18:44.006Z (10 months ago)
- Language: Python
- Size: 49.5 MB
- Stars: 58
- Watchers: 4
- Forks: 12
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- License: LICENSE
- Citation: CITATION.cff
Awesome Lists containing this project
- awesome-scientific-image-analysis - Trackastra - Tracking with Transformers. (🐾 Tracking / Software tools)
README
[](https://pypi.org/project/trackastra/)
[](https://github.com/weigertlab/trackastra/blob/main/LICENSE)
# *Trackastra* - Tracking by Association with Transformers
*Trackastra* is a cell tracking approach that links already segmented cells in a microscopy timelapse by predicting associations with a transformer model that was trained on a diverse set of microscopy videos.
## Reference
Paper: [Trackastra: Transformer-based cell tracking for live-cell microscopy](https://www.ecva.net/papers/eccv_2024/papers_ECCV/papers/09819.pdf)
```
@inproceedings{gallusser2024trackastra,
title={Trackastra: Transformer-based cell tracking for live-cell microscopy},
author={Gallusser, Benjamin and Weigert, Martin},
booktitle={European conference on computer vision},
pages={467--484},
year={2024},
organization={Springer}
}
```
## Examples
Nuclei tracking | Bacteria tracking
:-: | :-:
|
## Installation
This repository contains the Python implementation of Trackastra.
Please first set up a Python environment (with Python version 3.10 or higher), preferably via [conda](https://conda.io/projects/conda/en/latest/user-guide/install/index.html) or [mamba](https://mamba.readthedocs.io/en/latest/installation/mamba-installation.html#mamba-install).
### Simple installation
Trackastra can then be installed from PyPI using `pip`:
```bash
pip install trackastra
```
### With ILP support
For tracking with an integer linear program (ILP, which is optional)
```bash
conda create --name trackastra python=3.10 --no-default-packages
conda activate trackastra
conda install -c conda-forge -c gurobi -c funkelab ilpy
pip install "trackastra[ilp]"
```
### Installation with training support
```bash
pip install "trackastra[train]"
```
📄
Development installation
```bash
conda create --name trackastra python=3.10 --no-default-packages
conda activate trackastra
conda install -c conda-forge -c gurobi -c funkelab ilpy
git clone https://github.com/weigertlab/trackastra.git
pip install -e "./trackastra[all]"
```
📄
Notes/Troubleshooting
- For the optional ILP linking, this will install [`motile`](https://funkelab.github.io/motile/index.html) and binaries for two discrete optimizers:
1. The [Gurobi Optimizer](https://www.gurobi.com/). This is a commercial solver, which requires a valid license. Academic licenses are provided for free, see [here](https://www.gurobi.com/academia/academic-program-and-licenses/) for how to obtain one.
2. The [SCIP Optimizer](https://www.scipopt.org/), a free and open source solver. If `motile` does not find a valid Gurobi license, it will fall back to using SCIP.
- On MacOS, installing packages into the conda environment before installing `ilpy` can cause problems.
- 2024-06-07: On Apple M3 chips, you might have to use the nightly build of `torch` and `torchvision`, or worst case build them yourself.
## Usage: Tracking with a pretrained model
The input to Trackastra is a sequence of images and their corresponding cell (instance) segmentations.
> The available pretrained models are described in detail [here](trackastra/model/pretrained.json).
Tracking with Trackastra can be done via:
Napari plugin
For a quick try of Trackastra on your data, please use our [napari plugin](https://github.com/weigertlab/napari-trackastra/), which already comes with pretrained models included.
Python API
All you need are the following two `numpy` arrays:
- `imgs`: a microscopy time lapse of shape `time,(z),y,x`.
- `masks`: corresponding instance segmentation of shape `time,(z),y,x`.
The predicted associations can then be used for linking with several modes:
- `greedy_nodiv` (greedy linking with no division) - fast, no additional dependencies
- `greedy` (greedy linking with division) - fast, no additional dependencies
- `ilp` (ILP based linking) - slower but more accurate, needs [`motile`](https://github.com/funkelab/motile)
Apart from that, no hyperparameters to choose :)
📄 Show python example
```python
import torch
from trackastra.model import Trackastra
from trackastra.tracking import graph_to_ctc, graph_to_napari_tracks, write_to_geff
from trackastra.data import example_data_bacteria
device = "automatic" # explicit choices: [cuda, mps, cpu]
# load some test data images and masks
imgs, masks = example_data_bacteria()
# Load a pretrained model
model = Trackastra.from_pretrained("general_2d", device=device)
# or from a local folder
# model = Trackastra.from_folder('path/my_model_folder/', device=device)
# Track the cells
track_graph, masks_tracked = model.track(imgs, masks, mode="greedy") # or mode="ilp", or "greedy_nodiv"
# Relabel the masks and write to cell tracking challenge format (CTC),
ctc_tracks, ctc_masks = graph_to_ctc(
track_graph,
masks_tracked,
outdir="tracked_ctc",
)
# Or write to the graph exchange file format (GEFF)
write_to_geff(
track_graph,
masks_tracked,
outdir="tracked_geff.zarr",
)
```
You then can visualize the tracks with [napari](https://github.com/napari/napari):
```python
# Visualise in napari
napari_tracks, napari_tracks_graph, _ = graph_to_napari_tracks(track_graph)
import napari
v = napari.Viewer()
v.add_image(imgs)
v.add_labels(ctc_masks)
v.add_tracks(data=napari_tracks, graph=napari_tracks_graph)
```
Fiji (via TrackMate)
Trackastra is one of the available trackers in [TrackMate](https://imagej.net/plugins/trackmate/). For installation and usage instructions take a look at this [tutorial](
https://imagej.net/plugins/trackmate/trackers/trackmate-trackastra).
Docker images
Some of our models are available as docker images on [Docker Hub](https://hub.docker.com/r/bentaculum/trackastra-track/tags). Currently, we only provide CPU-based docker images.
Track within a docker container with the following command, filling the ``:
```bash
docker run -it -v :/data -v :/results bentaculum/trackastra-track: --input_test /data/ --detection_folder "
```
📄 Show example with Cell Tracking Challenge model:
```bash
wget http://data.celltrackingchallenge.net/training-datasets/Fluo-N2DH-GOWT1.zip
chmod -R 775 Fluo-N2DH-GOWT1
docker pull bentaculum/trackastra-track:model.ctc-linking.ilp
docker run -it -v ./:/data -v ./:/results bentaculum/trackastra-track:model.ctc-linking.ilp --input_test data/Fluo-N2DH-GOWT1/01 --detection_folder TRA
```
Command Line Interface
After installing Trackastra, simply run in your terminal
```bash
trackastra track --help
```
to build a command for tracking directly from images and corresponding instance segmentation masks saved on disk as two series of TIF files.
## Usage: Training a model on your own data
To run an example:
- Clone this repository and go into the scripts directory with `cd trackastra/scripts`.
- Download the [Fluo-N2DL-HeLa](http://data.celltrackingchallenge.net/training-datasets/Fluo-N2DL-HeLa.zip) dataset from the Cell Tracking Challenge into `data/ctc`.
Then run:
```bash
python train.py --config example_config.yaml
```
Generally, training data needs to be provided in the [Cell Tracking Challenge (CTC) format](http://public.celltrackingchallenge.net/documents/Naming%20and%20file%20content%20conventions.pdf), i.e. annotations are located in a folder containing one or several subfolders named `TRA`, with masks and tracklet information.