{"id":13441666,"url":"https://github.com/ocetintas/MPNTrackSeg","last_synced_at":"2025-03-20T12:32:21.665Z","repository":{"id":42226761,"uuid":"509443696","full_name":"ocetintas/MPNTrackSeg","owner":"ocetintas","description":"\"Multi-Object Tracking and Segmentation via Neural Message Passing\" (IJCV 2022) - Official Implementation","archived":false,"fork":false,"pushed_at":"2023-04-13T11:25:29.000Z","size":1570,"stargazers_count":20,"open_issues_count":0,"forks_count":2,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-08-01T03:36:57.490Z","etag":null,"topics":["graph-neural-networks","multi-object-tracking","neural-message-passing","segmentation"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ocetintas.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2022-07-01T12:18:36.000Z","updated_at":"2024-06-19T04:13:13.000Z","dependencies_parsed_at":"2024-01-16T02:45:53.338Z","dependency_job_id":"5e8dbac9-7bb5-41cc-b2a8-9e0b3cf85d24","html_url":"https://github.com/ocetintas/MPNTrackSeg","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ocetintas%2FMPNTrackSeg","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ocetintas%2FMPNTrackSeg/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ocetintas%2FMPNTrackSeg/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ocetintas%2FMPNTrackSeg/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ocetintas","download_url":"https://codeload.github.com/ocetintas/MPNTrackSeg/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":221760108,"owners_count":16876355,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["graph-neural-networks","multi-object-tracking","neural-message-passing","segmentation"],"created_at":"2024-07-31T03:01:36.684Z","updated_at":"2025-03-20T12:32:21.657Z","avatar_url":"https://github.com/ocetintas.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# Multi-Object Tracking and Segmentation via Neural Message Passing\n\nThis the official implementation of our **IJCV 2022** paper *Multi-Object Tracking and Segmentation via Neural Message Passing* ([Guillem Brasó*](https://dvl.in.tum.de/team/braso/), [Orcun Cetintas*](https://dvl.in.tum.de/team/cetintas/), [Laura Leal-Taixe](https://dvl.in.tum.de/team/lealtaixe/)) \n\nThis work builds upon our previous **CVPR 2020 (oral)** paper [*Learning a Neural Solver for Multiple Object Tracking*](https://arxiv.org/abs/1912.07515)\nand extends it by: \n1) integrating an attentive module to our neural message passing scheme to yield a unified model for multi-object tracking and segmentation \n2) providing an extensive evaluation of our tracking model over three challenging datasets, including MOT20, KITTI and the recently proposed Human in Events dataset.\n\n[[Paper]](tbd)\n![Method Visualization](visuals/overview.png)\n\n## Setup\n\n1. Clone and enter this repository:\n   ```\n   git clone https://github.com/ocetintas/MPNTrackSeg.git\n   cd MPNTrackSeg\n   ```\n\n2. Put the project directories in PYTHONPATH by copying the following lines in your .bashrc file:\n   ```\n    export PYTHONPATH=\"${PYTHONPATH}:[PATH_TO_YOUR_PROJECT]/MPNTrackSeg/src\"\n    export PYTHONPATH=\"${PYTHONPATH}:[PATH_TO_YOUR_PROJECT]/MPNTrackSeg/MOTChallengeEvalKit/src\"\n    export PYTHONPATH=\"${PYTHONPATH}:[PATH_TO_YOUR_PROJECT]/MPNTrackSeg/tracktor-mots/src\"\n   ```\n\n3. Create an [Anaconda environment](https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html) for this project:\n    1. `conda env create -f environment.yml`\n    2. `conda activate MPNTrackSeg`\n\n\n4. Modify the variables `DATA_PATH`, and `OUTPUT_PATH` in  `src/mot_neural_solver/path_cfg.py` so that they are set to\nyour preferred locations for storing datasets and output results, respectively.\n\n5. Download [MOTS20](https://motchallenge.net/data/MOTS/) and/or [KITTIMOTS](https://www.vision.rwth-aachen.de/page/mots) datasets. Expected folder structure: \n\n    ```\n    DATA_PATH\n    ├── KITTIMOTS\n    │   └── ...\n    └── MOTS20\n        └── train\n        │    ├── MOTS20-02\n        │    │   ├── det\n        │    │   │   └── det.txt\n        │    │   └── gt\n        │    │   │   └── gt.txt\n        │    │   └── img1 \n        │    │   │   └── ...\n        │    │   └── seqinfo.ini\n        │    └── ...\n        └── test\n            └── ...\n\n    ```\n\n6. (**OPTIONAL**) We provide our trained models and detections:\n\n    1. [[Models]](https://drive.google.com/file/d/1wjMTZnND3w7OtmSiJiKlp1tMc_TGOgUX/view?usp=sharing)\n    \n        Default folder structure is: \n\n        ```\n        MPNTrackSeg\n        └── output\n                └── trained_models\n                    └── mots\n                    │    ├── kitti.ckpt\n                    │    └── mots20.ckpt\n                    └── reid\n                        └── resnet50_market_cuhk_duke.tar-232\n\n        ```\n\n    2. [[Detections]](https://drive.google.com/file/d/1xdMAQLnM06MkawgaNCHr9OPuIJk3HUQB/view?usp=sharing)\n\n        Default location for each detections file is under det folder for each sequence with the naming det.txt (See step 5)\n\n\n## Training\nSpecify hyperparameters via [tracking_cfg.yaml](https://github.com/ocetintas/MPNTrackSeg/blob/main/configs/tracking_cfg.yaml) and choose train and validation splits via [train.py](https://github.com/ocetintas/MPNTrackSeg/blob/main/scripts/train.py#L39). \n```\npython scripts/train.py\n```\n\n## Evaluation\nSpecify hyperparameters via [tracking_cfg.yaml](https://github.com/ocetintas/MPNTrackSeg/blob/main/configs/tracking_cfg.yaml) and choose the trained model via [evaluate.py](https://github.com/ocetintas/MPNTrackSeg/blob/main/scripts/evaluate.py#L22). Our hyperparameters for MOTS20 and KITTIMOTS datasets are provided under configs folder. \n```\npython scripts/evaluate.py\n```\n\n## Citation\n If you use our work in your research, please cite our publications:\n\n 1. Multi-Object Tracking and Segmentation via Neural Message Passing (IJCV 2022)\n\n    ```\n         @article{MPNTrackSeg,\n         author = {Bras{\\'o}, Guillem and Cetintas, Orcun and Leal-Taix{\\'e}, Laura},\n         date = {2022/09/26},\n         doi = {10.1007/s11263-022-01678-6},\n         id = {Bras{\\'o}2022},\n         isbn = {1573-1405},\n         journal = {International Journal of Computer Vision},\n         title = {Multi-Object Tracking and Segmentation Via Neural Message Passing},\n         url = {https://doi.org/10.1007/s11263-022-01678-6},\n         year = {2022}}\n    ```\n2. Learning a Neural Solver for Multiple Object Tracking (CVPR 2020)\n\n    ```\n        @InProceedings{braso_2020_CVPR,\n        author={Guillem Brasó and Laura Leal-Taixé},\n        title={Learning a Neural Solver for Multiple Object Tracking},\n        booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},\n        month = {June},\n        year = {2020}\n    }\n    ```\n\n## Acknowledgements\nWe use the codebases of [Tracktor](https://github.com/phil-bergmann/tracking_wo_bnw) for preprocessing and [MOTChallengeEvalKit](https://github.com/dendorferpatrick/MOTChallengeEvalKit) and [TrackEval](https://github.com/JonathonLuiten/TrackEval) for evaluation. We thank the authors of these codebases for their great work!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Focetintas%2FMPNTrackSeg","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Focetintas%2FMPNTrackSeg","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Focetintas%2FMPNTrackSeg/lists"}