{"id":13443350,"url":"https://github.com/google-research/deeplab2","last_synced_at":"2025-05-16T11:03:57.603Z","repository":{"id":41118977,"uuid":"366871418","full_name":"google-research/deeplab2","owner":"google-research","description":"DeepLab2 is a TensorFlow library for deep labeling, aiming to provide a unified and state-of-the-art TensorFlow codebase for dense pixel labeling tasks.","archived":false,"fork":false,"pushed_at":"2023-04-17T22:02:27.000Z","size":8820,"stargazers_count":1020,"open_issues_count":39,"forks_count":158,"subscribers_count":21,"default_branch":"main","last_synced_at":"2025-04-19T14:04:23.318Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/google-research.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2021-05-12T22:39:37.000Z","updated_at":"2025-04-17T15:01:27.000Z","dependencies_parsed_at":"2024-01-16T20:29:58.238Z","dependency_job_id":"3b1b7d0a-e8c2-43c4-b96a-597c630e126e","html_url":"https://github.com/google-research/deeplab2","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-research%2Fdeeplab2","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-research%2Fdeeplab2/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-research%2Fdeeplab2/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-research%2Fdeeplab2/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/google-research","download_url":"https://codeload.github.com/google-research/deeplab2/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254518384,"owners_count":22084374,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T03:01:59.536Z","updated_at":"2025-05-16T11:03:57.576Z","avatar_url":"https://github.com/google-research.png","language":"Python","funding_links":[],"categories":["Python","🌐 Panoptic Segmentation","其他_机器视觉"],"sub_categories":["📚 Key Implementations","网络服务_其他"],"readme":"# DeepLab2: A TensorFlow Library for Deep Labeling\n\nDeepLab2 is a TensorFlow library for deep labeling, aiming to provide a\nunified and state-of-the-art TensorFlow codebase for dense pixel labeling tasks,\nincluding, but not limited to semantic segmentation, instance segmentation,\npanoptic segmentation, depth estimation, or even video panoptic segmentation.\n\nDeep labeling refers to solving computer vision problems by assigning a\npredicted value for each pixel in an image with a deep neural network. As\nlong as the problem of interest could be formulated in this way, DeepLab2\nshould serve the purpose. Additionally, this codebase includes our recent and\nstate-of-the-art research models on deep labeling. We hope you will find it\nuseful for your projects.\n\n## Change logs\n\n*   10/18/2022: Add kMaX-DeepLab ADE20K panoptic segmentation results in\n    [model zoo](g3doc/projects/kmax_deeplab.md).\n\n*   10/04/2022: Open-source MOAT model [code](model/pixel_encoder/moat.py) and\n    [ImageNet pretrained weights](g3doc/projects/moat_imagenet_pretrained_checkpoints.md).\n    We thank [Chenglin Yang](https://chenglin-yang.github.io/) for their\n    valuable contributions.\n\n*   08/26/2022: Add ViP-DeepLab support for [Waymo Open Dataset: Panoramic Video Panoptic Segmentation](https://arxiv.org/abs/2206.07704).\n    We thank [Jieru Mei](https://meijieru.com/),\n    [Alex Zhu](https://github.com/alexzzhu),\n    [Xinchen Yan](https://sites.google.com/site/skywalkeryxc/),\n    and [Hang Yan](https://scholar.google.com/citations?user=A4UXXLMAAAAJ\u0026hl=en),\n    for their valuable contributions.\n\n*   08/16/2022: Support Colab [demo](DeepLab_COCO_Demo.ipynb) for kMaX-DeepLab.\n\n*   07/12/2022: Open-source\n    [k-means Mask Transformer](https://arxiv.org/pdf/2207.04044.pdf)\n    (kMaX-DeepLab) code and [model zoo](g3doc/projects/kmax_deeplab.md).\n\n*   07/11/2022: Drop support of Tensorflow 2.5. Please update to 2.6.\n\n*   04/27/2022: Add ViP-DeepLab [demo](ViP_DeepLab_Demo.ipynb) and update\n    ViP-DeepLab [model zoo](g3doc/projects/vip_deeplab.md).\n\n*   09/07/2021: Add numpy implementation of Segmentation and Tracking Quality.\n    Find it [here](evaluation/numpy/segmentation_and_tracking_quality.py).\n\n*   09/06/2021: Update Panoptic-DeepLab w/ MobileNetv3 backbone results on\n    Cityscapes.\n\n*   08/13/2021: Open-source MaX-DeepLab-L COCO checkpoints (51.3% PQ on COCO val\n    set).\n\n*   07/26/2021: Add ViP-DeepLab support for SemKITTI-DVPS.\n\n*   07/07/2021: KITTI-STEP and MOTChallenge-STEP are ready to use.\n\n*   06/07/2021: Add hungarian matching support on TPU for MaX-DeepLab, thanks to\n    the help from [Jiquan Ngiam](https://cs.stanford.edu/~jngiam/)\n    and [Amil Merchant](https://scholar.google.com/citations?user=uRImMPoAAAAJ\u0026hl=en).\n\n*   06/01/2021: \"Hello, World!\", DeepLab2 made publicly available.\n\n## Installation\n\nSee [Installation](g3doc/setup/installation.md).\n\n## Dataset preparation\n\nThe dataset needs to be converted to TFRecord. We provide some examples below.\n\n* \u003ca href='g3doc/setup/ade20k.md'\u003eADE20K\u003c/a\u003e\u003cbr\u003e\n* \u003ca href='g3doc/setup/cityscapes.md'\u003eCityscapes\u003c/a\u003e\u003cbr\u003e\n* \u003ca href='g3doc/setup/coco.md'\u003eCOCO\u003c/a\u003e\u003cbr\u003e\n* \u003ca href='g3doc/setup/kitti_step.md'\u003eKITTI-STEP\u003c/a\u003e\u003cbr\u003e\n* \u003ca href='g3doc/setup/motchallenge_step.md'\u003eMOTChallenge-STEP\u003c/a\u003e\u003cbr\u003e\n\nSome guidances about how to convert your own dataset.\n\n* \u003ca href='g3doc/setup/your_own_dataset.md'\u003eYour Own Dataset\u003c/a\u003e\u003cbr\u003e\n\n## Projects\n\nWe list a few projects that use DeepLab2.\n\n* \u003ca href='g3doc/projects/panoptic_deeplab.md'\u003ePanoptic-DeepLab\u003c/a\u003e\u003cbr\u003e\n* \u003ca href='g3doc/projects/axial_deeplab.md'\u003eAxial-DeepLab\u003c/a\u003e\u003cbr\u003e\n* \u003ca href='g3doc/projects/max_deeplab.md'\u003eMaX-DeepLab\u003c/a\u003e\u003cbr\u003e\n* \u003ca href='g3doc/projects/motion_deeplab.md'\u003eSTEP (Motion-DeepLab)\u003c/a\u003e\u003cbr\u003e\n* \u003ca href='g3doc/projects/vip_deeplab.md'\u003eViP-DeepLab\u003c/a\u003e\u003cbr\u003e\n* \u003ca href='g3doc/projects/kmax_deeplab.md'\u003ekMaX-DeepLab\u003c/a\u003e\u003cbr\u003e\n\n## Colab Demo\n\n*   \u003ca href='https://colab.research.google.com/github/google-research/deeplab2/blob/main/DeepLab_COCO_Demo.ipynb'\u003ekMaX-DeepLab Colab notebook for off-the-shelf inference with COCO checkpoints.\u003c/a\u003e\u003cbr\u003e\n\n*   \u003ca href='https://colab.research.google.com/github/google-research/deeplab2/blob/main/DeepLab_Cityscsapes_Demo.ipynb'\u003ePanoptic-DeepLab Colab notebook for off-the-shelf inference with Cityscapes checkpoints.\u003c/a\u003e\u003cbr\u003e\n\n*   \u003ca href='https://colab.research.google.com/github/google-research/deeplab2/blob/main/ViP_DeepLab_Demo.ipynb'\u003eViP-DeepLab Colab notebook for off-the-shelf inference with Cityscapes-DVPS checkpoints.\u003c/a\u003e\u003cbr\u003e\n\nNote that the exported models used in all the demos are in **CPU** mode.\n\n## Running DeepLab2\n\nSee [Getting Started](g3doc/setup/getting_started.md). In short, run the\nfollowing command:\n\nTo run DeepLab2 on GPUs, the following command should be used:\n\n```bash\npython trainer/train.py \\\n    --config_file=${CONFIG_FILE} \\\n    --mode={train | eval | train_and_eval | continuous_eval} \\\n    --model_dir=${BASE_MODEL_DIRECTORY} \\\n    --num_gpus=${NUM_GPUS}\n```\n\n## Contacts (Maintainers)\n\nPlease check \u003ca href='g3doc/faq.md'\u003eFAQ\u003c/a\u003e if you have some questions before\nreporting the issues.\u003cbr\u003e\n\n* Mark Weber, github: [markweberdev](https://github.com/markweberdev)\n* Huiyu Wang, github: [csrhddlam](https://github.com/csrhddlam)\n* Siyuan Qiao, github: [joe-siyuan-qiao](https://github.com/joe-siyuan-qiao)\n* Jun Xie, github: [clairexie](https://github.com/clairexie)\n* Maxwell D. Collins, github: [mcollinswisc](https://github.com/mcollinswisc)\n* YuKun Zhu, github: [yknzhu](https://github.com/YknZhu)\n* Liangzhe Yuan, github: [yuanliangzhe](https://github.com/yuanliangzhe)\n* Dahun Kim, github: [mcahny](https://github.com/mcahny)\n* Qihang Yu, github: [yucornetto](https://github.com/yucornetto)\n* Liang-Chieh Chen, github: [aquariusjay](https://github.com/aquariusjay)\n\n## Disclaimer\n\n* Note that this library contains our **re-implemented** DeepLab models in\n*TensorFlow2*, and thus may have some minor differences from the published\npapers (e.g., learning rate).\n\n* This is not an official Google product.\n\n## Citing DeepLab2\n\nIf you find DeepLab2 useful for your project, please consider citing\n`DeepLab2` along with the relevant DeepLab series.\n\n* DeepLab2:\n\n```\n@article{deeplab2_2021,\n  author={Mark Weber and Huiyu Wang and Siyuan Qiao and Jun Xie and Maxwell D. Collins and Yukun Zhu and Liangzhe Yuan and Dahun Kim and Qihang Yu and Daniel Cremers and Laura Leal-Taixe and Alan L. Yuille and Florian Schroff and Hartwig Adam and Liang-Chieh Chen},\n  title={{DeepLab2: A TensorFlow Library for Deep Labeling}},\n  journal={arXiv: 2106.09748},\n  year={2021}\n}\n\n```\n\n### References\n\n1. Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus\n   Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele.\n   \"The cityscapes dataset for semantic urban scene understanding.\" In CVPR,\n   2016.\n\n2. Andreas Geiger, Philip Lenz, and Raquel Urtasun. \"Are we ready for\n   autonomous driving? the kitti vision benchmark suite.\" In CVPR, 2012.\n\n3. Jens Behley, Martin Garbade, Andres Milioto, Jan Quenzel, Sven Behnke,\n   Cyrill Stachniss, and Jurgen Gall. \"Semantickitti: A dataset for semantic\n   scene understanding of lidar sequences.\" In ICCV, 2019.\n\n4. Alexander Kirillov, Kaiming He, Ross Girshick, Carsten Rother, and Piotr\n   Dollar. \"Panoptic segmentation.\" In CVPR, 2019.\n\n5. Dahun Kim, Sanghyun Woo, Joon-Young Lee, and In So Kweon. \"Video panoptic\n   segmentation.\" In CVPR, 2020.\n\n6. Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona,\n   Deva Ramanan, Piotr Dollar, and C Lawrence Zitnick. \"Microsoft COCO:\n   Common objects in context.\" In ECCV, 2014.\n\n7. Patrick Dendorfer, Aljosa Osep, Anton Milan, Konrad Schindler, Daniel\n   Cremers, Ian Reid, Stefan Roth, and Laura Leal-Taixe. \"MOTChallenge: A\n   Benchmark for Single-camera Multiple Target Tracking.\" IJCV, 2020.\n\n8. Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso, and\n   Antonio Torralba. \"Scene Parsing through ADE20K Dataset.\" In CVPR, 2017.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoogle-research%2Fdeeplab2","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgoogle-research%2Fdeeplab2","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoogle-research%2Fdeeplab2/lists"}