{"id":13639466,"url":"https://github.com/OpenGVLab/M3I-Pretraining","last_synced_at":"2025-04-19T22:32:49.819Z","repository":{"id":101643931,"uuid":"568640805","full_name":"OpenGVLab/M3I-Pretraining","owner":"OpenGVLab","description":"[CVPR 2023] implementation of  Towards All-in-one Pre-training via Maximizing Multi-modal Mutual Information.","archived":false,"fork":false,"pushed_at":"2023-06-01T13:51:52.000Z","size":614,"stargazers_count":91,"open_issues_count":1,"forks_count":5,"subscribers_count":12,"default_branch":"main","last_synced_at":"2024-08-03T01:14:53.221Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2211.09807","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/OpenGVLab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2022-11-21T04:53:14.000Z","updated_at":"2024-07-29T06:02:57.000Z","dependencies_parsed_at":"2024-01-14T09:15:12.537Z","dependency_job_id":"63ee355d-c3e5-4bb3-a19b-bf4dae1c1712","html_url":"https://github.com/OpenGVLab/M3I-Pretraining","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenGVLab%2FM3I-Pretraining","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenGVLab%2FM3I-Pretraining/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenGVLab%2FM3I-Pretraining/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenGVLab%2FM3I-Pretraining/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/OpenGVLab","download_url":"https://codeload.github.com/OpenGVLab/M3I-Pretraining/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223810437,"owners_count":17206767,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-02T01:01:00.930Z","updated_at":"2024-11-09T09:31:04.201Z","avatar_url":"https://github.com/OpenGVLab.png","language":null,"funding_links":[],"categories":["Summary"],"sub_categories":[],"readme":"# M3I Pre-training\n\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/towards-all-in-one-pre-training-via/object-detection-on-coco)](https://paperswithcode.com/sota/object-detection-on-coco?p=towards-all-in-one-pre-training-via)\n\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/towards-all-in-one-pre-training-via/object-detection-on-coco-minival)](https://paperswithcode.com/sota/object-detection-on-coco-minival?p=towards-all-in-one-pre-training-via)\n\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/towards-all-in-one-pre-training-via/object-detection-on-lvis-v1-0-minival)](https://paperswithcode.com/sota/object-detection-on-lvis-v1-0-minival?p=towards-all-in-one-pre-training-via)\n\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/towards-all-in-one-pre-training-via/semantic-segmentation-on-ade20k)](https://paperswithcode.com/sota/semantic-segmentation-on-ade20k?p=towards-all-in-one-pre-training-via) [![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/towards-all-in-one-pre-training-via/image-classification-on-imagenet)](https://paperswithcode.com/sota/image-classification-on-imagenet?p=towards-all-in-one-pre-training-via)\n\nThis repository is an official implementation of CVPR 2023 paper [Towards All-in-one Pre-training via Maximizing Multi-modal Mutual Information](https://arxiv.org/abs/2211.09807).\n\nBy [Weijie Su](https://scholar.google.com/citations?user=ECDe6IIAAAAJ\u0026hl=en), [Xizhou Zhu](https://scholar.google.com/citations?user=02RXI00AAAAJ\u0026hl=en), [Chenxin Tao](https://scholar.google.com/citations?user=sXHFIBkAAAAJ\u0026hl=en), [Lewei Lu](https://scholar.google.com/citations?user=zdgKJXIAAAAJ\u0026hl=en), [Bin Li](http://staff.ustc.edu.cn/~binli/), [Gao Huang](http://www.gaohuang.net/), [Yu Qiao](https://scholar.google.com/citations?user=gFtI-8QAAAAJ\u0026hl=en), [Xiaogang Wang](https://scholar.google.com/citations?user=-B5JgjsAAAAJ\u0026hl=en), [Jie Zhou](https://scholar.google.com/citations?user=6a79aPwAAAAJ\u0026hl=en), [Jifeng Dai](https://jifengdai.org/).\n\nCode will be available.\n\n## Introduction\n\n**M**aximizing **M**ulti-modal **M**utual **I**nformation Pre-training (**M3I Pre-training**), initially described in [arxiv](https://arxiv.org/abs/2211.09807), is a simple yet effective one-stage pre-training paradigm. It can integrate existing pre-training methods (supervised pre-training, weakly-supervised pre-training and self-supervised pre-training) under an unified mutual information perspective and maintain all desired properties through a single-stage pre-training. Notably, we successfully pre-train a 1B model ([InternImage-H](https://arxiv.org/abs/2211.05778)) with M3I Pre-training and achieve new record `65.4 mAP` on COCO detection test-dev, `62.5 mAP` on LVIS detection minival, and `62.9 mIoU` on ADE20k.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./figs/fig1-comparison.png\" alt=\"m3i pre-training\" width=\"600\"/\u003e\n\u003c/p\u003e\n\n\u003c!-- ## Main Results\n\n**Results of InternImage-H**\n\n| Method          | Model         | #param | ImageNet | COCO | LVIS | ADE20k |\n|:---------------:|:-------------:|:------:|:--------:|:----:|:----:|:------:|\n| M3I Pre-training| InternImage-H | 1B     |          |      |      |        |\n\n**Results of ViT-B/16** --\u003e\n\n\n## Citation\n\nIf this work is helpful for your research, please consider citing the following BibTeX entry.\n\n```\n@InProceedings{Su_2023_CVPR,\n    author    = {Su, Weijie and Zhu, Xizhou and Tao, Chenxin and Lu, Lewei and Li, Bin and Huang, Gao and Qiao, Yu and Wang, Xiaogang and Zhou, Jie and Dai, Jifeng},\n    title     = {Towards All-in-One Pre-Training via Maximizing Multi-Modal Mutual Information},\n    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},\n    month     = {June},\n    year      = {2023},\n    pages     = {15888-15899}\n}\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FOpenGVLab%2FM3I-Pretraining","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FOpenGVLab%2FM3I-Pretraining","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FOpenGVLab%2FM3I-Pretraining/lists"}