{"id":18686273,"url":"https://github.com/mrgiovanni/modelsgenesis","last_synced_at":"2025-05-15T03:06:55.676Z","repository":{"id":43393640,"uuid":"198694912","full_name":"MrGiovanni/ModelsGenesis","owner":"MrGiovanni","description":"[MICCAI 2019 Young Scientist Award] [MEDIA 2020 Best Paper Award] Models Genesis","archived":false,"fork":false,"pushed_at":"2025-01-11T07:29:40.000Z","size":23069,"stargazers_count":760,"open_issues_count":28,"forks_count":144,"subscribers_count":14,"default_branch":"master","last_synced_at":"2025-04-15T00:48:49.726Z","etag":null,"topics":["3d-model","fine-tuning","foundation-models","pre-trained-model","representation-learning","self-supervised-learning","transfer-learning"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MrGiovanni.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-07-24T19:07:19.000Z","updated_at":"2025-04-13T03:08:35.000Z","dependencies_parsed_at":"2025-01-20T05:21:49.570Z","dependency_job_id":"f9d7c721-5725-4e32-aa67-81e0a997055c","html_url":"https://github.com/MrGiovanni/ModelsGenesis","commit_stats":{"total_commits":263,"total_committers":13,"mean_commits":20.23076923076923,"dds":"0.23574144486692017","last_synced_commit":"f7b8adcff59c918bf71216bd83f5bebd98cc6fa6"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MrGiovanni%2FModelsGenesis","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MrGiovanni%2FModelsGenesis/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MrGiovanni%2FModelsGenesis/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MrGiovanni%2FModelsGenesis/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MrGiovanni","download_url":"https://codeload.github.com/MrGiovanni/ModelsGenesis/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254264766,"owners_count":22041793,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d-model","fine-tuning","foundation-models","pre-trained-model","representation-learning","self-supervised-learning","transfer-learning"],"created_at":"2024-11-07T10:26:52.603Z","updated_at":"2025-05-15T03:06:50.636Z","avatar_url":"https://github.com/MrGiovanni.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\u003cimg width=\"70%\" src=\"figures/logo.png\" /\u003e\u003c/p\u003e\n\n\u003cdiv align=\"center\"\u003e\n\n![visitors](https://visitor-badge.laobi.icu/badge?page_id=MrGiovanni/ModelsGenesis)\n[![GitHub Repo stars](https://img.shields.io/github/stars/MrGiovanni/ModelsGenesis?style=social)](https://github.com/MrGiovanni/ModelsGenesis/stargazers)\n\n\u003c/div\u003e\n\nWe have built a set of pre-trained models called \u003cb\u003eGeneric Autodidactic Models\u003c/b\u003e, nicknamed \u003cb\u003eModels Genesis\u003c/b\u003e, because they are created \u003ci\u003eex nihilo\u003c/i\u003e (with no manual labeling), self-taught (learned by self-supervision), and generic (served as source models for generating application-specific target models). We envision that Models Genesis may serve as a primary source of transfer learning for 3D medical imaging applications, in particular, with limited annotated data. \n\n\u003cp align=\"center\"\u003e\u003cimg width=\"100%\" src=\"figures/patch_generator.png\" /\u003e\u003c/p\u003e\n\u003cp align=\"center\"\u003e\u003cimg width=\"85%\" src=\"figures/framework.png\" /\u003e\u003c/p\u003e\n\n\n## Paper\nThis repository provides the official implementation of training Models Genesis as well as the usage of the pre-trained Models Genesis in the following paper:\n\n\u003cb\u003eModels Genesis: Generic Autodidactic Models for 3D Medical Image Analysis\u003c/b\u003e \u003cbr/\u003e\n[Zongwei Zhou](https://www.zongweiz.com/)\u003csup\u003e1\u003c/sup\u003e, [Vatsal Sodha](https://github.com/vatsal-sodha)\u003csup\u003e1\u003c/sup\u003e, [Md Mahfuzur Rahman Siddiquee](https://github.com/mahfuzmohammad)\u003csup\u003e1\u003c/sup\u003e,  \u003cbr/\u003e\n[Ruibin Feng](https://chs.asu.edu/ruibin-feng)\u003csup\u003e1\u003c/sup\u003e, [Nima Tajbakhsh](https://www.linkedin.com/in/nima-tajbakhsh-b5454376/)\u003csup\u003e1\u003c/sup\u003e, [Michael B. Gotway](https://www.mayoclinic.org/biographies/gotway-michael-b-m-d/bio-20055566)\u003csup\u003e2\u003c/sup\u003e, and [Jianming Liang](https://chs.asu.edu/jianming-liang)\u003csup\u003e1\u003c/sup\u003e \u003cbr/\u003e\n\u003csup\u003e1 \u003c/sup\u003eArizona State University,   \u003csup\u003e2 \u003c/sup\u003eMayo Clinic \u003cbr/\u003e\nInternational Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI), 2019 \u003cbr/\u003e\n\u003cb\u003e[Young Scientist Award](http://www.miccai.org/about-miccai/awards/young-scientist-award/)\u003c/b\u003e  \u003cbr/\u003e\n[paper](http://www.cs.toronto.edu/~liang/Publications/ModelsGenesis/MICCAI_2019_Full.pdf) | [code](https://github.com/MrGiovanni/ModelsGenesis) | [slides](https://docs.wixstatic.com/ugd/deaea1_c5e0f8cd9cde4c3db339d866483cbcd3.pdf) | [poster](http://www.cs.toronto.edu/~liang/Publications/ModelsGenesis/Models_Genesis_Poster.pdf) | talk ([YouTube](https://youtu.be/5W_uGzBloZs), [YouKu](https://v.youku.com/v_show/id_XNDM5NjQ1ODAxMg==.html?sharefrom=iphone\u0026sharekey=496e1494c76ed263653aa3aada61c23e6)) | [blog](https://zhuanlan.zhihu.com/p/86366534)\n\n\u003cb\u003eModels Genesis\u003c/b\u003e \u003cbr/\u003e\n[Zongwei Zhou](https://www.zongweiz.com/)\u003csup\u003e1\u003c/sup\u003e, [Vatsal Sodha](https://github.com/vatsal-sodha)\u003csup\u003e1\u003c/sup\u003e, [Jiaxuan Pang](https://github.com/MRJasonP)\u003csup\u003e1\u003c/sup\u003e, [Michael B. Gotway](https://www.mayoclinic.org/biographies/gotway-michael-b-m-d/bio-20055566)\u003csup\u003e2\u003c/sup\u003e, and [Jianming Liang](https://chs.asu.edu/jianming-liang)\u003csup\u003e1\u003c/sup\u003e \u003cbr/\u003e\n\u003csup\u003e1 \u003c/sup\u003eArizona State University,   \u003csup\u003e2 \u003c/sup\u003eMayo Clinic \u003cbr/\u003e\nMedical Image Analysis (MedIA) \u003cbr/\u003e\n\u003cb\u003e[MedIA Best Paper Award](http://www.miccai.org/about-miccai/awards/medical-image-analysis-best-paper-award/)\u003c/b\u003e  \u003cbr/\u003e\n[paper](https://arxiv.org/pdf/2004.07882.pdf) | [code](https://github.com/MrGiovanni/ModelsGenesis) | [slides](https://d5b3ebbb-7f8d-4011-9114-d87f4a930447.filesusr.com/ugd/deaea1_5ecdfa48836941d6ad174dcfbc925575.pdf) | [graphical abstract](https://ars.els-cdn.com/content/image/1-s2.0-S1361841520302048-fx1_lrg.jpg)\n\n\u003cp float=\"center\"\u003e\n  \u003cimg width=\"30%\" src=\"figures/Young_Scientist_Award.JPG\" /\u003e\n  \u003cimg width=\"60%\" src=\"figures/MedIA_Best_Paper_Award.JPG\" /\u003e \n\u003c/p\u003e\n\n## Available implementation\n\n- keras/\n- pytorch/\n\n**\u0026#9733; News: Models Genesis, incorporated with nnU-Net, [rank # 1](https://decathlon-10.grand-challenge.org/evaluation/challenge/leaderboard/) in segmenting liver/tumor and hippocampus.**\n- competition/\n\n\n## Major results from our work\n\n1. **Models Genesis outperform 3D models trained from scratch**\n2. **Models Genesis top any 2D approaches, including ImageNet models and degraded 2D Models Genesis**\n3. **Models Genesis (2D) offer performances equivalent to supervised pre-trained models**\n\nThe par plots presented below are produced by Matlab code in `figures/plotsuperbar.m` and the helper functions in `figures/superbar`. Credit to [superbar](https://github.com/scottclowe/superbar) by Scott Lowe.\n\n\u003cp align=\"center\"\u003e\u003cimg width=\"90%\" src=\"figures/results.png\" /\u003e\u003c/p\u003e\n\nNote that learning from scratch \u003ci\u003esimply\u003c/i\u003e in 3D may not necessarily yield performance better than ImageNet-based transfer learning in 2D\n\n\n## Citation\nIf you use this code or use our pre-trained weights for your research, please cite our papers:\n```\n@InProceedings{zhou2019models,\n  author=\"Zhou, Zongwei and Sodha, Vatsal and Rahman Siddiquee, Md Mahfuzur and Feng, Ruibin and Tajbakhsh, Nima and Gotway, Michael B. and Liang, Jianming\",\n  title=\"Models Genesis: Generic Autodidactic Models for 3D Medical Image Analysis\",\n  booktitle=\"Medical Image Computing and Computer Assisted Intervention -- MICCAI 2019\",\n  year=\"2019\",\n  publisher=\"Springer International Publishing\",\n  address=\"Cham\",\n  pages=\"384--393\",\n  isbn=\"978-3-030-32251-9\",\n  url=\"https://link.springer.com/chapter/10.1007/978-3-030-32251-9_42\"\n}\n\n@article{zhou2021models,\n  title=\"Models Genesis\",\n  author=\"Zhou, Zongwei and Sodha, Vatsal and Pang, Jiaxuan and Gotway, Michael B and Liang, Jianming\",\n  journal=\"Medical Image Analysis\",\n  volume = \"67\",\n  pages = \"101840\",\n  year = \"2021\",\n  issn = \"1361-8415\",\n  doi = \"https://doi.org/10.1016/j.media.2020.101840\",\n  url = \"http://www.sciencedirect.com/science/article/pii/S1361841520302048\",\n}\n\n@phdthesis{zhou2021towards,\n  title={Towards Annotation-Efficient Deep Learning for Computer-Aided Diagnosis},\n  author={Zhou, Zongwei},\n  year={2021},\n  school={Arizona State University}\n}\n```\n\n\n## Acknowledgement\nThis research has been supported partially by ASU and Mayo Clinic through a Seed Grant and an Innovation Grant, and partially by the National Institutes of Health (NIH) under Award Number R01HL128785. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH. This work has utilized the GPUs provided partially by the ASU Research Computing and partially by the Extreme Science and Engineering Discovery Environment (XSEDE) funded by the National Science Foundation (NSF) under grant number ACI-1548562. This is a patent-pending technology.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmrgiovanni%2Fmodelsgenesis","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmrgiovanni%2Fmodelsgenesis","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmrgiovanni%2Fmodelsgenesis/lists"}