{"id":21816610,"url":"https://github.com/medmnist/experiments","last_synced_at":"2025-07-19T21:33:11.379Z","repository":{"id":170062548,"uuid":"385617266","full_name":"MedMNIST/experiments","owner":"MedMNIST","description":"Codebase for reproducible benchmarking experiments in MedMNIST v2","archived":false,"fork":false,"pushed_at":"2024-07-11T13:22:53.000Z","size":31,"stargazers_count":52,"open_issues_count":1,"forks_count":21,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-07T09:42:55.378Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MedMNIST.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-07-13T13:40:13.000Z","updated_at":"2025-04-02T05:23:52.000Z","dependencies_parsed_at":"2024-07-11T15:15:37.291Z","dependency_job_id":null,"html_url":"https://github.com/MedMNIST/experiments","commit_stats":null,"previous_names":["medmnist/experiments"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/MedMNIST/experiments","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MedMNIST%2Fexperiments","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MedMNIST%2Fexperiments/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MedMNIST%2Fexperiments/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MedMNIST%2Fexperiments/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MedMNIST","download_url":"https://codeload.github.com/MedMNIST/experiments/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MedMNIST%2Fexperiments/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266019657,"owners_count":23864916,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-27T15:35:31.791Z","updated_at":"2025-07-19T21:33:11.352Z","avatar_url":"https://github.com/MedMNIST.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# MedMNIST Experiments \n\nTraining and evaluation scripts to reproduce both 2D and 3D experiments in our [MedMNIST](https://github.com/MedMNIST/MedMNIST/) paper, including PyTorch, auto-sklearn, AutoKeras and Google AutoML Vision together with their weights ;)\n\n\n# Code Structure\n* [`MedMNIST2D/`](./MedMNIST2D/): training and evaluation scripts of MedMNIST2D\n  * [`models.py`](./MedMNIST2D/models.py): *ResNet-18* and *ResNet-50* models (for small-image datasets like CIFAR-10/100)\n  * [`train_and_eval_pytorch.py`](./MedMNIST2D/train_and_eval_pytorch.py): training and evaluation script implemented with PyTorch\n  * [`train_and_eval_autokeras.py`](./MedMNIST2D/train_and_eval_autokeras.py):  training and evaluation script of Autokeras\n  * [`train_and_eval_autosklearn.py`](./MedMNIST2D/train_and_eval_autosklearn.py): training and evaluation script of auto-sklearn\n  * [`eval_google_automl_vision.py`](./MedMNIST2D/eval_google_automl_vision.py): evaluation script of models trained by Google AutoML Vision\n\n* [`MedMNIST3D/`](./MedMNIST3D/): training and evaluation scripts of MedMNIST3D\n  * [`models.py`](./MedMNIST3D/models.py): *ResNet-18* and *ResNet-50* models (for small-image datasets like CIFAR-10/100), basically same as [`MedMNIST2D/models.py`](./MedMNIST2D/models.py)\n  * [`train_and_eval_pytorch.py`](./MedMNIST3D/train_and_eval_pytorch.py): training and evaluation script implemented with PyTorch\n  * [`train_and_eval_autokeras.py`](./MedMNIST3D/train_and_eval_autokeras.py):  training and evaluation script of Autokeras\n  * [`train_and_eval_autosklearn.py`](./MedMNIST3D/train_and_eval_autosklearn.py): training and evaluation script of auto-sklearn\n\n    \n# Installation and Requirements\nThis repository is working with [MedMNIST official code](https://github.com/MedMNIST/MedMNIST/) and PyTorch.\n\n1. Setup the required environments and install `medmnist` as a standard Python package:\n\n        pip install medmnist\n\n2. Check whether you have installed the latest [version](https://github.com/MedMNIST/MedMNIST/tree/main/medmnist/info.py):\n\n        \u003e\u003e\u003e import medmnist\n        \u003e\u003e\u003e print(medmnist.__version__)\n\n3. The code requires common Python environments for machine learning; Basically, it was tested with\n\n- Python 3 (Anaconda 3.6.3 specifically)\n- PyTorch==1.3.1\n- autokeras\\==1.0.15\n- auto-sklearn\\==0.10.0\n- tensorflow==2.3.0\n\n  Higher (or lower) versions should also work (perhaps with minor modifications).\n\n4. For *MedMNIST3D*, our code additionally requires [ACSConv](https://github.com/M3DV/ACSConv). Install it through `pip` via the command bellow:\n\n        pip install git+git://github.com/M3DV/ACSConv.git\n    \n    Then you can check use the cool `2.5D`, `3D` and `ACS` model convertor as follows:\n\n    ```python\n    from torchvision.models import resnet18\n    from acsconv.converters import ACSConverter\n    # model_2d is a standard pytorch 2D model\n    model_2d = resnet18(pretrained=True)\n    B, C_in, H, W = (1, 3, 64, 64)\n    input_2d = torch.rand(B, C_in, H, W)\n    output_2d = model_2d(input_2d)\n\n    model_3d = ACSConverter(model_2d)\n    # once converted, model_3d is using ACSConv and capable of processing 3D volumes.\n    B, C_in, D, H, W = (1, 3, 64, 64, 64)\n    input_3d = torch.rand(B, C_in, D, H, W)\n    output_3d = model_3d(input_3d)\n    ```\n\n5. Download the model weights and predictions from [Zenodo](https://doi.org/10.5281/zenodo.7782113).\n    -  `weights_*.zip`: \n        - PyTorch, AutoKeras and Google AutoML Vision are provided for MedMNIST2D.\n        - PyTorch and AutoKeras are provided for MedMNIST3D. \n        - If you are using PyTorch model weights, please note that the ResNet18_224 / ResNet50_224 models are trained with images resized to 224 x 224 by `PIL.Image.NEAREST`. \n        - Snapshots for `auto-sklearn` are not uploaded due to the embarrassingly large model sizes (lots of model ensemble).\n    -  `predictions.zip`: We also provide all standard prediction files by PyTorch, auto-sklearn, AutoKeras and Google AutoML Vision, which works with `medmnist.Evaluator`. Each file is named as `{flag}_{split}_[AUC]{auc:.3f}_[ACC]{acc:.3f}@{run}.csv`, e.g., `bloodmnist_test_[AUC]0.997_[ACC]0.957@autokeras_3.csv`.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmedmnist%2Fexperiments","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmedmnist%2Fexperiments","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmedmnist%2Fexperiments/lists"}