{"id":27379516,"url":"https://github.com/jiagengchang/myevae","last_synced_at":"2026-05-16T08:02:40.094Z","repository":{"id":258277209,"uuid":"870187968","full_name":"JiaGengChang/MyeVAE","owner":"JiaGengChang","description":"MyeVAE. A multi-modal variation autoencoder with sub-task supervised latent subspace. Used for predicting mortality risk in newly diagnosed myeloma patients.","archived":false,"fork":false,"pushed_at":"2025-04-07T06:37:59.000Z","size":13393,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-07T07:31:02.589Z","etag":null,"topics":["multi-omics-integration","pytorch","scikit-learn-pipelines","survival-regression","vae"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/JiaGengChang.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-10-09T15:40:26.000Z","updated_at":"2025-04-07T06:38:02.000Z","dependencies_parsed_at":"2025-04-07T07:41:08.399Z","dependency_job_id":null,"html_url":"https://github.com/JiaGengChang/MyeVAE","commit_stats":null,"previous_names":["jiagengchang/cancer-survival-ml","jiagengchang/myevae"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JiaGengChang%2FMyeVAE","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JiaGengChang%2FMyeVAE/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JiaGengChang%2FMyeVAE/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JiaGengChang%2FMyeVAE/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/JiaGengChang","download_url":"https://codeload.github.com/JiaGengChang/MyeVAE/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248724592,"owners_count":21151559,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["multi-omics-integration","pytorch","scikit-learn-pipelines","survival-regression","vae"],"created_at":"2025-04-13T13:50:30.393Z","updated_at":"2025-10-05T22:50:58.560Z","avatar_url":"https://github.com/JiaGengChang.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# MyeVAE\n\n[![PyPI version](https://badge.fury.io/py/myevae.svg)](https://badge.fury.io/py/myevae)\n[![GitHub latest commit](https://badgen.net/github/last-commit/JiaGengChang/myevae)](https://GitHub.com/JiaGengChang/myevae/commit/)\n[![GitHub license](https://img.shields.io/github/license/JiaGengChang/myevae.svg)](https://github.com/JiaGengChang/myevae/blob/master/LICENSE)\n[![made-with-python](https://img.shields.io/badge/Made%20with-Python-1f425f.svg)](https://www.python.org/)\n\nMyeVAE is a variational autoencoder leveraging multi-omics for risk prediction in newly diagnosed multiple myeloma patients.\n\nThis repository contains the Python source code to preprocess multimoodal features, train MyeVAE, score its performance, and perform SHAP analysis. \n\nAll computation is done on the CPU, using the PyTorch library.\n\n\u003cp align=\"left\"\u003e\u003cimg src=\"https://raw.githubusercontent.com/JiaGengChang/myevae/refs/heads/main/assets/myeVAE.png\" alt=\"Illustration of MyeVAE architecture, hosted on github\" width=\"600\"\u003e\u003c/p\u003e\n\n# Setup\n\n## Install MyeVAE\nRequires **Python 3.9** or later\n\nInstall through PyPI \n```bash\npip install myevae\n```\n\nInstall through github\n```bash\npip install git+https://github.com/JiaGengChang/myevae.git\n```\n\n## Download data\n\nDownload the raw multi-omics dataset and example outputs from the following link:\n\n1. Input datasets are stored at https://myevae.s3.us-east-1.amazonaws.com/example_inputs.tar.gz\n\n2. Example output files are stored at https://myevae.s3.us-east-1.amazonaws.com/example_outputs.tar.gz\n\nPlease use the download links sparingly.\n\nIf you use the AWS S3 CLI, the bucket ARN is `arn:aws:s3:::myevae` - you can list its contents and download specific files.\n\n# Step-by-step guide\n## 1. (Optional) Feature preprocessing\n\nAs there are over 50,000 raw features and contains missing values, this step performs supervised-feature selection, scaling, and imputation.\n\nFor feature selection, elastic net regularized Cox proportional hazards is used.\n\nFor imputation, round-robin imputation with scikit-learn IterativeImputer is used, with random forest regressor and classifier as based estimators.\n\n```bash\nmyevae preprocess \\\n    -e [endpoint: os | pfs] \\\n    -i [/inputs/folder]\n```\n\nThe following output files will be created in the same directory as the inputs.\n```bash\nproject\n└──inputs\n    ├──...\n    ├──train_features_os_processed.csv\n    ├──train_features_pfs_processed.csv\n    ├──valid_features_os_processed.csv\n    └──valid_features_pfs_processed.csv\n```\n\nPreprocessing of features as computationally expensive than hyperparameter tuning. On 10 cores and 64GB RAM, it uses a wall time of **~12 hours**.\n\nAlternatively, if you are just testing this package, you may use pre-processed features from the S3 bucket link and proceed to the next step.\n\n## 2. Fit model\n\nSee [requirements](#requirements) for the required files before training.\n\nBasically, you need to place the processed features and labels in `/inputs/folder`, and create a hyperparameter file named `param_grid.py` in `/params/folder`. See [this section](#3-hyperparameter-file) for the default hyperparamter grid.\n\n```bash\nmyevae train \\\n    -e [endpoint: os | pfs] \\\n    -n [model_name] \\\n    -i [/inputs/folder] \\\n    -p [/params/folder] \\\n    -t [nthreads] \\\n    -o [/output/folder]\n```\n\nIf the hyperparameter file is not specified, the default will be used (`/src/myevae/param_grid.py`), the process should take only **5 minutes**. \n\nWith an actual hyperparameter grid, the training can take up to **12 hours**.\n\n## 3. Score model\n\n```bash\nmyevae score \\\n    -e [endpoint: os | pfs] \\\n    -n [model_name] \\\n    -i [/inputs/folder] \\\n    -o [/output/folder]\n```\n\nModel scoring should be done in under **a minute**.\n\n## 4. SHAP analysis\n```bash\nmyevae shap \\\n    -e [endpoint: os | pfs] \\\n    -n [model_name] \\\n    -i [/inputs/folder] \\\n    -o [/output/folder]\n```\n\nApproximation of SHAP values and generation of summary plots should take **1-2 minutes**.\n\n## End result\nFor a model named \"default\", the final output should look something like this. \n\n\u003cp align=\"left\"\u003e\u003cimg src=\"https://raw.githubusercontent.com/JiaGengChang/myevae/refs/heads/main/assets/directory.png\" alt=\"Illustration of folder structure, hosted on github\" width=\"300\"\u003e\u003c/p\u003e\n\nValidation scores and best parameters can be found in the `.json` files. The `.pth` files are model weights.\n\n# Requirements\n\n## 1. Raw features\nRequired for [preprocessing](#1-optional-feature-preprocessing) only.\n\nPlace the following `.csv` files in the inputs folder:\n```bash\nproject\n└──inputs\n    ├──train_features_os_processed.csv\n    ├──valid_features_os_processed.csv\n    └──train_labels.csv\n```\n\nNote the preprocessing does not use valid labels (`valid_labels.csv`), but it is fine if the file is in the directory.\n\n## 2. Processed features\nRequired for [training](#2-fit-model), [scoring](#3-score-model) and [shap analysis](#4-shap-analysis)\n\nPlace the following `.csv` files inside your inputs folder (`/inputs/folder`), and the `param_grid.py` inside your preferred folder. Also create an empty folder for the outputs.\n```bash\nproject\n├──inputs\n│     ├──train_features_os_processed.csv\n│     ├──valid_features_os_processed.csv\n│     ├──train_labels.csv\n│     └──valid_labels.csv\n├──params\n│     └──param_grid.py\n└──outputs\n      └──[output files created here]\n```\n\nFor `features*.csv` and `labels*.csv`, column 0 is read in as the index, which should be the patient IDs.\n\nExample input csv files can be downloaded from AWS S3 link above.\n\n## 3. Hyperparameter file\nRequired for [training](#2-fit-model) only.\n\nPlace a python file named `param_grid.py` containing the hyperparameter grid dictionary in the params folder (specified with `-p`). \n\nThis contains the set of hyperparameters that grid search will be performed on. \n\nOtherwise, use the default provided in `src/myevae/param_grid.py`. This default hyperparameter grid is only meant for testing purposes:\n\n```python\nfrom torch.nn import LeakyReLU, Tanh\n\n# the default hyperparameter grid for debugging uses\n# this is not meant to be used for real training, as the search space is only on z_dim\nparam_grid = {\n    'z_dim': [8,16,32],\n    'lr': [5e-4], \n    'batch_size': [1024],\n    'input_types': [['exp','cna','gistic','fish','sbs','ig']],\n    'input_types_subtask': [['clin']],\n    'layer_dims': [[[32, 4],[16,4],[4,1],[4,1],[4,1],[4,1]]],\n    'layer_dims_subtask' : [[4,1]],\n    'kl_weight': [1],\n    'activation': [LeakyReLU()],\n    'subtask_activation': [Tanh()],\n    'epochs': [100],\n    'burn_in': [20],\n    'patience': [5],\n    'dropout': [0.3],\n    'dropout_subtask': [0.3]\n}\n```\n\nAn actual paramater file is available at https://myevae.s3.us-east-1.amazonaws.com/param_grid.py\n\nUsing the actual hyperparameter file, the training time will be significantly longer (~12 hours)\n\n## Software\nThese dependencies will be automatically installed.\n```\npython \u003e= 3.9\ntorch \u003e= 1.9.0\nscikit-learn \u003e= 0.24.1\nscikit-survival \u003e= 0.23.1\nimportlib_resources\nmatplotlib\nshap-0.47.3.dev8-offline-fork-for-myevae==0.0.1\n```\n\nThe last dependency is a offline fork of shap (https://pypi.org/project/shap/) which has been modified to work MyeVAE.\n\n\n## Recommended hardware\n\n1. Minimum 4 CPU cores (8 cores is recommended)\n\n2. Minimum 16 GB RAM (64GB is required for feature preprocessing)\n\nNo GPU is required.\n\n\n# Citation\n\nIf you use MyeVAE in your research, please consider citing:\n\nChang, J.G., Chen, J., Chew, GL. et al. MyeVAE: a multi-modal variational autoencoder for risk profiling of newly diagnosed multiple myeloma. BMC Artif. Intell. 1, 8 (2025). https://doi.org/10.1186/s44398-025-00009-2\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjiagengchang%2Fmyevae","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjiagengchang%2Fmyevae","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjiagengchang%2Fmyevae/lists"}