{"id":13490201,"url":"https://github.com/mirzaevinom/data_science_bowl_2018","last_synced_at":"2025-03-28T05:32:05.760Z","repository":{"id":201286467,"uuid":"138928592","full_name":"mirzaevinom/data_science_bowl_2018","owner":"mirzaevinom","description":"My 5th place (out of 816 teams) solution to The 2018 Data Science Bowl organized by Booz Allen Hamilton","archived":false,"fork":false,"pushed_at":"2019-03-29T20:49:47.000Z","size":1217,"stargazers_count":157,"open_issues_count":3,"forks_count":63,"subscribers_count":8,"default_branch":"master","last_synced_at":"2024-10-31T03:35:31.770Z","etag":null,"topics":["data-science-bowl-2018","deep-learning","instance-segmentation","kaggle-competition","object-detection"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mirzaevinom.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2018-06-27T20:15:44.000Z","updated_at":"2024-06-20T06:49:27.000Z","dependencies_parsed_at":"2023-10-21T05:30:30.012Z","dependency_job_id":null,"html_url":"https://github.com/mirzaevinom/data_science_bowl_2018","commit_stats":null,"previous_names":["mirzaevinom/data_science_bowl_2018"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mirzaevinom%2Fdata_science_bowl_2018","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mirzaevinom%2Fdata_science_bowl_2018/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mirzaevinom%2Fdata_science_bowl_2018/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mirzaevinom%2Fdata_science_bowl_2018/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mirzaevinom","download_url":"https://codeload.github.com/mirzaevinom/data_science_bowl_2018/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245978200,"owners_count":20703675,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-science-bowl-2018","deep-learning","instance-segmentation","kaggle-competition","object-detection"],"created_at":"2024-07-31T19:00:42.671Z","updated_at":"2025-03-28T05:32:05.366Z","avatar_url":"https://github.com/mirzaevinom.png","language":"Python","funding_links":[],"categories":["Prediction"],"sub_categories":["[[2018] Data Science Bowl](https://www.kaggle.com/c/data-science-bowl-2018)"],"readme":"# The 2018 Data Science Bowl: \"Spot Nuclei. Speed Cures.\"\n\nThis repository contains scripts of my solution to [The 2018 Data Science Bowl](https://www.kaggle.com/c/data-science-bowl-2018). Goal of the competition was to create an algorithm to automate nucleus detection from biomedical images.\n\nWeights are now available [here](https://www.dropbox.com/s/1kql7tsug876xfn/kaggle_bowl.h5?dl=0).\n\n## Model overview\n\nFor this competition, I modified [Matterport's](https://github.com/matterport/Mask_RCNN) implementation of [Mask-RCNN](https://arxiv.org/abs/1703.06870) deep neural network for object instance segmentation. I adapted the existing model configurations to detect small nuclei in images with varying size and modality. To ensure that the model doesn't overfit, I used an [external dataset](https://www.kaggle.com/voglinio/external-h-e-data-with-mask-annotations) and relied heavily on image augmentation. Moreover, generated mosaics from train images based on [this notebook](https://www.kaggle.com/bonlime/train-test-image-mosaic). To improve generalizability of the model, I split (using stratification) the `stage1_train` dataset into train and validation sets based on 5 image modalities provided by [Allen Goodman](https://www.kaggle.com/c/data-science-bowl-2018/discussion/48130). After training the model using Resnet101 as a backbone encoder and Adam as an optimizer, I improved prediction accuracy by test time augmentation and post-processing the masks.\n\n## Training Method(s)\n\n### Pre-processing\n- I noticed some issues with the provided masks. Therefore, used the annotations and mask provided by [Konstantin Lopuhin](https://github.com/lopuhin/kaggle-dsbowl-2018-dataset-fixes) in [data quality issues](https://www.kaggle.com/c/data-science-bowl-2018/discussion/47572) thread.\n- Removed the alpha channel from the images.\n- Filled holes in the masks\n- Split (using stratification) the `stage1_train` dataset into 90% train and 10% validation sets based on 5 image modalities provided by [Allen Goodman](https://www.kaggle.com/c/data-science-bowl-2018/discussion/48130).\n- Used an [external dataset](https://www.kaggle.com/voglinio/external-h-e-data-with-mask-annotations) provided in the forum. Divided the images and the masks into 4 pieces due their large sizes. External dataset [download links](https://nucleisegmentationbenchmark.weebly.com/dataset.html).\n- Generated mosaics from train images based on [Emil's](https://www.kaggle.com/bonlime/train-test-image-mosaic) notebook.\n\n\n### Model and Training\n* Modified [Matterport's](https://github.com/matterport/Mask_RCNN) implementation of [Mask-RCNN](https://arxiv.org/abs/1703.06870) deep neural network for object instance segmentation.\n* Tuned hyperparameters to detect small nuclei from the images. (I found [this tutorial](https://engineering.matterport.com/splash-of-color-instance-segmentation-with-mask-r-cnn-and-tensorflow-7c761e238b46) very useful for understanding the model hyperparameters)\n    + Original Matterport implementation was validating only on one image so fixed this [validation issue](https://github.com/matterport/Mask_RCNN/issues/89).\n    + Reduced RPN (region proposal network) anchor sizes since the nuclei are mostly small.\n    + Increased number of anchors to be used since the nuclei are small and can be found anywhere on an image.\n    + Increased maximum number of predicted objects since an image can contain 300 or more nuclei.\n    + Increased `POST_NMS_ROIS_TRAINING` to get more region proposals during training.\n    + Added extra parameter `DETECTION_MASK_THRESHOLD` to model configuration. Default was hardcoded in the model as 0.5 but setting it to 0.35 helped in detection of small nuclei boundaries.\n    + Resized images and masks to 512x512\n* Relied heavily on image augmentation due to small training set:\n    - Random horizontal or vertical flips\n    - Random 90 or -90 degrees rotation\n    - [Random rotations](https://www.kaggle.com/c/data-science-bowl-2018/discussion/49692) in the range of (-15, 15) degrees\n    - [Random cropping](https://www.kaggle.com/c/data-science-bowl-2018/discussion/49692) of bigger images and masks to 256x256x3.\n    - [Random scaling](https://www.kaggle.com/c/data-science-bowl-2018/discussion/49692) of image and mask scaling in the range (0.5, 2.0)\n\n* Used Resnet101 architecture as a backbone encoder but initialized the first 50 layers of the model with pre-trained Resnet50 weights from [ImageNet competition](https://github.com/fchollet/deep-learning-models/releases/).\n* Trained the model with [Adam](https://arxiv.org/abs/1412.6980) optimizer for 75 epochs:\n    - 25 epochs with learning rate 1e-4\n    - 25 epochs with learning rate 1e-5\n    - 25 epochs with learning rate 1e-6\n* Did not retrain the model with stage1 test data during stage 2 as I was afraid of overfitting on black and white images.\n\n\n### Post-processing\n\n- Combined predictions on actual image and horizontally flipped image: took unions of masks with maximum overlap and removed false positive masks with small overlap.\n- Due to configured RPN anchor sizes, the model predicts small nuclei very well. However, it struggles at predicting large nuclei. Therefore, if a model predicts no masks for an image then I scale down the image and predict once again.\n- Removed overlaps between predicted nuclei based on their objectness score. In other words, removed intersections from the masks with lower scores.\n    - If this intersection removal results in multiple objects in that mask, then removing all the small pieces.\n- Closing small holes inside the masks using morphological operations (dilation followed by erosion).\n\n## Interesting findings\n\n- Mask-RCNN model overfits easily without image augmentation.\n- Removing false positive mask predictions improves the overall score significantly.\n- Since images are on different scales, predicting masks on scaled images helps with the model generalizability.\n- Dilating and then eroding individual masks helped me achieve slightly better result.\n- Matterport's original implementation was only [validating on only one image](https://github.com/matterport/Mask_RCNN/issues/89). Fixing this issue made the training process reproducible.\n- I found that the model reaches a local minima faster when trained using Adam optimizer compared to default SGD optimizer.\n\n## Unsuccessful approaches tried\n- Trained the model with Dice Coefficient Loss instead of default binary cross-entropy loss for the masks heads. Although got comparable results couldn't beat prediction accuracy of binary cross-entropy on my validation set.\n- Trained with random Gaussian and Poisson (or shot) noise for image augmentation. It actually hurt overall model performance.\n- Tried ensembling actual image predictions with horizontal and vertical flip predictions. Used non-maximum suppression for removing overlaps. Did not improve prediction accuracy on the validation set.\n- Trained end-to-end without initializing with pre-trained ImageNet weights. Mostly got to Mean IoU score of 0.35 on stage1 test set.\n- Trained on preprocessed images with adaptive histogram equalization (CLAHE). The model performed way worse.\n\n## Example model predictions\n\nFor the following figures red lines represent ground truth boundaries and blue lines represent prediction boundaries.\n\n* Model predictions for some stage 1 test image samples:\n\u003cp float=\"left\"\u003e\n\u003cimg width=\"200\" height=\"200\" src=\"images/sample_1.png\"/\u003e\u003cimg width=\"200\" height=\"200\" src=\"images/sample_2.png\"/\u003e\u003cimg width=\"200\" height=\"200\" src=\"images/sample_3.png\"/\u003e\n\u003c/p\u003e\n\n* Model predictions for some stage 2 test image samples:\n\u003cp float=\"left\"\u003e\n\u003cimg width=\"200\" height=\"200\" src=\"images/sample_4.png\" /\u003e\u003cimg width=\"200\" height=\"200\" src=\"images/sample_5.png\"/\u003e\u003cimg width=\"200\" height=\"200\" src=\"images/sample_6.png\"/\u003e\n\u003c/p\u003e\n\n# Appendix\n\n\n## A1. Model Execution Time\n\nThe following execution times are measured on Nvidia P100 GPUs provided by [Ohio Supercomputer Center](https://www.osc.edu/)\n\n+ Each training epoch takes about 12 minutes.\n+ It takes about 15 hours to train the model from scratch.\n+ It takes 1 to 4 seconds to predict all the nuclei on a single image.\n+ Plotting boundaries of each nuclei and saving plots adds an extra 1 second on average\n\n## A2. Dependencies\n\nThe codes are written in Python (3.6.3) and tested on  Red Hat Enterprise Linux Server (7.4). The scripts depend on the following python libraries available on `PyPi`:\n\n* `tensorflow (1.3.0), keras (2.1.3), numpy (1.13.3), scipy (0.19.1)` for computations\n* `cv2 (3.4.0), skimage (0.13.0), matplotlib (2.1.0)` for image processing and plotting\n* `tqdm (4.11.2)` for progress bar\n\n\n## A3.  How To Generate the Solution\n\n1. Download/extract/place the training (external dataset [download links](https://nucleisegmentationbenchmark.weebly.com/dataset.html) ) datasets in the following folder structure:\n\n~~~~~~~\n        project\n          |-- codes\n          |-- data\n               |--- stage1_train\n                    |-- imageID\n                          |-- images\n                          |--- masks\n               |-- stage1_test\n                    |-- similar to stage1_train\n               |-- stage2_test\n                    |-- similar to stage1_train\n               |-- external_data\n                    |-- tissue_images\n                          |-- imageID.tif\n                    |-- annotations\n                          |-- imageID.xml\n~~~~~~~\n\n2. Run `python augment_preprocess.py` to pre-process external data and create mosaics from the dataset. (You can skip this step if you only want to train on provided train set)\n\n3. Run `python train.py` to train the model. Model weights are saved at `../data/logs/kaggle_bowl/mask_rcnn.h5`.\n\n4.  Run `python predict.py` to evaluate model performance on validation set and predict nuclei boundaries on test set.\n\n### Acknowledgements\n\n* This material is based upon work supported by the National Science Foundation under Agreement No. 0931642 ([The Ohio Supercomputer Center](https://www.osc.edu/))\n* I would like to also thank [Mathematical Biosciences Institue](http://mbi.osu.edu) (MBI) at Ohio State University, for partially supporting this research. MBI receives its funding through the National Science Foundation grant DMS 1440386\n\n\n[1]: https://github.com/matterport/Mask_RCNN\n[2]: https://www.osc.edu/\n[3]: https://arxiv.org/abs/1703.06870\n[4]: https://www.kaggle.com/voglinio/external-h-e-data-with-mask-annotations\n[5]: https://www.kaggle.com/c/data-science-bowl-2018/discussion/48130\n[6]: https://github.com/matterport/Mask_RCNN/issues/89\n[7]: https://engineering.matterport.com/splash-of-color-instance-segmentation-with-mask-r-cnn-and-tensorflow-7c761e238b46\n[8]: https://www.kaggle.com/bonlime/train-test-image-mosaic\n[10]: https://nucleisegmentationbenchmark.weebly.com/dataset.html\n[11]: http://mbi.osu.edu\n[12]: http://www.lerner.ccf.org/thor/scott/lab/\n[13]: https://promise12.grand-challenge.org\n[14]: https://www.kaggle.com/c/data-science-bowl-2018/discussion/47572\n[15]: https://github.com/lopuhin/kaggle-dsbowl-2018-dataset-fixes\n[16]: https://github.com/fchollet/deep-learning-models/releases/\n[17]: https://arxiv.org/abs/1412.6980\n[18]: https://www.kaggle.com/c/data-science-bowl-2018/discussion/49692\n[19]: https://nucleisegmentationbenchmark.weebly.com/dataset.html\n[20]: https://github.com/matterport/Mask_RCNN/issues/89\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmirzaevinom%2Fdata_science_bowl_2018","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmirzaevinom%2Fdata_science_bowl_2018","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmirzaevinom%2Fdata_science_bowl_2018/lists"}