{"id":25895637,"url":"https://github.com/diamondlightsource/gas-hydrate-segmentation-unets","last_synced_at":"2025-08-02T20:34:47.081Z","repository":{"id":93804152,"uuid":"345756620","full_name":"DiamondLightSource/gas-hydrate-segmentation-unets","owner":"DiamondLightSource","description":"Code for segmentation of 3-dimensional X-ray Computed Tomography (XCT) images of methane bearing sand using 2d and 3d U-net models","archived":false,"fork":false,"pushed_at":"2022-04-14T17:39:28.000Z","size":49,"stargazers_count":4,"open_issues_count":0,"forks_count":0,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-03-02T22:33:11.261Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DiamondLightSource.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-03-08T18:33:21.000Z","updated_at":"2022-10-17T11:58:20.000Z","dependencies_parsed_at":"2023-03-05T18:45:28.097Z","dependency_job_id":null,"html_url":"https://github.com/DiamondLightSource/gas-hydrate-segmentation-unets","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/DiamondLightSource/gas-hydrate-segmentation-unets","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DiamondLightSource%2Fgas-hydrate-segmentation-unets","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DiamondLightSource%2Fgas-hydrate-segmentation-unets/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DiamondLightSource%2Fgas-hydrate-segmentation-unets/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DiamondLightSource%2Fgas-hydrate-segmentation-unets/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DiamondLightSource","download_url":"https://codeload.github.com/DiamondLightSource/gas-hydrate-segmentation-unets/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DiamondLightSource%2Fgas-hydrate-segmentation-unets/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":268448362,"owners_count":24252019,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-02T02:00:12.353Z","response_time":74,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-03-02T22:32:17.151Z","updated_at":"2025-08-02T20:34:47.073Z","avatar_url":"https://github.com/DiamondLightSource.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"This repository has been published to Zenodo [![DOI](https://zenodo.org/badge/345756620.svg)](https://zenodo.org/badge/latestdoi/345756620)\n\n# Gas Hydrate Segmentation Using U-nets. Code Repository.\n\nThis code repository supports the article manuscript entitled 'U-Net Segmentation Methods for Variable-Contrast XCT Images of Methane-Bearing Sand Using Small Training Datasets'\nAuthored by F. J. Alvarez-Borges, O. N. F. King, B. N. Madhusudhan, T. Connolley, M. Basham and S. I. Ahmed.\nApril 2022\n\nCopyright 2022 Diamond Light Source Ltd.\nLicensed under the Apache License, Version 2.0\n\nFunders:\nNatural Environment Research Council (UK) grant No. NE/K00008X/1\n\n## Description\n\n### 2d and 3d U-net training and prediction scripts \n\nThese can be found in the `unet_methods` directory.\nOn Linux, For ease of installation and use we recommend downloading the [singularity container](#singularity-container).\n\n#### 2d methods\n\nFor 2d U-net training and prediction, settings are specified by editing the YAML files in `unet_2d/settings`. Paths for image data (HDF5 format), labels(HDF5 format) and input models are given via the command line.\n\n##### For 2d U-net training\n\n```shell\npython train_2d_unet.py path/to/image/data.h5 path/to/corresponding/segmentation/labels.h5\n```\n\nA model will be trained and saved to your working directory\n\n##### For 2d U-net prediction\n\n```shell\npython predict_2d_unet.py path/to/model_file.zip path/to/data_for_prediction.h5\n```\n\nA directory of segmented volumes will be saved to your working directory.\n\n#### 3d methods\n\nFor 3d U-net training and prediction, settings are specified by editing the YAML files in `unet_3d/settings`. Paths for image data(HDF5 format), labels(HDF5 format) and input models are also given in the YAML files.\n\nThe 3d methods utilise the [Torchio](https://github.com/fepegar/torchio) library for data sampling and augmentation:\nPérez-García, Fernando, Rachel Sparks, and Sebastien Ourselin. ‘TorchIO: A Python Library for Efficient Loading, Preprocessing, Augmentation and Patch-Based Sampling of Medical Images in Deep Learning’. ArXiv:2003.04696 [Cs, Eess, Stat], 9 March 2020. http://arxiv.org/abs/2003.04696.\n\nAnd the 3d U-net utilises the [pytorch-3dunet](https://github.com/wolny/pytorch-3dunet) library:\nWolny, Adrian, Lorenzo Cerrone, Athul Vijayan, Rachele Tofanelli, Amaya Vilches Barro, Marion Louveaux, Christian Wenzl, et al. ‘Accurate and Versatile 3D Segmentation of Plant Tissues at Cellular Resolution’. Edited by Christian S Hardtke, Dominique C Bergmann, Dominique C Bergmann, and Moritz Graeff. ELife 9 (29 July 2020): e57613. https://doi.org/10.7554/eLife.57613.\n\n\n##### For 3d U-net training\n\n```shell\npython train_3d_unet.py \n```\n\nA model will be trained and saved to the directory where your input data resides.\n\n##### For 3d U-net prediction\n\n```shell\npython predict_3d_unet.py\n```\n\nA segmented volume will be saved to the directory where your input data resides.\n\n#### Installation\n\nIn order to run the scripts you will need an NVIDIA GPU, ideally with at least 8GB of memory and that supports CUDA 10 or greater. \nFor U-Net segmentation, we recommend using the [singularity container](#singularity-container).  \nOtherwise, If you want to install locally, create an [Anaconda](https://www.anaconda.com/products/individual#Downloads) environment using the `unet_environment.yaml` file found in `unet_methods`.\n\n```shell\nconda env create -n unet_env -f unet_methods/unet_environment.yaml\n```\n\nOnce the environment is installed you can activate it\n\n```shell\nconda activate unet_env\n```\nYou should now be able to run the U-net training and prediction by calling the relevant scripts using Python.\n\n### Singularity Container\n\nRather than installing your own anaconda environment and downloading the codebase, both can be retrieved in a singularity container.\n```shell\nsingularity pull library://ollyking/unet-segmentation/unet_conda_container\n```\nTo run the container, first a writable data directory needs to be created to be bound to the container. This directory should contain:\n- the data and label files for model training\n- a subdirectory named `unet-settings`, this will contain the YAML settings files. This directory can be copied from the repository and is found [here](https://github.com/DiamondLightSource/gas-hydrate-segmentation-unets/tree/main/unet_methods/unet-settings). \n\nIn addition, the trained models and any predictions will also be output to this folder.\n\nThe following examples assume that the data directory was named `my_data_dir` and sits in the same directory as the singularity image. The container is run with `singularity run` and the data directory is bound with the commandline argument `-B my_data_dir/:/data`, this binds it at the location `/data` within the container. In addition the flag `--nv` is needed in order to give the container access to the GPU. \n\n#### For 2d U-net training using the container image\n\n```shell\nsingularity run --nv -B my_data_dir/:/data unet_conda_container_latest.sif 2dunet-train --data /data/\u003cmy_data_file.h5\u003e --labels /data/\u003cmy_label_file.h5\u003e --data_dir /data\n```\n\nA model will be trained and saved to `my_data_dir` along with a preview image.\n\n#### For 2d U-net prediction using the container image\n\n```shell\nsingularity run --nv -B my_data_dir/:/data unet_conda_container_latest.sif 2dunet-predict /data/\u003cmy_trained_2d_model.zip\u003e /data/\u003cdata_file_to_predict.h5\u003e --data_dir /data\n```\n\nA directory of segmented volumes will be saved to `my_data_dir`.\n\n#### For 3d U-net training using the container image\n\n```shell\nsingularity run --nv -B my_data_dir/:/data unet_conda_container_latest.sif 3dunet-train /data/\u003cmy_data_file.h5\u003e /data/\u003cmy_label_file.h5\u003e /data/\u003cmy_validation_data_file.h5\u003e /data/\u003cmy_validation_label_file.h5\u003e --data_dir /data\n\n```\nA model will be trained and saved to `my_data_dir` along with a preview image and a graph of training and validation loss. In addition, a segmentation of the validation volume is saved. \n\n#### For 3d U-net prediction using the container image\n\n```shell\nsingularity run --nv -B my_data_dir/:/data unet_conda_container_latest.sif 3dunet-predict /data/\u003cmy_trained_3d_model.pytorch\u003e /data/\u003cdata_file_to_predict.h5\u003e --data_dir /data\n```\nThe segmented volume will be saved to `my_data_dir`.\n\n### Data preparation and Rootpainter training data scripts\n\nThese can be found in the `matlab_scripts` directory.\n\nFor computation of segmentation performance based on the central 40 XY slices:\n'segmentation_performance.m' \n \nTo convert 572x572x572 voxel training and validation HDF5 subvolumes into RootPainter-readable slices:\n'rootpainter_annot_CH4.m'\n'rootpainter_annot_sand.m'\n \nTo convert HDF5 file data to tiff stacks for visualisation:\n'h5_to_tiff_stack.m'\n \nAll MATLAB scripts are run using the MATLAB app and include a graphical user interface for file input/output.\n\nMATLAB code was created using MATLAB 2020b.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdiamondlightsource%2Fgas-hydrate-segmentation-unets","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdiamondlightsource%2Fgas-hydrate-segmentation-unets","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdiamondlightsource%2Fgas-hydrate-segmentation-unets/lists"}