{"id":51319980,"url":"https://github.com/vanvalenlab/torch-dynamicnuclearnet","last_synced_at":"2026-07-01T12:03:11.115Z","repository":{"id":368226498,"uuid":"1232352309","full_name":"vanvalenlab/torch-dynamicnuclearnet","owner":"vanvalenlab","description":"DynamicNuclearNet implemented in PyTorch","archived":false,"fork":false,"pushed_at":"2026-06-29T14:57:09.000Z","size":543,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2026-06-29T16:32:17.046Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vanvalenlab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-05-07T21:04:11.000Z","updated_at":"2026-06-29T14:57:28.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/vanvalenlab/torch-dynamicnuclearnet","commit_stats":null,"previous_names":["vanvalenlab/torch-dynamicnuclearnet"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/vanvalenlab/torch-dynamicnuclearnet","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vanvalenlab%2Ftorch-dynamicnuclearnet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vanvalenlab%2Ftorch-dynamicnuclearnet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vanvalenlab%2Ftorch-dynamicnuclearnet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vanvalenlab%2Ftorch-dynamicnuclearnet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vanvalenlab","download_url":"https://codeload.github.com/vanvalenlab/torch-dynamicnuclearnet/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vanvalenlab%2Ftorch-dynamicnuclearnet/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35005413,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-01T02:00:05.325Z","response_time":130,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-07-01T12:03:10.215Z","updated_at":"2026-07-01T12:03:11.096Z","avatar_url":"https://github.com/vanvalenlab.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Torch Mesmer\n\nA PyTorch implementation of DynamicNuclearNet for segmenting live nuclei for cell tracking.\n\n## The Network\n\nDynamicNuclearNet (DNN) is built on a Panoptic network, consisting of an EfficientNetV2BL backbone connected to a [feature pyramid network](https://arxiv.org/abs/1612.03144). The levels of the backbone and feature pyramid network can be selected, but for the pre-trained DNN model, we use backbone levels `C1-C5`, and pyramid levels `P1-P7`. Pyramid levels are then upsampled to match the input resolution (256 x 256 px) and delivered to the semantic heads of the model.\n\n In the pre-trained DNN model, there are three semantic heads:\n\n```markdown\nHead 1 (1, 256, 256)\n└─ Inner distance transform for the nucleus\n\nHead 2 (1, 256, 256)\n└─ Outer distance transform for the nucleus\n\nHead 3 (2, 256, 256)\n├─ Foreground pixels for nucleus\n└─ Background pixels for nucleus\n```\n\nAfter softmax on the semantic head convolutions, the model concatenates all predictions into an output tensor of shape `(4, 256, 256)` and returns it.\n\n## The Dataset\n\nEach dataset contains one nuclear and one cytosol channel, as well as the labeled ground truth mask for each channel, and metadata that contains the source tissue and experiment of each image.\n\n- **Training**: 4950 square images (512 x 512)\n\n- **Validation**: 1417 square images (512 x 512 px)\n\n- **Test**: 717 square images (512 x 512 px)\n\n## The Training\n\n### Data loaders\n\nThe training and validation data are loaded into a PyTorch `Dataset` object, which conducts preprocessing under the hood for each batch. This `Dataset` is then used in the construction of a `Dataloder`. The preprocessing and augmentation pipeline is outlined below:\n\n1. Each item (one image and ground truth mask) is selected from the full dataset.\n2. The image is normalized with two steps:\n    1. Images are thresholded in order to reduce the influence of bright pixels.\n    2. Imaes are then normalized using Contrast Limited Adaptive Histogram Equalization (CLAHE) with the `equalize_adapthist` function from [scikit-image](https://scikit-image.org/).\n3. The labels are then transformed to generate the inner distance transform, outer distance transform, and foreground/background pixels.\n4. The normalized images and mask transformations are then augmented using a random combination of rotations, flips, crops and zooms.\n5. The images and masks are then returned to the model.\n\n### Optimizer, Learning Rate, and Loss Function\n\nWe use the Adam optimizer with a learning rate of 0.0001. Upon a plateau in validation loss that lasted longer than 5 epochs, the model's learning rate is reduced by a factor of 10.\n\nThe loss function is a combination of weighted categorical cross entropy (WCCE) and mean squared error (MSE) loss. For continuous predictions (inner distance transforms), MSE loss was used. For categorical predictions (foreground and background), WCCE loss was used with class weights calculated for each batch. Loss from continuous heads was weighted with 0.01 to increase stability during training. The loss calculated from each head was summed and then used in backpropagation.\n\nWe used a batch size of 10 images, and an augmented version of each images was seen only once during each epoch. The model was trained for 50 epochs, and we test the model that returned the lowest validation loss.\n\n## The Testing\n\nThe model was used to segment 717 test images. These segmentations were then compared to the ground truth using a custom metrics pipeline that analyzes the following:\n\n- **Recall**\n- **Precision**\n- **Jaccard index (IoU)** - The index of overlap between the ground truth and the prediction\n- **Gained detections** - objects segmented but not present in the ground truth\n- **Missed detections** - objects present in the ground truth that were missed by the model\n- **Splits** - number of \"one to many\" errors\n- **Merges** - number of \"many to one\" errors\n- **Catastrophes** - number of \"many to many\" errors\n\nEach of these metrics was calculated for every image, allowing us to identify areas of weakness in each trained model.\n\n## How to use DNN\n\n### Instantiate the model\n\n```python\nfrom tifffile import imread\nimport matplotlib.pyplot as plt\nimport numpy as np\nfrom torch_dnn.dnn import DNN\n\n\nmodel = DNN(\n    model_path='../.deepcell/dnn/saved_model_best_dict.pth',\n    device='cuda:0'\n)\n```\n\n### Read in the example image and view\n\n```python\nframe = imread('example/example_image.tiff')\n\nfig, ax = plt.subplots()\nax.imshow(frame)\nax.set_axis_off()\nax.set_title('Nuclear Image')\nplt.show()\n```\n\n![image](example/nuclear_image.png)\n\n### Predict\n\n```python\nframe = frame[np.newaxis, np.newaxis] # model requires a time and channel axis\n\nmask = model.predict(frame)\n```\n\n### Plot the results\n\n```python\nfig, ax = plt.subplots(1,2)\n\nax[0].imshow(frame.squeeze())\nax[0].set_title('Nuclear Image')\n\nax[1].imshow(mask.squeeze())\nax[1].set_title('Predicted segmentation')\n\nfor axis in ax:\n    axis.set_axis_off()\n\nfig.tight_layout()\nplt.show()\n```\n\n![image](example/nuclear_image_and_seg.png)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvanvalenlab%2Ftorch-dynamicnuclearnet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvanvalenlab%2Ftorch-dynamicnuclearnet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvanvalenlab%2Ftorch-dynamicnuclearnet/lists"}