{"id":20874450,"url":"https://github.com/imagingdatacommons/idc-sm-annotations-conversion","last_synced_at":"2026-07-20T19:33:31.052Z","repository":{"id":175008164,"uuid":"653172050","full_name":"ImagingDataCommons/idc-sm-annotations-conversion","owner":"ImagingDataCommons","description":null,"archived":false,"fork":false,"pushed_at":"2025-03-01T22:22:32.000Z","size":474,"stargazers_count":0,"open_issues_count":0,"forks_count":1,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-03-01T23:22:55.433Z","etag":null,"topics":["conversion","dicom","pathomics"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ImagingDataCommons.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-06-13T14:33:19.000Z","updated_at":"2025-02-20T14:27:22.000Z","dependencies_parsed_at":"2023-12-14T15:27:24.489Z","dependency_job_id":"ab2c75f8-725c-4b65-bbd0-4eebcd77a4fc","html_url":"https://github.com/ImagingDataCommons/idc-sm-annotations-conversion","commit_stats":null,"previous_names":["imagingdatacommons/idc-pan-cancer-annotations-conversion","imagingdatacommons/idc-sm-annotations-conversion"],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ImagingDataCommons%2Fidc-sm-annotations-conversion","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ImagingDataCommons%2Fidc-sm-annotations-conversion/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ImagingDataCommons%2Fidc-sm-annotations-conversion/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ImagingDataCommons%2Fidc-sm-annotations-conversion/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ImagingDataCommons","download_url":"https://codeload.github.com/ImagingDataCommons/idc-sm-annotations-conversion/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243248251,"owners_count":20260752,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["conversion","dicom","pathomics"],"created_at":"2024-11-18T06:32:39.289Z","updated_at":"2026-07-20T19:33:31.045Z","avatar_url":"https://github.com/ImagingDataCommons.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# IDC Annotation Conversion\n\nPython project for converting various pathology annotations into DICOM\nformat for ingestion into the Imaging Data Commons.\n\nThe code in this repository is currently under development.\n\n### Installation\n\nThis repository is structured to be directly installable as a Python\ndistribution named `idc-annotation-conversion` via pip. You should be able to\nrun this command from the root of the cloned repository to install the packages\nalong with all its dependencies (defined in `pyproject.toml`) in your current\nPython environment:\n\n```bash\npip install .\n```\nAlternatively, you can install the package directly from remote with:\n\n```bash\npip install https://github.com/ImagingDataCommons/idc-sm-annotations-conversion.git\n```\n\n### Cloud Authentication\n\nYou need to authenticate to the relevant Google cloud buckets to run the code\nin this package. Specifically, access to the following resources is required:\n\n- Project `idc-etl-processing`\n- Bucket `public-datasets-idc`, the public bucket containing DICOM-format whole\n  slide images.\n- Bucket `idc-annotation-conversion-outputs`, or any other bucket specified\n  as the output bucket, if any.\n\nDepending on the conversion process that you are running, you may also need\naccess to:\n\n- Bucket `tcia-nuclei-seg`, which contains the original (CSV format)\n  segmentations for the `pan_cancer_nuclei_seg` conversion process.\n- Project `idc-external-031` and bucket `rms_annotation_test_oct_2023`, which contains the\n  original (XML format) annotations for the `rms` conversion process.\n\nIf you are using an IDC cloud VM, this should be handled\nautomatically for you. Otherwise, you should run:\n\n```\ngcloud auth application-default login --billing-project idc-etl-processing\n```\n\nand then once you are finished:\n\n```\ngcloud auth application-default revoke\n```\n\n### Use\n\nEach conversion process is implemented as a submodule of the `idc_annotation_conversion`\nmodule, which is installed when you installed this package. Each submodule has an\nan entrypoint (a `__main__.py` file), meaning that to run the process once this\npackage is installed you run:\n\n```bash\npython -m idc_annotation_conversion.\u003cmodule\u003e \u003cargs\u003e\n```\n\nSo for example to run the `pan_cancer_nuclei_seg` conversion process:\n\n```bash\npython -m idc_annotation_conversion.pan_cancer_nuclei_seg \u003cargs\u003e\n```\n\nIn each case, the default parameters should be sufficient to run a conversion processon\non the entire collection but there a number of optional arguments to control the process.\nYou can see the options by running `--help` when calling the submodule. E.g.:\n\n```bash\npython -m idc_annotation_conversion.pan_cancer_nuclei_seg --help\n```\n\n### Collection Details\n\nThe following modules are currently available:\n\n- `pan_cancer_nuclei_seg`: This module implements conversion of Pan Cancer\n  Nuclei Segmentations for several collections within TCGA. The original data are supplied\n  in a non-standard CSV format giving the image coordinates points on the contours of\n  nuclei as segmented by a deep-learning based segmentation model. These data were previously released\n  [here](https://www.cancerimagingarchive.net/analysis-result/pan-cancer-nuclei-seg/) as part of\n  The Cancer Imaging Archive. These coordinates are converted to DICOM Microscopy Bulk Simple\n  Annotation objects, and in addition, the contours are converted to masks and stored as\n  a pyramid of binary DICOM Segmentation objects. Since this \"raster conversion\" takes place at the\n  highest resolution, this process is very slow and memory intensive.\n\n- `rms`: Conversion of annotations related to the rhabdomyosarcoma mutation prediction project from the Frederick National Laboratory.\n  Both hand annotated regions for tissue type (necrosis, stroma, ARMS, ERMS), used as training data in the project, and model-generated prediction results (for the same tissue classes) are available.\n  Hand annotated regions are provided as ImageScope format XML annotations and are converted to DICOM Structured Report objects with the `convert-xml-annotations` sub-command.\n  Model-generated probabilistic segmentation maps are provided as serialized NumPy arrays (`.npy` files) and converted to both binary and fractional DICOM Segmentation objects with the `convert-segmentations` sub-command.\n\n- `tcga_til_maps`: There are two versions of this collection, both containing patch-level maps of tumor-infiltrating lymphocytes (TILs) predicted by a neural network for several collections within TCGA. The two versions correspond to two different versions of the model, published in 2018 and 2022 by the same lab at Stony Brook University. Conversion routines for these two versions implemented as two separate sub-commands within this module.\n\n  The 2018 versions covers a smaller subset of the TCGA collections. The algorithm is published in [this paper](https://www.cell.com/cell-reports/pdf/S2211-1247(18)30447-9.pdf) and the source files are available [here](https://stonybrookmedicine.app.box.com/v/cellreportspaper). The collection was also described by TCIA on [this page](https://www.cancerimagingarchive.net/analysis-result/til-wsi-tcga/). The files are supplied as low-resolution PNG images, where each pixel in the PNG corresponds to a 50 micron patch in the original slide and the pixel value indicates the presence of TILs within the patch. The `convert-2018` command converts these to binary DICOM segmentation objects.\n\n  The 2022 versions covers a wider range of TCGA images and additionally has probabilistic segmentations (before thresholding) available in addition to binarized versions. This algorithm is described in [this paper](https://www.frontiersin.org/journals/oncology/articles/10.3389/fonc.2021.806603/full) and the source data are available [here](https://stonybrookmedicine.box.com/v/til-results-new-model). These are supplied as a non-standard text file containing a list of patch coordinates and associated binary or probabilistic pixel values. The `convert-2022` command coverts these to pixel arrays and stores them as DICOM Segmentation objects, giving one binary and one fractional (probabilistic) segmentation object for each slide.\n\n- `gbm_transcriptional_subtypes`: This module relates to a collection of results from [this paper](https://www.nature.com/articles/s41467-023-39933-0) from Stanford University on transcriptional subtypes within glioblastoma. There are two data types of interest here: transcriptional subtype maps classifying an image patch into a set of transcriptional subtypes, and aggressiveness maps giving the aggressiveness of each image patch. While the conversion process for both is implemented in this repository, only the aggressiveness maps have been released at this time. The source data are not publicly available elsewhere. The aggressiveness maps are supplied as arrays of image coordinates and corresponding aggressiveness scores (between 0 and 1) within an h5 format file, with one aggressiveness score for an entire image patch. These are converted to DICOM Parametric Map objects using the `convert-aggressiveness-maps` sub-command of this module.\n\n- `catch`: This module relates to the conversion of pathologist-drawn annotations of tumor and tissue regions in the [CAnine CuTaneous Cancer Histology](https://www.cancerimagingarchive.net/collection/catch/) dataset. Annotations are supplied in a SQLITE database created by the SlideRunner annotation tool. These are converted to DICOM Microscopy Bulk Simple Annotation objects.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fimagingdatacommons%2Fidc-sm-annotations-conversion","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fimagingdatacommons%2Fidc-sm-annotations-conversion","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fimagingdatacommons%2Fidc-sm-annotations-conversion/lists"}