{"id":13716058,"url":"https://github.com/angelolab/ark-analysis","last_synced_at":"2026-01-14T17:35:34.880Z","repository":{"id":37057397,"uuid":"164378401","full_name":"angelolab/ark-analysis","owner":"angelolab","description":"Integrated pipeline for multiplexed image analysis","archived":false,"fork":false,"pushed_at":"2025-12-18T08:25:22.000Z","size":54658,"stargazers_count":100,"open_issues_count":54,"forks_count":31,"subscribers_count":9,"default_branch":"main","last_synced_at":"2025-12-21T10:44:55.755Z","etag":null,"topics":["multiplexed-imaging","python"],"latest_commit_sha":null,"homepage":"https://ark-analysis.readthedocs.io/en/latest/","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/angelolab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2019-01-07T04:31:09.000Z","updated_at":"2025-12-18T08:25:24.000Z","dependencies_parsed_at":"2023-10-14T18:08:52.943Z","dependency_job_id":"e4345245-a336-4456-a655-ea4704eb3294","html_url":"https://github.com/angelolab/ark-analysis","commit_stats":{"total_commits":587,"total_committers":23,"mean_commits":25.52173913043478,"dds":0.666098807495741,"last_synced_commit":"05e7714d4f1f3cb7416b62c29da632b04750ca72"},"previous_names":[],"tags_count":42,"template":false,"template_full_name":null,"purl":"pkg:github/angelolab/ark-analysis","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/angelolab%2Fark-analysis","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/angelolab%2Fark-analysis/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/angelolab%2Fark-analysis/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/angelolab%2Fark-analysis/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/angelolab","download_url":"https://codeload.github.com/angelolab/ark-analysis/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/angelolab%2Fark-analysis/sbom","scorecard":{"id":195892,"data":{"date":"2025-08-11","repo":{"name":"github.com/angelolab/ark-analysis","commit":"b26713b970ed13c98adb2280624f2e89773bef3c"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":5.7,"checks":[{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Code-Review","score":9,"reason":"Found 29/30 approved changesets -- score normalized to 9","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Token-Permissions","score":10,"reason":"GitHub workflow tokens follow principle of least privilege","details":["Info: jobLevel 'contents' permission set to 'read': .github/workflows/ci.yml:61","Info: jobLevel 'contents' permission set to 'read': .github/workflows/ci.yml:70","Warn: jobLevel 'contents' permission set to 'write': .github/workflows/release-drafter.yml:22","Info: jobLevel 'contents' permission set to 'read': .github/workflows/release.yml:20","Info: jobLevel 'contents' permission set to 'read': .github/workflows/release.yml:28","Info: jobLevel 'contents' permission set to 'read': .github/workflows/release.yml:44","Info: jobLevel 'contents' permission set to 'read': .github/workflows/release.yml:72","Warn: jobLevel 'contents' permission set to 'write': .github/workflows/release.yml:96","Info: topLevel 'contents' permission set to 'read': .github/workflows/build.yml:8","Info: topLevel 'contents' permission set to 'read': .github/workflows/ci.yml:18","Info: topLevel 'contents' permission set to 'read': .github/workflows/release-drafter.yml:16","Info: topLevel 'contents' permission set to 'read': .github/workflows/release.yml:12","Info: topLevel 'contents' permission set to 'read': .github/workflows/test.yml:12"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: MIT License: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/build.yml:21: update your workflow using https://app.stepsecurity.io/secureworkflow/angelolab/ark-analysis/build.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/build.yml:26: update your workflow using https://app.stepsecurity.io/secureworkflow/angelolab/ark-analysis/build.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/build.yml:49: update your workflow using https://app.stepsecurity.io/secureworkflow/angelolab/ark-analysis/build.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:29: update your workflow using https://app.stepsecurity.io/secureworkflow/angelolab/ark-analysis/ci.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:34: update your workflow using https://app.stepsecurity.io/secureworkflow/angelolab/ark-analysis/ci.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/ci.yml:41: update your workflow using https://app.stepsecurity.io/secureworkflow/angelolab/ark-analysis/ci.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:82: update your workflow using https://app.stepsecurity.io/secureworkflow/angelolab/ark-analysis/ci.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:87: update your workflow using https://app.stepsecurity.io/secureworkflow/angelolab/ark-analysis/ci.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/ci.yml:93: update your workflow using https://app.stepsecurity.io/secureworkflow/angelolab/ark-analysis/ci.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/release-drafter.yml:26: update your workflow using https://app.stepsecurity.io/secureworkflow/angelolab/ark-analysis/release-drafter.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/release.yml:49: update your workflow using https://app.stepsecurity.io/secureworkflow/angelolab/ark-analysis/release.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/release.yml:55: update your workflow using https://app.stepsecurity.io/secureworkflow/angelolab/ark-analysis/release.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/release.yml:79: update your workflow using https://app.stepsecurity.io/secureworkflow/angelolab/ark-analysis/release.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/release.yml:85: update your workflow using https://app.stepsecurity.io/secureworkflow/angelolab/ark-analysis/release.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/release.yml:101: update your workflow using https://app.stepsecurity.io/secureworkflow/angelolab/ark-analysis/release.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/release.yml:107: update your workflow using https://app.stepsecurity.io/secureworkflow/angelolab/ark-analysis/release.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/test.yml:28: update your workflow using https://app.stepsecurity.io/secureworkflow/angelolab/ark-analysis/test.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/test.yml:33: update your workflow using https://app.stepsecurity.io/secureworkflow/angelolab/ark-analysis/test.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/test.yml:41: update your workflow using https://app.stepsecurity.io/secureworkflow/angelolab/ark-analysis/test.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/test.yml:59: update your workflow using https://app.stepsecurity.io/secureworkflow/angelolab/ark-analysis/test.yml/main?enable=pin","Info:   0 out of  12 GitHub-owned GitHubAction dependencies pinned","Info:   0 out of   8 third-party GitHubAction dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Branch-Protection","score":-1,"reason":"internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration","details":null,"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Signed-Releases","score":8,"reason":"2 out of the last 2 releases have a total of 2 signed artifacts.","details":["Info: signed release artifact: ark_analysis-0.7.2-py3-none-any.whl.sigstore.json: https://github.com/angelolab/ark-analysis/releases/tag/v0.7.2","Info: signed release artifact: ark_analysis-0.7.1-cp310-cp310-macosx_10_9_universal2.whl.sigstore: https://github.com/angelolab/ark-analysis/releases/tag/v0.7.1","Warn: release artifact v0.7.2 does not have provenance: https://api.github.com/repos/angelolab/ark-analysis/releases/179403009","Warn: release artifact v0.7.1 does not have provenance: https://api.github.com/repos/angelolab/ark-analysis/releases/133499345"],"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Vulnerabilities","score":7,"reason":"3 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: GHSA-9hjg-9r4m-mvj7","Warn: Project is vulnerable to: GHSA-9wx4-h78v-vm56","Warn: Project is vulnerable to: PYSEC-2023-74 / GHSA-j8r2-6x86-q33q"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 30 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-16T21:49:09.665Z","repository_id":37057397,"created_at":"2025-08-16T21:49:09.665Z","updated_at":"2025-08-16T21:49:09.665Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28428680,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-14T16:38:47.836Z","status":"ssl_error","status_checked_at":"2026-01-14T16:34:59.695Z","response_time":107,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["multiplexed-imaging","python"],"created_at":"2024-08-03T00:01:06.614Z","updated_at":"2026-01-14T17:35:34.864Z","avatar_url":"https://github.com/angelolab.png","language":"Jupyter Notebook","funding_links":[],"categories":["Image processing and segmentation"],"sub_categories":[],"readme":"# ark-analysis\n\n\u003cdiv align=\"center\"\u003e\n\n|         |                                                                                                                            |\n| ------- | -------------------------------------------------------------------------------------------------------------------------- |\n| Status  | [![CI][badge-ci]][link-badge-ci] [![Docs][badge-docs]][link-badge-docs] [![Coverage][badge-coverage]][link-badge-coverage] |\n| Package | [![PyPI - Version][badge-pypi]][link-badge-pypi] [![PyPI - Python Version][badge-python-versions]][link-badge-pypi]        |\n| Meta    | [![License][badge-license]][link-badge-license] [![uv][badge-uv]][link-uv]                                                 |\n\n\u003c!-- Badges --\u003e\n\n\u003c!-- Status --\u003e\n[badge-ci]: https://github.com/angelolab/ark-analysis/actions/workflows/ci.yml/badge.svg\n[badge-docs]: https://img.shields.io/readthedocs/ark-analysis?logo=readthedocs\n[badge-coverage]: https://img.shields.io/coverallsCoverage/github/angelolab/ark-analysis?logo=coveralls\n\u003c!-- Package --\u003e\n[badge-pypi]: https://img.shields.io/pypi/v/ark-analysis.svg?logo=pypi\u0026label=PyPI\u0026logoColor=gold\n[badge-python-versions]: https://img.shields.io/pypi/pyversions/ark-analysis.svg?logo=python\u0026label=Python\u0026logoColor=gold\n\u003c!-- Meta / License --\u003e\n[badge-license]: https://img.shields.io/badge/License-MIT-yellow.svg\n[badge-uv]: https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/uv/main/assets/badge/v0.json\n\u003c/div\u003e\n\nToolbox for analyzing multiplexed imaging data.\n\nFull documentation for the project can be found [here](https://ark-analysis.readthedocs.io/en/latest/).\n\n## Table of Contents\n- [Pipeline Flowchart](#pipeline-flowchart)\n- [Getting Started](#getting-started)\n  - [Overview](#overview)\n    - [1. Segmentation](#1-segmentation)\n    - [2. Pixel clustering with Pixie](#2-pixel-clustering-with-pixie)\n    - [3. Cell clustering with Pixie](#3-cell-clustering-with-pixie)\n    - [4. Post clustering tasks](#4-post-clustering-tasks)\n    - [5. Spatial analysis](#5-spatial-analysis)\n  - [Installation Steps.](#installation-steps)\n    - [Download the Repo](#download-the-repo)\n    - [Running on Windows](#running-on-windows)\n    - [Using the Repository](#using-the-repository)\n    - [Updating the repo](#updating-the-repo)\n- [External Tools](#external-tools)\n  - [Mantis Viewer](#mantis-viewer)\n    - [Mantis directory structure](#mantis-directory-structure)\n    - [Loading image-specific files](#loading-image-specific-files)\n    - [Loading project-wide files](#loading-project-wide-files)\n    - [View cell features](#view-cell-features)\n  - [External Hard Drives and Google File Stream](#external-hard-drives-and-google-file-stream)\n- [Updating the Repository](#updating-the-repository)\n- [Example Dataset](#example-dataset)\n- [Questions?](#questions)\n- [Want to contribute?](#want-to-contribute)\n- [How to Cite](#how-to-cite)\n\n\n## Pipeline Flowchart\n\n![](docs/_images/ark_processing.png)\n\n## Getting Started\n\n### Overview\nThis repo contains tools for analyzing multiplexed imaging data. The assumption is that you've already performed any necessary image processing on your data (such as denoising, background subtraction, autofluorescence correction, etc), and that it is ready to be analyzed. For MIBI data, we recommend using the [toffy](https://github.com/angelolab/toffy) processing pipeline.\n\nWe have recorded workshop talks which complement the repository. [MIBI Workshop Playlist](https://youtube.com/playlist?list=PLjNbkEm4vA26o5YvWKeyHXF8HjTJc7yB0).\n\n#### 1. Segmentation  \nThe [**segmentation notebook**](./templates/1_Segment_Image_Data.ipynb) will walk you through the process of using [Mesmer](https://www.nature.com/articles/s41587-021-01094-0) to segment your image data. This includes selecting the appropriate channel(s) for segmentation, running your data through the network, and then extracting single-cell statistics from the resulting segmentation mask. [Workshop Talk - Session V - Part 1: Segmentation](https://youtu.be/4_AJxrxPYlk?t=231)\n  - *Note:* It is assumed that the cell table uses the default column names as in `ark/settings.py`. Refer to the [docs](docs/_rtd/data_types.md) to get descriptions of the cell table columns, and methods to adjust them if necessary.\n  - If you plan to segment out non-traditional cellular structures such as protein aggregates or cytoplasmic projections often found in brain cells (e.g. microglia, astrocytes, and neuropil), try out the companion notebook [**ezSegmenter**](./templates/ez_segmenter.ipynb) either as a stand-alone or in combination with the above standard cell segmentation process.\n\n#### 2. Pixel clustering with Pixie  \nThe first step in the [Pixie](https://doi.org/10.1038/s41467-023-40068-5) pipeline is to run the [**pixel clustering notebook**](./templates/2_Pixie_Cluster_Pixels.ipynb). The notebook walks you through the process of generating pixel clusters for your data, and lets you specify what markers to use for the clustering, train a model, use it to classify your entire dataset, and generate pixel cluster overlays. The notebook includes a GUI for manual cluster adjustment and annotation. [Workshop Talk - Session IV - Pixel Level Analysis](https://youtu.be/e7C1NvaPLaY)\n\n#### 3. Cell clustering with Pixie  \nThe second step in the [Pixie](https://doi.org/10.1038/s41467-023-40068-5) pipeline is to run the [**cell clustering notebook**](./templates/3_Pixie_Cluster_Cells.ipynb). This notebook will use the pixel clusters generated in the first notebook to cluster the cells in your dataset. The notebook walks you through generating cell clusters for your data and generates cell cluster overlays. The notebook includes a GUI for manual cluster adjustment and annotation. [Workshop Talk - Session V - Cell-level Analysis - Part 2: Cell Clustering](https://youtu.be/4_AJxrxPYlk?t=2704)\n\n#### 4. Post Clustering Tasks\nAfter the Pixie Pipeline, the user can inspect and fine tune their results with the [**post clustering notebook**](./templates/4_Post_Clustering.ipynb). This notebook will go over cleaning up artifacts left from clustering, and working with functional markers.\n\n#### 5. Spatial Analysis  \n[Workshop Talk - Session VI - Spatial Analysis - Part 1: Choosing the Right Analysis Tool](https://youtu.be/HQAkS7kbzH8).\n\n  1. **Pairwise Enrichment Analysis**\n\n     The [**pairwise enrichment notebook**](templates/example_pairwise_spatial_enrichment.ipynb) allows the user to investigate the\n     interaction between the phenotypes present in their data. In addition users can\n     cluster based on phenotypes around a particular feature such as *artery* or *gland*. [Workshop Talk - Session VI - Spatial Analysis - Part 2: Pairwise Spatial Enrichment](https://youtu.be/HQAkS7kbzH8?t=619).\n\n  2. **K-means Neighborhood Analysis**\n\n     The [**neighborhood analysis notebook**](templates/example_neighborhood_analysis_script.ipynb) sheds light on neighborhoods made of micro-environments which consist of a collection of cell phenotypes. [Workshop Talk - Session VI - Spatial Analysis - Part 3: K-means Neighborhood Analysis](https://youtu.be/HQAkS7kbzH8?t=2117).\n\n  1. **Spatial LDA**\n\n     The [**preprocessing**](templates/LDA_Preprocessing.ipynb) and [**training / inference**](templates/LDA_Training_and_Inference.ipynb) draws from language analysis, specifically topic modelling. Spatial LDA overlays a probability distribution on cells belonging to a any particular micro-environment. [Workshop Talk - Session VI - Spatial Analysis - Part 4: Spatial LDA](https://youtu.be/HQAkS7kbzH8?t=3087).\n\n\n### Installation Steps\n\n\n#### Pip Installation\n\nYou can install the latest version of `ark` with:\n\n```sh\npip install ark-analysis\n```\n\nHowever, the repository will still need to be cloned if you wish to use the Jupyter Notebooks.\n\n#### Download the Repo\n\nYou can find all the versions available in the [Releases Section](https://github.com/angelolab/ark-analysis/releases).\nOpen terminal and navigate to where you want the code stored. \n\n\nIf you would like to use the latest version of `ark` simply clone the project and create the Conda environment.\n\n```sh\ngit clone -b v0.7.2 https://github.com/angelolab/ark-analysis.git\ncd ark-analysis\nconda env create -f environment.yml\n```\n\n\n#### Running on Windows\n\nOur repo runs best on Linux-based systems (including MacOS). If you need to run on Windows, please consult our [Windows guide](https://ark-analysis.readthedocs.io/en/latest/_rtd/windows_setup.html) for additional instructions.\n\n#### Using the Repository\n\nFirst, activate the `conda` environment inside the root of the `ark-analysis` folder:\n\n```sh\nconda activate ark_env\n```\n\nOnce activated, notebooks can be used via this command for Windows:\n\n```sh\nstart_jupyter.sh\n``` \n\nor this command for macOS:\n\n```sh\n./start_jupyter.sh\n```\n\nThis will automatically open your browser with a link to our Jupyter notebooks.\n\nBe sure to keep this terminal open.  **Do not exit the terminal or enter `control-c` until you are finished with the notebooks**. \n\nYou can shut down the notebooks by entering `control-c` in the terminal window.\n\n**REMEMBER TO DUPLICATE AND RENAME NOTEBOOKS**\n\nIf you didn't change the name of the notebooks within the `templates` folder, they will be overwritten when you decide to update the repo. Read about updating Ark [here](#updating-the-repository)\n\n### Updating the repo\n\nThe `ark-analysis` repo is constantly being updated. In order to get those changes to your version, you'll need to tell git to update with the following command:\n\n```\ngit pull\n```\n\nAfter performing the above command, you will sometimes need to update your environment:\n\n```shell\nconda remove --name ark_env --all\nconda env create -f environment.yml\n```\n\nTo update the notebooks, run this command for Windows:\n\n```shell\nstart_jupyter.sh -u\n```\n\nor this command for macOS:\n\n```shell\n./start_jupyter.sh -u\n```\n\n## External Tools\n\n### Mantis Viewer\n[Mantis](https://mantis.parkerici.org/) is a multiplexed image viewer developed by the Parker Institute. It has built in functionality for easily\nviewing multichannel images, creating overlays, and concurrently displaying image features alongisde raw channels. We have found it to be extremely useful for analying the output of our analysis pipeline. There are detailed instructions on \ntheir [download page](https://github.com/ParkerICI/mantis-viewer/releases) for how to install and use the tool. Below are some details specifically related to how we use it in `ark`. [Workshop Talk - Session V - Cell-level Analysis - Part 3: Assessing Accuracy with Mantis Viewer](https://youtu.be/4_AJxrxPYlk?t=4639).\n\n#### Mantis directory structure\nMantis expects image data to have a specific organization in order to display it. It is quite similar to how MIBI data is already stored, \nwith a unique folder for each FOV and all channels as individual tifs within that folder. Any notebooks that suggest using Mantis Viewer to inspect results\nwill automatically format the data in the format shown below. \n\n\n```sh\nmantis\n│ \n├── fov0\n│   ├── cell_segmentation.tiff\n│   ├── chan0.tiff\n│   ├── chan1.tiff\n│   ├── chan2.tiff\n│   ├── ...\n│   ├── population_mask.csv\n│   └── population_mask.tiff\n├── fov1\n│   ├── cell_segmentation.tiff\n│   ├── chan0.tiff\n│   ├── chan1.tiff\n│   ├── chan2.tiff\n│   ├── ...\n│   ├── population_mask.csv\n│   └── population_mask.tiff\n└── marker_counts.csv\n\n```\n\n#### Loading image-specific files\nIn addition to the images, there are additional files in the directory structure which can be read into mantis.  \n\n`cell_segmentation`: This file contains the predicted segmentation for each cell in the image, and allows mantis to identify individual cells.\n\n`population_pixel_mask`: This file maps the individual pixel clusters generated by Pixie in the [pixel clustering notebook](link) to the image data. \n\n`population_cell_mask`: Same as above, but for cell clusters instead of pixel clusters\n\nThese files should be specified when first initializing a project in mantis as indicated below:\n\n![](docs/_images/mantis_loading.png)\n\n#### Loading project-wide files\nWhen inspecting the output of the clustering notebooks, it is often useful to add project-wide .csv files, such as `marker_counts.csv`. These files contain \ninformation, such as the average expression of a given marker, across all the cells in the project. Project-wide files can either be loaded at project initialization, as shown below:\n\n![](docs/_images/mantis_feature_file.png)\n\nOr they can be loaded into an existing project via Import -\u003e Segment Features -\u003e For project from CSV\n\n\n#### View cell features\nOnce you have loaded the project-wide files into Mantis, you'll need to decide which of the features you want to view. \nClick on `Show Plot Plane` at the bottom right, then select the marker you want to assess. This will then allow you to view\nthe cell expression of that marker when you mouse over the cell in Mantis. \n\n![](docs/_images/mantis_feature_view.png)\n\n## Updating the Repository\n\nThis project is still under development, and we are making frequent changes and improvements. If you want to update the version on your computer to have the latest changes, perform the following steps. Otherwise, we recommend waiting for new releases.\n\nFirst, get the latest version of the repository.\n\n```sh\ngit pull\n```\n\nThen, run the command below to update the Jupyter notebooks to the latest version\n\n\nIf you have made changes to these notebooks that you would like to keep (specific file paths, settings, custom routines, etc), rename them before updating! \n\nFor example, rename your existing copy of `1_Segment_Image_Data.ipynb` to `1_Segment_Image_Data_old.ipynb`. Then, after running the update command, a new version of `1_Segment_Image_Data.ipynb` will be created with the newest code, and your old copy will exist with the new name that you gave it. \n\nAfter updating, you can copy over any important paths or modifications from the old notebooks into the new notebook.\n\n## Example Dataset\n\nIf you would like to test out the pipeline, then we have incorporated an example dataset within the notebooks. Currently the dataset contains 11 FOVs with 22 channels (CD3, CD4, CD8, CD14, CD20, CD31, CD45, CD68, CD163, CK17, Collagen1, ECAD, Fibronectin, GLUT1, H3K9ac, H3K27me3, HLADR, IDO, Ki67, PD1, SMA, Vim), and intermediate data necessary for each notebook in the pipeline.\n\nThe dataset is split into several smaller components, with each Jupyter Notebook using a combination of those components. We utilize [**Hugging Face**](https://huggingface.co) for storing the dataset and using their API's for creating these configurations. You can view the [dataset](https://huggingface.co/datasets/angelolab/ark_example)'s repository as well.\n\n### Dataset Compartments\n\n**Image Data:** This compartment stores the tiff files for each channel, for every FOV.\n```sh\nimage_data/\n├── fov0/\n│  ├── CD3.tiff\n│  ├── ...\n│  └── Vim.tiff\n├── fov1/\n│  ├── CD3.tiff\n│  ├── ...\n│  └── Vim.tiff\n├── .../\n```\n\n**Cell Table:** This compartment stores the various cell tables which get generated by Notebook 1.\n\n```sh\nsegmentation/cell_table/\n├── cell_table_arcsinh_transformed.csv\n├── cell_table_size_normalized.csv\n└── cell_table_size_normalized_cell_labels.csv\n```\n\n**Deepcell Output:** This compartment stores the segmentation images after running deepcell.\n```sh\nsegmentation/deepcell_output/\n├── fov0_whole_cell.tiff\n├── fov0_nuclear.tiff\n├── ...\n├── fov10_whole_cell.tiff\n└── fov10_nuclear.tiff\n```\n\n**Example Pixel Output:** This compartment stores feather files, csvs and pixel masks generated by pixel clustering.\n\n```sh\nsegmentation/example_pixel_output_dir/\n├── cell_clustering_params.json\n├── channel_norm.feather\n├── channel_norm_post_rowsum.feather\n├── pixel_thresh.feather\n├── pixel_channel_avg_meta_cluster.csv\n├── pixel_channel_avg_som_cluster.csv\n├── pixel_masks/\n│  ├── fov0_pixel_mask.tiff\n│  └── fov1_pixel_mask.tiff\n├── pixel_mat_data/\n│  ├── fov0.feather\n│  ├── ...\n│  └── fov10.feather\n├── pixel_mat_subset/\n│  ├── fov0.feather\n│  ├── ...\n│  └── fov10.feather\n├── pixel_meta_cluster_mapping.csv\n└── pixel_som_weights.feather\n```\n\n**Example Cell Output:** This compartment stores feather files, csvs and cell masks generated by cell clustering.\n\n```sh\nsegmentation/example_cell_output_dir/\n├── cell_masks/\n│  ├── fov0_cell_mask.tiff\n│  └── fov1_cell_mask.tiff\n├── cell_meta_cluster_channel_avg.csv\n├── cell_meta_cluster_count_avg.csv\n├── cell_meta_cluster_mapping.csv\n├── cell_som_cluster_channel_avg.csv\n├── cell_som_cluster_count_avg.csv\n├── cell_som_weights.feather\n├── cluster_counts.feather\n├── cluster_counts_size_norm.feather\n└── weighted_cell_channel.csv\n```\n\n### Dataset Configurations\n\n* **1 - Segment Image Data:**\n  * Image Data\n* **2 - Pixie Cluster Pixels:**\n  * Image Data\n  * Cell Table\n  * Deepcell Output\n* **3 - Pixie Cluster Cells:**\n  * Image Data\n  * Cell Table\n  * Deepcell Output\n  * Example Pixel Output\n* **4 - Post Clustering:**\n  * Image Data\n  * Cell Table\n  * Deepcell Output\n  * Example Cell Output\n\n## Questions?\n\nIf you have a general question or are having trouble with part of the repo, you can refer to our [FAQ](https://ark-analysis.readthedocs.io/en/latest/_rtd/faq.html) or head to the [discussions](https://github.com/angelolab/ark-analysis/discussions) tab to get help. If you've found a bug with the codebase, first make sure there's not already an [open issue][issue-tracker], and if not, you can then [open an issue](https://github.com/angelolab/ark-analysis/issues/new/choose) describing the bug.\n\n\n## Want to contribute?  \n\nIf you would like to help make `ark` better, please take a look at our [contributing guidelines](https://ark-analysis.readthedocs.io/en/latest/_rtd/contributing.html). \n\n## How to Cite\nPlease directly cite the `ark` repo (https://github.com/angelolab/ark-analysis) if it was a part of your analysis. In addition, please cite the relevant paper(s) below where applicable to your study. \n\n1. [Greenwald, Miller et al. Whole-cell segmentation of tissue images with human-level performance using large-scale data annotation and deep learning [2021]](https://www.nature.com/articles/s41587-021-01094-0)\n2. [Liu et al. Robust phenotyping of highly multiplexed tissue imaging data using pixel-level clustering [2023]](https://doi.org/10.1038/s41467-023-40068-5)\n\n\n\u003c!-- Links --\u003e\n\n\u003c!-- Badges --\u003e\n\n\u003c!-- Status --\u003e\n[link-badge-ci]: https://github.com/angelolab/ark-analysis/actions/workflows/ci.yml\n[link-badge-docs]: https://ark-analysis.readthedocs.io\n[link-badge-coverage]: https://coveralls.io/github/angelolab/ark-analysis?branch=main\n\n\u003c!-- Package  --\u003e \n[link-badge-pypi]: https://pypi.org/project/ark-analysis\n\n\n\u003c!-- Meta / License --\u003e\n[link-badge-license]: https://opensource.org/licenses/MIT\n[link-uv]: https://github.com/astral-sh/uv\n\n\u003c!-- Other --\u003e\n\n[issue-tracker]: https://github.com/angelolab/ark-analysis/issues","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fangelolab%2Fark-analysis","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fangelolab%2Fark-analysis","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fangelolab%2Fark-analysis/lists"}