{"id":20063184,"url":"https://github.com/alan-turing-institute/grace","last_synced_at":"2025-05-05T17:32:29.634Z","repository":{"id":154758098,"uuid":"563785485","full_name":"alan-turing-institute/grace","owner":"alan-turing-institute","description":"Graph Representation Analysis for Connected Embeddings","archived":false,"fork":false,"pushed_at":"2023-12-01T16:21:37.000Z","size":63439,"stargazers_count":34,"open_issues_count":50,"forks_count":0,"subscribers_count":6,"default_branch":"main","last_synced_at":"2024-06-11T16:27:10.791Z","etag":null,"topics":["computer-vision","data-science","feature-extraction","graphical-models","image-processing","latent-representations","machine-learning","neural-networks","object-detection"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/alan-turing-institute.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2022-11-09T10:32:14.000Z","updated_at":"2024-05-10T21:22:59.000Z","dependencies_parsed_at":"2023-10-14T16:42:44.010Z","dependency_job_id":"0eebed0c-aaae-443a-87e9-4ab290e60413","html_url":"https://github.com/alan-turing-institute/grace","commit_stats":{"total_commits":595,"total_committers":9,"mean_commits":66.11111111111111,"dds":"0.36302521008403366","last_synced_commit":"070f3303dc144cc972b541e4327c9059cf8e55ec"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alan-turing-institute%2Fgrace","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alan-turing-institute%2Fgrace/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alan-turing-institute%2Fgrace/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alan-turing-institute%2Fgrace/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/alan-turing-institute","download_url":"https://codeload.github.com/alan-turing-institute/grace/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224457264,"owners_count":17314441,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["computer-vision","data-science","feature-extraction","graphical-models","image-processing","latent-representations","machine-learning","neural-networks","object-detection"],"created_at":"2024-11-13T13:40:52.901Z","updated_at":"2024-11-13T13:40:53.482Z","avatar_url":"https://github.com/alan-turing-institute.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/charliermarsh/ruff/main/assets/badge/v1.json)](https://github.com/charliermarsh/ruff)\n[![Black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)\n[![pre-commit](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit\u0026logoColor=white)](https://github.com/pre-commit/pre-commit)\n[![Actions status](https://github.com/alan-turing-institute/grace/workflows/CI/badge.svg)](https://github.com/alan-turing-institute/grace/actions)\n\n\n# GRACE - Graph Representation Analysis for Connected Embeddings 🌐 📊 🤓\n\n\u003cimg width=\"40%\" align=\"right\" alt=\"project logo\" src=\"./assets/logo_white.png\"/\u003e\n\n\nThis `grace` repository contains a Python library 🐍 for identification of patterns in imaging data. The package provides a method 🖥️ to find connected objects \u0026 regions of interest in images by constructing graph-like representations 🌐 .\n\n*Read more about:*\n+ the [science](#science) behind this project 👩‍🔬👨‍🔬,\n+ the [workflow](#workflow) of the individual steps 👩‍💻👨‍💻\n+ the [contributors](#contributors) participating in the project design\n    + how to bring in your own ideas\n    + how to provide your feedback\n+ don't forget to give us a '⭐' -\u003e 😉\n\n---\n\n\n## Science\n\nThe acronym `grace` stands for  __G__ raph __R__ epresentation __A__ nalysis for __C__ onnected __E__ mbeddings 📈📉. This tool was developed by researchers as a scientific project at The Alan Turing Institute in the [Data Science for Science programme](https://www.turing.ac.uk/research/research-programmes/data-science-science-and-humanities).\n\nAs the initial use case, we (see the [list of contributors](#contributors) below) developed `grace` for localising filaments in cryo-electron microscopy (cryoEM) imaging datasets as an image processing tool that automatically identifies filamentous proteins and locates the regions of interest, an accessory or binding protein.\n\nFind out more details about the project aims \u0026 objectives [here](https://www.turing.ac.uk/research/research-projects/machine-learning-and-large-cryogenic-electron-microscopy-data-sets) \u0026 [here](https://www.turing.ac.uk/research/research-projects/molecular-structure-images-under-physical-constraints) or visit the [citation](#citation) panel below to check out the overarching research projects.\n\n---\n\n\n## Workflow\n\n\u003cimg width=\"100%\" align=\"left\" alt=\"workflow steps\" src=\"./assets/workflow_steps.png\"/\u003e\n\nThe `grace` workflow consists of the following steps:\n\n1. Image data acquisition (_e.g._ cryo-electron microscopy)\n2. Object detection via bounding boxes (_e.g._ [crYOLO](https://cryolo.readthedocs.io/en/stable/), [RELION](https://github.com/3dem/relion), or [FasterRCNN](https://arxiv.org/pdf/1506.01497.pdf))\n3. Organisation of the bounding boxes as nodes connected via edges as a 2D graph structure (_e.g._ [Delaunay triangulation](https://docs.scipy.org/doc/scipy/reference/generated/scipy.spatial.Delaunay.html))\n4. Cropping of image patches (at various scales) from each bounding box detected in the image\n5. Latent feature extraction from image patches (_e.g._ pre-trained neural network, such as [_ResNet-152_](https://pytorch.org/vision/main/models/generated/torchvision.models.resnet152.html))\n6. *'Human-in-the-loop'* annotation of the desired pattern in the image data (see the [napari plugin](#development) below)\n7. Classification of each 'nodeness' and 'edgeness' confidence via deep neural network classifiers. The neural network can be applied to a full graph, or subgraphs around each node (_e.g._ using immediate [1-hop neighbourhood](https://arxiv.org/pdf/1907.06051.pdf)).\n8. Combinatorial optimisation via integer linear programming (ILP) to connect the candidate object nodes via edges (see the [expected outcomes](#outcomes) below)\n9. Quantitative evaluation of the filament detection performance\n10. Ta-da! 🥳\n\n---\n\n\n## Installation\n\n`grace` has been tested with Python 3.8+ on OS X.\n\nFor local development, clone the repo and install in editable mode following these guidelines:\n\n*Note:* Choose which conda environment you'd like to use:\n* If you need to annotate your data (images) in *napari*, we recommend the `grace-env-with-napari` -\u003e `environment-with-napari.yaml`\n* If you rely on previously annotated data and do not require *napari*, we recommend the `grace-env-napari-free` -\u003e `environment-napari-free.yaml`\n\nSpecify your preference \u0026 follow the steps below:\n\n```sh\n# clone the grace GitHub repository\n\ngit clone https://github.com/alan-turing-institute/grace.git\ncd ./grace\n\n# create a conda playground from the respective environment.yaml\nconda env create -f YOUR-CHOSEN-ENVIRONMENT.yaml\n\n# To activate this environment, use\n#\n#     $ conda activate grace-env-with-napari\n#     OR\n#     $ conda activate grace-env-napari-free\n#\n# To deactivate an active environment, use\n#\n#     $ conda deactivate\n\nconda activate grace-env-OF-YOUR-CHOICE\n\n# install grace from local folder (not on pypi yet)\n\npip install -e \".[dev]\"\n\n# install pre-commit separately\n\nconda install -c conda-forge pre_commit\n\n# follow the hooks from .pre-commit-config.yaml\n\npre-commit install\n\n```\n\n*Note:*  when exporting your own grace conda environment, use the following:\n\n```sh\nconda env export --no-builds \u003e new_environment.yaml\n```\n\nThis will allow environments to be shared between different platforms and OS. For a new install with a grace version not on pypi, please remove `grace` from the requirements under `pip` within the newly created yaml file.\n\n---\n\nIf you currently do not have any data to test / implement GRACE on, have a look at the option of **simulating a synthetic dataset** as described in [this](./grace/simulator/README.md) README. An accessible link to some pre-annotated simulated images is coming soon! 🚧\n\n\n## Annotator GUI\n\nOur repository contains a graphical user interface (GUI) which allows the user to manually annotate the regions of interests (motifs) in their cryo-EM data.\n\nTo test the annotator, make sure you've [installed](#installation) the repository using the annotation environment \u0026 navigate to:\n\n```sh\npython examples/show_data.py\n```\n\n\nhttps://user-images.githubusercontent.com/48791041/233156173-cf2a69d3-d4be-4ba1-ae57-aebf6b9501cc.mov\n\n_Demonstration of the napari widget to annotate cryo-EM images._\n\n\nThe recording above 👆 shows a *napari*-based GUI widget for annotation of the desired motifs, in our case, filamentous proteins. Follow these steps to test the plugin out:\n\n1. Build the graph from all vertices (node, white circle) using the `'build graph'` function in the right-hand panel.\n2. Navigates the triangulated graph by zooming in/out or moving along the image from either the `'nodes_...'` or `'edges_...'` layer list.\n3. Choose the `'annotation_...'` layer in the left-hand layer list and click on the 'brush'🖌️ icon at the top of the layer control.\n4. Annotate nodes belonging to object instances by drawing over the nodes in a continuous line.\n5. Identify edges within connected objects (green 🟩 lines) _versus_ edges outside of annotated objects (magenta 🟪 lines) by cutting the graph using the `'cut graph'` function in the right-hand panel.\n6. In case of an annotation error ❌, choose the eraser icon at the top of the layer control to erase incorrect annotations. Re-cut the graph until you are happy with the overall annotation of the image.\n7. _Note:_ Not every single node / object has to be accounted for when annotating, take it easy 😎.\n8. Once happy with the annotations, save them out by exporting via the `'export...'` button on the right-hand side. Inversely, you can load previously saved annotations using the `'import...'` button.\n9. Ta-da! 🥳\n\n---\n\n\n## Outcomes\n\n🚧 **Work in progress** 🚧\n\nThe expected outcome of the `grace` workflow is to identify all connected objects as individual filament instances. We tested the combinatorial optimisation step on simulated data with 3 levels of 'line-seeding' densities: dense, medium and sparse.\n\n\u003cimg width=\"100%\" align=\"left\" alt=\"optimising dummy graphs\" src=\"./assets/optimiser_dummy.png\"\u003e\n\nAs you can see, the optimiser works well to identify filamentous object instances simulated at various densities, and appears to work across object cross-overs (middle image, pink objects).\n\nMore details about how this type of graph representation analysis could be applied to other image data processing will become available soon - stay tuned! 😎👌\n\n---\n\n\n## Contributors\n\n**Methodology / software development [The Alan Turing Institute]:**\n\n+ 👩‍💻 [Beatriz Costa Gomes](https://github.com/mooniean \"mooniean\")\n+ 👩‍💻 [Kristina Ulicna](https://github.com/KristinaUlicna \"KristinaUlicna\")\n+ 👨‍💻 [Christorpher J Soelistyo](https://github.com/chris-soelistyo \"chris-soelistyo\")\n+ 👩‍💻 [Camila Rangel Smith](https://github.com/crangelsmith \"crangelsmith\")\n+ 👩‍💻 [Marjan Famili](https://github.com/marjanfamili \"marjanfamili\")\n+ 👨‍💻 [Alan R Lowe](https://github.com/quantumjot \"quantumjot\")\n\n\n**Dataset generation / processing [The University of Bristol]:**\n\n+ 👨‍🔬 [Marston Bradshaw](https://research-information.bris.ac.uk/en/persons/marston-bradshaw \"Marston Bradshaw\")\n+ 👩‍🔬 [Danielle Paul](https://www.turing.ac.uk/people/researchers/danielle-paul \"Danielle Paul\")\n\n\n...and many others...\n\nIf you'd like to contribute to our ongoing work, please do not hesitate to let us know your suggestions for potential improvements by [raising an issue on GitHub](https://github.com/alan-turing-institute/grace/issues \"Grace GitHub | Issues\").\n\n---\n\n\n## Citation\n\n🚧 **Work in progress** 🚧\n\n[![Project:ML_for_CryoEM](https://img.shields.io/badge/Project-Machine_Learning_for_CryoEM-blue)](https://www.turing.ac.uk/research/research-projects/machine-learning-and-large-cryogenic-electron-microscopy-data-sets)\n\n[![Project:Mol_Structures](https://img.shields.io/badge/Project-Molecular_Structure_Imaging-blue)](https://www.turing.ac.uk/research/research-projects/molecular-structure-images-under-physical-constraints)\n\n\nWe are currently writing up our methodology and key results, so please stay tuned for future updates!\n\nIn the meantime, please use the template below to cite our work:\n\n```\n@unpublished{grace_repository,\n    year = {2023},\n    month = {April},\n    publisher = {{CCP-EM} Collaborative Computational Project for Electron cryo-Microscopy},\n    howpublished = {Paper presented at the 2023 {CCP-EM} Spring Symposium},\n    url = {https://www.ccpem.ac.uk/downloads/symposium/ccp-em_symp_schedule_2023.pdf},\n    author = {Beatriz Costa-Gomes, Kristina Ulicna, Christorpher Soelistyo, Marjan Famili, Alan Lowe​},\n    title = {Deconstructing cryoEM micrographs with a graph-based analysis for effective structure detection},\n    abstract = {Reliable detection of structures is a fundamental step in analysis of cryoEM micrographs.\n    Despite intense developments of computational approaches in recent years, time-consuming hand annotating\n    remains inevitable and represents a rate-limiting step in the analysis of cryoEM data samples with\n    heterogeneous objects. Furthermore, many of the current solutions are constrained by image characteristics:\n    the large sizes of individual micrographs, the need to perform extensive re-training of the detection models\n    to find objects of various categories in the same image dataset, and the presence of artefacts that might\n    have similar shapes to the intended targets.\n    To address these challenges, we developed GRACE (Graph Representation Analysis for Connected Embeddings),\n    a computer vision-based Python package for identification of structural motifs in complex imaging data.\n    GRACE sources from large images populated with low-fidelity object detections to build a graph representation\n    of the entire image. This global graph is then traversed to find structured regions of interest via extracting\n    latent node representations from the local image patches and connecting candidate objects in a supervised manner\n    with a graph neural network.\n    Using a human-in-the-loop approach, the user is encouraged to annotate the desired motifs of interest, making\n    our tool agnostic to the type of object detections. The user-nominated structures are then localised and\n    connected using a combinatorial optimisation step, which uses the latent embeddings to decide whether the\n    graph nodes belong to an object instance.\n    Importantly, GRACE reduces the search space from millions of pixels to hundreds of nodes, which allows for\n    fast and efficient implementation and potential tool customisation. In addition, our method can be repurposed\n    to search for different motifs of interest within the same dataset in a significantly smaller time scale to\n    the currently available open-source methods. We envisage that our end-to-end approach could be extended to\n    other types of imaging data where object segmentation and detection remains challenging.}\n}\n```\n\n---\n\n### _Happy graphing!_ 🎮\n- Your GRACE development research team 👋\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falan-turing-institute%2Fgrace","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Falan-turing-institute%2Fgrace","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falan-turing-institute%2Fgrace/lists"}