{"id":21675798,"url":"https://github.com/nemoandrea/cryocare-hpc04","last_synced_at":"2025-03-20T09:22:24.112Z","repository":{"id":164588152,"uuid":"246308225","full_name":"NemoAndrea/cryoCARE-hpc04","owner":"NemoAndrea","description":"repository holding the files to be run on the HPC04 compute cluster","archived":false,"fork":false,"pushed_at":"2021-02-11T12:08:36.000Z","size":81,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-01-25T10:11:14.644Z","etag":null,"topics":["publication-code","tu-delft","tudelft"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/NemoAndrea.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-03-10T13:28:32.000Z","updated_at":"2024-07-23T08:48:58.000Z","dependencies_parsed_at":null,"dependency_job_id":"260a5e16-4940-479d-a141-3d566742c03d","html_url":"https://github.com/NemoAndrea/cryoCARE-hpc04","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NemoAndrea%2FcryoCARE-hpc04","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NemoAndrea%2FcryoCARE-hpc04/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NemoAndrea%2FcryoCARE-hpc04/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NemoAndrea%2FcryoCARE-hpc04/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/NemoAndrea","download_url":"https://codeload.github.com/NemoAndrea/cryoCARE-hpc04/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244583265,"owners_count":20476286,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["publication-code","tu-delft","tudelft"],"created_at":"2024-11-25T14:10:57.863Z","updated_at":"2025-03-20T09:22:24.078Z","avatar_url":"https://github.com/NemoAndrea.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# cryoCARE-hpc04\n\nThis repository holds an adapter version of the cryoCARE scripts originally developed by [the Jug Group](https://github.com/juglab) used to denoise cryo-electron tomograms.  Scripts require that the provided images originate from a `direct electron detector` (e.g. Gatan K2 Summit), in subframe form. This is because we need to be able to generate 2 images containing half the total signal at each tomogram `tilt angle`.\n\nThe scripts were built to be run on a `hpc`, but there is no reason why they cannot be run on a personal compute device. It will just take a while. Having a **fast graphics card** will really help in managing training time.\n\n## Installation\n\nIt is recommended that you install the required dependencies via a separate **[Anaconda/Miniconda](https://docs.anaconda.com/anaconda/install/)** environment. The required dependencies are listed in `requirements.yml` and can easily be installed in a new conda environment using:\n\n``` \nconda env create -f requirements.yml -n \u003cname_for_your_new_environment\u003e\n```\n\nThen run \n\n`conda activate \u003cname_for_your_new_environment\u003e` every time you want to use the scripts to make sure you are using that environment.\n\nIn addition, you need to have `IMOD` and `MotionCor2` installed on your system. \n\nMotionCor2 (tested with `v1.0.5`) can be downloaded from the [UCSF EM core download page](https://emcore.ucsf.edu/ucsf-software). You will have to complete a form and after that you will be provided with a download.\n\nIMOD (tested with `v4.9.2` ) can be downloaded from the [IMOD website](https://bio3d.colorado.edu/imod/download.html). _If you are running from Windows, you will also need to install [Cygwin](https://www.cygwin.com/) to use it_\n\n\u003e It is possible to make things work for IMOD `4.11.x` using the `etomo --namingstyle 0` compatibility argument. \n\n## Structure and use\n\nThe main workflow is based on `Jupyter notebooks`. These can be found in the `T2T` directory. \n\n\u003e The `.sh` files (root directory) are specific to the cluster an its installed packages and are therefore not very useful for general use\n\nI recommend you copy the `T2T` directory (and its contents) for each dataset you want to process. That way you can always come back to the notebooks and check what options you picked, etc.\n\n\u003e So, clone this repository, do not touch the code in it, and just **copy the `T2T` folder** every time you want to process a new dataset. Treat this repository as a template, avoid editing files directly here\n\nSo, for each tilt series, create a folder with a name for that tomogram (e.g. `tomogram_\u003ccondition\u003e_\u003cdate\u003e_number_4`), and copy the `T2T` folder into it. You should then have the following structure.\n```\ntomogram_name_or_similar\n|   └─── T2T\n|           └─── frames/\n|           |---- [xx]_notebook.ipynb \n|           |---- [xy]_notebook.ipynb \n|           ... \n```\n\nThen copy your tilt images in stack form (`.tif` or `.mrc`) into the `frames/` folder. After having copied the data, you run the numbered notebook files (i.e. `.ipynb`) in numerical order. This means starting with `01_Split_Frames.ipynb`. \n\nIf you have never used a `Jupyter notebook`, there are many online introductions such as [this one](https://realpython.com/jupyter-notebook-introduction/#starting-the-jupyter-notebook-server). To start a notebook, simply navigate into the `T2T` directory in your`Anaconda` terminal and type\n\n```\njupyter notebook\n```\n\nThis should open a notebook in browser, and you can follow the instructions from there until completion. Go through the files and adjust paths as needed, and move on to the next one after completion.\n\nA short summary of what these notebooks do can be found below. I suggest you read that first before running anything.\n\n## Procedure in brief\n\nThe list of steps we have to do (with corresponding notebook number) is as follows:\n* [01] Split and align the raw DED frames into `even` and `odd` set (alignment using MotionCor2) \n* [02] Reconstruct two tomograms \n* [03] Sample subvolumes of the tomograms into training and validation set \n* [04] Train the network **(this takes the longest)**\n* [05] Run the denoiser network on tomograms \n\n### Extras\n\n* [05b] this will allow you to batch process a series of reconstructed tomograms. (denoise many using a single network)\n* [06] free up storage space\n\n## 2D training mode\n\nThis modified version of cryoCARE has the option to denoise using only 2D slices. This can be beneficial in combination with `SIRT` tomogram reconstruction  (an option in IMOD). This is done by selecting `type2D` in `[03]_training_data_generation` .\n\nGive it a shot in combination with `SIRT` and compare it to the baseline 3D without SIRT. \n\n## Credit\n\nThis repository is based on the original code available [at the juglab github](https://github.com/juglab/cryoCARE_T2T), and their [associated publication](https://ieeexplore.ieee.org/document/8759519): \n\n```\nBuchholz, Tim-Oliver, et al. \"Cryo-care: Content-aware image restoration for cryo-transmission electron microscopy data.\" 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019). IEEE, 2019.\n```\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnemoandrea%2Fcryocare-hpc04","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnemoandrea%2Fcryocare-hpc04","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnemoandrea%2Fcryocare-hpc04/lists"}