{"id":13687922,"url":"https://github.com/theislab/single-cell-tutorial","last_synced_at":"2025-05-15T20:02:00.813Z","repository":{"id":42204020,"uuid":"157229912","full_name":"theislab/single-cell-tutorial","owner":"theislab","description":"Single cell current best practices tutorial case study for the paper:Luecken and Theis, \"Current best practices in single-cell RNA-seq analysis: a tutorial\" ","archived":false,"fork":false,"pushed_at":"2022-12-11T23:19:01.000Z","size":47280,"stargazers_count":1465,"open_issues_count":21,"forks_count":473,"subscribers_count":44,"default_branch":"master","last_synced_at":"2025-04-08T01:37:53.159Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/theislab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-11-12T14:55:58.000Z","updated_at":"2025-04-03T15:28:58.000Z","dependencies_parsed_at":"2022-07-10T04:00:36.467Z","dependency_job_id":null,"html_url":"https://github.com/theislab/single-cell-tutorial","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/theislab%2Fsingle-cell-tutorial","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/theislab%2Fsingle-cell-tutorial/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/theislab%2Fsingle-cell-tutorial/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/theislab%2Fsingle-cell-tutorial/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/theislab","download_url":"https://codeload.github.com/theislab/single-cell-tutorial/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254414461,"owners_count":22067264,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-02T15:01:03.405Z","updated_at":"2025-05-15T20:01:59.760Z","avatar_url":"https://github.com/theislab.png","language":"Jupyter Notebook","funding_links":[],"categories":["Jupyter Notebook","Useful Links:","Tutorials and workflows"],"sub_categories":["Data Analysis Pipeline","Spatial transcriptomics"],"readme":"# Scripts for \"Current best-practices in single-cell RNA-seq: a tutorial\"\n\n\u003e **Note**\n\u003e The \"current\" best practices that are detailed in this workflow were set up in 2019. Thus, they do not necessarily follow the latest best practices for scRNA-seq analysis anymore.\n\u003e For an up-to-date version of the latest best practices for single-cell RNA-seq analysis (and more modalities) please see our consistently updated online book: https://www.sc-best-practices.org.\n\u003e\n\u003e For more information and contribution guidelines please visit the associated Github repository: https://github.com/theislab/single-cell-best-practices\n\n![image](https://drive.google.com/uc?export=view\u0026id=1YoX3F8gNGH5K0AFu4wFdd5ccEXT-mP_J)\n\nThis repository is complementary to the publication:\n\nM.D. Luecken, F.J. Theis, \"Current best practices in single-cell RNA-seq analysis: a tutorial\", Molecular Systems Biology 15(6) (2019): e8746\n\nThe paper was recommended on F1000 prime as being of special significance in the field.\n\n\u003ca href=\"https://f1000.com/prime/736010853?bd=1 \u0026ui=150648 \" target=\"_blank\"\u003e\u003cimg src=\"https://s3.us-east-1.amazonaws.com/cdn.f1000.com/images/badges/badgef1000.gif\" alt=\"Access the recommendation on F1000Prime\" id=\"bg\" /\u003e\u003c/a\u003e\n\n\nThe repository contains:\n- scripts to generate the paper figures\n- a case study which complements the manuscript\n- the code for the marker gene detection study from the supplementary material\n\nThe main part of this repository is a case study where the best-practices established in the manuscript are applied to a mouse intestinal epithelium regions dataset from Haber et al., Nature 551 (2018) available from the GEO under GSE92332. This case study can be found in different versions in the `latest_notebook/` and `old_releases/` directories.\n\nThe scripts in the plotting_scripts/ folder reproduce the figures that are shown in the manuscript and the supplementary materials. These scripts contain comments to explain each step. Each figure that does not have a corresponding script in the plotting_scripts/ folder was taken from the case study or the marker gene study.\n\nIn case of questions or issues, please get in touch by posting an issue in this repository.\n\nIf the materials in this repo are of use to you, please consider citing the above publication.\n\n\n## Environment set up\n\nA docker container with a working sc-tutorial environment is now available [here](https://hub.docker.com/r/leanderd/single-cell-analysis) thanks to [Leander Dony](https://github.com/le-ander). If you would like to set up the environment via conda or manually outside of the docker container, please follow the instructions below.\n\nTo run the tutorial case study, several packages must be installed. As both `R` and `python` packages are required, we prefer using a conda environment. To facilitate the setup of a conda environment, we have provided the `sc_tutorial_environment.yml` file, which contains all conda and pip installable dependencies. R dependencies, which are not already available as conda packages, must be installed into the environment itself.\n\n\nTo set up a conda environment, the following instructions must be followed.\n\n1. Set up the conda environment from the `sc_tutorial_environment.yml` file.\n\n    ```\n    conda env create -f sc_tutorial_environment.yml\n    ```\n\n2. Ensure that the environment can find the `gsl` libraries from R. This is done by setting the `CFLAGS` and `LDFLAGS` environment variables (see https://bit.ly/2CjJsgn). Here we set them so that they are correctly set every time the environment is activated.\n\n    ```\n    cd YOUR_CONDA_ENV_DIRECTORY\n    mkdir -p ./etc/conda/activate.d\n    mkdir -p ./etc/conda/deactivate.d\n    touch ./etc/conda/activate.d/env_vars.sh\n    touch ./etc/conda/deactivate.d/env_vars.sh\n    ```\n\n    Where YOUR_CONDA_ENV_DIRECTORY can be found by running `conda info --envs` and using the directory that corresponds to your conda environment name (default: sc-tutorail).\n\n    WHILE NOT IN THE ENVIRONMENT(!!!!) open the `env_vars.sh` file at `./etc/conda/activate.d/env_vars.sh` and enter the following into the file:\n\n    ```\n    #!/bin/sh\n    \n    CFLAGS_OLD=$CFLAGS\n    export CFLAGS_OLD\n    export CFLAGS=\"`gsl-config --cflags` ${CFLAGS_OLD}\"\n     \n    LDFLAGS_OLD=$LDFLAGS\n    export LDFLAGS_OLD\n    export LDFLAGS=\"`gsl-config --libs` ${LDFLAGS_OLD}\"\n    ```\n    \n    Also change the `./etc/conda/deactivate.d/env_vars.sh` file to:\n\n    ```\n    #!/bin/sh\n     \n    CFLAGS=$CFLAGS_OLD\n    export CFLAGS\n    unset CFLAGS_OLD\n     \n    LDFLAGS=$LDFLAGS_OLD\n    export LDFLAGS\n    unset LDFLAGS_OLD\n    ```\n    \n    Note again that these files should be written WHILE NOT IN THE ENVIRONMENT. Otherwise you may overwrite the CFLAGS and LDFLAGS environment variables in the base environment!\n\n3. Enter the environment by `conda activate sc-tutorial` or `conda activate ENV_NAME` if you changed the environment name in the `sc_tutorial_environment.yml` file.\n\n4. Open R and install the dependencies via the commands:\n\n    ```\n    install.packages(c('devtools', 'gam', 'RColorBrewer', 'BiocManager'))\n    update.packages(ask=F)\n    BiocManager::install(c(\"scran\",\"MAST\",\"monocle\",\"ComplexHeatmap\",\"slingshot\"), version = \"3.8\")\n    ```\n \nThese steps should set up an environment to perform single cell analysis with the tutorial workflow on a Linux system. Please note that we have encountered issues with conda environments on Mac OS. When using Mac OS we recommend installing the packages without conda using separately installed `python` and `R` versions. Alternatively, you can try using the base conda environment and installing all packages as described in the `conda_env_instructions_for_mac.txt` file. In the base environment, R should be able to find the relevant gsl libraries, so `LDFLAGS` and `CFLAGS` should not need to be set.\n\nAlso note that conda and pip doesn't always play nice together. Conda developers have suggested first installing all conda packages and then installing pip packages on top of this where conda packages are not available. Thus, installing further conda packages into the environment may cause issues. Instead, start a new environment and reinstall all conda packages first.\n\nIf you prefer to set up an environment manually, a list of all package requirements are given at the end of this document.\n\n\n## Downloading the data\n\nAs mentioned above the data for the case study comes from GSE92332. To run the case study as shown, you must download this data and place it in the correct folder. Unpacking the data requires `tar` and `gunzip`, which should already be available on most systems. If you are cloning the github repository and have the case study script in a `latest_notebook/` folder, then from the location where you store the case study ipynb file, this can be done via the following commands:\n\n```\ncd ../  #To get to the main github repo folder\nmkdir -p data/Haber-et-al_mouse-intestinal-epithelium/\ncd data/Haber-et-al_mouse-intestinal-epithelium/\nwget ftp://ftp.ncbi.nlm.nih.gov/geo/series/GSE92nnn/GSE92332/suppl/GSE92332_RAW.tar\nmkdir GSE92332_RAW\ntar -C GSE92332_RAW -xvf GSE92332_RAW.tar\ngunzip GSE92332_RAW/*_Regional_*\n```\n\nThe annotated dataset with which we briefly compare the results at the end of the notebook, is available from the same GEO accession ID (GSE92332). It can be obtained using the following command:\n\n```\ncd data/Haber-et-al_mouse-intestinal-epithelium/\nwget ftp://ftp.ncbi.nlm.nih.gov/geo/series/GSE92nnn/GSE92332/suppl/GSE92332_Regional_UMIcounts.txt.gz\ngunzip GSE92332_Regional_UMIcounts.txt.gz\n```\n\n\n## Case study notes\n\nWe have noticed that results such as visualization, dimensionality reduction, and clustering (and hence all downstream results as well) can give slightly different results on different systems. This has to do with the numerical libraries that are used in the backend. Thus, we cannot guarantee that a rerun of the notebook will generate exactly the same clusters.\n\nWhile all results are qualitatively similar, the assignment of cells to clusters especialy for stem cells, TA cells, and enterocyte progenitors can differ between runs across systems. To show the diversity that can be expected, we have uploaded shortened case study notebooks to the `alternative_clustering_results/` folder.\n\nNote that running `sc.pp.pca()` with the parameter `svd_solver='arpack'` drastically reduces the variability between systems, however the output is not exactly the same.\n\n\n## Adapting the pipeline for other datasets:\n\nThe pipeline was designed to be easily adaptable to new datasets. However, there are several limitations to the general applicability of the current workflow. When adapting the pipeline for your own dataset please take into account the following:\n\n1. Sparse data formats are not supported by `rpy2` and therefore do not work with any of the integrated R commands. Datasets can be turned into a dense format using the code: `adata.X = adata.X.toarray()`\n\n2. The case study assumes that the input data is count data obtained from a single-cell protocol with UMIs. If the input data is full-length read data, then one could consider replacing the normalization method with another method that includes gene length normalization (e.g., TPM).\n\n\n## Manual installation of package requirements\n\nThe following packages are required to run the first version of the case study notebook. For further versions see the README.md in the latest_notebook/ and old_releases/ folders.\n\nGeneral:\n- Jupyter notebook\n- IRKernel\n- rpy2\n- R \u003e= 3.4.3\n- Python \u003e= 3.5\n\nPython:\n- scanpy\n- numpy\n- scipy\n- pandas\n- seaborn\n- louvain\u003e=0.6\n- python-igraph\n- gprofiler-official (from Case study notebook 1906 version)\n- python-gprofiler from Valentine Svensson's github (vals/python-gprofiler)\n  - only needed for notebooks before version 1906\n- ComBat python implementation from Maren Buettner's github (mbuttner/maren_codes/combat.py)\n  - only needed for scanpy versions before 1.3.8 which don't include `sc.pp.combat()`\n\nR:\n- scater\n- scran\n- MAST\n- gam\n- slingshot (change DESCRIPTION file for R version 3.4.3)\n- monocle 2\n- limma\n- ComplexHeatmap\n- RColorBrewer\n- clusterExperiment\n- ggplot2\n- IRkernel\n\n## Possible sources of error in the manual installation:\n\n#### For R 3.4.3:\n\nWhen using Slingshot in R 3.4.3, you must pull a local copy of slingshot via the github repository and change the `DESCRIPTION` file to say `R\u003e=3.4.3` instead of `R\u003e=3.5.0`.\n\n#### For R \u003e= 3.5 and bioconductor \u003e= 3.7:\n\nThe clusterExperiment version that comes for bioconductor 3.7 has slightly changed naming convention. `clusterExperiment()` is now called `ClusterExperiment()`. The latest version of the notebook includes this change, but when using the original notebook, please note that this may throw an error.\n\n#### For rpy2 \u003c 3.0.0:\n\nPandas 0.24.0 is not compatible with rpy2 \u003c 3.0.0. When using old versions of rpy2, please downgrade pandas to 0.23.X. Please also note that Pandas 0.24.0 requires anndata version 0.6.18 and scanpy version \u003e 1.37.0.\n\n#### For enrichment analysis with g:profiler:\n\nEnsure that the correct g:profiler package is used for the notebook. Notebooks until 1904 use `python-gprofiler` from valentine svensson's github, and Notebooks from 1906 use the `gprofiler-official` package from the g:profiler team.\n\n#### If not R packages can be found:\n\nEnsure that IRkernel has linked the correct version of R with your jupyter notebook. Check instructions at `https://github.com/IRkernel/IRkernel`.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftheislab%2Fsingle-cell-tutorial","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftheislab%2Fsingle-cell-tutorial","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftheislab%2Fsingle-cell-tutorial/lists"}