{"id":22794807,"url":"https://github.com/menchelab/bioprofilingnotebooks","last_synced_at":"2025-10-10T21:07:19.681Z","repository":{"id":77421016,"uuid":"335374291","full_name":"menchelab/BioProfilingNotebooks","owner":"menchelab","description":null,"archived":false,"fork":false,"pushed_at":"2022-10-14T13:15:42.000Z","size":50217,"stargazers_count":0,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-30T17:46:24.604Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/menchelab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-02-02T17:46:15.000Z","updated_at":"2021-11-09T18:13:57.000Z","dependencies_parsed_at":null,"dependency_job_id":"3fb2c7b6-ada8-4d19-83b1-a2d4325f2afc","html_url":"https://github.com/menchelab/BioProfilingNotebooks","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/menchelab/BioProfilingNotebooks","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/menchelab%2FBioProfilingNotebooks","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/menchelab%2FBioProfilingNotebooks/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/menchelab%2FBioProfilingNotebooks/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/menchelab%2FBioProfilingNotebooks/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/menchelab","download_url":"https://codeload.github.com/menchelab/BioProfilingNotebooks/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/menchelab%2FBioProfilingNotebooks/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":270940963,"owners_count":24671745,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-18T02:00:08.743Z","response_time":89,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-12T04:10:30.589Z","updated_at":"2025-10-10T21:07:14.663Z","avatar_url":"https://github.com/menchelab.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# BioProfiling.jl analysis notebooks\n\n\n[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.5659932.svg)](https://doi.org/10.5281/zenodo.5659932)\n\nDemonstration of [_BioProfiling.jl_](https://github.com/menchelab/BioProfiling.jl) and robust statistics for morphological cell profiling using high-content imaging. Analyses described in the article [\"BioProfiling.jl: Profiling biological perturbations with high-content imaging in single cells and heterogeneous populations\"](https://doi.org/10.1093/bioinformatics/btab853) by Loan Vulliard, Joel Hancock, Anton Kamnev, Christopher W. Fell, Joana Ferreira da Silva, Joanna I. Loizou, Vanja Nagy, Loïc Dupré and Jörg Menche.\n\n## Summary\n\nThis collection of notebooks includes the analysis of a high-content imaging chemical screen, the comparison of the morphological activity of compounds on cells seeded at two densities, as well as the comparison of genetic overexpression in two plates in a genetic screen following the Cell Painting assay. This demonstrates how to curate morphological profiles and perform common tasks for downstream analyses using _BioProfiling.jl_. All details can be found [in the corresponding publication](https://doi.org/10.1093/bioinformatics/btab853). For the content of each notebook, see the section \"Reproducing example analyses\" below, and the header of each notebook.\n\n## Prerequisites\n\n* Create a subfolder called `fig`, where the generated figures will be saved.\n* Create a subfolder called `data`, where the input and intermediate files will be stored.\n* Download the input data [from Figshare](https://doi.org/10.6084/m9.figshare.14784678.v2). Only the file `data.zip` is necessary to reproduce the core results of the analysis described in this notebook.\n* Copy the transfer list and morphological measurement files in the `data` folder.\n* (Optional) Mount or copy the raw images in a folder if you want to use the visual diagnostics features.\n* (Optional) If you want to run the minimal example in the *FigS3_LUAD*, you will need additional external data. The instructions are provided in the notebook itself.\n* (Optional) If you only want to reproduce parts of the analysis, you can copy the intermediate files [from Figshare](https://doi.org/10.6084/m9.figshare.14784678.v2) into the `data` folder.\n\n## Running the code using Docker\n\n### General instructions\n\nThis repository compiles a collection of scripts and Jupyter notebooks. For reproducibility, it is designed to run in a Docker container based on the [jupyter/datascience-notebook image](https://hub.docker.com/r/jupyter/datascience-notebook). The following steps describe how to run the code in the same development environment as intended:\n\n#### Set up your directory\n* [Install and run Docker Desktop](https://www.docker.com/get-started) on your machine (the Community Edition is available for free).\n* Clone this repository and set its root folder as your working directory.\n* Make sure you followed the _Prerequisites_ section and that the input data is in the repository, in the `data` subfolder.\n\n#### Option 1 - Pulling the image from DockerHub [fast]\n* You can obtain a pre-built image to run this notebooks from the DockerHub image repository:\n\n\t\tdocker pull koalive/bioprofilingnotebooks:v4\n\n#### Option 2 - Building the image from the Dockerfile [robust]\n* Alternatively, run the following command the first time you want to run code from this repository - which might take some time to download all requirements:\n\n\t\tdocker build --rm -t bioprofilingnotebooks .\n\t\tdocker tag bioprofilingnotebooks koalive/bioprofilingnotebooks:v4\n\n#### Start a docker container running a Jupyter server\n* Run the following each time you want to start a notebook server to run code from this repository:\n\n\t\tdocker run -p 9999:8888 -v `pwd`:/home/jovyan koalive/bioprofilingnotebooks:v4\n\n* (Optional) If you wish to test the visual diagnostic features, you need to specify the folder in which the images can be found:\n\n\t\tdocker run -p 9999:8888 -v `pwd`:/home/jovyan -v /Local/Path/To/Images/:/images/ koalive/bioprofilingnotebooks:v4\n\n* Find the token needed to connect to the Jupyter notebook in the console output and go to the corresponding address in your browser:\n\n\t\thttp://127.0.0.1:9999/?token=\u003cyourToken\u0026gt;\n\n* You can now choose a notebook to run (see the next section for details).\n\n* When you are done, close the notebook server and the docker container by pressing CTRL+C in your terminal.\n\n#### Reproducing example analyses\nThe notebooks are split in four different categories as follows:\n\n* The **main analyses**, describing the complete analyses of a drug screen using _BioProfiling.jl_ and contextualizing the profiles using external annotations of the compounds. The following order is recommended:\n\t* *Fig2a_Profiling.ipynb*\n\t* *Fig2b_HitDetection.ipynb*\n\t* *Fig3_HitEnrichment.ipynb*\t\n* The **additional analyses**, describing more precisely results from the drug screen and exploring alternative approaches. They require some files from the main analyses and can be run either after completing the *Fig2b_HitDetection.ipynb* notebook or using the [intermediate files provided on FigShare](https://doi.org/10.6084/m9.figshare.14784678.v2). The following order is recommended:\n\t* *FigS1_NoCellFilter.ipynb*\n\t* *FigS2a_Profiling.ipynb*\n\t* *FigS2b_HitDetection.ipynb*\n\t* *FigS3_ProfilingApproaches.ipynb*\n\t* *STables.ipynb*\n* The notebook *Fig1_Common_Artifacts.ipynb* is independent and simply represents the **prevalence of common imaging artifacts** and biological outliers in high-content imaging datasets. \n* The notebook *FigS4_LUAD.ipynb* is independent and can provide a **simpler example** based on an external genetic screen made available as part of the [CytoData Hackathon 2018](https://github.com/cytodata/cytodata-hackathon-2018) (with smaller data and easily running on a laptop).\n\n\n#### Warning\n\nThe notebooks *Fig2a_Profiling.ipynb*, *FigS1_NoCellFilter.ipynb* and *FigS2a_Profiling.ipynb* require to load individual cell measurements for whole plates and might run out of memory on a desktop machine. We recommend up to 80GB of memory for these steps.  \nNote that without mounting a folder with the raw images, the notebook cells demonstrating the use of the visual diagnostics in these same notebooks cannot be run. The overall analysis should be reproducible nonetheless.\n\n### Note for Windows users\n\nYou can follow the same instructions in a PowerShell. After installing Docker desktop, you might need to:\n\n* Run and complete the following procedure:\n\t\t\n\t\tdocker login\n\n* Share the drive in which you cloned this repository in Docker's settings\n* Run the notebook server explicitely stating the path to this repository:\n\n\t\tdocker run -p 9999:8888 -v C:\\\u003cpathOnYourComputer\u003e\\BioProfilingNotebooks:/home/jovyan koalive/bioprofilingnotebooks:v4\n\t\t\n### Note for Linux users\n\nYou can follow the general instructions. You might need to run Docker with super-user privileges depending on your setup, *i.e.* using *sudo docker* in all calls to Docker.  \nIf the following happens when you try to run the notebook server: `PermissionError: [Errno 13] Permission denied: '/home/jovyan/.local'`, try the following options:\n`sudo docker run --user $(id -u):$(id -g) --group-add users -p 9999:8888 -v `pwd`:/home/jovyan koalive/bioprofilingnotebooks:v4`\n\n### Note for MacOS users\n\nYou can follow the general instructions.\n\n### Note for Singularity users\n\nYou can use `singularity pull docker://koalive/bioprofilingnotebooks:v4` to get the image from DockerHub, then convert it to a sandbox with `singularity build --sandbox sandbox/ bioprofilingnotebooks_v4.sif` and run the sandbox with the --no-home and --writable flags on a given port (*6789* in this example), while binding your current directory to `/home/jovyan/`:\n```singularity exec -B /your/working/directory/:/home/jovyan --writable --no-home sandbox/ jupyter-notebook --port 6789```\nWe recommend commenting out the following lines in the notebooks, which requires to load temporary fonts used for plotting results, which will not be available in the singularity sandbox:\n```\nttf_import(paths = \"/tmp/.fonts/\")\nloadfonts()\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmenchelab%2Fbioprofilingnotebooks","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmenchelab%2Fbioprofilingnotebooks","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmenchelab%2Fbioprofilingnotebooks/lists"}