{"id":24360481,"url":"https://github.com/simonsfoundation/scaip","last_synced_at":"2026-01-24T21:36:34.171Z","repository":{"id":80673989,"uuid":"70708207","full_name":"simonsfoundation/SCaIP","owner":"simonsfoundation","description":"Tools for Calcium Imaging Analysis","archived":false,"fork":false,"pushed_at":"2016-10-12T17:32:36.000Z","size":117055,"stargazers_count":4,"open_issues_count":0,"forks_count":1,"subscribers_count":8,"default_branch":"master","last_synced_at":"2025-04-10T09:48:22.879Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/simonsfoundation.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2016-10-12T14:30:00.000Z","updated_at":"2019-03-12T16:33:30.000Z","dependencies_parsed_at":null,"dependency_job_id":"d92340e0-c3c7-46a5-8059-c78529650479","html_url":"https://github.com/simonsfoundation/SCaIP","commit_stats":null,"previous_names":[],"tags_count":8,"template":false,"template_full_name":null,"purl":"pkg:github/simonsfoundation/SCaIP","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simonsfoundation%2FSCaIP","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simonsfoundation%2FSCaIP/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simonsfoundation%2FSCaIP/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simonsfoundation%2FSCaIP/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/simonsfoundation","download_url":"https://codeload.github.com/simonsfoundation/SCaIP/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simonsfoundation%2FSCaIP/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28737436,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-24T21:19:41.845Z","status":"ssl_error","status_checked_at":"2026-01-24T21:13:38.675Z","response_time":89,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-01-18T21:29:56.574Z","updated_at":"2026-01-24T21:36:34.155Z","avatar_url":"https://github.com/simonsfoundation.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n\n\n\nPlease refer to the following wiki [page](https://github.com/agiovann/Constrained_NMF/wiki/Processing-large-datasets) or read in the testing section below\n\nConstrainedNMF\n==============\n\nPython translation of Constrained Non-negative Matrix Factorization algorithm for source extraction from calcium imaging data.\n\n[![Join the chat at https://gitter.im/agiovann/SOURCE_EXTRACTION_PYTHON](https://badges.gitter.im/agiovann/SOURCE_EXTRACTION_PYTHON.svg)](https://gitter.im/agiovann/SOURCE_EXTRACTION_PYTHON?utm_source=badge\u0026utm_medium=badge\u0026utm_campaign=pr-badge\u0026utm_content=badge)\n\n\u003ca href='https://travis-ci.org/agiovann/Constrained_NMF'\u003e\u003cimg src='https://secure.travis-ci.org/agiovann/Constrained_NMF.png?branch=master'\u003e\u003c/a\u003e\n\n# Deconvolution and demixing of calcium imaging data\n\nThe code implements a method for simultaneous source extraction and spike inference from large scale calcium imaging movies. The code is suitable for the analysis of somatic imaging data. Implementation for the analysis of dendritic/axonal imaging data will be added in the future.\n\nThe algorithm is presented in more detail in\n\nPnevmatikakis, E.A., Soudry, D., Gao, Y., Machado, T., Merel, J., ... \u0026 Paninski, L. (2016). Simultaneous denoising, deconvolution, and demixing of calcium imaging data. Neuron 89(2):285-299, http://dx.doi.org/10.1016/j.neuron.2015.11.037\n\nPnevmatikakis, E.A., Gao, Y., Soudry, D., Pfau, D., Lacefield, C., ... \u0026 Paninski, L. (2014). A structured matrix factorization framework for large scale calcium imaging data analysis. arXiv preprint arXiv:1409.2903. http://arxiv.org/abs/1409.2903\n\n# Contributors\n\nAndrea Giovannucci, Eftychios Pnevmatikakis, \nCenter for Computational Biology, Simons Foundation, New York, NY\n\nValentina Staneva\neScience Institute. University of Washinghton. Seattle, WA.\n\nJohannes Friedrich\nColumbia University, New York, NY. \n\nCode description and related packages\n=======\n\nThis repository contains a Python implementation of the spatio-temporal demixing, i.e., (source extraction) code for large scale calcium imaging data. Related code can be found in the following links:\n\n## Python\n- [Source extraction with CNMF (this package)](https://github.com/agiovann/SOURCE_EXTRACTION_PYTHON)\n- [Group LASSO initialization and spatial CNMF](https://github.com/danielso/ROI_detect)\n\n## Matlab\n- [Constrained deconvolution and source extraction with CNMF](https://github.com/epnev/ca_source_extraction)\n- [MCMC spike inference](https://github.com/epnev/continuous_time_ca_sampler)\n- [Group LASSO initialization and spatial CNMF](https://github.com/danielso/ROI_detect)\n\n## Integration with other libraries\n- [SIMA](http://www.losonczylab.org/sima/1.3/): The [constrained deconvolution](https://github.com/losonczylab/sima/blob/master/sima/spikes.py) method has been integrated with SIMA, a Python based library for calcium imaging data analysis.\n- [Thunder](http://thunder-project.org/): The [group LASSO initialization and spatial CNMF](https://github.com/j-friedrich/thunder/tree/LocalNMF) method has been integrated with Thunder, a library for large scale neural data analysis with Spark.\n\nInstallation\n===================================================\n\nInstallation on MAC OS\n----------------------\n\nDownload and install Anaconda (Python 2.7) \u003chttp://docs.continuum.io/anaconda/install\u003e\n\n    ```\n    git clone --recursive https://github.com/agiovann/Constrained_NMF.git\n    cd Constrained_NMF/CalBlitz/\n    git checkout master\n    git pull\n    conda create -n CNMF ipython --file requirements_conda.txt    \n    source activate CNMF\n    pip install -r requirements_pip.txt\n    conda install -c menpo opencv3=3.1.0\n    \n    To make the package available from everywhere:\n    ```\n    export PYTHONPATH=\"/path/to/Constrained_NMF:$PYTHONPATH\"\n    ```\n\n\n\nTroubleshooting\n----------------\n**SCS**:\n\nif you get errors compiling scs when installing cvxpy you probably need to create a link to openblas or libgfortran in\n/usr/local/lib/, for instance:\n\n`sudo ln -s  /Library/Frameworks/R.framework/Libraries/libgfortran.3.dylib  /usr/local/lib/libgfortran.2.dylib`\n\n\n\n**debian fortran compiler problems:**\nif you get the error  gcc: error trying to exec 'cc1plus': execvp: No such file or directory in ubuntu run\nor issues related to SCS type\n\n ```\n sudo apt-get install g++ libatlas-base-dev gfortran  libopenblas-dev\n conda install openblas atlas\n ```\n\n if still there are issues try\n\n  `export LD_LIBRARY_PATH=/path_to_your_home/anaconda2/lib/`\n\n if more problems try \n\n ```\n conda install  atlas (only Ubuntu)\n pip install 'tifffile\u003e=0.7'\n conda install accelerate\n conda install openblas \n ```\n \n**CVXOPT**:\n\nIf you are on Windows and don't manage to install or compile cvxopt, a simple solution is to download the right binary [there](http://www.lfd.uci.edu/~gohlke/pythonlibs/#cvxopt) and install the library by typing:\n\n```\npip install cvxopt-1.1.7-XXXX.whl\n```\n\nTest the system\n----------------------\n\n**SINGLE PATCH**\n\nIn case you used installation af point 1 above you will need to download the test files from\n\u003chttps://github.com/agiovann/Constrained_NMF/releases/download/v0.3/Demo.zip\u003e\n\nA. Go into the cloned folder, type `python demo.py`\n\nB. Using the Spyder (type `conda install spyder`) IDE.\n\n    1. Unzip the file Demo.zip (you do not need this step if you installed dusing method 2 above, just enter the Constrained_NMF folder and you will find all the required files there).\n    2. Open the file demo.py with spyder\n    3. change the base_folder variable to point to the folder you just unzipped\n    3. Run the cells one by one inspecting the output\n    4. Remember to stop the cluster (last three lines of file). You can also stop it manually by typing in a terminal\n    'ipcluster stop'\n\nC. Using notebook.\n\n    1. Unzip the file Demo.zip (you do not need this step if you installed dusing method 3 above, just enter the Constrained_NMF folder and you will find all the required files there).\n    2. type `ipython notebook`\n    3. open the notebook called demoCNMF.ipynb \n    4. change the base_folder variable to point to the folder you just unzipped\n    5. and run cell by cell inspecting the result\n    6. Remember to stop the cluster (last three lines of file). You can also stop it manually by typing in a terminal\n    'ipcluster stop'\n\n\n**MULTI PATCH**\n+ Download the two demo movies [here](https://github.com/agiovann/Constrained_NMF/releases/download/v0.4-alpha/Patch_demo.zip) (courtesy of Dr. Sue Ann Koay from the Tank Lab, Princeton Neuroscience Institute, Princeton. NJ). Unzip the folder. Then in Spyder open the file demo_patches.py, and change the base_folder variable to point to the folder you just unzipped. \n+ Run one by one the cells (delimited by '#%%') \n+ Inspect the results. The demo will start a cluster and process pathes of the movie (more details [here](https://github.com/agiovann/Constrained_NMF/wiki/Processing-large-datasets)) in parallel (cse.map_reduce.run_CNMF_patches). Afterwards, it will merge the results back together and proceed to firstly merge potentially overlaping components (cse.merge_components) from different patches, secondly to update the spatial extent of the joined spatial components (cse.spatial.update_spatial_components), and finally denoising the traces (cse.temporal.update_temporal_components). THe final bit is used for visualization. \n\nDocumentation\n========\n\nDocumentation of the code can be found [here](http://agiovann.github.io/Constrained_NMF)\n\nDependencies\n========\nThe code uses the following libraries\n- [NumPy](http://www.numpy.org/)\n- [SciPy](http://www.scipy.org/)\n- [Matplotlib](http://matplotlib.org/)\n- [Scikit-Learn](http://scikit-learn.org/stable/)\n- [Tifffile](https://pypi.python.org/pypi/tifffile) For reading tiff files. Other choices can work there too.\n- [cvxpy](http://www.cvxpy.org/) for solving optimization problems\n- [ipyparallel](http://ipyparallel.readthedocs.org/en/latest/) for parallel processing\n\nExternal Dependencies\n============\n\nFor the constrained deconvolution method (```deconvolution.constrained_foopsi```)  various solvers can be used, each of which requires some additional packages:\n\n1. ```'cvxpy'```: (default) For this option, the following packages are needed:\n  * [CVXOPT](http://cvxopt.org/) Required.\n  * [CVXPY](http://www.cvxpy.org/) Required.\n2. ```'cvx'```: For this option, the following packages are needed:\n  * [CVXOPT](http://cvxopt.org/) Required.\n  * [PICOS](http://picos.zib.de/) Required.\n  * [MOSEK](https://www.mosek.com/) Optional but strongly recommended for speed improvement, free for academic use.\n3. ```'spgl1'```: For this option, the SPGL1 python implementation is required. It is by default imported as a submodule. The original implementation can be found at (https://github.com/mpf/spgl1).  \n\nIn general ```'cvxpy'``` can be faster, when using the 'ECOS' or 'SCS' sovlers, which are included with the CVXPY installation. ```'spgl1'``` can also be very fast but the python implementation is not as fast as in Matlab and not thoroughly tested.\n\nQuestions, comments, issues\n=======\nPlease use the gitter chat room (use the button above) for questions and comments and create an issue for any bugs you might encounter.\n\nImportant note\n======\nThe implementation of this package is based on the matlab implementation which can be found [here](https://github.com/epnev/ca_source_extraction). Some of the Matlab features are currently lacking, but will be included in future releases.\n\nLicense\n=======\n\nThis program is free software; you can redistribute it and/or\nmodify it under the terms of the GNU General Public License\nas published by the Free Software Foundation; either version 2\nof the License, or (at your option) any later version.\n\nThis program is distributed in the hope that it will be useful,\nbut WITHOUT ANY WARRANTY; without even the implied warranty of\nMERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the\nGNU General Public License for more details.\n\nYou should have received a copy of the GNU General Public License\nalong with this program.  If not, see \u003chttp://www.gnu.org/licenses/\u003e.\n\n\nExperimental\n=============\n\nThe package comes with a toolbox to manipulate movies written in Python, Calblitz. If you want to give it a try use the demo_pipeline.py file. Before that you need to install some packages:\n\n```\npip install pims\nconda install opencv\nconda install h5py\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsimonsfoundation%2Fscaip","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsimonsfoundation%2Fscaip","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsimonsfoundation%2Fscaip/lists"}