{"id":18496606,"url":"https://github.com/mouseland/pykilosort","last_synced_at":"2025-06-26T12:32:08.404Z","repository":{"id":39566010,"uuid":"218056497","full_name":"MouseLand/pykilosort","owner":"MouseLand","description":"WIP: Python port of Kilosort2","archived":false,"fork":false,"pushed_at":"2023-07-08T07:12:12.000Z","size":809,"stargazers_count":51,"open_issues_count":31,"forks_count":27,"subscribers_count":12,"default_branch":"master","last_synced_at":"2025-05-24T22:06:37.107Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://github.com/MouseLand/Kilosort2/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MouseLand.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2019-10-28T13:45:13.000Z","updated_at":"2025-05-04T21:50:34.000Z","dependencies_parsed_at":"2022-08-09T15:02:28.930Z","dependency_job_id":"c934d435-4e55-4c20-b3b3-e7bc19e72786","html_url":"https://github.com/MouseLand/pykilosort","commit_stats":null,"previous_names":["rossant/pykilosort"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/MouseLand/pykilosort","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MouseLand%2Fpykilosort","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MouseLand%2Fpykilosort/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MouseLand%2Fpykilosort/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MouseLand%2Fpykilosort/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MouseLand","download_url":"https://codeload.github.com/MouseLand/pykilosort/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MouseLand%2Fpykilosort/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262068139,"owners_count":23253745,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-06T13:30:13.005Z","updated_at":"2025-06-26T12:32:08.380Z","avatar_url":"https://github.com/MouseLand.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Python port of KiloSort2\n\nThis is a work-in-progress literal Python port of the original MATLAB version of Kilosort 2, written by Marius Pachitariu.\nThe code is still being debugged and is not ready for use.\n\n## Scope\n### Why an IBL port of pykilosort? (draft)\nPreprocessing functions and standardization of outputs\n\nKush Banga, Cyrille Rossant, Olivier Winter\n\n## Installation \n\n### System Requirements\n\nThe code makes extensive use of the GPU via the CUDA framework. A high-end NVIDIA GPU with at least 8GB of memory is required.\n\nA good CPU and a large amount of RAM (minimum 32GB or 64GB) is also required.\n\nSee [the Wiki on the Matlab version](https://github.com/MouseLand/Kilosort2/wiki/8.-Hardware-guide) for more information.\n\n\u003c!-- TODO: What OS's does this work on? I am testing with Ubuntu .04. --\u003e\n\nYou will need NVIDIA drivers and cuda-toolkit installed on your computer too. This can be the hardest part of the installation. To test if your is working OK you should be able to run the following:\n```\nnvidia-smi # Should show how much your GPU is being used right now\nnvcc # This is the CUDA compiler\n```\n\n### Doing the install using Anaconda\n\nOnly on Linux, first install fftw by running the following \n    \n    sudo apt-get install -y libfftw3-dev\n\nNavigate to the desired location for the repository and clone it\n\n    git clone -b drift_test_stable https://github.com/kushbanga/pykilosort.git\n    cd pykilosort\n\nCreate a conda environment\n\n    conda env create -f pyks2.yml\n    conda activate pyks2\n    conda develop .\n\n### Managing CUDA Errors\n\nErrors with the CUDA installation can sometimes be fixed by downgrading\nthe version of cudatoolkit installed. Currently tested versions are 9.2,\n10.0, 10.2, 11.0 and 11.5\n\nTo check the current version run the following:\n\n    conda activate pyks2\n    conda list cudatoolkit\n\nTo install version 10.0 for example run the following\n\n    conda activate pyks2\n    conda remove cupy, cudatoolkit\n    conda install -c conda-forge cupy cudatoolkit=10.0\n\n\n## Usage\n\n### Example\n\nThis is how to run for general users\n```python\nfrom pathlib import Path\nfrom pykilosort import run, add_default_handler, np1_probe, np2_probe\n\n# Run standard ks2.5 algorithm for a np1 probe\ndata_path = Path('path/to/data/data.bin')\ndir_path = Path('path/to/output/folder') # by default uses the same folder as the dataset\nadd_default_handler(level='INFO') # print output as the algorithm runs\nrun(data_path, dir_path=dir_path, probe=np1_probe())\n\n# Run chronic recordings for a np2 probe\n# For now this still uses ks2.5 clustering, chronic clustering algorithm coming soon!\ndata_paths = [\n    Path('path/to/first/dataset/dataset.bin'),\n    Path('path/to/second/dataset/dataset.bin'),\n    Path('path/to/third/dataset/dataset.bin'),\n]\ndir_path = Path('path/to/output/folder') # by default uses the same folder as the first dataset\nadd_default_handler(level='INFO')\nrun(data_paths, dir_path=dir_path, probe=np2_probe(), low_memory=True)\n```\n\nThis is how to run for NP1.0 probe (for IBL)\n```python\nimport shutil\nfrom pathlib import Path\nimport numpy as np\n\nimport pykilosort\nfrom pykilosort.ibl import run_spike_sorting_ibl, ibl_pykilosort_params\n\nINTEGRATION_DATA_PATH = Path(\"/datadisk/Data/spike_sorting/pykilosort_tests\")\nSCRATCH_DIR = Path.home().joinpath(\"scratch\", 'pykilosort')\nshutil.rmtree(SCRATCH_DIR, ignore_errors=True)\nSCRATCH_DIR.mkdir(exist_ok=True)\nDELETE = True  # delete the intermediate run products, if False they'll be copied over\nbin_file = INTEGRATION_DATA_PATH.joinpath(\"imec_385_100s.ap.bin\")\n# this is the output of the pykilosort data, unprocessed after the spike sorter\nks_output_dir = INTEGRATION_DATA_PATH.joinpath(\"results\")\nks_output_dir.mkdir(parents=True, exist_ok=True)\n# this is the output standardized as per IBL standards (SI units, ALF convention)\nalf_path = ks_output_dir.joinpath('alf')\n\n\nparams = ibl_pykilosort_params()\nrun_spike_sorting_ibl(bin_file, delete=DELETE, scratch_dir=SCRATCH_DIR,\n                      ks_output_dir=ks_output_dir, alf_path=alf_path, log_level='DEBUG', params=params)\n```\n\n### Disk cache (serialized results \u0026 parameter objects)\n\nThe MATLAB version used a big `rez` structured object containing the input data, the parameters, intermediate and final results.\n\nThe Python version makes the distinction between:\n\n- `raw_data`: a NumPy-like object of shape `(n_channels_total, n_samples)`\n- `probe`: a Bunch instance (dictionary) with the channel coordinates, the indices of the \"good channels\"\n- `params`: a Bunch instance (dictionary) with optional user-defined parameters. It can be empty. Any missing parameter is transparently replaced by the default as found in `default_params.py` file in the repository.\n- `intermediate`: a Bunch instance (dictionary) with intermediate arrays.\n\nThese objects are accessible via the *context* (`ctx`) which replaces the MATLAB `rez` object: `ctx.raw_data`, etc.\n\nThis context also stores a special object called `ctx.intermediate` which stores intermediate arrays. This object derives from `Bunch` and implements special methods to save and load arrays in a temporary folder. By default, an intermediate result called `ctx.intermediate.myarray` is stored in `./.kilosort/context/myarray.npy`.\n\nThe main `run()` function checks the existence of some of these intermediate arrays to skip some steps that might have run already, for a given dataset.\n\nThe suffixes `_m` (merge), `_s` (split), `_c` (cutoff) are used to disambiguate between multiple processing steps for the same arrays (they would be overwritten otherwise).\n\n\n## Technical notes about the port\n\nThe following differences between MATLAB and Python required special care during the port:\n\n* Discrepancy between 0-based and 1-based indexing.\n* MATLAB uses Fortran ordering for arrays, whereas NumPy uses C ordering by default. The Python code therefore uses Fortran ordering exclusively so that the custom CUDA kernels can be used with no modification.\n* In MATLAB, arrays can be extended transparently with indexing, whereas NumPy/CuPy requires explicit concatenation.\n* The MATLAB code used mex C files to launch CUDA kernels, whereas the Python code uses CuPy directly.\n* A few workarounds around limitations of CuPy compared to MATLAB: no `cp.median()`, no GPU version of the `lfilter()` LTI filter in CuPy (a custom CUDA kernel had to be written), etc.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmouseland%2Fpykilosort","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmouseland%2Fpykilosort","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmouseland%2Fpykilosort/lists"}