{"id":32177381,"url":"https://github.com/klugerlab/fit-sne","last_synced_at":"2026-02-21T13:01:27.096Z","repository":{"id":37396898,"uuid":"115304736","full_name":"KlugerLab/FIt-SNE","owner":"KlugerLab","description":"Fast Fourier Transform-accelerated Interpolation-based  t-SNE (FIt-SNE)","archived":false,"fork":false,"pushed_at":"2021-09-13T00:21:33.000Z","size":32822,"stargazers_count":606,"open_issues_count":20,"forks_count":111,"subscribers_count":33,"default_branch":"master","last_synced_at":"2025-12-08T05:20:12.548Z","etag":null,"topics":["big-data","fast-algorithm","t-sne","visualization"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/KlugerLab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-12-25T03:48:38.000Z","updated_at":"2025-11-26T07:46:08.000Z","dependencies_parsed_at":"2022-08-08T20:15:26.372Z","dependency_job_id":null,"html_url":"https://github.com/KlugerLab/FIt-SNE","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/KlugerLab/FIt-SNE","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KlugerLab%2FFIt-SNE","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KlugerLab%2FFIt-SNE/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KlugerLab%2FFIt-SNE/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KlugerLab%2FFIt-SNE/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/KlugerLab","download_url":"https://codeload.github.com/KlugerLab/FIt-SNE/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KlugerLab%2FFIt-SNE/sbom","scorecard":{"id":79376,"data":{"date":"2025-08-11","repo":{"name":"github.com/KlugerLab/FIt-SNE","commit":"47ff14f1defc1ff3a8065c8b7baaf45c33e7e0b2"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":2.8,"checks":[{"name":"Code-Review","score":7,"reason":"Found 7/10 approved changesets -- score normalized to 7","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Binary-Artifacts","score":4,"reason":"binaries present in source code","details":["Warn: binary detected: src/winlibs/fftw/libfftw3-3.dll:1","Warn: binary detected: src/winlibs/fftw/libfftw3-3.lib:1","Warn: binary detected: src/winlibs/fftw/libfftw3f-3.dll:1","Warn: binary detected: src/winlibs/fftw/libfftw3f-3.lib:1","Warn: binary detected: src/winlibs/fftw/libfftw3l-3.dll:1","Warn: binary detected: src/winlibs/fftw/libfftw3l-3.lib:1"],"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":9,"reason":"license file detected","details":["Info: project has a license file: LICENSE.txt:0","Warn: project license file does not contain an FSF or OSI license."],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Pinned-Dependencies","score":-1,"reason":"no dependencies found","details":null,"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Signed-Releases","score":0,"reason":"Project has not signed or included provenance with any releases.","details":["Warn: release artifact v1.2.1 not signed: https://api.github.com/repos/KlugerLab/FIt-SNE/releases/25647161","Warn: release artifact v1.1.0 not signed: https://api.github.com/repos/KlugerLab/FIt-SNE/releases/15455522","Warn: release artifact v1.0.0 not signed: https://api.github.com/repos/KlugerLab/FIt-SNE/releases/13693665","Warn: release artifact v1.2.1 does not have provenance: https://api.github.com/repos/KlugerLab/FIt-SNE/releases/25647161","Warn: release artifact v1.1.0 does not have provenance: https://api.github.com/repos/KlugerLab/FIt-SNE/releases/15455522","Warn: release artifact v1.0.0 does not have provenance: https://api.github.com/repos/KlugerLab/FIt-SNE/releases/13693665"],"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 28 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-15T05:28:48.504Z","repository_id":37396898,"created_at":"2025-08-15T05:28:48.505Z","updated_at":"2025-08-15T05:28:48.505Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29681468,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-21T12:30:22.644Z","status":"ssl_error","status_checked_at":"2026-02-21T12:29:55.402Z","response_time":107,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["big-data","fast-algorithm","t-sne","visualization"],"created_at":"2025-10-21T20:04:00.767Z","updated_at":"2026-02-21T13:01:27.087Z","avatar_url":"https://github.com/KlugerLab.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# FFT-accelerated Interpolation-based t-SNE (FIt-SNE)\n## Introduction\nt-Stochastic Neighborhood Embedding ([t-SNE](https://lvdmaaten.github.io/tsne/)) is a highly successful method for dimensionality reduction and visualization of high dimensional datasets.  A popular [implementation](https://github.com/lvdmaaten/bhtsne) of t-SNE uses the Barnes-Hut algorithm to approximate the gradient at each iteration of gradient descent. We accelerated this implementation as follows:\n\n* Computation of the N-body Simulation: Instead of approximating the N-body simulation using Barnes-Hut, we interpolate onto an equispaced grid and use FFT to perform the convolution, dramatically reducing the time to compute the gradient at each iteration of gradient descent. See the [this](http://gauss.math.yale.edu/~gcl22/blog/numerics/low-rank/t-sne/2018/01/11/low-rank-kernels.html) post for some intuition on how it works.\n* Computation of Input Similarities: Instead of computing nearest neighbors using vantage-point trees, we approximate nearest neighbors using the [Annoy](https://github.com/spotify/annoy) library. The neighbor lookups are multithreaded to take advantage of machines with multiple cores. Using \"near\" neighbors as opposed to strictly \"nearest\" neighbors is faster, but also has a smoothing effect, which can be useful for embedding some datasets (see [Linderman et al. (2017)](https://arxiv.org/abs/1711.04712)). If subtle detail is required (e.g. in identifying small clusters), then use vantage-point trees (which is also multithreaded in this implementation). \n\n\nCheck out our [paper](https://www.nature.com/articles/s41592-018-0308-4) or [preprint](https://arxiv.org/abs/1712.09005) for more details and some benchmarks.\n\n## Features\nAdditionally, this implementation includes the following features:\n* Early exaggeration: In [Linderman and Steinerberger (2018)](https://epubs.siam.org/doi/abs/10.1137/18M1216134), we showed that appropriately choosing the early exaggeration coefficient can lead to improved embedding of swissrolls and other synthetic datasets. Early exaggeration is built into all t-SNE implementations; here we highlight its importance as a parameter. \n* Late exaggeration: Increasing the exaggeration coefficient late in the optimization process can improve separation of the clusters. [Kobak and Berens (2019)](https://www.nature.com/articles/s41467-019-13056-x) suggest starting late exaggeration immediately following early exaggeration. \n* Initialization: Custom initialization can be provided from Python/Matlab/R. As suggested by [Kobak and Berens (2019)](https://www.nature.com/articles/s41467-019-13056-x), initializing t-SNE with the first two principal components (scaled to have standard deviation 0.0001) results in an embedding which often preserves the global structure more effectively than the default random normalization. See there for other initialisation tricks.\n* Variable degrees of freedom: [Kobak et al. (2019)](https://ecmlpkdd2019.org/downloads/paper/327.pdf) show that decreasing the degree of freedom (df) of the t-distribution (resulting in heavier tails)  reveals fine structure that is not visible in standard t-SNE.\n* Perplexity combination: The perplexity parameter determines the width of the Gaussian kernel, such that small perplexity values uncover local structure while larger values reveal global structure. [Kobak and Berens (2019)](https://www.nature.com/articles/s41467-019-13056-x) show that using combination of several perplexity values, resulting in a multi-scale embedding, can be useful. \n* All optimisation parameters can be controlled from Python/Matlab/R. For example, [Belkina et al. (2019)](https://www.nature.com/articles/s41467-019-13055-y) highlight the importance of increasing the learning rate when embedding large data sets. \n\n\n## Installation\nR, Matlab, and Python wrappers are `fast_tsne.R`, `fast_tsne.m`, and `fast_tsne.py` respectively. Each of these wrappers can be used after installing FFTW and compiling the C++ code, as below. [Gioele La Manno](https://twitter.com/GioeleLaManno) implemented a Python (Cython) wrapper, which is available on PyPI [here](https://pypi.python.org/pypi/fitsne).\n\n**Note:** If you update to a new version of FIt-SNE using `git pull`, be sure to recompile. \n\n### OSX and Linux\nThe only prerequisite is [FFTW](http://www.fftw.org/), which can be downloaded and installed from the website. Then, from the root directory compile the code as:\n```bash\ng++ -std=c++11 -O3  src/sptree.cpp src/tsne.cpp src/nbodyfft.cpp  -o bin/fast_tsne -pthread -lfftw3 -lm -Wno-address-of-packed-member\n```\nSee [here](https://github.com/KlugerLab/FIt-SNE/issues/35) for instructions in case one does not have `sudo` rights (one can install `FFTW` in the home directory and provide its path to `g++`).\n\nCheck out `examples/` for usage. The [Python demo notebook](https://github.com/KlugerLab/FIt-SNE/blob/master/examples/test.ipynb) walks one through the most of the available options using the MNIST data set.\n\n\n### Windows\nA Windows binary is available [here](https://github.com/KlugerLab/FIt-SNE/releases/download/v1.2.1/FItSNE-Windows-1.2.1.zip). Please extract to the `bin/` folder, and you should be all set.\n\nIf you would like to compile it yourself see below. The code has been currently tested with MS Visual Studio 2015 (i.e., MS Visual Studio Version 14).\n\n1.  First open the provided FItSNE solution (FItSNE.sln) using MS Visual Studio and build the Release configuration. \n2.  Copy the binary file ( e.g. `x64/Release/FItSNE.exe`) generated by the build process to the `bin/` folder \n3.  For Windows, we have added all dependencies, including the [FFTW library](http://www.fftw.org/), which is distributed under the GNU General Public License. For the binary to find the FFTW DLLs, you need to either add `src/winlibs/fftw/` to your PATH, or to copy the DLLs into the `bin/` directory.\n\nAs of this commit, only the R wrapper properly calls the Windows executable. The Python and Matlab wrappers can be trivially changed to call it (just changing `bin/fast_tsne` to `bin/FItSNE.exe` in the code), and will be changed in future commits.\n \nMany thanks to [Josef Spidlen](https://github.com/jspidlen) for this Windows implementation!\n\n## Acknowledgements and References\nWe are grateful for members of the community who have [contributed](https://github.com/KlugerLab/FIt-SNE/graphs/contributors) to improving FIt-SNE, especially [Dmitry Kobak](https://github.com/dkobak), [Pavlin Poličar](https://github.com/pavlin-policar), and [Josef Spidlen](https://github.com/jspidlen).\n\nIf you use FIt-SNE, please cite:\n\nGeorge C. Linderman, Manas Rachh, Jeremy G. Hoskins, Stefan Steinerberger, Yuval Kluger. (2019). Fast interpolation-based t-SNE for improved visualization of single-cell RNA-seq data. Nature Methods. ([link](https://www.nature.com/articles/s41592-018-0308-4))\n\nOur implementation is derived from the Barnes-Hut implementation:\n\nLaurens van der Maaten (2014). Accelerating t-SNE using tree-based algorithms. Journal of Machine Learning Research, 15(1):3221–3245. ([link](https://dl.acm.org/citation.cfm?id=2627435.2697068))\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fklugerlab%2Ffit-sne","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fklugerlab%2Ffit-sne","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fklugerlab%2Ffit-sne/lists"}