{"id":17100887,"url":"https://github.com/vbharadwaj-bk/fast_tensor_leverage","last_synced_at":"2026-03-04T00:08:12.156Z","repository":{"id":77428608,"uuid":"548068060","full_name":"vbharadwaj-bk/fast_tensor_leverage","owner":"vbharadwaj-bk","description":"Code for the Neurips 2023 Paper \"Fast Exact Leverage Score Sampling from Khatri-Rao Products\"","archived":false,"fork":false,"pushed_at":"2024-10-23T21:21:49.000Z","size":261564,"stargazers_count":5,"open_issues_count":0,"forks_count":2,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-12T23:54:45.037Z","etag":null,"topics":["khatri-rao","leverage-score","randomized-algorithm","sparse-tensors"],"latest_commit_sha":null,"homepage":"https://arxiv.org/pdf/2301.12584.pdf","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vbharadwaj-bk.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-10-08T21:39:29.000Z","updated_at":"2024-10-31T01:51:13.000Z","dependencies_parsed_at":"2023-11-27T07:29:52.508Z","dependency_job_id":"6fe8fb76-6520-47e7-82d3-fd0fa94d0a53","html_url":"https://github.com/vbharadwaj-bk/fast_tensor_leverage","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vbharadwaj-bk%2Ffast_tensor_leverage","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vbharadwaj-bk%2Ffast_tensor_leverage/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vbharadwaj-bk%2Ffast_tensor_leverage/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vbharadwaj-bk%2Ffast_tensor_leverage/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vbharadwaj-bk","download_url":"https://codeload.github.com/vbharadwaj-bk/fast_tensor_leverage/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248647232,"owners_count":21139083,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["khatri-rao","leverage-score","randomized-algorithm","sparse-tensors"],"created_at":"2024-10-14T15:22:56.159Z","updated_at":"2026-03-04T00:08:12.127Z","avatar_url":"https://github.com/vbharadwaj-bk.png","language":"Jupyter Notebook","readme":"# Fast Exact Leverage Score Sampling from Khatri-Rao Products with Applications to Tensor Decomposition\nThis repository contains the code for\nthe paper [Fast Exact Leverage Score Sampling\nfrom Khatri-Rao Products](https://arxiv.org/pdf/2301.12584),\nto appear at Neurips 2023 (main conference, poster).\n\n## What can I do with it?\nYou can draw samples from the Khatri-Rao product of matrices\nwith tens of millions of rows each, provided the rank is at most\n100-200 (and you have enough memory to store the inputs). These\nsamples are drawn according to the exact distribution of leverage\nscores on the rows of the Khatri-Rao product, which have attractive\nproperties for randomized linear least-squares.\n\nOn LBNL Perlmutter CPU nodes, our code can draw 50,000 samples from\na Khatri-Rao product with $\\approx 10^{21}$ total rows and 32 columns\nin under 0.33 seconds.  \n\nHere's a smaller test case that can run on a laptop:\n```python\nimport numpy as np\nimport cppimport.import_hook\nfrom cpp_ext.als_module import Sampler\n\nif __name__=='__main__':\n    # Take J=10,000 samples from the KRP of N=4 matrices,\n    # each with dimensions I x R, where I = 1000 and R = 8.\n    I, N, J, R = 1000, 4, 10000, 8\n    matrices = [np.random.normal(size=(I, R)) for i in range(N)]\n\n    sampler = Sampler(matrices, J, R, \"efficient\")\n    samples = np.zeros((N, J), dtype=np.uint64)\n    sampler.KRPDrawSamples(N+1, samples)\n```\n\n## How do I reproduce the figures in the paper from the data?\nThe raw data is stored in `json` format in the folder\n`submission_data`. These files were output directly by the\nscripts in the `benchmarks` folder. You can reproduce the\nfigures using the material in the `plotting` folder. From\nwithin the folder, run \n\n```shell\n[zsh]\u003e python generate_images.py\n```\nwhich will output all figures in PDF format to \n`plotting/paper_images`. This script will also print tables\nused in the work to `STDOUT`. Alternatively, \nuse the Jupyter file `generate_images.ipynb` to generate\nthe plots inside a notebook.\n\nIf you would like to generate the data yourself, use the\nscripts in the `benchmarks` folder. The accuracy and runtime\nbenchmarks are easy to reproduce, but you may need a machine\nwith several hundred gigabytes of RAM to reproduce \nexperiments involving the largest sparse tensors \n(on our benchmark platform, each LBNL Perlmutter \nnode has 512GB of RAM). \n\n## Details\nThis repository contains two implementations of the data \nstructure in the paper. The first, available in the \nfolder `reference_implementation`, is written entirely in \nPython with Numpy. It is slow, but the structure of the \ncode matches the pseudocode in our paper closely. You \ncan use it to verify the correctness of our algorithm\nor pick apart how the data structure works. You only need\nPython, Numpy, and Matplotlib to test the reference\nimplementation - see the README inside the folder for\nmore instructions.\n\nThe second implementation is a fast version written\nin C++, with Pybind11 Python bindings \nand compatibility with Numpy for easy \ntesting. All benchmarks were conducted\nwith the fast version, and all instructions detail\nhow to build the fast version.\n\nThis repository contains a copy of `json.hpp`\nfrom Niels Lohmann's repository \n\u003chttps://github.com/nlohmann/json\u003e. It is published\nunder an MIT license.\n\n## Requirements\nYou need GCC 11.2+, OpenMP, and an install of the BLAS\nand LAPACK. This code has been tested with OpenBLAS, and\nwe make no guarantees for Intel MKL or other BLAS\nimplementations. If you decide to use MKL, either a) \ntry using the Intel C++ compiler instead of GCC or b) if you use GCC, \nlink MKL in sequential mode. We have observed segfaults\nwhen OpenMP threading is enabled in conjunction with\nMKL (when compiled with GCC).\n\nWe strongly recommended that you install Intel\nThread Building Blocks (TBB), but this is not\nrequired. We rely on Intel TBB for fast parallel \nsorting during sparse tensor decomposition.\n\nOur C++ code could stand alone, but right now, you\nneed Python as well to run it. \n\n## Building our code\n\n### Step 0: Clone the Repository\nClone the repository and `cd` into it. \n```shell\n[zsh]\u003e git clone https://github.com/vbharadwaj-bk/fast_tensor_leverage.git\n[zsh]\u003e cd fast_tensor_leverage\n```\n\n### Step 1: Install Python packages\nInstall Python dependencies with the following command:\n```shell\n[zsh]\u003e pip install -r requirements.txt\n```\nWe rely on the Pybind11 and cppimport packages. We\nuse the HDF5 format to store sparse tensors, so\nyou need the h5py package if you want to perform\nsparse tensor decomposition. \n\n### Step 2: Configure the compile and runtime environments \nWithin the repository, run the following command:\n```shell\n[zsh]\u003e python configure.py\n```\nThis will create two files in the repository root:\n`config.json` and `env.sh`. Edit the configuration\nJSON file with include / link flags for your LAPACK\ninstall. If you have TBB installed (strongly\nrecommended for good performance), fill in the appropriate\nentries. If you do not have TBB installed (or if you\nare using a compiler that automatically links the BLAS),\nset those JSON entries to empty lists. \n\nThe file `env.sh` sets up the runtime environment,\nand must be called every time you start a new shell \nto run our code. First, set the variables CC\nand CXX to your C and C++ compilers. The C++ extension\nmodule is compiled with\nthese when it is imported by Python at runtime. \n\n### Step 3: Test the code \nYou're ready to test! The C++ extension\ncompiles automatically the first time you run\nthe code, and is not compiled subsequently. Run\nthe following code:\n```shell\n[zsh]\u003e python compare_distributions.py\n```\nIf all goes well, you should see a new graph in\nthe `plotting` folder called\n`distribution_comparison_test_generated.png`\n(ignore the file `distribution_comparison.png`, which\nis the control image for the README).\nThe graph should look like this:\n![Distribution Comparison](plotting/distribution_comparison.png)\n\nThis image compares the true probability\ndistribution of leverage scores of the\nKhatri-Rao product \n$U_1 \\odot U_2 \\odot U_3$ (all\nchosen as $8 \\times 8$ matrices to make\nthe graph readable) to a\nhistogram of samples taken by our sampler.\nThere is excellent agreement between the\ndistribution that our sampler draws from\nand the true leverage score distribution.\n\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvbharadwaj-bk%2Ffast_tensor_leverage","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvbharadwaj-bk%2Ffast_tensor_leverage","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvbharadwaj-bk%2Ffast_tensor_leverage/lists"}