{"id":17623668,"url":"https://github.com/zachgrayio/rank1_covariance","last_synced_at":"2025-07-11T11:35:40.721Z","repository":{"id":252360289,"uuid":"840209021","full_name":"zachgrayio/rank1_covariance","owner":"zachgrayio","description":"incremental (rank-1) covariance matrix calculation on the GPU","archived":false,"fork":false,"pushed_at":"2024-08-20T16:13:31.000Z","size":81,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-30T01:36:59.690Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Cuda","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zachgrayio.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-08-09T07:46:27.000Z","updated_at":"2024-08-20T16:13:34.000Z","dependencies_parsed_at":"2024-08-20T18:17:43.083Z","dependency_job_id":"a39e5238-0bd6-4cd8-9866-8298ff338118","html_url":"https://github.com/zachgrayio/rank1_covariance","commit_stats":null,"previous_names":["zachgrayio/rank1_covariance"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/zachgrayio/rank1_covariance","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zachgrayio%2Frank1_covariance","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zachgrayio%2Frank1_covariance/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zachgrayio%2Frank1_covariance/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zachgrayio%2Frank1_covariance/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zachgrayio","download_url":"https://codeload.github.com/zachgrayio/rank1_covariance/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zachgrayio%2Frank1_covariance/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":264795838,"owners_count":23665241,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-22T21:23:27.354Z","updated_at":"2025-07-11T11:35:40.504Z","avatar_url":"https://github.com/zachgrayio.png","language":"Cuda","funding_links":[],"categories":[],"sub_categories":[],"readme":"# rank-1 covariance matrix calculation\n\nWithin this repo there are 2 simple programs that calculate covariances for a set of inputs on the GPU. \n\nEach program calculates these matrices either in the simple, traditional manner, or in an \"incremental\" or \"rank-1\" mode\nwhere _only_ the latest row included in the calculation is factored in as it appears, leading to large performance gains\nover a traditional matmul based approach.\n\n- A Python reference implementation: [main.py](main.py)\n  - PyTorch + CUDA if available\n  - also uses `numpy`'s `.cov()` for a simple sanity check\n- A CUDA C++ binary [main.cu](main.cu)\n  - includes a custom rank1 kernel\n  - also a \"1shot\" covariance function that does roughly the same as `.cov` AFAIK\n    - makes use of cuBLAS's `cublasSgemm_v2` and a few small kernels to speed up taking col means and centering\n  - test coverage to ensure the results are correct\n  - a comparison function that compares timing of the 2 approaches\n\n## Results on my RTX 3080 16Gi\n\nWith the 1shot work factor set to 1:\n\n```text\n...\ntotal time for rank1 update: 21.4057 seconds\ntotal time for 1shot update: 83.4697 seconds\niterations per second for rank1 update: 117.7259 iterations/second\niterations per second for 1shot update: 30.1906 iterations/second\n```\n\nBut note that for some domains, the work factor for this function may be lower (closer to `0.5`), but this is domain and application specific.\n\n## Run it yourself\n\n### Requirements\nRequires NVIDIA hardware of course as well as a working CUDA setup but edit `CMakeLists.txt` as needed if it doesn't work\nwith your setup. Python code should be portable enough to run anywhere, if you have `torch` and `numpy` present.\n\n### Running\nTo run everything:\n\n```bash\nmake\n```\n\nFor advanced usage, see the [Makefile](Makefile).\n\n## Profiling\nI haven't had a chance to profile this yet, there's certainly a few bottlenecks to chase down.\n\n## WTF is Rank-1?\nA rank-1 update in linear algebra refers to the process of modifying a matrix by adding or subtracting a matrix that can be expressed as the outer product of two vectors.\nSpecifically, if you have an existing matrix $A$, a rank-1 update to this matrix would look like:\n\n$A' = A + u \\cdot v^T$\n\nwhere $u$ and $v$ are column vectors, and $v^T$ is the transpose of $v$. The product $u \\cdot v^T$ results in a matrix \nof the same dimensions as $A$, but it is a rank-1 matrix because it is formed by the outer product of two vectors.\n\nThis operation is called a rank-1 update because it increases (or decreases) the rank of the matrix by at most 1.\n\n### How that is applied in these 2 programs\n\nIn the context of calculating a covariance matrix, a rank-1 update is particularly useful when you have a sequence of data points arriving one at a time, and you want to update the covariance matrix incrementally without recalculating it from scratch.\n\nGiven a covariance matrix $\\Sigma$, a new data point $x_{\\text{new}}$, and the current mean vector $\\mu$ based on $n-1$ samples, the rank-1 update can be mathematically described as follows:\n\n1. **Calculate the new mean**:\n  \n   $\\mu_{\\text{new}} = \\mu + \\frac{x_{\\text{new}} - \\mu}{n}$\n\n2. **Compute the deviation vectors**:\n   \n    $\\delta_{\\text{old}} = x_{\\text{new}} - \\mu$\n\n    $\\delta_{\\text{new}} = x_{\\text{new}} - \\mu_{\\text{new}}$\n\n3. **Update the covariance matrix**:\n\n   $\\Sigma_{\\text{new}} = \\frac{n-2}{n-1} \\Sigma + \\frac{\\delta_{\\text{old}} \\cdot \\delta_{\\text{new}}^T}{n-1}$\n\nThis update rule efficiently incorporates the new data point into the existing covariance matrix $\\Sigma$ by adjusting it with the outer product of the deviation vectors $\\delta_{\\text{old}}$ and $\\delta_{\\text{new}}$.\n\n### In practice\n\nUpdating a covariance matrix with a new row in this case is as simple as the following, taken directly from the python reference impl:\n\n```python\ndef rank1_update(cov_matrix, new_row, mean, n):\n    # calculate the new mean\n    new_mean = mean + (new_row - mean) / n\n    # update the covariance matrix\n    delta_old = new_row - mean\n    delta_new = new_row - new_mean\n    cov_matrix = ((n - 2) / (n - 1)) * cov_matrix + torch.outer(delta_old, delta_new) / (n - 1)\n    return cov_matrix, new_mean\n```\n\nand while it's a bit more noisy, the CUDA kernel bears a lot of resemblance:\n\n```c++\n__global__ void rank1_update_kernel(float* d_cov_matrix, const float* d_new_row, float* d_mean, int cols, int n) {\n    const int idx = blockIdx.x * blockDim.x + threadIdx.x;\n    if (idx \u003e= cols) return;\n\n    const float new_mean = fmaf(d_new_row[idx] - d_mean[idx], 1.0f / n, d_mean[idx]);\n\n    // update the covariance matrix\n    const int row_start = idx * cols;\n    for (int j = 0; j \u003c cols; ++j) {\n        const float delta_old = d_new_row[j] - d_mean[j];\n        const float delta_new = d_new_row[j] - new_mean;\n        d_cov_matrix[row_start + j] = fmaf(delta_old, delta_new / (n - 1), (n - 2) / static_cast\u003cfloat\u003e(n - 1) * d_cov_matrix[row_start + j]);\n    }\n\n    d_mean[idx] = new_mean;\n}\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzachgrayio%2Frank1_covariance","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzachgrayio%2Frank1_covariance","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzachgrayio%2Frank1_covariance/lists"}