{"id":39030111,"url":"https://github.com/pivlab/ccc-gpu","last_synced_at":"2026-01-17T17:37:26.532Z","repository":{"id":303307861,"uuid":"929170940","full_name":"pivlab/ccc-gpu","owner":"pivlab","description":null,"archived":false,"fork":false,"pushed_at":"2026-01-12T02:23:36.000Z","size":60681,"stargazers_count":1,"open_issues_count":3,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-01-12T07:51:01.877Z","etag":null,"topics":["correlation-analysis","gpu","gpu-acceleration"],"latest_commit_sha":null,"homepage":"https://ccc-gpu.readthedocs.io/en/latest/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-2-clause-patent","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pivlab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-02-07T23:58:26.000Z","updated_at":"2026-01-09T04:43:40.000Z","dependencies_parsed_at":"2025-07-06T23:32:47.222Z","dependency_job_id":"7909fcee-c96f-4128-ab7f-50f0463162cc","html_url":"https://github.com/pivlab/ccc-gpu","commit_stats":null,"previous_names":["pivlab/ccc-gpu"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/pivlab/ccc-gpu","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pivlab%2Fccc-gpu","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pivlab%2Fccc-gpu/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pivlab%2Fccc-gpu/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pivlab%2Fccc-gpu/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pivlab","download_url":"https://codeload.github.com/pivlab/ccc-gpu/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pivlab%2Fccc-gpu/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28513688,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-17T13:38:16.342Z","status":"ssl_error","status_checked_at":"2026-01-17T13:37:44.060Z","response_time":85,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["correlation-analysis","gpu","gpu-acceleration"],"created_at":"2026-01-17T17:37:26.029Z","updated_at":"2026-01-17T17:37:26.515Z","avatar_url":"https://github.com/pivlab.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Clustermatch Correlation Coefficient GPU (CCC-GPU)\n\n[![License](https://img.shields.io/badge/License-BSD%202--Clause-orange.svg)](https://opensource.org/licenses/BSD-2-Clause)\n[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)\n[![CUDA](https://img.shields.io/badge/CUDA-12.0+-green.svg)](https://developer.nvidia.com/cuda-downloads)\n[![Documentation](https://img.shields.io/badge/docs-readthedocs-blue.svg)](https://ccc-gpu.readthedocs.io/en/latest/)\n\nThe **Clustermatch Correlation Coefficient (CCC)** is a highly-efficient, next-generation correlation coefficient that captures not-only-linear relationships and can work on numerical and categorical data types. **CCC-GPU** is a GPU-accelerated implementation that provides significant performance improvements for large-scale datasets using CUDA.\n\n📖 **Full documentation available at: [https://ccc-gpu.readthedocs.io/en/latest/](https://ccc-gpu.readthedocs.io/en/latest/)**\n\nCCC is based on the simple idea of clustering data points and then computing the Adjusted Rand Index (ARI) between the two clusterings. It is a robust and efficient method that can detect linear and non-linear relationships, making it suitable for a wide range of applications in genomics, machine learning, and data science.\n\nOriginal notebooks and scripts used for the manuscript, as well as other analyses are available in the [analysis](https://github.com/pivlab/ccc-gpu/tree/main/analysis) directory.\n\n## Code Structure\n\n- **libs/ccc**: Python code for CCC-GPU\n- **libs/ccc_cuda_ext**: CUDA C++ code for CCC-GPU\n- **tests**: Test suits\n- **analysis**: Notebooks and scripts for analysis and visualization\n\n## Installation\n\n### Requirements\n\n**Hardware:**\n- Nvidia GPU with CUDA Compute Capability 8.6 or higher\n\n**Software:**\n- OS: Linux x86_64 distributions using glibc 2.28 or later, including:\n  - Debian 10+\n  - Ubuntu 18.10+\n  - Fedora 29+\n  - CentOS/RHEL 8+\n- Python 3.10 to 3.13 (3.14 will be supported soon)\n- Nvidia driver with CUDA 12.5 or higher (for GPU acceleration)\n\n\u003e **Note**: You can use command `nvidia-smi` to check your Nvidia driver and CUDA version.\n\n\u003e **Note**: If you are using another operating system, or architecture other than those noted above, you need to build from source.\n\n### Quick Install with pip\n\n\n```bash\n# Create conda environment if you want to test it out in a separate environment\n# conda create -n ccc-gpu -c conda-forge python=3.10 (or 3.11, 3.12, 3.13)\n# conda activate ccc-gpu\n\n# Install cccgpu from PyPI\npip install cccgpu\n```\n\nThen we can verify the installation using a simple one-liner:\n```bash\npython -c \"from ccc.coef.impl_gpu import ccc as ccc_gpu; import numpy as np; print(ccc_gpu(np.random.rand(100), np.random.rand(100)))\"\n```\n\nSupport for more Python versions and architectures requires extra effort, and will be added soon.\n\n### Install from Source\n\nInstall from source using the provided conda-lock environment:\n\n#### 1. Clone Repository\n\n```bash\n# Clone the repository\ngit clone https://github.com/pivlab/ccc-gpu\ncd ccc-gpu\n```\n\n#### 2. Setup Environment with conda-lock\n\nThis process uses [pipx](https://pipx.pypa.io/stable/) to install conda-lock in an isolated environment, keeping your base environment clean:\n\n\u003e **Why conda-lock?** We use conda-lock to ensure **reproducible installations** across different systems. Unlike regular `environment.yml` files, conda-lock provides exact version pins for all packages and their dependencies, preventing version conflicts and ensuring you get the same environment that was tested during development.\n\n```bash\n# Install conda-lock using pipx (installs in isolated environment)\npipx install conda-lock\n\n# Create the main ccc-gpu environment from lock file\nconda-lock install --name ccc-gpu conda-lock.yml  # or: conda-lock install --name ccc-gpu conda-lock.yml --conda mamba\n\n# Activate the main environment\nconda activate ccc-gpu\n\n# Install the package from source\npip install .\n```\n\n\n\u003e **Note**: If you prefer to use Mamba for faster package resolution, you can install MiniForge which includes Mamba:\n\u003e ```bash\n\u003e curl -L -O \"https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-$(uname)-$(uname -m).sh\"\n\u003e bash Miniforge3-$(uname)-$(uname -m).sh -b\n\u003e ```\n\u003e Then replace `conda` with `mamba` in the commands above.\n\n\n## Testing\nTo execute all the test suites, install the package from source first (or you can use the pip install command to install both cccgpu and pytest), then at the root of the repository, run:\n\n```bash\nconda activate ccc-gpu\nbash ./scripts/run_tests.sh python\n```\n\n## Usage\n### End-to-End Tutorial\n\nYou can find a tutorial showing simplified analysis steps for those we used in our paper in this [notebook](/analysis/99-tutorials/05-walkthrough-with-gtex-data.ipynb) using the public GTEx v8 data.\n\n\n### Basic Usage\n\nCCC-GPU provides a simple API identical to the original CCC implementation:\n\n```python\nimport numpy as np\n# New CCC-GPU implementation import\nfrom ccc.coef.impl_gpu import ccc\n# Original CCC implementation import\n# from ccc.coef.impl import ccc\n\n# Generate sample data\nnp.random.seed(0)\nx = np.random.randn(1000)\ny = x**2 + np.random.randn(1000) * 0.1  # Non-linear relationship\n\n# Compute CCC coefficient\ncorrelation = ccc(x, y)\nprint(f\"CCC coefficient: {correlation:.3f}\")\n```\n\n### API Specification\n\n```python\nfrom ccc.coef.impl_gpu import ccc\n\nccc(\n    x: NDArray,\n    y: NDArray = None,\n    internal_n_clusters: Union[int, Iterable[int]] = None,\n    return_parts: bool = False,\n    n_chunks_threads_ratio: int = 1,\n    n_jobs: int = 1,\n    pvalue_n_perms: int = None,\n    partitioning_executor: str = \"thread\",\n) -\u003e Union[float, NDArray, tuple]\n```\n\n**Parameters:**\n\n**Note:** Several parameters control CPU parallelization for non-core computations (e.g., `n_jobs`, `partitioning_executor`), maintaining compatibility with the original CCC implementation. Future iterations may implement these procedures with CUDA, potentially deprecating these parameters.\n\n- **`x`** *(NDArray, required)*: 1d or 2d numerical array with the data. NaN values are not supported. If 2d numpy array: features in rows, objects in columns. If pandas DataFrame: features in columns, objects in rows. If 1d: must provide `y` parameter.\n\n- **`y`** *(NDArray, optional)*: Optional 1d numerical array. Only used when `x` is 1d. Must have the same shape as `x`. Computes coefficient between `x` and `y`.\n\n- **`internal_n_clusters`** *(int or Iterable[int], optional)*: Controls the number of clusters used for partitioning. If integer: maximum number of clusters (starts from k=2). If iterable: custom list of k values. Default: range from 2 to sqrt(n_objects), capped at 10.\n\n- **`return_parts`** *(bool, default=False)*: If True, returns the partitions that maximized the coefficient for each object pair. If False, only returns CCC values.\n\n- **`n_chunks_threads_ratio`** *(int, default=1)*: Modifies how pairwise comparisons are split across threads. Controls the ratio of chunks to threads for better load balancing.\n\n- **`n_jobs`** *(int, default=1)*: Number of CPU cores/threads for internal parallelization. `None` uses all available cores. Negative values use `os.cpu_count() + n_jobs`. Must yield a value \u003e= 1.\n\n- **`pvalue_n_perms`** *(int, optional)*: If provided and \u003e 0, computes p-value using the specified number of permutations. If None or 0, p-values are not computed.\n\n- **`partitioning_executor`** *(str, default=\"thread\")*: Executor type for data partitioning. `\"thread\"` uses ThreadPoolExecutor (less memory), `\"process\"` uses ProcessPoolExecutor (potentially faster), any other value disables parallelization for partitioning.\n\n**Returns:**\n\nReturn type varies based on input dimensionality and parameters:\n\n- **For 1d input (x and y pair)**:\n  - Default: `float` (scalar CCC value)\n  - With `pvalue_n_perms`: `tuple[float, float]` (coefficient, p-value)\n  - With `return_parts=True`: adds partition arrays to the tuple\n\n- **For 2d input (multiple pairs)**:\n  - Default: `NDArray[float]` (condensed 1d array of size `n*(n-1)/2`)\n  - With `pvalue_n_perms`: `tuple[NDArray[float], NDArray[float]]` (coefficients, p-values)\n  - With `return_parts=True`: adds partition arrays to the tuple\n\n**Notes:**\n\n- Coefficient range: [0, 1] inclusive\n- Returns `np.nan` if a variable has no variation\n- Uses GPU acceleration (CUDA) for coefficient computation\n- NaN values in input data are not supported\n\n### Working with Gene Expression Data\n\nCCC-GPU is particularly useful for genomics applications:\n\n```python\nimport pandas as pd\n# New CCC-GPU implementation import\nfrom ccc.coef.impl_gpu import ccc\n# Original CCC implementation import\n# from ccc.coef.impl import ccc\n\n# Load gene expression data\n# Assume genes are in columns, samples in rows\ngene_expr = pd.read_csv('gene_expression.csv', index_col=0)\n\n# Compute gene-gene correlations\ngene_correlations = ccc(gene_expr.T)  # Transpose so genes are in rows\n\n# Find highly correlated gene pairs\nimport numpy as np\nfrom scipy.spatial.distance import squareform\n\n# Convert to square matrix\ncorr_matrix = squareform(gene_correlations)\nnp.fill_diagonal(corr_matrix, 0)  # Remove self-correlations\n\n# Find top correlations\ntop_indices = np.unravel_index(np.argsort(corr_matrix.ravel())[-10:], corr_matrix.shape)\ngene_names = gene_expr.columns.tolist()\n\nprint(\"Top 10 gene pairs by CCC:\")\nfor i, j in zip(top_indices[0], top_indices[1]):\n    print(f\"{gene_names[i]} - {gene_names[j]}: {corr_matrix[i, j]:.3f}\")\n```\n\nRefer to the original CCC Repository for more usage examples: [https://github.com/greenelab/ccc](https://github.com/greenelab/ccc)\n\n### Controlling Debug Logging\n\nBy default, CCC-GPU runs silently without debug output. You can enable detailed logging (including CUDA device information, memory usage, and processing details) using the `CCC_GPU_LOGGING` environment variable:\n\n```bash\n# Run with default behavior (no debug output)\npython your_script.py\n\n# Enable debug logging for troubleshooting\nCCC_GPU_LOGGING=1 python your_script.py\n\n# Or set it for the session\nexport CCC_GPU_LOGGING=1\npython your_script.py\n```\n\nThis is particularly useful for:\n- Debugging GPU memory issues\n- Understanding CUDA device utilization\n- Monitoring batch processing performance\n- Troubleshooting installation problems\n\n## Performance Benchmarks\n\nCCC-GPU provides significant performance improvements over CPU-only implementations:\n\n| Number of genes | CCC-GPU vs. CCC (12 cores) |\n|---|---|\n| 500 | 16.52 |\n| 1000 | 30.65 |\n| 2000 | 45.72 |\n| 4000 | 59.46 |\n| 6000 | 67.46 |\n| 8000 | 71.48 |\n| 10000 | 72.38 |\n| 16000 | 73.83 |\n| 20000 | 73.88 |\n\n*Benchmarks performed on synthetic gene expression data with 1000 fixed samples. Hardware: AMD Ryzen Threadripper 7960X CPU and an NVIDIA RTX 4090 GPU. Git commit on which the benchmark results were collected: 05f129dfa47ad801eff963b4189484c7c64bd28e*\n\n## Documentation\n\nBuild and view the full documentation locally:\n\n```bash\ncd docs\nmake html\n```\n\nThen open `docs/build/html/index.html` in your browser.\n\nIf using VS Code, the `Live Preview` extension provides convenient in-editor viewing.\n\n## Citation\n\nIf you use CCC-GPU in your research, please cite the original CCC paper:\n\n```bibtex\n@article{zhang2025cccgpu,\n  title={CCC-GPU: A graphics processing unit (GPU)-optimized nonlinear correlation coefficient for large transcriptomic analyses},\n  author={Zhang, Haoyu and Fotso, Kevin and Pividori, Milton},\n  journal={bioRxiv},\n  year={2025},\n  doi={10.1101/2025.06.03.657735},\n}\n```\n\n## Contributing\n\nContributions are welcome! Please feel free to submit a Pull Request. For major changes, please open an issue first to discuss what you would like to change.\n\n## License\n\nThis project is licensed under the BSD 2-Clause License - see the [LICENSE](LICENSE) file for details.\n\n## Acknowledgments\n\n- Original CCC implementation: [https://github.com/greenelab/ccc](https://github.com/greenelab/ccc)\n- CUDA development team for the excellent CUDA toolkit\n- pybind11 for seamless Python-C++ integration\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpivlab%2Fccc-gpu","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpivlab%2Fccc-gpu","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpivlab%2Fccc-gpu/lists"}