{"id":49221754,"url":"https://github.com/ekin-kahraman/rustscenic","last_synced_at":"2026-05-08T00:02:19.660Z","repository":{"id":352651220,"uuid":"1215149288","full_name":"Ekin-Kahraman/rustscenic","owner":"Ekin-Kahraman","description":"Rust + PyO3 reimplementation of the SCENIC+ single-cell regulatory-network pipeline. ATAC preproc, peak calling, GRN, AUCell, topics, cistarget, eRegulon — one pip install, no Java, no MACS2, no dask.","archived":false,"fork":false,"pushed_at":"2026-05-04T03:06:05.000Z","size":11406,"stargazers_count":6,"open_issues_count":2,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-04T04:17:18.065Z","etag":null,"topics":["atac-seq","aucell","bioinformatics","gene-regulatory-network","pyo3","pyscenic","regulon","rust","scatac-seq","scenic","scenicplus","scrna-seq","single-cell"],"latest_commit_sha":null,"homepage":"https://github.com/Ekin-Kahraman/rustscenic#readme","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Ekin-Kahraman.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":".github/CODEOWNERS","security":"SECURITY.md","support":null,"governance":null,"roadmap":"docs/roadmap-v0.2.md","authors":"AUTHORS.md","dei":null,"publiccode":null,"codemeta":null,"zenodo":".zenodo.json","notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-04-19T14:47:24.000Z","updated_at":"2026-05-04T03:05:24.000Z","dependencies_parsed_at":null,"dependency_job_id":"334bd21c-9be2-4ba2-864a-9d162e9e8ca5","html_url":"https://github.com/Ekin-Kahraman/rustscenic","commit_stats":null,"previous_names":["ekin-kahraman/rustscenic"],"tags_count":16,"template":false,"template_full_name":null,"purl":"pkg:github/Ekin-Kahraman/rustscenic","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ekin-Kahraman%2Frustscenic","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ekin-Kahraman%2Frustscenic/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ekin-Kahraman%2Frustscenic/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ekin-Kahraman%2Frustscenic/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Ekin-Kahraman","download_url":"https://codeload.github.com/Ekin-Kahraman/rustscenic/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ekin-Kahraman%2Frustscenic/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32760963,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-07T02:14:30.463Z","status":"ssl_error","status_checked_at":"2026-05-07T02:14:29.405Z","response_time":62,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["atac-seq","aucell","bioinformatics","gene-regulatory-network","pyo3","pyscenic","regulon","rust","scatac-seq","scenic","scenicplus","scrna-seq","single-cell"],"created_at":"2026-04-24T04:04:36.541Z","updated_at":"2026-05-08T00:02:19.647Z","avatar_url":"https://github.com/Ekin-Kahraman.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# rustscenic\n\n[![CI](https://github.com/Ekin-Kahraman/rustscenic/actions/workflows/audit.yml/badge.svg)](https://github.com/Ekin-Kahraman/rustscenic/actions/workflows/audit.yml)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)\n[![Python](https://img.shields.io/badge/Python-3.10%2B-blue)](https://www.python.org/)\n[![Rust](https://img.shields.io/badge/Rust-stable-orange)](https://www.rust-lang.org/)\n\nA Rust + PyO3 replacement for the SCENIC / SCENIC+ compute stack: one install, modern Python, low-memory CPU execution, and atlas-scale regulatory-network analysis without Java, dask, CUDA, or fragile multi-tool environments.\n\n```bash\npip install rustscenic\n```\n\nFive runtime dependencies (numpy, pandas, pyarrow, scipy, anndata). Python 3.10–3.13, Linux + macOS (x86_64 + aarch64). No dask, no Java, no CUDA.\n\nThe practical SCENIC+ compute path in one package:\n\n```mermaid\nflowchart LR\n    rna[\"RNA\u003cbr/\u003eAnnData\"] --\u003e grn[\"GRN\"]\n    atac[\"ATAC\u003cbr/\u003eAnnData/fragments\"] --\u003e chrom[\"topics\u003cbr/\u003ecisTarget\u003cbr/\u003eenhancer links\"]\n    grn --\u003e ereg[\"eRegulons\"]\n    chrom --\u003e ereg\n    ereg --\u003e auc[\"AUCell\u003cbr/\u003ecells x regulons\"]\n```\n\n## Status\n\n**Current release: v0.4.1** on PyPI. v0.4.0 established publishable real-data end-to-end on PBMC and mouse brain E18 multiome via the public `pipeline.run`; v0.4.1 fixes `pipeline.run(tfs=\"hs\"/\"mm\")` species shortcuts. See [CHANGELOG](CHANGELOG.md) and [`validation/`](validation/) for evidence and caveats.\n\n## Goal\n\nrustscenic is being built as the single-install replacement for the practical SCENIC / SCENIC+ workflow: RNA GRN inference, AUCell regulon activity, motif enrichment, ATAC fragment preprocessing, topic modelling, enhancer-gene linking, and eRegulon assembly in one package.\n\nThe project is intentionally not a thin wrapper around the old stack. The target is a simpler architecture that makes regulatory-network analysis easier to install, cheaper to run on CPU, deterministic under a fixed seed, and robust to real atlas conventions such as ENSEMBL `var_names`, duplicate gene symbols, backed AnnData, and UCSC/Ensembl chromosome mismatches.\n\n## What it does\n\nRust-native replacements for the compute stages plus the glue that scenicplus builds eRegulons from:\n\n| Stage | **rustscenic** | Replaces |\n|---|---|---|\n| Gene-regulatory network inference | `rustscenic.grn.infer` | `arboreto.grnboost2` |\n| Per-cell regulon activity scoring | `rustscenic.aucell.score` | `pyscenic.aucell.aucell` |\n| Topic modelling on scATAC peaks (Online VB) | `rustscenic.topics.fit` | `pycisTopic` (gensim VB) |\n| Topic modelling K ≥ 30 (Mallet-class collapsed Gibbs) | `rustscenic.topics.fit_gibbs` | `pycisTopic` (Mallet, Java) |\n| Motif-regulon enrichment | `rustscenic.cistarget.enrich` | `pycistarget` AUC kernel |\n| ATAC fragments → cells × peaks matrix | `rustscenic.preproc.fragments_to_matrix` | `pycisTopic` fragment loader |\n| Cell QC (TSS enrichment, FRiP, insert size) | `rustscenic.preproc.qc` | `pycisTopic.qc` |\n| Enhancer → gene correlation | `rustscenic.enhancer.link_peaks_to_genes` | `scenicplus` p2g linking |\n| eRegulon assembly (TF × enhancers × target genes) | `rustscenic.eregulon.build_eregulons` | `scenicplus` eRegulon builder |\n| End-to-end pipeline orchestrator | `rustscenic.pipeline.run` | `scenicplus` snakemake |\n\nBundled with the wheel: HGNC (1,839 human) and MGI (1,721 mouse) TF lists via `rustscenic.data.tfs(species)`. Motif rankings can be fetched and cached via `rustscenic.data.download_motif_rankings`. Cellxgene-curated h5ads (ENSEMBL IDs in `var_names`, gene symbols in `var[\"feature_name\"]`) are auto-detected so atlas data works without manual patching.\n\n## Quick example (PBMC-3k, RNA GRN + AUCell)\n\n```python\nimport anndata as ad\nimport rustscenic.grn, rustscenic.aucell\n\nadata = ad.read_h5ad(\"rna.h5ad\")\ntfs = rustscenic.grn.load_tfs(\"hs_hgnc_tfs.txt\")\n\n# 1. GRN inference\ngrn = rustscenic.grn.infer(adata, tf_names=tfs, n_estimators=5000, seed=777)\n\n# 2. Build top-50-target regulons and score per-cell activity\nregulons = [\n    (f\"{tf}_regulon\", grn[grn[\"TF\"] == tf].nlargest(50, \"importance\")[\"target\"].tolist())\n    for tf in grn[\"TF\"].unique()\n]\nauc = rustscenic.aucell.score(adata, regulons, top_frac=0.05)\n```\n\nFull RNA example script: [`examples/pbmc3k_end_to_end.py`](examples/pbmc3k_end_to_end.py). Runs in about 3 minutes on an 8-core laptop with `n_estimators=500`. [`docs/tester-quickstart.md`](docs/tester-quickstart.md) is the collaborator smoke-test path.\n\n## Measured against the pyscenic / arboreto reference\n\nSame input on both sides. Every row has a log file under [`validation/`](validation/).\n\n| Axis | pyscenic / arboreto | **rustscenic** |\n|---|---|---|\n| Installs on fresh Python 3.10–3.13 venv | arboreto: `TypeError: Must supply at least one delayed object` (dask_expr); pyscenic: `ModuleNotFoundError: pkg_resources` in current stacks | PyPI wheels and sdist install; core APIs import |\n| AUCell wall-time, Ziegler 2021 atlas (31,602 × 59) | 6.81 s (pyscenic) | 0.25 s |\n| AUCell wall-time, 10x Multiome (10,290 × 1,457) | 18.6 s (pyscenic) | 0.21 s |\n| Peak RSS, 4 stages on 100,000 cells × 20,292 genes | \u003e 40 GB (reported) | 6.3 GB |\n| Cistarget kernel vs `ctxcore.recovery.aucs` | reference | Pearson 1.0000, mean abs diff 2.4 × 10⁻⁵ |\n| AUCell per-cell Pearson vs pyscenic (Ziegler, 31,602 cells) | reference | 0.984 mean, 91.7 % of cells \u003e 0.95 |\n| Canonical airway TFs matching literature (Ziegler, n=14) | 8 / 14 (pyscenic, unit weights) | 8 / 14 — same hits, same 5/14 misses |\n| Bit-identical output under same seed across threaded runs | no (dask non-determinism) | yes |\n| Runtime dependencies | 40 + | 5 |\n\nTool-to-tool variation (same hits, same misses on the same 14 canonical TFs) is smaller than the dataset-inherent noise, consistent with rustscenic being numerically equivalent to pyscenic at the per-cell level.\n\n## Per-stage detail\n\nNumbers are **rustscenic**'s values. The measurement context (dataset, `n_cells`, etc.) is in each row.\n\n### GRN — `arboreto.grnboost2` replacement\n\n| Measurement | Value |\n|---|---|\n| Per-edge Spearman vs arboreto (PBMC-3k scanpy, n_estimators=5000, 480,680 shared edges, v0.3.10) | 0.611 |\n| Within-TF Spearman, mean across 1,274 TFs (same fixture) | 0.632 (median 0.649) |\n| Per-edge Spearman vs arboreto (multiome3k, n_estimators=5000, 816 k common edges, 2026-04) | 0.58 |\n| Per-target TF-ranking Spearman mean | 0.57 |\n| TRRUST known TF→target edges recovered (PBMC-3k) | 17 / 18 (94 %) |\n| Lineage TFs correctly enriched in expected cell types (PBMC-10k) | 8 / 8 (SPI1, PAX5, EBF1, TCF7, LEF1, TBX21, CEBPD, IRF8) |\n| Cortex marker TFs present in regulon set (E18 multiome, 4,770 cells, v0.3.10; name-presence, not cell-type enrichment) | 9 / 9 (Pax6, Neurod2, Sox2, Ascl1, Tbr1, Neurog2, Fezf2, Eomes, Foxg1) |\n| MITF regulon activity, Tirosh 2016 melanoma — malignant vs TME | 3.48× |\n| Wall vs pyscenic on PBMC-3k (n_estimators=5000, seed 777, Apple M5, v0.3.10; pyscenic in sync mode — not apples-to-apples against dask-parallel) | 214 s vs 381 s (1.78×) |\n| 100k-cell bootstrap, n_estimators=100 | 17 min / 5.0 GB peak RSS |\n\nEdge rankings disagree with arboreto at fine grain (per-edge Spearman 0.611 on PBMC-3k v0.3.10 / 0.58 on multiome3k 2026-04, top-10k Jaccard 0.20) — expected consequence of independent histogram-GBM quantisation. Coarse biology converges (per-TF Spearman ≈ 0.65, all canonical lineage TFs recovered on both human PBMC and mouse cortex). Downstream AUCell is 0.99 per-cell with pyscenic, so edge-ranking differences do not propagate.\n\n### AUCell — `pyscenic.aucell` replacement\n\n| Measurement | Value |\n|---|---|\n| Per-cell Pearson vs pyscenic (10x Multiome, 2,588 × 1,457) | 0.988 mean, 99.5 % of cells \u003e 0.95 |\n| Per-cell Pearson vs pyscenic (Ziegler atlas, 31,602 × 59) | 0.984 mean, 91.7 % of cells \u003e 0.95 |\n| Per-regulon Pearson (10x Multiome) | 0.87 mean, 90.5 % \u003e 0.80 |\n| Exact top-regulon-per-cell match (Multiome) | 88.4 % |\n| Wall-time, 10k cells × 1,457 regulons | 0.21 s (vs 18.6 s pyscenic) |\n| 100 k cells × 500 regulons | 10 s, 5.6 GB peak RSS |\n\n### Topics — `pycisTopic` LDA replacement (Online VB + collapsed Gibbs)\n\nTwo algorithms ship side-by-side:\n- `rustscenic.topics.fit` — Online VB LDA, fastest at K ≤ 10.\n- `rustscenic.topics.fit_gibbs` — collapsed Gibbs (Mallet's algorithm class). Add `n_threads=N` for parallel AD-LDA.\n\nReal PBMC 3k Multiome ATAC, 1,500 cells × 98,319 peaks, K = 30, intrinsic top-10 NPMI on the training corpus:\n\n| Tool | Wall | Unique topics (of 30) | Top-10 NPMI mean |\n|---|---|---|---|\n| `rustscenic.topics.fit` (Online VB) | 104 s | 2 / 30 (collapsed) | +0.012 |\n| `rustscenic.topics.fit_gibbs` (serial) | 191 s | **22 / 30** | **+0.031** |\n| `rustscenic.topics.fit_gibbs` (8-thread) | **84 s** | 25 / 30 | +0.019 |\n| Mallet (pycisTopic reference) | n/a | 24 / 30 | 0.196 (extrinsic) |\n\nCollapsed Gibbs gives ~11× more distinct topics than Online VB on sparse scATAC at K = 30 and ~2.7× higher intrinsic NPMI; the parallel AD-LDA path adds a 2.56× wall-clock speedup at 8 threads while preserving topic diversity. Mallet's published 0.196 is an extrinsic NPMI (different protocol, not directly comparable in absolute scale). See [`docs/topic-collapse.md`](docs/topic-collapse.md) and [`docs/bench-vs-references.md`](docs/bench-vs-references.md). Reproduce with `python validation/scaling/bench_npmi_head_to_head.py` and `python validation/scaling/bench_gibbs_parallel.py`.\n\n### Cistarget — `pycistarget` AUC kernel replacement\n\nValidated on the aertslab hg38 v10 feather database (5,876 motifs × 27,015 genes):\n\n| Measurement | Value |\n|---|---|\n| Per-regulon Pearson vs `ctxcore.recovery.aucs` (58 TRRUST regulons) | 1.0000 (all \u003e 0.9999, abs diff 2.4 × 10⁻⁵) |\n| Self-consistency (motif's own top-500 genes → rank #1) | 10 / 10 |\n| TRRUST at scale (166 TFs ≥ 10 targets): TF-annotated motif ranks #1 | 19 % |\n| Same benchmark: any TF-motif in top-100 | 68 – 100 % (rises with regulon size) |\n| Mouse mm10 cross-species (5 TRRUST TFs) | 2 / 5 rank #1, 4 / 5 in top-5 |\n| 100 k-cell workload × 100 regulons | 2.6 s, 6.3 GB peak RSS |\n\nBit-identical to `ctxcore.recovery.aucs` at float32 precision. The 19 % rank-#1 rate is the scaled-out TRRUST-vs-motif-binding benchmark, a property of the gold-standard mismatch, not the implementation.\n\n### End-to-end + determinism\n\n| Pipeline | Wall | Peak RSS | Stages |\n|---|---|---|---|\n| Reference (arboreto + pyscenic + tomotopy), 10x Multiome 3k | 11.8 min | n/a | 4 |\n| rustscenic, 10x Multiome 3k | 9.1 min | n/a | 4 |\n| rustscenic, **100k synthetic multiome E2E** | **12.7 min** | **7.09 GB** | **7 (all)** |\n| rustscenic, **200k synthetic multiome E2E** | **16.8 min** | **7.44 GB** | **7 (all)** |\n\nMemory: 100k synthetic multiome 7-stage E2E peaks at **7.09 GB RSS**, vs scenicplus stack's reported \u003e 40 GB at comparable scale. Bit-identical output under the same seed across threaded runs, verified across three consecutive runs per stage. 10 / 10 robustness edge-case tests pass (foreign genes, NaN input, duplicate gene names, all-zero cells, large regulons, object-dtype rankings, n_topics = 0, very-sparse matrices). Reproduce with `python validation/scaling/bench_e2e_100k_synthetic.py`; reproduce the 200k synthetic run with `python validation/scaling/bench_e2e_200k_synthetic.py`.\n\n## Scope and alternatives\n\nrustscenic covers the practical SCENIC / SCENIC+ compute path on CPU. Adjacent tools with different scope:\n\n- **GPU, CUDA** — [flashSCENIC](https://github.com/haozhu233/flashscenic) (uses RegDiffusion, a different algorithm from GENIE3 / GRNBoost2, so outputs are not pyscenic-numerical).\n- **Multiomic enhancer-aware GRN** — [scenicplus](https://github.com/aertslab/scenicplus) (joint scRNA + scATAC enhancer inference; superset of this scope).\n- **TF-activity scoring from prebuilt regulons, no GRN inference** — [decoupler-py](https://saezlab.github.io/decoupler-py/) with CollecTRI.\n- **R Bioconductor ecosystem** — the original R-SCENIC or [Epiregulon](https://www.nature.com/articles/s41467-025-62252-5).\n\nrustscenic does not bundle the aertslab motif ranking feather databases (300 MB – 35 GB). Users fetch them from [`resources.aertslab.org`](https://resources.aertslab.org/) and pass the resulting DataFrame to `cistarget.enrich`.\n\n## CLI\n\n```bash\nrustscenic grn       --expression data.h5ad --tfs tfs.txt --output grn.parquet\nrustscenic aucell    --expression data.h5ad --regulons grn.parquet --output auc.parquet\nrustscenic topics    --expression atac.h5ad --output topics --n-topics 30\nrustscenic cistarget --rankings motifs.feather --regulons grn.parquet --output enrichment.tsv\n```\n\n## Repo layout\n\n- `crates/` — Rust workspace: `rustscenic-{grn, aucell, topics, preproc, py}`\n- `python/rustscenic/` — Python package, CLI entry point, type stubs\n- `examples/pbmc3k_end_to_end.py` — RNA GRN + AUCell script on real PBMC-3k\n- `validation/` — reproducible benchmark scripts + measurement reports for every number above, plus `VALIDATION_SUMMARY.md`\n- `tests/` — pytest suite (152 Python tests, 1 skipped) + Rust crate tests (57)\n- `manuscript/` — preprint source\n- `docs/topic-collapse.md` — known algorithmic caveat\n\n## License\n\nMIT. Algorithm implementations follow the aertslab Python references — original method credit to Aibar et al. 2017 (SCENIC), Bravo González-Blas et al. 2023 (SCENIC+), Hoffman-Blei-Bach 2010 (Online VB LDA).\n\n## Citation and attribution\n\nIf you use rustscenic in a paper, report, benchmark, derivative package, or lab workflow, cite the exact release used. GitHub citation metadata is in [`CITATION.cff`](CITATION.cff).\n\nrustscenic was created and is maintained by Ekin Kahraman. See [`AUTHORS.md`](AUTHORS.md) and [`docs/collaboration-and-authorship.md`](docs/collaboration-and-authorship.md) for contribution and authorship expectations.\n\n## Contact\n\nFile issues at [github.com/Ekin-Kahraman/rustscenic/issues](https://github.com/Ekin-Kahraman/rustscenic/issues). Bug, correctness, and validation-report templates pre-fill the fields we need. If you ran the pipeline on real data and want the result folded into the v0.4.x sweep, see [`docs/tester-reporting.md`](docs/tester-reporting.md). Coordinated vulnerability disclosure: see [SECURITY.md](SECURITY.md).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fekin-kahraman%2Frustscenic","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fekin-kahraman%2Frustscenic","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fekin-kahraman%2Frustscenic/lists"}