{"id":30799812,"url":"https://github.com/imoonlab/hghc","last_synced_at":"2025-09-05T19:54:35.636Z","repository":{"id":310733068,"uuid":"1041015148","full_name":"iMoonLab/HGHC","owner":"iMoonLab","description":"This repository contains the code and data for the paper “Hypergraph-Based High-Order Correlation Analysis for Large-Scale Long-Tailed Data.”","archived":false,"fork":false,"pushed_at":"2025-08-19T21:41:02.000Z","size":28,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-08-19T23:29:33.130Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/iMoonLab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-08-19T21:13:41.000Z","updated_at":"2025-08-19T21:41:05.000Z","dependencies_parsed_at":"2025-08-19T23:29:35.639Z","dependency_job_id":"54b1da77-f15d-4fff-b057-df6e350b134f","html_url":"https://github.com/iMoonLab/HGHC","commit_stats":null,"previous_names":["imoonlab/hghc"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/iMoonLab/HGHC","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iMoonLab%2FHGHC","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iMoonLab%2FHGHC/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iMoonLab%2FHGHC/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iMoonLab%2FHGHC/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/iMoonLab","download_url":"https://codeload.github.com/iMoonLab/HGHC/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iMoonLab%2FHGHC/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":273813275,"owners_count":25172892,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-05T02:00:09.113Z","response_time":402,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-09-05T19:54:32.098Z","updated_at":"2025-09-05T19:54:35.627Z","avatar_url":"https://github.com/iMoonLab.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003e This repository contains the code and data for the paper “Hypergraph-Based High-Order Correlation Analysis for Large-Scale Long-Tailed Data.”\n\n## Highlights\n\n- **HSMOTE**: Lightweight dual encoder–decoder (`models.HSMOTE`) trained with BCE reconstruction and negative sampling on node–hyperedge incidence.\n- **Tail oversampling via synthesis**: Interpolation in embedding space + **Bernoulli edge membership sampling** over class-aware candidate sets (`synthesize_vertices_bernoulli`), with **Top-M per-class popular edges**, candidate capping, and class-specific degree targets.\n- **Global structure aggregation**: Build **row-normalized Top-k PPR** on the item–user bipartite graph, keep item→item mass, and **fine-tune `encoder_node + LinearHead` with fixed P**.\n- **Inference-time diffusion**: 3-step lightweight diffusion (`utils.diffuse_3steps`) to smooth representations before classification.\n\n## Repository Structure\n\n```\nrepo/\n├── config.py                 # Hyperparameters \u0026 paths (edit to your data locations)\n├── main.py                   # End-to-end: load → hypergraph → HSMOTE → synth → PPR → finetune → eval\n├── models.py                 # HSMOTE and LinearHead\n├── load_data.py              # Loading/alignment, subgraph builder, BCE w/ negative sampling, helpers\n├── utils.py                  # Training utils, metrics, PPR builders, diffusion, memmap, etc.\n├── ppr.py                    # PPR\n├── data/                     # raw features/labels/user JSON\n├── cache/                    # runtime caches\n└── README.md\n```\n\n## Data \u0026 Formats\n\n`config.py` points to your files (update paths as needed):\n\n```python\n# Example fields — adapt to your setup\ndata_dir   = \"Datasets/subset\"\nasins_pkl  = \"asins_**.pkl\"       \nfeat_pkl   = \"features_**.pkl\"    \nlab_pkl    = \"labels_**.pkl\"      \nuser_json  = \"user_products_1000.json\"\n```\n\n**Expected formats**\n\n- `features_**.pkl`: `{'features': float32 [N, d], 'asins': List[str]}`\n\n- `labels_**.pkl`:   `{'labels': int64 [N]}`\n\n- `asins_**.pkl`: several schemas are supported; `load_idx2node_from_asins_pkl` auto-parses and aligns\n\n- `user_products_1000.json`: user–item interactions, e.g.:\n\n  ```json\n  [\n    {\"user\": \"u1\", \"items\": [{\"prefix\":\"P\",\"asin\":\"A1\"}, {\"prefix\":\"P\",\"asin\":\"A2\"}]},\n    {\"user\": \"u2\", \"items\": [{\"asin\":\"B3\"}]}\n  ]\n  ```\n\n**Hypergraph construction \u0026 cache**\n\n- Datasets download: https://drive.google.com/file/d/1v8nXKoIrd7bmfGZyW6N3_WT0dZUASDrw/view?usp=sharing\n\n- `load_hypergraph` builds CSR for item→user (`offsets_i`, `indices_users`) and user→item (`offsets_u`, `indices_items`).\n- A stable cache key (from `kept_asin2row`) is used to write `.csr-\u003chash\u003e.npz`, `.users.json`, and `.meta.json` for fast reloads.\n\n## Quick Start (End-to-End)\n\nRun:\n\n```bash\npython main.py\n```\n\nPipeline in `main.py`:\n\n1. **Load \u0026 align**: `load_and_align` aligns features/labels to `idx2node` → `X, y`.\n2. **Build hypergraph**: `load_hypergraph` → CSR `H`; compute full edge features `Xe_full`.\n3. **Split**: `stratified_split_ratio` per class.\n4. **Train HSMOTE**: `train_hsmote_on_tails` on tail classes via batch subgraphs (BCE + MSE).\n5. **Synthesize**: `compute_ze_full_memmap` encodes edge embeddings `Ze` (memmap). `synthesize_vertices_bernoulli` creates new tail nodes + merges dataset.\n6. **Build PPR**: `build_P_items_topk_bipartite` on the bipartite graph, keep item→item rows, re-normalize → `P_hat`.\n7. **Fine-tune \u0026 evaluate**: `finetune_with_P_fixed` (fixed `P_hat`) on train/val; inference uses 3-step diffusion.\n\nConsole logs include `[HSMOTE]`, `[PPR]`, `[FT]` (fine-tuning), and final `[TEST]` metrics.\n\n## Configuration (Key Hyperparameters)\n\nAdjust in `config.py` :\n\n- **Model**: `d_in`, `d_hid`, `d_embed`\n- **HSMOTE pretraining**: `lr_hsmote`, `wd_hsmote`, `beta_mse`, `neg_per_pos`, `hsmote_batches`, `per_class_train`, `log_every`\n- **Tail threshold**: `tail_threshold` (relative to the largest class)\n- **Synthesis**: `target_tail_ratio`, `topM`, `cand_cap_per_syn`, `max_edges_per_syn`, `ze_score_chunk`, `block_new_nodes`\n- **PPR**: `ppr_alpha`, `ppr_eps`, `ppr_topk`, `ppr_chunk`\n- **Fine-tuning**: `lr_enc`, `wd_enc`, `lr_head`, `wd_head`, `ft_epochs`, `precls_patience`, `bs_seeds`\n- **Inference**: `infer_alpha`, `infer_steps`, `encode_bs`\n\n------\n\n## Contact\n\n- Maintainer: **Xiangmin Han / Tsinghua University**\n- Email: **simon.xmhan@gmail.com_**\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fimoonlab%2Fhghc","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fimoonlab%2Fhghc","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fimoonlab%2Fhghc/lists"}