{"id":37070381,"url":"https://github.com/lalalabox/healformers","last_synced_at":"2026-01-14T08:12:44.040Z","repository":{"id":297012985,"uuid":"995162306","full_name":"lalalabox/healformers","owner":"lalalabox","description":"Mask-Aware HEALPix Transformers","archived":false,"fork":false,"pushed_at":"2025-06-12T05:34:37.000Z","size":4937,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-09-28T04:21:40.266Z","etag":null,"topics":["cosmology","denoising","healpix","inpainting","spherical","transformer","weak-lensing"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lalalabox.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-06-03T04:11:22.000Z","updated_at":"2025-06-12T05:34:23.000Z","dependencies_parsed_at":"2025-06-03T22:14:00.349Z","dependency_job_id":"10bd5dd5-a47e-49c6-adb9-1e8f4e148466","html_url":"https://github.com/lalalabox/healformers","commit_stats":null,"previous_names":["lalalabox/healformers"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/lalalabox/healformers","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lalalabox%2Fhealformers","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lalalabox%2Fhealformers/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lalalabox%2Fhealformers/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lalalabox%2Fhealformers/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lalalabox","download_url":"https://codeload.github.com/lalalabox/healformers/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lalalabox%2Fhealformers/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28413633,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-14T05:26:33.345Z","status":"ssl_error","status_checked_at":"2026-01-14T05:21:57.251Z","response_time":107,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cosmology","denoising","healpix","inpainting","spherical","transformer","weak-lensing"],"created_at":"2026-01-14T08:12:43.421Z","updated_at":"2026-01-14T08:12:44.018Z","avatar_url":"https://github.com/lalalabox.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🌌 HealFormers: Mask-Aware HEALPix Transformers\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./imgs/healformer_architecture.jpg\"  alt=\"architecture\" width=\"500\"\u003e\n\u003c/p\u003e\n\n**HealFormer** is a cutting-edge transformer model specifically designed for data structured on the **HEALPix grid**, widely used in **cosmology**, **astrophysics**, and **large-scale structure analysis**. HealFormer natively manages incomplete sky observations with state-of-the-art precision, eliminating the need for projections or specialized spherical convolutions, and scales effortlessly to large astronomical surveys.\n\n\u003e [!NOTE]  \n\u003e\n\u003e * Source code will be publicly available after paper acceptance.\n\u003e * Pretrained models and datasets will be hosted on [🤗 HuggingFace](https://huggingface.co/).\n\n---\n\n## 🌠 Why Choose HealFormer?\n\nTraditional spherical analysis methods often struggle with partial-sky coverage and computational efficiency. HealFormer is uniquely designed to solve these challenges:\n\n| 🚩 Pain Points                        | 🎯 HealFormer Solutions                              |\n| ------------------------------------- | ---------------------------------------------------   |\n| Inefficient mask handling             | ✅ Direct mask-aware learning                        |\n| Distortion from projections           | ✅ Native HEALPix operations (no projections needed) |\n| Poor scalability for high resolutions | ✅ Efficient from Nside=256 up to Nside=1024+        |\n| Expensive model training              | ✅ LoRA-based tuning reduces cost by 90%+            |\n| Limited generalization                | ✅ Strong transfer learning and generalization       |\n\n---\n\n## 🌟 Key Features\n\n* **Mask Awareness:** Directly processes masked regions; adapts to arbitrary mask sizes and shapes.\n* **Native HEALPix Integration:** No need for projection or spherical approximation; maintains full data integrity.\n* **State-of-the-Art Performance:** Exceeds Wiener filter and Kaiser-Squires both in pixel space and harmonic space.\n* **Unified Masking:** A single model supports various mask patterns and sky coverage (e.g. KiDS, DES, DECaLS, Planck), without retraining.\n* **Efficient Transfer Learning:** LoRA-based fine-tuning reduces trainable parameters to ~10%, enabling efficient transfer learning.\n* **Scalable \u0026 Generalizable:** Efficiently scales from low (Nside=256) to high (Nside=1024+) resolutions; generalizes robustly across different cosmological parameters.\n\n---\n\n## 📦 Installation\n\nInstall HealFormer easily via pip:\n\n```bash\npip install healformers\n```\n\nRequirements: Python 3.11+, healpy, torch, transformers, etc. (See `pyproject.toml` for details)\n\n---\n\n## 🚀 Quickstart Example: Weak-Lensing Mass Mapping\n\nMinimal working example to reconstruct a kappa map:\n\n```python\nimport healpy as hp\nimport torch\nfrom healformers import HealFormerModel, Mock\n\n# Generate mock data (gamma1, gamma2)\nbatch = Mock.generate_full_batch(\n    nside=256, mask_type=\"decals\", batch_size=1, return_type=\"torch\"\n)\nkappa_true = batch[\"targets\"][0, -1]\n\n# Load pretrained model\nmodel_path = \"path_to_model_directory\"\nmodel = HealFormerModel.from_pretrained(model_path)\n\n# Predict kappa map\nwith torch.no_grad():\n    kappa_pred = model(**batch)[\"logits\"][0, 0]\n\n# Visualization\nhp.mollview(kappa_true.numpy(), nest=True, title=\"True Kappa\", sub=(121))\nhp.mollview(kappa_pred.numpy(), nest=True, title=\"Reconstructed Kappa\", sub=(122))\n```\n\n---\n\n## 🛰️ Scientific Applications\n\n- **Weak lensing mass mapping** under realistic, incomplete sky coverage – ✅ **Ready**\n- **Power spectrum estimation** on irregular spherical masks – 🔜 *Coming soon*\n- **Field-level cosmological inference** from partial-sky data – 🔜 *Coming soon*\n\n---\n\n## 🎨 Visualization Showcase\n\n**1. Clean Map Reconstruction (w/ mask)**\n\n*Kaiser-Squires (KS) vs Wiener filter (WF) vs HealFormer (HF)*\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./imgs/mask_effect_nside256_maskDECaLS_noiseFalse.jpg\"  alt=\"mask_effect\" width=\"500\"\u003e\n\u003c/p\u003e\n\n**2. Noisy Map Reconstruction**\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./imgs/noise_effect_nside256_maskDECaLS_noiseTrue.jpg\"  alt=\"noise_effect\" width=\"500\"\u003e\n\u003c/p\u003e\n\n**3. Residuals Across Diverse Masks**\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./imgs/compare_allMask_residual_nside256.jpg\"  alt=\"residual_allMask\" width=\"500\"\u003e\n\u003c/p\u003e\n\n---\n\n## 🧩 Model Zoo \u0026 Resources\n\n* 📦 **Pretrained models:** *Coming soon*\n* 📚 **Fine-tuning guides:** *Coming soon*\n\n---\n\n## 📄 Citation\n\nIf you utilize HealFormer, please cite:\n\n```\n[Your citation here after publication]\n```\n\n---\n\n## 🤝 Contribution \u0026 Community\n\nWe warmly welcome contributions, feedback, and bug reports!\n\n* Open an issue on [GitHub Issues](https://github.com/lalalabox/healformers/issues)\n* Submit pull requests for direct contributions.\n\n---\n\n## ⚙️ Built With\n\nSpecial thanks to frameworks and models enabling this work:\n\n* [**PyTorch**](https://pytorch.org/)\n* [**🤗 Transformers \u0026 PEFT (LoRA)**](https://github.com/huggingface/transformers)\n* [**Masked Autoencoders (MAE)**](https://github.com/facebookresearch/mae)\n\n---\n\n## 📜 License\n\nLicensed under **Apache-2.0**. See [LICENSE](https://github.com/lalalabox/healformers/blob/main/LICENSE) for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flalalabox%2Fhealformers","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flalalabox%2Fhealformers","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flalalabox%2Fhealformers/lists"}