{"id":28375678,"url":"https://github.com/farukalpay/information-bottleneck-beta-optimization","last_synced_at":"2025-06-26T05:31:29.557Z","repository":{"id":291948260,"uuid":"979288531","full_name":"farukalpay/information-bottleneck-beta-optimization","owner":"farukalpay","description":"This repository hosts a progressive series of implementations (Code_v1, Code_v2, and beyond) for deterministic β*-optimization in the Information Bottleneck framework. Includes symbolic fusion, multi-path inference, and Alpay Algebra–driven critical point validation (β* = 4.14144).","archived":false,"fork":false,"pushed_at":"2025-05-15T19:23:11.000Z","size":3714,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-06-05T23:26:28.915Z","etag":null,"topics":["alpay-algebra","beta-optimization","deep-theory","ib-framework","information-bottleneck","machine-learning-theory","mutual-information","phase-transition","statistical-inference","symbolic-algebra"],"latest_commit_sha":null,"homepage":"https://summary.md","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/farukalpay.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-05-07T09:30:21.000Z","updated_at":"2025-05-15T19:23:14.000Z","dependencies_parsed_at":"2025-05-11T20:38:07.742Z","dependency_job_id":null,"html_url":"https://github.com/farukalpay/information-bottleneck-beta-optimization","commit_stats":null,"previous_names":["farukalpay/betabottle","farukalpay/ib-beta-star-validation","farukalpay/information-bottleneck-beta-optimization"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/farukalpay/information-bottleneck-beta-optimization","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/farukalpay%2Finformation-bottleneck-beta-optimization","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/farukalpay%2Finformation-bottleneck-beta-optimization/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/farukalpay%2Finformation-bottleneck-beta-optimization/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/farukalpay%2Finformation-bottleneck-beta-optimization/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/farukalpay","download_url":"https://codeload.github.com/farukalpay/information-bottleneck-beta-optimization/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/farukalpay%2Finformation-bottleneck-beta-optimization/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262009031,"owners_count":23244300,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["alpay-algebra","beta-optimization","deep-theory","ib-framework","information-bottleneck","machine-learning-theory","mutual-information","phase-transition","statistical-inference","symbolic-algebra"],"created_at":"2025-05-29T23:06:27.794Z","updated_at":"2025-06-26T05:31:29.548Z","avatar_url":"https://github.com/farukalpay.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# β-Optimization in the Information Bottleneck Framework  \n[![ORCID](https://img.shields.io/badge/ORCID-0009--0009--2207--6528-brightgreen)](https://orcid.org/0009-0009-2207-6528)\n\n**Author:** Faruk Alpay\n\n| Version | Title | Date | DOI / License |\n|---------|-------|------|---------------|\n| **V1** | *β-Optimization in the Information Bottleneck Framework: A Theoretical Analysis* | 7 – 11 May 2025 | 10.22541/au.174664105.57850297/v1 / CC BY 4.0 |\n| **V2** | *β-Optimization … Multi-Path Extension* | 12 – 27 May 2025 | 10.5281/zenodo.15384382 / MIT |\n| **V3 (current)** | *Stable and Convexified Information Bottleneck Optimization via Symbolic Continuation and Entropy-Regularized Trajectories* | ≥ 12 May 2025 | arXiv:2505.09239 / arXiv Non-Exclusive Distribution License |\n| **V4 (planned)** | *Proof-Tight \u0026 Large-Scale Continuation IB* | Q4 2025 (target) | T B A |\n\n\u003e **Please cite V3** for new work; older DOIs remain valid for archival purposes.\n\n---\n\n## 📂 Repository map\n\n```\nCode_v1/                      # β* validation framework\ncode_v2_Multi_Path/           # multi-path incremental-β solver\ncode_v3_Stable_Continuation/  # NEW: convex + entropy + continuation\ndocs/                         # legacy citations, notes\nLICENSE\nREADME.md\n```\n\n### Quick PDFs\n\n| Version | Path |\n|---------|------|\n| **V1** | `Code_v1/paper/enhanced_ib_framework.pdf` |\n| **V2** | `code_v2_Multi_Path/paper/enhanced_ib_framework.pdf` |\n| **V3** | `code_v3_Stable_Continuation/paper/stable_convex_ib.pdf` |\n\n---\n\n## Code_v3 — Stable Continuation IB (Convex + Entropy)  \n\n*(directory `code_v3_Stable_Continuation/`)*\n\n| File | Role |\n|------|------|\n| `stable_continuation_ib.py` | Predictor–corrector solver implementing \\(u(t)=t^2\\) and small entropy penalty |\n| `requirements.txt` | `numpy`, `scipy`, `jax` (GPU optional), `matplotlib` |\n| `ib_plots/` | `bsc_critical_region.png`, `bsc_phase_transition_detection.png`, `continuation_ib_results.png`, `encoder_comparison.png`, `encoder_evolution.png`, `enhanced_multipath_best_encoder.png`, `enhanced_multipath_beta_trajectories.png`, `enhanced_multipath_convergence.png`, `enhanced_multipath_info_plane.png`, `ib_curve_comparison.png`, `izy_vs_beta_continuation.png` |\n| `paper/stable_convex_ib.pdf` | V3 manuscript (same as DOI) |\n\nRun the demo:\n\n```bash\npython code_v3_Stable_Continuation/stable_continuation_ib.py \n```\n\nOutputs the figures above and reproduces the BSC \u0026 8×8 experiments (see Figures 1–5 in the PDF).\n\n---\n\n## 🔄 Improvements: Version Comparison\n\n### Code_v1 vs. code_v2_Multi_Path vs. code_v3_Stable_Continuation\n| Feature                         | Code_v1 (Validation Framework)         | Code_v2 (Multi-Path Framework)         | Code_v3 (Stable Continuation)          |\n|---------------------------------|----------------------------------------|----------------------------------------|----------------------------------------|\n| **Primary Goal** | Validate symbolic $\\beta^*$ (4.14144)  | Prevent encoder collapse \u0026 robust IB optimization across $\\beta$ spectrum | Eliminate phase jumps via symbolic continuation \u0026 convexification |\n| $\\beta$ Scheduling              | Static / Focused on $\\beta^*$          | Incremental \u0026 adaptive with gradual increase | Predictor-corrector ODE with continuation |\n| Encoder Collapse Prevention     | Structural KL convergence criteria     | ✅ Multi-path stability (multiple parallel solutions) | ✅ Entropy regularization + convex surrogate |\n| Critical $\\beta^*$ Detection    | Deterministic, high-precision          | Multi-method estimation with gradient tracking | Guaranteed via Hessian eigenvalue monitoring |\n| Information Plane Path Tracking | Basic dynamics plot                    | Multi-path visualization with solution trajectories | Continuous trajectory \u0026 bifurcation visualization |\n| Damping \u0026 Stabilization         | Adaptive based on convergence behavior | Adaptive per path with local iterations | ✅ Automatic via ODE continuation |\n| Convex Surrogate Function       | ❌ None                                | ❌ None                                | ✅ $u(t)=t^2$ |\n| Entropy Regularization          | ❌ None                                | ❌ None                                | ✅ Constant small $\\varepsilon$ |\n| Bifurcation Handling            | ❌ Limited                             | Path selection \u0026 merging               | ✅ Explicit detection via Hessian eigenvalues |\n| Core Algorithm                  | Staged optimization, symbolic β*       | JIT-compiled multi-path incremental evolution | Predictor-corrector ODE with implicit function continuation |\n| **Dependencies** | numpy, scipy, scikit-learn, matplotlib | numpy, scipy, matplotlib, **jax, jaxlib**, (sympy optional) | numpy, scipy, **jax, jaxlib**, matplotlib |\n| JAX Acceleration                | ❌ No                                  | ✅ Yes (JIT-compiled core functions)   | ✅ Yes (64-bit precision enabled) |\n| Visualization                   | Static plots, convergence tracking     | Solution paths, β trajectories, multi-path info plane | Solution trajectories \u0026 bifurcation visualization |\n\n---\n\n## 🔄 Improvements across versions\n\n| Feature | V1 | V2 | V3 | V4 (planned) |\n|---------|----|----|----|--------------| \n| Goal | β* proof | Multi-path robustness | Eliminate phase jumps | Proof-tight, large-scale |\n| Convex surrogate (u(t)) | — | — | (t^2) | Adaptive slope |\n| Entropy regulator (ε) | — | — | constant small | Annealed ε(β) |\n| Continuation | — | β-grid multi-path | Predictor-corrector ODE | Arc-length continuation |\n| Dataset scale | 2×2, 8×8 | 8×8 | 2×2, 8×8 | MNIST, CIFAR-10 |\n| JAX / GPU | — | ✅ | ✅ | ✅+TPU |\n| Package | script | script | script | pip package |\n| Proof rigor | β* lemma | empirical | convexity lemma | full theorem set |\n| Target venue | Authorea | Zenodo | arXiv | Springer-Nature |\n\n---\n\n## 🔮 v4 Roadmap (Q4 2025)\n\n- Full formal proof of global convexity + uniqueness.\n- Adaptive entropy schedule linked to Hessian condition number.\n- Gaussian/Variational IB demo on MNIST \u0026 CIFAR-10.\n- Arc-length continuation for automatic step control.\n- Package ib-continuation on PyPI with CLI ib-trace.\n- Submit Springer-Nature manuscript (sn-article.cls).\n\n---\n\n## 📜 Citation\n\n```\n@misc{alpay2025stableconvexifiedinformationbottleneck,\n      title={Stable and Convexified Information Bottleneck Optimization via Symbolic Continuation and Entropy-Regularized Trajectories}, \n      author={Faruk Alpay},\n      year={2025},\n      eprint={2505.09239},\n      archivePrefix={arXiv},\n      primaryClass={cs.LG},\n      url={https://arxiv.org/abs/2505.09239}, \n}\n```\n\n(Legacy BibTeX for V1 and V2 lives in docs/old_citations.bib.)\n\n---\n\n## 📄 License\n\nMIT for academic/educational use.\nCommercial enquiries → alpay@lightcap.ai\n\n---\n\n## 📬 Contact\n\nFaruk Alpay · ORCID 0009-0009-2207-6528 · alpay@lightcap.ai\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffarukalpay%2Finformation-bottleneck-beta-optimization","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffarukalpay%2Finformation-bottleneck-beta-optimization","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffarukalpay%2Finformation-bottleneck-beta-optimization/lists"}