{"id":40168258,"url":"https://github.com/thosvarley/syntropy","last_synced_at":"2026-01-19T17:00:31.044Z","repository":{"id":313254760,"uuid":"1050695843","full_name":"thosvarley/syntropy","owner":"thosvarley","description":"A python package for information-theoretic analysis of discrete and continuous data.","archived":false,"fork":false,"pushed_at":"2026-01-16T21:39:23.000Z","size":2750,"stargazers_count":2,"open_issues_count":1,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-01-17T06:37:07.684Z","etag":null,"topics":["data-science","information-theory","scientific-computing","toolset"],"latest_commit_sha":null,"homepage":"https://syntropy.readthedocs.io/en/latest/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/thosvarley.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-09-04T19:49:14.000Z","updated_at":"2026-01-16T21:39:27.000Z","dependencies_parsed_at":"2025-10-18T23:24:09.093Z","dependency_job_id":"e8654f0c-bfe9-46a2-b616-f3f8732d5d2e","html_url":"https://github.com/thosvarley/syntropy","commit_stats":null,"previous_names":["thosvarley/syntropy"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/thosvarley/syntropy","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thosvarley%2Fsyntropy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thosvarley%2Fsyntropy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thosvarley%2Fsyntropy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thosvarley%2Fsyntropy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/thosvarley","download_url":"https://codeload.github.com/thosvarley/syntropy/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thosvarley%2Fsyntropy/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28577020,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-19T16:29:19.148Z","status":"ssl_error","status_checked_at":"2026-01-19T16:29:17.772Z","response_time":67,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-science","information-theory","scientific-computing","toolset"],"created_at":"2026-01-19T17:00:19.528Z","updated_at":"2026-01-19T17:00:31.020Z","avatar_url":"https://github.com/thosvarley.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# Syntropy\n\n[![Python 3.8+](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/downloads/)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n[![Documentation](https://readthedocs.org/projects/syntropy/badge/?version=latest)](https://syntropy.readthedocs.io/en/latest/)\n\n**Syntropy** is a Python library for multivariate information-theoretic analysis of discrete and continuous data. It provides efficient implementations of information measures ranging from basic quantities like entropy and mutual information to modern constructs like the partial information decomposition, O-information, and information rates for time series.\n\n## Features\n\n- **Multiple estimators**: Discrete, Gaussian, KNN (Kraskov), and neural (normalizing flow) estimators\n- **Pointwise measures**: Access local/pointwise values, not just expected values\n- **Higher-order information**: Total correlation, dual total correlation, O-information, S-information\n- **Information decomposition**: Partial information decomposition (PID) and partial entropy decomposition\n- **Time series**: Information rates and Lempel-Ziv complexity measures\n- **Consistent API**: Same interface across all estimator types\n\n## Installation\n\n```bash\npip install syntropyx\n```\n\nThen import as:\n```python\nimport syntropy\n```\n\nNote: The package is `syntropyx` on PyPI because `syntropy` was already taken. The `x` is just a workaround—the actual library is called Syntropy.\n\nFor development:\n```bash\ngit clone https://github.com/thosvarley/syntropy.git\ncd syntropy\npip install -e \".[dev]\"\n```\n\n## Quick Start\n\n### Discrete Distributions\n\nDiscrete estimators work with probability distributions represented as dictionaries:\n\n```python\nfrom syntropy.discrete import mutual_information, o_information\n\n# XOR distribution: pure synergy\nxor = {\n    (0, 0, 0): 0.25,\n    (0, 1, 1): 0.25,\n    (1, 0, 1): 0.25,\n    (1, 1, 0): 0.25,\n}\n\n# Mutual information between inputs (0,1) and output (2)\nptw, mi = mutual_information(idxs_x=(0, 1), idxs_y=(2,), joint_distribution=xor)\nprint(f\"I(X0,X1 ; X2) = {mi:.3f} bits\")  # 1.0 bit\n\n# O-information (negative = synergy-dominated)\nptw, omega = o_information(idxs=(0, 1, 2), joint_distribution=xor)\nprint(f\"Omega = {omega:.3f} bits\")  # -1.0 bit\n```\n\n### Gaussian Estimator\n\nFor continuous data with approximately Gaussian distributions:\n\n```python\nimport numpy as np\nfrom syntropy.gaussian import mutual_information, total_correlation\n\n# Generate correlated Gaussian data\nn = 10_000\nx = np.random.randn(n)\ny = 0.8 * x + 0.6 * np.random.randn(n)\nz = 0.5 * x + 0.866 * np.random.randn(n)\ndata = np.vstack([x, y, z])\n\ncov = np.cov(data)\nmi = mutual_information(idxs_x=(0,), idxs_y=(1,), cov=cov)\ntc = total_correlation(idxs=(0, 1, 2), cov=cov)\n\nprint(f\"I(X ; Y) = {mi:.3f} nats\")\nprint(f\"TC(X, Y, Z) = {tc:.3f} nats\")\n```\n\n### KNN Estimator (Kraskov)\n\nNon-parametric estimation for continuous data:\n\n```python\nimport numpy as np\nfrom syntropy.knn import mutual_information\n\n# Non-linear relationship\nn = 5_000\nx = np.random.randn(n)\ny = x**2 + 0.5 * np.random.randn(n)\ndata = np.vstack([x, y])\n\nptw, mi = mutual_information(idxs_x=(0,), idxs_y=(1,), data=data, k=5)\nprint(f\"I(X ; Y) = {mi:.3f} nats\")\n```\n\n### Neural Estimator\n\nFor complex, high-dimensional distributions using normalizing flows:\n\n```python\nimport torch\nfrom syntropy.neural import mutual_information\n\n# Generate data (samples x features format)\nn = 10_000\nx = torch.randn(n)\ny = 0.7 * x + 0.714 * torch.randn(n)\ndata = torch.stack([x, y], dim=1)\n\nptw, mi = mutual_information(idxs_x=(0,), idxs_y=(1,), data=data, verbose=True)\nprint(f\"I(X ; Y) = {mi:.3f} nats\")\n```\n\n### Mixed Discrete-Continuous\n\nFor mutual information between discrete and continuous variables:\n\n```python\nimport numpy as np\nfrom syntropy.mixed import mutual_information\n\nn = 10_000\ncontinuous = np.random.randn(1, n)\ndiscrete = (continuous \u003e 0).astype(int)\n\nptw, mi = mutual_information(discrete_vars=discrete, continuous_vars=continuous)\nprint(f\"I(discrete ; continuous) = {mi:.3f} nats\")\n```\n\n## Available Measures\n\n| Measure | Discrete | Gaussian | KNN | Neural | Mixed |\n|---------|:--------:|:--------:|:---:|:------:|:-----:|\n| Entropy | x | x | x | x | x |\n| Conditional Ent. | x | x | x | x | x |\n| Mutual Information | x | x | x | x | x |\n| Conditional MI | x | x | x | x |\n| KL Divergence | x | x | | |\n| Total Correlation | x | x | x | x |\n| Dual Total Correlation | x | x | x | x |\n| O-Information | x | x | x | x |\n| S-Information | x | x | x | x |\n| Co-Information | x | | | |\n| TSE Complexity | x | x | | | \n| Partial Info. Decomp. | x | x | | |\n| Partial Entropy Decomp. | x | x | | |\n| Generalized Info. Decomp. | x | x | | |\n| Information Rates | x | x | | |\n| Connected Information | x| | | |\n\n### Optimizations and Utilities\n\nSyntropy also includes a number of optimization algorithms. \n\n* Finding optimally-synergistic submatrices from a covariance matrix (as done by [Varley, Pope et al., 2023](https://www.nature.com/articles/s42003-023-04843-w)).\n* Finding the maximum-entropy discrete distribution consistent with k-order marginals (as done in the [DIT package](https://dit.readthedocs.io/en/latest/optimization.html?highlight=maxentoptimizer)).\n\nIn the ```utils.py``` files, you can also find a number of utility functions for interacting with discrete and continuous probability distributions. \n\n## Documentation\n\nFull documentation is available at [syntropy.readthedocs.io](https://syntropy.readthedocs.io).\n\n- [Quickstart Guide](https://syntropy.readthedocs.io/en/latest/quickstart.html)\n- [Theory Primer](https://syntropy.readthedocs.io/en/latest/theory.html)\n- [API Reference](https://syntropy.readthedocs.io/en/latest/api/syntropy.html)\n\n## Testing\n\n```bash\npytest tests/\n```\n\n## Citation\n\nIf you use Syntropy in your research, please cite:\n\n```bibtex\n@software{syntropy,\n  author = {Varley, Thomas F.},\n  title = {Syntropy: Multivariate Information Theory for Python},\n  url = {https://github.com/thosvarley/syntropy},\n}\n```\n\n## License\n\nMIT License. See [LICENSE](LICENSE) for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthosvarley%2Fsyntropy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthosvarley%2Fsyntropy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthosvarley%2Fsyntropy/lists"}