{"id":36886336,"url":"https://github.com/kwatcharasupat/latte","last_synced_at":"2026-01-12T15:30:28.970Z","repository":{"id":45129527,"uuid":"423844819","full_name":"kwatcharasupat/latte","owner":"kwatcharasupat","description":"Latte: Cross-framework Python Package for Evaluation of Latent-based Generative Models","archived":false,"fork":false,"pushed_at":"2025-07-29T20:54:07.000Z","size":37893,"stargazers_count":36,"open_issues_count":1,"forks_count":3,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-09-17T10:33:11.469Z","etag":null,"topics":["deep-learning","machine-learning","python"],"latest_commit_sha":null,"homepage":"https://latte.readthedocs.io/en/latest","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kwatcharasupat.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null},"funding":{"github":["karnwatcharasupat"]}},"created_at":"2021-11-02T13:00:14.000Z","updated_at":"2025-07-29T20:54:11.000Z","dependencies_parsed_at":"2024-05-09T04:51:22.739Z","dependency_job_id":null,"html_url":"https://github.com/kwatcharasupat/latte","commit_stats":null,"previous_names":["kwatcharasupat/latte","karnwatcharasupat/latte"],"tags_count":8,"template":false,"template_full_name":null,"purl":"pkg:github/kwatcharasupat/latte","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kwatcharasupat%2Flatte","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kwatcharasupat%2Flatte/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kwatcharasupat%2Flatte/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kwatcharasupat%2Flatte/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kwatcharasupat","download_url":"https://codeload.github.com/kwatcharasupat/latte/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kwatcharasupat%2Flatte/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28340866,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-12T12:22:26.515Z","status":"ssl_error","status_checked_at":"2026-01-12T12:22:10.856Z","response_time":98,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","machine-learning","python"],"created_at":"2026-01-12T15:30:28.884Z","updated_at":"2026-01-12T15:30:28.956Z","avatar_url":"https://github.com/kwatcharasupat.png","language":"Python","funding_links":["https://github.com/sponsors/karnwatcharasupat"],"categories":[],"sub_categories":[],"readme":"\u003e ### Please consider giving back to the community if you have benefited from this work.\n\u003e\n\u003e If you've **benefited commercially from this work**, which we've poured significant effort into and released under permissive licenses, we hope you've found it valuable! While these licenses give you lots of freedom, we believe in nurturing a vibrant ecosystem where innovation can continue to flourish.\n\u003e\n\u003e So, as a gesture of appreciation and responsibility, we strongly urge commercial entities that have gained from this software to consider making voluntary contributions to music-related non-profit organizations of your choice. Your contribution directly helps support the foundational work that empowers your commercial success and ensures open-source innovation keeps moving forward.\n\u003e\n\u003e Some suggestions for the beneficiaries are provided [here](https://github.com/the-secret-source/nonprofits). Please do not hesitate to contribute to the list by opening pull requests there.\n\n---\n\n\n\n\u003cp align=center\u003e\u003cimg width=\"50%\" src=\"https://raw.githubusercontent.com/karnwatcharasupat/latte/main/assets/logo.png\"/\u003e\u003c/p\u003e\n\u003cp align=center\u003e\u003cb\u003eCross-framework Python Package for Evaluation of Latent-based Generative Models\u003c/b\u003e\u003c/p\u003e\n\n[![Documentation Status](https://readthedocs.org/projects/latte/badge/?version=latest)](https://latte.readthedocs.io/en/latest/?badge=latest)\n[![CircleCI](https://circleci.com/gh/karnwatcharasupat/latte/tree/main.svg?style=shield)](https://circleci.com/gh/karnwatcharasupat/latte/tree/main)\n[![codecov](https://codecov.io/gh/karnwatcharasupat/latte/branch/main/graph/badge.svg)](https://codecov.io/gh/karnwatcharasupat/latte/branches/main)\n[![CodeFactor](https://www.codefactor.io/repository/github/karnwatcharasupat/latte/badge/main)](https://www.codefactor.io/repository/github/karnwatcharasupat/latte/overview/main)\n[![License](https://img.shields.io/badge/license-MIT-brightgreen)](https://opensource.org/licenses/MIT)\n[![PyPI version](https://badge.fury.io/py/latte-metrics.svg)](https://badge.fury.io/py/latte-metrics)\n[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.5786402.svg)](https://doi.org/10.5281/zenodo.5786402)\n[![arXiv](https://img.shields.io/badge/arXiv-2112.10638-b31b1b.svg)](https://arxiv.org/abs/2112.10638)\n\n\n# Latte\n\nLatte (for _LATent Tensor Evaluation_) is a cross-framework Python package for evaluation of latent-based generative models. Latte supports calculation of disentanglement and controllability metrics in both PyTorch (via TorchMetrics) and TensorFlow.\n\n\n## Installation\n\nFor developers working on local clone, `cd` to the repo and replace `latte` with `.`. For example, `pip install .[tests]`\n\n```console\npip install latte-metrics           # core (numpy only)\npip install latte-metrics[pytorch]  # with torchmetrics wrapper\npip install latte-metrics[keras]    # with tensorflow wrapper\npip install latte-metrics[tests]    # for testing\n```\n\n### Running tests locally\n```\npip install .[tests]\npytest tests/ --cov=latte\n```\n\n## Quick Examples\n\n### Functional API\n```python\nimport latte\nfrom latte.functional.disentanglement.mutual_info import mig\nimport numpy as np\n\nlatte.seed(42)\n\nz = np.random.randn(16, 8)\na = np.random.randn(16, 2)\n\nmutual_info_gap = mig(z, a, discrete=False, reg_dim=[4, 3])\n```\n\n\n### Modular API\n```python\nimport latte\nfrom latte.metrics.core.disentanglement import MutualInformationGap\nimport numpy as np\n\nlatte.seed(42)\n\nmig = MutualInformationGap()\n\n# ... \n# initialize data and model\n# ...\n\nfor data, attributes in range(batches):\n  recon, z = model(data)\n\n  mig.update_state(z, attributes)\n\nmig_val = mig.compute()\n```\n\n### TorchMetrics API\n```python\nimport latte\nfrom latte.metrics.torch.disentanglement import MutualInformationGap\nimport torch\n\nlatte.seed(42)\n\nmig = MutualInformationGap()\n\n# ... \n# initialize data and model\n# ...\n\nfor data, attributes in range(batches):\n  recon, z = model(data)\n\n  mig.update(z, attributes)\n\nmig_val = mig.compute()\n```\n\n### Keras Metric API\n```python\nimport latte\nfrom latte.metrics.keras.disentanglement import MutualInformationGap\nfrom tensorflow import keras as tfk\n\nlatte.seed(42)\n\nmig = MutualInformationGap()\n\n# ... \n# initialize data and model\n# ...\n\nfor data, attributes in range(batches):\n  recon, z = model(data)\n\n  mig.update_state(z, attributes)\n\nmig_val = mig.result()\n```\n\n## Example Notebooks\n\nSee Latte in action with Morpho-MNIST example notebooks on Google Colab:\n- \u003ca href=\"https://colab.research.google.com/github/karnwatcharasupat/latte/blob/main/examples/morphomnist/morphomnist-torch.ipynb\"\u003e**PyTorch (vanilla)**\u003c/a\u003e\n- \u003ca href=\"https://colab.research.google.com/github/karnwatcharasupat/latte/blob/main/examples/morphomnist/morphomnist-lightning.ipynb\"\u003e**PyTorch Lightning**\u003c/a\u003e\n- \u003ca href=\"https://colab.research.google.com/github/karnwatcharasupat/latte/blob/main/examples/morphomnist/morphomnist-keras.ipynb\"\u003e**TensorFlow/Keras**\u003c/a\u003e\n\n## Documentation\n\nhttps://latte.readthedocs.io/en/latest\n\n## Supported metrics\n\n🧪 Beta support | ✔️ Stable | 🔨 In Progress | 🕣 In Queue | 👀 KIV |\n\n| Metric                                        | Latte Functional  | Latte Modular | TorchMetrics   | Keras Metric | \n| :---                                          | :--:        | :--:      | :--:       | :--:       |\n| _Disentanglement Metrics_                     |\n| [📝](https://arxiv.org/abs/1802.04942) Mutual Information Gap (MIG)                          |🧪|🧪|🧪|🧪|\n| [📝](https://arxiv.org/abs/2110.05587) Dependency-blind Mutual Information Gap (DMIG)         |🧪|🧪|🧪|🧪|\n| [📝](https://www.researchgate.net/publication/356259963_Controllable_Music_Supervised_Learning_of_Disentangled_Representations_for_Music_Generation) Dependency-aware Mutual Information Gap (XMIG)                                                |🧪|🧪|🧪|🧪|\n| [📝](https://www.researchgate.net/publication/356259963_Controllable_Music_Supervised_Learning_of_Disentangled_Representations_for_Music_Generation) Dependency-aware Latent Information Gap (DLIG)                                                |🧪|🧪|🧪|🧪|\n| [📝](https://arxiv.org/abs/1711.00848) Separate Attribute Predictability (SAP)                |🧪|🧪|🧪|🧪|\n| [📝](https://arxiv.org/abs/1802.05312) Modularity                                             |🧪|🧪|🧪|🧪|\n| [📝](https://openreview.net/forum?id=Sy2fzU9gl) β-VAE Score    |👀|👀|👀|👀|\n| [📝](https://arxiv.org/abs/1802.05983) FactorVAE Score   |👀|👀|👀|👀|\n| [📝](https://openreview.net/forum?id=By-7dz-AZ) DCI Score    |👀|👀|👀|👀|\n| [📝](https://arxiv.org/abs/1811.00007) Interventional Robustness Score (IRS)   |👀|👀|👀|👀|\n| [📝](https://arxiv.org/abs/1910.09772) Consistency   |👀|👀|👀|👀|\n| [📝](https://arxiv.org/abs/1910.09772) Restrictiveness   |👀|👀|👀|👀|\n| _Interpolatability Metrics_                     |\n| [📝](https://www.researchgate.net/publication/356259963_Controllable_Music_Supervised_Learning_of_Disentangled_Representations_for_Music_Generation) Smoothness                                                |🧪|🧪|🧪|🧪|\n| [📝](https://www.researchgate.net/publication/356259963_Controllable_Music_Supervised_Learning_of_Disentangled_Representations_for_Music_Generation) Monotonicity                                              |🧪|🧪|🧪|🧪|\n| [📝](https://archives.ismir.net/ismir2021/paper/000064.pdf) Latent Density Ratio                                              |🕣|🕣|🕣|🕣|\n| [📝](https://arxiv.org/abs/2007.15474) Linearity                                        |👀|👀|👀|👀|\n\n## Bundled metric modules\n🧪 Experimental (subject to changes) | ✔️ Stable | 🔨 In Progress | 🕣 In Queue\n\n| Metric Bundle                                 | Latte Functional  | Latte Modular | TorchMetrics   | Keras Metric | Included\n| :---                                          | :--: | :--:        | :--:      | :--:       | :---|\n| Dependency-aware Disentanglement              |🧪|🧪|🧪|🧪| MIG, DMIG, XMIG, DLIG |\n| LIAD-based Interpolatability                  |🧪|🧪|🧪|🧪| Smoothness, Monotonicity |\n\n## Cite \n\nFor individual metrics, please cite the paper according to the link in the 📝 icon in front of each metric.\n\nIf you find our package useful, please cite open access [paper](https://www.sciencedirect.com/science/article/pii/S2665963822000033) on _Software Impacts_ (Elsevier) as\n```bibtex\n@article{\n  watcharasupat2021latte,\n  author = {Watcharasupat, Karn N. and Lee, Junyoung and Lerch, Alexander},\n  title = {{Latte: Cross-framework Python Package for Evaluation of Latent-based Generative Models}},\n  journal = {Software Impacts},\n  volume = {11},\n  pages = {100222},\n  year = {2022},\n  issn = {2665-9638},\n  doi = {https://doi.org/10.1016/j.simpa.2022.100222},\n  url = {https://www.sciencedirect.com/science/article/pii/S2665963822000033},\n}\n```\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkwatcharasupat%2Flatte","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkwatcharasupat%2Flatte","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkwatcharasupat%2Flatte/lists"}