{"id":15065037,"url":"https://github.com/rishit-dagli/glom-tensorflow","last_synced_at":"2025-10-15T15:18:09.462Z","repository":{"id":53507984,"uuid":"351805079","full_name":"Rishit-dagli/GLOM-TensorFlow","owner":"Rishit-dagli","description":"An attempt at the implementation of GLOM, Geoffrey Hinton's paper for emergent part-whole hierarchies from data","archived":false,"fork":false,"pushed_at":"2021-03-27T06:59:14.000Z","size":290,"stargazers_count":37,"open_issues_count":0,"forks_count":5,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-05-08T16:46:49.333Z","etag":null,"topics":["capsule-neural-networks","convolutional-neural-networks","deep-learning","embedding","geoffrey-hinton","glom","keras","keras-tensorflow","machine-learning","neural-network","representation-learning","tensorflow","tensorflow2"],"latest_commit_sha":null,"homepage":"https://pypi.org/project/glom-tf/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Rishit-dagli.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null}},"created_at":"2021-03-26T14:14:42.000Z","updated_at":"2024-10-11T16:16:40.000Z","dependencies_parsed_at":"2022-09-04T15:32:56.373Z","dependency_job_id":null,"html_url":"https://github.com/Rishit-dagli/GLOM-TensorFlow","commit_stats":null,"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"purl":"pkg:github/Rishit-dagli/GLOM-TensorFlow","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Rishit-dagli%2FGLOM-TensorFlow","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Rishit-dagli%2FGLOM-TensorFlow/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Rishit-dagli%2FGLOM-TensorFlow/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Rishit-dagli%2FGLOM-TensorFlow/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Rishit-dagli","download_url":"https://codeload.github.com/Rishit-dagli/GLOM-TensorFlow/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Rishit-dagli%2FGLOM-TensorFlow/sbom","scorecard":{"id":121166,"data":{"date":"2025-08-11","repo":{"name":"github.com/Rishit-dagli/GLOM-TensorFlow","commit":"9d8e4562414eedf69161a4c799e66798dee762b1"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":3.8,"checks":[{"name":"Code-Review","score":0,"reason":"Found 0/26 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Warn: no topLevel permission defined: .github/workflows/flake8-lint.yml:1","Warn: no topLevel permission defined: .github/workflows/python-publish.yml:1","Info: no jobLevel write permissions found"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/flake8-lint.yml:18: update your workflow using https://app.stepsecurity.io/secureworkflow/Rishit-dagli/GLOM-TensorFlow/flake8-lint.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/flake8-lint.yml:21: update your workflow using https://app.stepsecurity.io/secureworkflow/Rishit-dagli/GLOM-TensorFlow/flake8-lint.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/python-publish.yml:17: update your workflow using https://app.stepsecurity.io/secureworkflow/Rishit-dagli/GLOM-TensorFlow/python-publish.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/python-publish.yml:20: update your workflow using https://app.stepsecurity.io/secureworkflow/Rishit-dagli/GLOM-TensorFlow/python-publish.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/python-publish.yml:25: update your workflow using https://app.stepsecurity.io/secureworkflow/Rishit-dagli/GLOM-TensorFlow/python-publish.yml/main?enable=pin","Warn: pipCommand not pinned by hash: .github/workflows/flake8-lint.yml:27","Warn: pipCommand not pinned by hash: .github/workflows/flake8-lint.yml:28","Warn: pipCommand not pinned by hash: .github/workflows/flake8-lint.yml:29","Warn: pipCommand not pinned by hash: .github/workflows/python-publish.yml:36","Warn: pipCommand not pinned by hash: .github/workflows/python-publish.yml:37","Info:   0 out of   5 GitHub-owned GitHubAction dependencies pinned","Info:   0 out of   5 pipCommand dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: Apache License 2.0: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":-1,"reason":"internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration","details":null,"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 5 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-16T02:27:40.506Z","repository_id":53507984,"created_at":"2025-08-16T02:27:40.506Z","updated_at":"2025-08-16T02:27:40.506Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":274231507,"owners_count":25245601,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-08T02:00:09.813Z","response_time":121,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["capsule-neural-networks","convolutional-neural-networks","deep-learning","embedding","geoffrey-hinton","glom","keras","keras-tensorflow","machine-learning","neural-network","representation-learning","tensorflow","tensorflow2"],"created_at":"2024-09-25T00:29:53.606Z","updated_at":"2025-10-15T15:18:04.430Z","avatar_url":"https://github.com/Rishit-dagli.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# GLOM TensorFlow [![Twitter](https://img.shields.io/twitter/url?style=social\u0026url=https%3A%2F%2Fgithub.com%2FRishit-dagli%2FGLOM-TensorFlow)](https://twitter.com/intent/tweet?text=Wow:\u0026url=https%3A%2F%2Fgithub.com%2FRishit-dagli%2FGLOM-TensorFlow)\n\n![PyPI](https://img.shields.io/pypi/v/glom-tf)\n[![Flake8 Lint](https://github.com/Rishit-dagli/GLOM-TensorFlow/actions/workflows/flake8-lint.yml/badge.svg)](https://github.com/Rishit-dagli/GLOM-TensorFlow/actions/workflows/flake8-lint.yml)\n[![Upload Python Package](https://github.com/Rishit-dagli/GLOM-TensorFlow/actions/workflows/python-publish.yml/badge.svg)](https://github.com/Rishit-dagli/GLOM-TensorFlow/actions/workflows/python-publish.yml)\n![Python Version](https://img.shields.io/badge/python-3.7%20%7C%203.8%20%7C%203.9-blue)\n\n[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/Rishit-dagli/GLOM-TensorFlow/HEAD)\n[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Rishit-dagli/GLOM-TensorFlow/blob/main/examples/glom_example.ipynb)\n\n[![GitHub license](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](LICENSE)\n[![PEP8](https://img.shields.io/badge/code%20style-pep8-orange.svg)](https://www.python.org/dev/peps/pep-0008/)\n[![GitHub stars](https://img.shields.io/github/stars/Rishit-dagli/GLOM-TensorFlow?style=social)](https://github.com/Rishit-dagli/GLOM-TensorFlow/stargazers)\n[![GitHub followers](https://img.shields.io/github/followers/Rishit-dagli?label=Follow\u0026style=social)](https://github.com/Rishit-dagli)\n[![Twitter Follow](https://img.shields.io/twitter/follow/rishit_dagli?style=social)](https://twitter.com/intent/follow?screen_name=rishit_dagli)\n\nThis Python package attempts to implement GLOM in TensorFlow, which allows advances made by several different groups \ntransformers, neural fields, contrastive representation learning, distillation and capsules to be combined. This was \nsuggested by Geoffrey Hinton in his paper \n[\"How to represent part-whole hierarchies in a neural network\"](https://arxiv.org/abs/2102.12627).\n\nFurther, [Yannic Kilcher's video](https://youtu.be/cllFzkvrYmE) and [Phil Wang's repo](https://github.com/lucidrains/glom-pytorch)\nwas very helpful for me to implement this project.\n\n## Installation\n\nRun the following to install:\n\n```sh\npip install glom-tf\n```\n\n## Developing `glom-tf`\n\nTo install `glom-tf`, along with tools you need to develop and test, run the following in your virtualenv:\n\n```sh\ngit clone https://github.com/Rishit-dagli/GLOM-TensorFlow.git\n# or clone your own fork\n\ncd GLOM-TensorFlow\npip install -e .[dev]\n```\n\n## A bit about GLOM\n\nThe GLOM architecture is composed of a large number of columns which\nall use exactly the same weights. Each column is a stack of spatially local\nautoencoders that learn multiple levels of representation for what is happening\nin a small image patch. Each autoencoder transforms the embedding at one level\ninto the embedding at an adjacent level using a multilayer bottom-up encoder\nand a multilayer top-down decoder. These levels correspond to the levels in a\npart-whole hierarchy.\n\n![](images/interactions.png)\n\u003cp align=\"center\"\u003e\n\u003csmall\u003eInteractions among the 3 levels in one column\u003c/small\u003e\n\u003c/p\u003e\n\nAn example shared by the author was as an example when show a face image, a single column might converge on embedding \nvectors representing a nostril, a nose, a face, and a person.\n\nAt each discrete time and in each column separately, the embedding at a\nlevel is updated to be the weighted average of:\n- bottom-up neural net acting on the embedding at the level below at the previous time\n- top-down neural net acting on the embedding at the level above at the previous time\n- embedding vector at the previous time step\n- attention-weighted average of the embeddings at the same level in nearby columns at the previous time\n\nFor a static image, the embeddings at a level should settle down over time to produce similar vectors.\n\n![](images/embeddings.png)\n\u003cp align=\"center\"\u003e\n\u003csmall\u003eA picture of the embeddings at a particular time\u003c/small\u003e\n\u003c/p\u003e\n\n## Usage\n\n```python\nfrom glomtf import Glom\n\nmodel = Glom(dim = 512,\n             levels = 5,\n             image_size = 224,\n             patch_size = 14)\n\nimg = tf.random.normal([1, 3, 224, 224])\nlevels = model(img, iters = 12) # (1, 256, 5, 12)\n# 1 - batch\n# 256 - patches\n# 5 - levels\n# 12 - dimensions\n```\n\nUse the `return_all = True` argument to get all the column and level states per iteration. This also gives you access \nto all the level data across iterations for clustering, from which you can inspect the islands too.\n\n```python\nfrom glomtf import Glom\n\nmodel = Glom(dim = 512,\n             levels = 5,\n             image_size = 224,\n             patch_size = 14)\n\nimg = tf.random.normal([1, 3, 224, 224])\nall_levels = model(img, iters = 12, return_all = True) # (13, 1, 256, 5, 12)\n# 13 - time\n\n# top level outputs after iteration 6\ntop_level_output = all_levels[7, :, :, -1] # (1, 256, 512)\n# 1 - batch\n# 256 - patches\n# 512 - dimensions\n```\n\n## Want to Contribute 🙋‍♂️?\n\nAwesome! If you want to contribute to this project, you're always welcome! See [Contributing Guidelines](CONTRIBUTING.md). You can also take a look at [open issues](https://github.com/Rishit-dagli/GLOM-TensorFlow/issues) for getting more information about current or upcoming tasks.\n\n## Want to discuss? 💬\n\nHave any questions, doubts or want to present your opinions, views? You're always welcome. You can [start discussions](https://github.com/Rishit-dagli/GLOM-TensorFlow/discussions).\n\n## Citations\n\n```bibtex\n@misc{hinton2021represent,\n    title   = {How to represent part-whole hierarchies in a neural network}, \n    author  = {Geoffrey Hinton},\n    year    = {2021},\n    eprint  = {2102.12627},\n    archivePrefix = {arXiv},\n    primaryClass = {cs.CV}\n}\n```\n\n## License\n\n```\nCopyright 2020 Rishit Dagli\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n    http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frishit-dagli%2Fglom-tensorflow","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frishit-dagli%2Fglom-tensorflow","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frishit-dagli%2Fglom-tensorflow/lists"}