{"id":16916896,"url":"https://github.com/althonos/torch-treecrf","last_synced_at":"2025-10-08T17:19:03.498Z","repository":{"id":65336680,"uuid":"588944294","full_name":"althonos/torch-treecrf","owner":"althonos","description":"A PyTorch implementation of Tree-structured Conditional Random Fields.","archived":false,"fork":false,"pushed_at":"2025-04-06T14:00:36.000Z","size":50,"stargazers_count":6,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-11T16:24:38.242Z","etag":null,"topics":["conditional-random-fields","machine-learning","python-library","pytorch","torch"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/althonos.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"COPYING","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-01-14T15:04:43.000Z","updated_at":"2025-04-06T14:00:40.000Z","dependencies_parsed_at":"2025-01-02T17:45:13.379Z","dependency_job_id":"382d9d4a-55a6-4fbb-911a-5b3054199b82","html_url":"https://github.com/althonos/torch-treecrf","commit_stats":{"total_commits":40,"total_committers":1,"mean_commits":40.0,"dds":0.0,"last_synced_commit":"540e1bfa653e32e1bb5a129d93e2a4d043046db6"},"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/althonos%2Ftorch-treecrf","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/althonos%2Ftorch-treecrf/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/althonos%2Ftorch-treecrf/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/althonos%2Ftorch-treecrf/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/althonos","download_url":"https://codeload.github.com/althonos/torch-treecrf/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248439032,"owners_count":21103525,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["conditional-random-fields","machine-learning","python-library","pytorch","torch"],"created_at":"2024-10-13T19:31:10.334Z","updated_at":"2025-10-08T17:18:58.464Z","avatar_url":"https://github.com/althonos.png","language":"Python","readme":"# 🌲 `torch-treecrf`\n\n*A [PyTorch](https://pytorch.org/) implementation of Tree-structured Conditional Random Fields.*\n\n[![Actions](https://img.shields.io/github/actions/workflow/status/althonos/torch-treecrf/test.yml?branch=main\u0026logo=github\u0026style=flat-square\u0026maxAge=300)](https://github.com/althonos/torch-treecrf/actions)\n[![Coverage](https://img.shields.io/codecov/c/gh/althonos/torch-treecrf?style=flat-square\u0026maxAge=3600)](https://codecov.io/gh/althonos/torch-treecrf/)\n[![License](https://img.shields.io/badge/license-GPLv3-blue.svg?style=flat-square\u0026maxAge=2678400)](https://choosealicense.com/licenses/gpl-3.0/)\n[![PyPI](https://img.shields.io/pypi/v/torch-treecrf.svg?style=flat-square\u0026maxAge=3600)](https://pypi.org/project/torch-treecrf)\n[![Wheel](https://img.shields.io/pypi/wheel/torch-treecrf.svg?style=flat-square\u0026maxAge=3600)](https://pypi.org/project/torch-treecrf/#files)\n[![Python Versions](https://img.shields.io/pypi/pyversions/torch-treecrf.svg?style=flat-square\u0026maxAge=3600)](https://pypi.org/project/torch-treecrf/#files)\n[![Python Implementations](https://img.shields.io/badge/impl-universal-success.svg?style=flat-square\u0026maxAge=3600\u0026label=impl)](https://pypi.org/project/torch-treecrf/#files)\n[![Source](https://img.shields.io/badge/source-GitHub-303030.svg?maxAge=2678400\u0026style=flat-square)](https://github.com/althonos/torch-treecrf/)\n[![GitHub issues](https://img.shields.io/github/issues/althonos/torch-treecrf.svg?style=flat-square\u0026maxAge=600)](https://github.com/althonos/torch-treecrf/issues)\n[![Changelog](https://img.shields.io/badge/keep%20a-changelog-8A0707.svg?maxAge=2678400\u0026style=flat-square)](https://github.com/althonos/torch-treecrf.py/blob/master/CHANGELOG.md)\n[![Downloads](https://img.shields.io/badge/dynamic/json?style=flat-square\u0026color=303f9f\u0026maxAge=86400\u0026label=downloads\u0026query=%24.total_downloads\u0026url=https%3A%2F%2Fapi.pepy.tech%2Fapi%2Fprojects%2Ftorch-treecrf)](https://pepy.tech/project/torch-treecrf)\n\n## 🗺️ Overview\n\n[Conditional Random Fields](https://en.wikipedia.org/wiki/Conditional_random_field)\n(CRF) are a family of discriminative graphical learning models that can be used\nto model the dependencies between variables. The most common\nform of CRFs are Linear-chain CRF, where a prediction depends on\nan observed variable, as well as the prediction before and after it\n(the *context*). Linear-chain CRFs are widely used in Natural Language Processing.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg height=\"150\" src=\"https://github.com/althonos/torch-treecrf/raw/main/static/linear-chain-crf.svg?raw=true\"\u003e\n\u003c/p\u003e\n\n$$\nP(Y | X) = \\frac{1}{Z(X)} \\prod_{i=1}^n{ \\Psi_i(y_i, x_i) } \\prod_{i=2}^n{ \\Psi_{i-1,i}(y_{i-1}, y_i)}\n$$\n\nIn 2006, Tang *et al.*[[1]](#ref1) introduced Tree-structured CRFs to model hierarchical\nrelationships between predicted variables, allowing dependencies between\na prediction variable and its parents and children.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg height=\"280\" src=\"https://github.com/althonos/torch-treecrf/raw/main/static/tree-structured-crf.svg?raw=true\"\u003e\n\u003c/p\u003e\n\n$$\nP(Y | X) = \\frac{1}{Z(X)} \\prod_{i=1}^{n}{ \\Psi_i(y_i, x_i) } \\prod_{j \\in \\mathcal{N}(i)}{ \\Psi_{j,i}(y_j, y_i)}\n$$\n\nThis package implements a generic Tree-structured CRF layer in PyTorch. The\nlayer can be stacked on top of a [linear layer](https://pytorch.org/docs/stable/generated/torch.nn.Linear.html) to implement a proper Tree-structured CRF, or on any other kind of model\nproducing emission scores in log-space for every class of each label. Computation\nof marginals is implemented using Belief Propagation[[2]](#ref2), allowing for\nexact inference on trees[[3]](#ref3):\n\n$$\n\\begin{aligned}\nP(y_i | X)\n\u0026 =\n    \\frac{1}{Z(X)} \\Psi_i(y_i, x_i)\n    \u0026 \\underbrace{\\prod_{j \\in \\mathcal{C}(i)}{\\mu_{j \\to i}(y_i)}} \u0026\n    \u0026 \\underbrace{\\prod_{j \\in \\mathcal{P}(i)}{\\mu_{j \\to i}(y_i)}} \\\\\n\u0026 = \\frac1Z \\Psi_i(y_i, x_i)\n    \u0026 \\alpha_i(y_i) \u0026\n    \u0026 \\beta_i(y_i)  \\\\\n\\end{aligned}\n$$\n\nwhere for every node $i$, the message from the parents $\\mathcal{P}(i)$ and\nthe children $\\mathcal{C}(i)$ is computed recursively with the sum-product algorithm[[4]](#ref4):\n\n$$\n\\begin{aligned}\n\\forall j \\in \\mathcal{C}(i), \\mu_{j \\to i}(y_i) = \\sum_{y_j}{\n  \\Psi_{i,j}(y_i, y_j)\n  \\Psi_j(y_j, x_j)\n  \\prod_{k \\in \\mathcal{C}(j)}{\\mu_{k \\to j}(y_j)}\n} \\\\\n\\forall j \\in \\mathcal{P}(i), \\mu_{j \\to i}(y_i) = \\sum_{y_j}{\n  \\Psi_{i,j}(y_i, y_j)\n  \\Psi_j(y_j, x_j)\n  \\prod_{k \\in \\mathcal{P}(j)}{\\mu_{k \\to j}(y_j)}\n} \\\\\n\\end{aligned}\n$$\n\n\n*The implementation should be generic enough that any kind of [Directed acyclic graph](https://en.wikipedia.org/wiki/Directed_acyclic_graph) can be used as a label hierarchy,\nnot just trees.*\n\n## 🔧 Installing\n\nInstall the `torch-treecrf` package directly from [PyPi](https://pypi.org/project/peptides)\nwhich hosts universal wheels that can be installed with `pip`:\n```console\n$ pip install torch-treecrf\n```\n\n## 📋 Features\n\n- Encoding of directed graphs in an adjacency matrix, with $\\mathcal{O}(1)$ retrieval of children and parents for any node, and $\\mathcal{O}(N+E)$ storage.\n- Support for any acyclic hierarchy representable as a [Directed Acyclic Graph](https://en.wikipedia.org/wiki/Directed_acyclic_graph) and not just directed trees, allowing prediction of classes such as the [Gene Ontology](https://geneontology.org).\n- Multiclass output, provided all the target labels have the same number of classes: $Y \\in \\left\\\\{ 0, .., C \\right\\\\}^L$.\n- Minibatch support, with vectorized computation of the messages $\\alpha_i(y_i)$ and $\\beta_i(y_i)$.\n\n\n## 💡 Example\n\nTo create a Tree-structured CRF, you must first define the tree encoding the\nrelationships between variables. Let's build a simple CRF for a root variable\nwith two children:\n\n\u003cp align=\"center\"\u003e\n  \u003cimg height=\"150\" src=\"https://github.com/althonos/torch-treecrf/raw/main/static/example.svg?raw=true\"\u003e\n\u003c/p\u003e\n\nFirst, define an adjacency matrix $M$ representing the hierarchy, such that\n$M_{i,j}$ is $1$ if $j$ is a parent of $i$:\n```python\nadjacency = torch.tensor([\n    [0, 0, 0],\n    [1, 0, 0],\n    [1, 0, 0]\n])\n```\n\nThen create the CRF by giving it the adjacency matrix as the hyperparameter:\n```python\ncrf = torch_treecrf.TreeCRF(adjacency)\n```\n\nThe `TreeCRF` expects local emission scores as a tensor of shape $(\\star, L)$\nwhere $\\star$ is the minibatch size and $L$ the number of labels, and returns \na tensor of logits of the same shape.\n\nYou can also use the CRF layer for cases where labels have more than two\nclasses; in which case use the `TreeCRFLayer` module, which expects an \nemission tensor of shape $(\\star, C, L)$, where $\\star$ is the minibatch size, $L$ the number of labels and $C$ the number of class per label, and returns\na tensor $log P(Y | X)$ of the same shape.\n\n## 💭 Feedback\n\n### ⚠️ Issue Tracker\n\nFound a bug ? Have an enhancement request ? Head over to the [GitHub issue\ntracker](https://github.com/althonos/torch-treecrf/issues) if you need to report\nor ask something. If you are filing in on a bug, please include as much\ninformation as you can about the issue, and try to recreate the same bug\nin a simple, easily reproducible situation.\n\n### 🏗️ Contributing\n\nContributions are more than welcome! See\n[`CONTRIBUTING.md`](https://github.com/althonos/torch-treecrf/blob/main/CONTRIBUTING.md)\nfor more details.\n\n## ⚖️ License\n\nThis library is provided under the [MIT License](https://choosealicense.com/licenses/mit/).\n\n*This library was developed by [Martin Larralde](https://github.com/althonos/)\nduring his PhD project at the [European Molecular Biology Laboratory](https://www.embl.de/)\nin the [Zeller team](https://github.com/zellerlab).*\n\n## 📚 References\n\n- \u003ca id=\"ref1\"\u003e[1]\u003c/a\u003e Tang, Jie, Mingcai Hong, Juanzi Li, and Bangyong Liang. ‘Tree-Structured Conditional Random Fields for Semantic Annotation’. In The Semantic Web - ISWC 2006, edited by Isabel Cruz, Stefan Decker, Dean Allemang, Chris Preist, Daniel Schwabe, Peter Mika, Mike Uschold, and Lora M. Aroyo, 640–53. Lecture Notes in Computer Science. Berlin, Heidelberg: Springer, 2006. [doi:10.1007/11926078_46](https://doi.org/10.1007/11926078_46).\n- \u003ca id=\"ref2\"\u003e[2]\u003c/a\u003e Pearl, Judea. ‘Reverend Bayes on Inference Engines: A Distributed Hierarchical   Approach’. In Proceedings of the Second AAAI Conference on Artificial Intelligence, 133–136. AAAI’82. Pittsburgh, Pennsylvania: AAAI Press, 1982.\n- \u003ca id=\"ref3\"\u003e[3]\u003c/a\u003e Bach, Francis, and Guillaume Obozinski. ‘Sum Product Algorithm and Hidden Markov Model’, ENS Course Material, 2016. http://imagine.enpc.fr/%7Eobozinsg/teaching/mva_gm/lecture_notes/lecture7.pdf.\n- \u003ca id=\"ref4\u003e\"\u003e[4]\u003c/a\u003e Kschischang, Frank R., Brendan J. Frey, and Hans-Andrea Loeliger. ‘Factor Graphs and the Sum-Product Algorithm’. IEEE Transactions on Information Theory 47, no. 2 (February 2001): 498–519. [doi:10.1109/18.910572](https://doi.org/10.1109/18.910572).\n\n\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falthonos%2Ftorch-treecrf","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Falthonos%2Ftorch-treecrf","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falthonos%2Ftorch-treecrf/lists"}