{"id":27431431,"url":"https://github.com/dugu9sword/torch_random_fields","last_synced_at":"2025-04-14T15:29:33.102Z","repository":{"id":81752875,"uuid":"544079708","full_name":"dugu9sword/torch_random_fields","owner":"dugu9sword","description":"A highly optimized library for building markov random fields with pytorch.","archived":false,"fork":false,"pushed_at":"2023-11-02T07:26:37.000Z","size":167,"stargazers_count":9,"open_issues_count":0,"forks_count":2,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-03-14T15:56:43.812Z","etag":null,"topics":["crf","mrf","pytorch"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dugu9sword.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2022-10-01T15:36:59.000Z","updated_at":"2024-01-07T07:32:50.000Z","dependencies_parsed_at":"2023-11-02T08:35:39.489Z","dependency_job_id":null,"html_url":"https://github.com/dugu9sword/torch_random_fields","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dugu9sword%2Ftorch_random_fields","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dugu9sword%2Ftorch_random_fields/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dugu9sword%2Ftorch_random_fields/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dugu9sword%2Ftorch_random_fields/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dugu9sword","download_url":"https://codeload.github.com/dugu9sword/torch_random_fields/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248906128,"owners_count":21181118,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["crf","mrf","pytorch"],"created_at":"2025-04-14T15:29:32.215Z","updated_at":"2025-04-14T15:29:33.077Z","avatar_url":"https://github.com/dugu9sword.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\" width=\"30%\"\u003e\n\u003cimg src=\"images/logo.svg\" width=\"30%\" alt=\"logo\"/\u003e \n\u003c/p\u003e\n\n\u003c!-- # torch_random_fields --\u003e\n\n`torch_random_fields` is a library for building markov random fields (MRF) with complex topology \\[1\\] \\[2\\] with `pytorch`, it is optimized for batch training on GPU. \n\nThe key features include:\n\n- Easy to **plug** into your research code\n- Support for **batch** acceleration of any random field with arbitary **binary** or **ternary** connections on the **GPU**\n- Fast training/inference with **top-K** logits, do not worry about too large label space\n- Support for **context-aware** transition matrix and **low-rank** factorization\n\n\nYou may cite this project by:\n\n```\n@inproceedings{\n  wang2022regularized,\n  title={Regularized Molecular Conformation Fields},\n  author={Lihao Wang and Yi Zhou and Yiqun Wang and Xiaoqing Zheng and Xuanjing Huang and Hao Zhou},\n  booktitle={Advances in Neural Information Processing Systems},\n  editor={Alice H. Oh and Alekh Agarwal and Danielle Belgrave and Kyunghyun Cho},\n  year={2022},\n  url={https://openreview.net/forum?id=7XCFxnG8nGS}\n}\n```\n\n## Cases\n\n### Linear-Chain CRF\n\nCheck out the [tutorial](tests/test_linear_chain_crf.ipynb).\n\nThe well known linear-chain CRF which is heavily adopted in sequence labeling (POS-tagging, chunking, NER, etc.) is supported.\n\n\u003cimg src=\"images/linear_chain_crf.svg\" alt=\"logo\" style=\"zoom:100%;\" /\u003e \n\n### Top-K Skip-Chain CRF\n\nCheck out the [tutorial](tests/test_skip_chain_crf.ipynb).\n\nIn `torch_random_fields`, any random field with arbitary topology is supported. To be more precise, we require binary connections, although in some case ternary connections are also supported (yes, I am lazy).\n\nHere we show a case of Dynamic Skip-Chain CRF, where:\n\n- Some nodes (e.g., two nodes with the same words) are connected, which looks **skipping** the linear connection \\[3\\]\n- Only the **top-3** labels for each node are kept, greatly speeding up training and inference \\[4\\]\n\n\u003cimg src=\"images/top_k_skip_chain_crf.svg\" alt=\"logo\" style=\"zoom:100%;\" /\u003e \n\n### Ising/Potts Model\n\nIsing model (or Potts model) is widely used in statistical physics and computational biology \\[5\\]. In this case, the random variables form a grid, but it can be fully connected.\n\n\u003cimg src=\"images/potts.svg\" alt=\"logo\" style=\"zoom:100%;\" /\u003e \n\n## Features\n\n### Learning\n\n- **Linear-Chain CRF**: \n  - [x] maximum likelihood estimation\n  - [x] structured perceptron\n  - [x] piecewise training\n  - [x] pseudo-likelihood\n- **General CRF**: \n  - [x] structured perceptron\n  - [x] piecewise training\n  - [x] pseudo-likelihood\n\n\n### Inference\n\n- **Linear-Chain CRF**: \n  - [x] viterbi decoding\n  - [x] **batch** loopy belief propagation\n  - [x] **batch** mean field variational inference\n\n- **General CRF**: \n  - [x] **batch** loopy belief propagation\n  - [x] naive mean field variational inference\n  - [ ] **batch** naive mean field inference\n\n### Sampling\n\n- [x] Gibbs Sampling\n\n\n## Acknowledgement\n\nSome implementation borrows from these great projects with modifications:\n\n- linear-chain crf: [pytorch-crf](https://github.com/kmkurn/pytorch-crf), [fairseq](https://github.com/facebookresearch/fairseq)\n- loopy belief propagation: [pystruct](https://github.com/pystruct/pystruct)\n\n\n## Reference\n\n\\[1\\] [An Introduction to Conditional Random Fields](https://arxiv.org/abs/1011.4088) (Sutton and McCallum, 2010) \n\n\\[2\\] [Graphical Models, Exponential Families, and Variational Inference](https://people.eecs.berkeley.edu/~wainwrig/Papers/WaiJor08_FTML.pdf) (Wainwright and Jordan, 2008)\n\n\\[3\\] [A Skip-Chain Conditional Random Field for Ranking Meeting Utterances by Importance](https://aclanthology.org/W06-1643/) (Galley, 2006)\n\n\\[4\\] [Fast Structured Decoding for Sequence Models](http://papers.neurips.cc/paper/8566-fast-structured-decoding-for-sequence-models.pdf) (Sun, 2020)\n\n\\[5\\] [Improved contact prediction in proteins: Using pseudolikelihoods to infer Potts models](https://arxiv.org/abs/1211.1281) (Ekeberg, 2013)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdugu9sword%2Ftorch_random_fields","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdugu9sword%2Ftorch_random_fields","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdugu9sword%2Ftorch_random_fields/lists"}