https://github.com/dugu9sword/torch_random_fields
A highly optimized library for building markov random fields with pytorch.
https://github.com/dugu9sword/torch_random_fields
crf mrf pytorch
Last synced: about 1 year ago
JSON representation
A highly optimized library for building markov random fields with pytorch.
- Host: GitHub
- URL: https://github.com/dugu9sword/torch_random_fields
- Owner: dugu9sword
- Created: 2022-10-01T15:36:59.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2023-11-02T07:26:37.000Z (over 2 years ago)
- Last Synced: 2024-03-14T15:56:43.812Z (over 2 years ago)
- Topics: crf, mrf, pytorch
- Language: Jupyter Notebook
- Homepage:
- Size: 163 KB
- Stars: 9
- Watchers: 2
- Forks: 2
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
`torch_random_fields` is a library for building markov random fields (MRF) with complex topology \[1\] \[2\] with `pytorch`, it is optimized for batch training on GPU.
The key features include:
- Easy to **plug** into your research code
- Support for **batch** acceleration of any random field with arbitary **binary** or **ternary** connections on the **GPU**
- Fast training/inference with **top-K** logits, do not worry about too large label space
- Support for **context-aware** transition matrix and **low-rank** factorization
You may cite this project by:
```
@inproceedings{
wang2022regularized,
title={Regularized Molecular Conformation Fields},
author={Lihao Wang and Yi Zhou and Yiqun Wang and Xiaoqing Zheng and Xuanjing Huang and Hao Zhou},
booktitle={Advances in Neural Information Processing Systems},
editor={Alice H. Oh and Alekh Agarwal and Danielle Belgrave and Kyunghyun Cho},
year={2022},
url={https://openreview.net/forum?id=7XCFxnG8nGS}
}
```
## Cases
### Linear-Chain CRF
Check out the [tutorial](tests/test_linear_chain_crf.ipynb).
The well known linear-chain CRF which is heavily adopted in sequence labeling (POS-tagging, chunking, NER, etc.) is supported.
### Top-K Skip-Chain CRF
Check out the [tutorial](tests/test_skip_chain_crf.ipynb).
In `torch_random_fields`, any random field with arbitary topology is supported. To be more precise, we require binary connections, although in some case ternary connections are also supported (yes, I am lazy).
Here we show a case of Dynamic Skip-Chain CRF, where:
- Some nodes (e.g., two nodes with the same words) are connected, which looks **skipping** the linear connection \[3\]
- Only the **top-3** labels for each node are kept, greatly speeding up training and inference \[4\]
### Ising/Potts Model
Ising model (or Potts model) is widely used in statistical physics and computational biology \[5\]. In this case, the random variables form a grid, but it can be fully connected.
## Features
### Learning
- **Linear-Chain CRF**:
- [x] maximum likelihood estimation
- [x] structured perceptron
- [x] piecewise training
- [x] pseudo-likelihood
- **General CRF**:
- [x] structured perceptron
- [x] piecewise training
- [x] pseudo-likelihood
### Inference
- **Linear-Chain CRF**:
- [x] viterbi decoding
- [x] **batch** loopy belief propagation
- [x] **batch** mean field variational inference
- **General CRF**:
- [x] **batch** loopy belief propagation
- [x] naive mean field variational inference
- [ ] **batch** naive mean field inference
### Sampling
- [x] Gibbs Sampling
## Acknowledgement
Some implementation borrows from these great projects with modifications:
- linear-chain crf: [pytorch-crf](https://github.com/kmkurn/pytorch-crf), [fairseq](https://github.com/facebookresearch/fairseq)
- loopy belief propagation: [pystruct](https://github.com/pystruct/pystruct)
## Reference
\[1\] [An Introduction to Conditional Random Fields](https://arxiv.org/abs/1011.4088) (Sutton and McCallum, 2010)
\[2\] [Graphical Models, Exponential Families, and Variational Inference](https://people.eecs.berkeley.edu/~wainwrig/Papers/WaiJor08_FTML.pdf) (Wainwright and Jordan, 2008)
\[3\] [A Skip-Chain Conditional Random Field for Ranking Meeting Utterances by Importance](https://aclanthology.org/W06-1643/) (Galley, 2006)
\[4\] [Fast Structured Decoding for Sequence Models](http://papers.neurips.cc/paper/8566-fast-structured-decoding-for-sequence-models.pdf) (Sun, 2020)
\[5\] [Improved contact prediction in proteins: Using pseudolikelihoods to infer Potts models](https://arxiv.org/abs/1211.1281) (Ekeberg, 2013)