{"id":15032401,"url":"https://github.com/lucasb-eyer/pydensecrf","last_synced_at":"2025-05-14T13:07:01.018Z","repository":{"id":37549893,"uuid":"46139143","full_name":"lucasb-eyer/pydensecrf","owner":"lucasb-eyer","description":"Python wrapper to Philipp Krähenbühl's dense (fully connected) CRFs with gaussian edge potentials.","archived":false,"fork":false,"pushed_at":"2024-03-05T08:38:53.000Z","size":2745,"stargazers_count":1973,"open_issues_count":42,"forks_count":420,"subscribers_count":38,"default_branch":"master","last_synced_at":"2025-04-11T06:03:28.798Z","etag":null,"topics":["computer-vision","crf","cython","eigen","machine-learning","pairwise-potentials","unary-potentials"],"latest_commit_sha":null,"homepage":null,"language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lucasb-eyer.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2015-11-13T18:15:38.000Z","updated_at":"2025-04-08T14:13:25.000Z","dependencies_parsed_at":"2022-07-12T16:22:55.247Z","dependency_job_id":"ccde61e2-11a2-4dc2-a406-2e793589c71b","html_url":"https://github.com/lucasb-eyer/pydensecrf","commit_stats":{"total_commits":74,"total_committers":16,"mean_commits":4.625,"dds":0.3513513513513513,"last_synced_commit":"dd070546eda51e21ab772ee6f14807c7f5b1548b"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lucasb-eyer%2Fpydensecrf","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lucasb-eyer%2Fpydensecrf/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lucasb-eyer%2Fpydensecrf/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lucasb-eyer%2Fpydensecrf/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lucasb-eyer","download_url":"https://codeload.github.com/lucasb-eyer/pydensecrf/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254149955,"owners_count":22022851,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["computer-vision","crf","cython","eigen","machine-learning","pairwise-potentials","unary-potentials"],"created_at":"2024-09-24T20:18:17.932Z","updated_at":"2025-05-14T13:06:55.983Z","avatar_url":"https://github.com/lucasb-eyer.png","language":"C++","funding_links":[],"categories":["对象检测_分割"],"sub_categories":["资源传输下载"],"readme":"PyDenseCRF\n==========\n\nThis is a (Cython-based) Python wrapper for [Philipp Krähenbühl's Fully-Connected CRFs](http://web.archive.org/web/20161023180357/http://www.philkr.net/home/densecrf) (version 2, [new, incomplete page](http://www.philkr.net/2011/12/01/nips/)).\n\nIf you use this code for your reasearch, please cite:\n\n```\nEfficient Inference in Fully Connected CRFs with Gaussian Edge Potentials\nPhilipp Krähenbühl and Vladlen Koltun\nNIPS 2011\n```\n\nand provide a link to this repository as a footnote or a citation.\n\nInstallation\n============\n\nThe package is on PyPI, so simply run `pip install pydensecrf` to install it.\n\nIf you want the newest and freshest version, you can install it by executing:\n\n```\npip install git+https://github.com/lucasb-eyer/pydensecrf.git\n```\n\nand ignoring all the warnings coming from Eigen.\n\nNote that you need a relatively recent version of Cython (at least version 0.22) for this wrapper,\nthe one shipped with Ubuntu 14.04 is too old. (Thanks to Scott Wehrwein for pointing this out.)\nI suggest you use a [virtual environment](https://virtualenv.readthedocs.org/en/latest/) and install\nthe newest version of Cython there (`pip install cython`), but you may update the system version by\n\n```\nsudo apt-get remove cython\nsudo pip install -U cython\n```\n\n### Problems on Windows/VS\n\nSince this library needs to compile C++ code, installation can be a little more problematic than pure Python packages.\nMake sure to [have Cython installed](https://github.com/lucasb-eyer/pydensecrf/issues/62#issuecomment-400563257) or try [installing via conda instead](https://github.com/lucasb-eyer/pydensecrf/issues/69#issuecomment-400639881) if you are getting problems.\nPRs that improve Windows support are welcome.\n\n### Problems on Colab/Kaggle Kernel\n\n`pydensecrf` does not come pre-installed in Colab or Kaggle Kernel. Running `pip install pydensecrf` will result into\nbuild failures. Follow these steps instead for Colab/Kaggle Kernel:\n\n```\npip install -U cython\npip install git+https://github.com/lucasb-eyer/pydensecrf.git\n```\n\nUsage\n=====\n\nFor images, the easiest way to use this library is using the `DenseCRF2D` class:\n\n```python\nimport numpy as np\nimport pydensecrf.densecrf as dcrf\n\nd = dcrf.DenseCRF2D(640, 480, 5)  # width, height, nlabels\n```\n\nUnary potential\n---------------\n\nYou can then set a fixed unary potential in the following way:\n\n```python\nU = np.array(...)     # Get the unary in some way.\nprint(U.shape)        # -\u003e (5, 480, 640)\nprint(U.dtype)        # -\u003e dtype('float32')\nU = U.reshape((5,-1)) # Needs to be flat.\nd.setUnaryEnergy(U)\n\n# Or alternatively: d.setUnary(ConstUnary(U))\n```\n\nRemember that `U` should be negative log-probabilities, so if you're using\nprobabilities `py`, don't forget to `U = -np.log(py)` them.\n\nRequiring the `reshape` on the unary is an API wart that I'd like to fix, but\ndon't know how to without introducing an explicit dependency on numpy.\n\n**Note** that the `nlabels` dimension is the first here before the reshape;\nyou may need to move it there before reshaping if that's not already the case,\nlike so:\n\n```python\nprint(U.shape)  # -\u003e (480, 640, 5)\nU = U.transpose(2, 0, 1).reshape((5,-1))\n```\n\n### Getting a Unary\n\nThere's two common ways of getting unary potentials:\n\n1. From a hard labeling generated by a human or some other processing.\n   This case is covered by `from pydensecrf.utils import unary_from_labels`.\n\n2. From a probability distribution computed by, e.g. the softmax output of a\n   deep network. For this, see `from pydensecrf.utils import unary_from_softmax`.\n\nFor usage of both of these, please refer to their docstrings or have a look at [the example](examples/inference.py).\n\nPairwise potentials\n-------------------\n\nThe two-dimensional case has two utility methods for adding the most-common pairwise potentials:\n\n```python\n# This adds the color-independent term, features are the locations only.\nd.addPairwiseGaussian(sxy=(3,3), compat=3, kernel=dcrf.DIAG_KERNEL, normalization=dcrf.NORMALIZE_SYMMETRIC)\n\n# This adds the color-dependent term, i.e. features are (x,y,r,g,b).\n# im is an image-array, e.g. im.dtype == np.uint8 and im.shape == (640,480,3)\nd.addPairwiseBilateral(sxy=(80,80), srgb=(13,13,13), rgbim=im, compat=10, kernel=dcrf.DIAG_KERNEL, normalization=dcrf.NORMALIZE_SYMMETRIC)\n```\n\nBoth of these methods have shortcuts and default-arguments such that the most\ncommon use-case can be simplified to:\n\n```python\nd.addPairwiseGaussian(sxy=3, compat=3)\nd.addPairwiseBilateral(sxy=80, srgb=13, rgbim=im, compat=10)\n```\n\nThe parameters map to those in the paper as follows: `sxy` in the `Gaussian` case is `$\\theta_{\\gamma}$`,\nand in the `Bilateral` case, `sxy` and `srgb` map to `$\\theta_{\\alpha}$` and `$\\theta_{\\beta}$`, respectively.\nThe names are shorthand for \"x/y standard-deviation\" and \"rgb standard-deviation\" and for reference, the formula is:\n\n![Equation 3 in the original paper](https://user-images.githubusercontent.com/10962198/36150757-01122bf2-10c5-11e8-97d2-2e833c1c9461.png)\n\n### Non-RGB bilateral\n\nAn important caveat is that `addPairwiseBilateral` only works for RGB images, i.e. three channels.\nIf your data is of different type than this simple but common case, you'll need to compute your\nown pairwise energy using `utils.create_pairwise_bilateral`; see the [generic non-2D case](https://github.com/lucasb-eyer/pydensecrf#generic-non-2d) for details.\n\nA good [example of working with Non-RGB data](https://github.com/lucasb-eyer/pydensecrf/blob/master/examples/Non%20RGB%20Example.ipynb) is provided as a notebook in the examples folder.\n\n### Compatibilities\n\nThe `compat` argument can be any of the following:\n\n- A number, then a `PottsCompatibility` is being used.\n- A 1D array, then a `DiagonalCompatibility` is being used.\n- A 2D array, then a `MatrixCompatibility` is being used.\n\nThese are label-compatibilites `µ(xi, xj)` whose parameters could possibly be [learned](https://github.com/lucasb-eyer/pydensecrf#learning).\nFor example, they could indicate that mistaking `bird` pixels for `sky` is not as bad as mistaking `cat` for `sky`.\nThe arrays should have `nlabels` or `(nlabels,nlabels)` as shape and a `float32` datatype.\n\n### Kernels\n\nPossible values for the `kernel` argument are:\n\n- `CONST_KERNEL`\n- `DIAG_KERNEL` (the default)\n- `FULL_KERNEL`\n\nThis specifies the kernel's precision-matrix `Λ(m)`, which could possibly be learned.\nThese indicate correlations between feature types, the default implying no correlation.\nAgain, this could possiblty be [learned](https://github.com/lucasb-eyer/pydensecrf#learning).\n\n### Normalizations\n\nPossible values for the `normalization` argument are:\n\n- `NO_NORMALIZATION`\n- `NORMALIZE_BEFORE`\n- `NORMALIZE_AFTER`\n- `NORMALIZE_SYMMETRIC` (the default)\n\n### Kernel weight\n\nI have so far not found a way to set the kernel weights `w(m)`.\nAccording to the paper, `w(2)` was set to 1 and `w(1)` was cross-validated, but never specified.\nLooking through Philip's code (included in [pydensecrf/densecrf](https://github.com/lucasb-eyer/pydensecrf/tree/master/pydensecrf/densecrf)),\nI couldn't find such explicit weights, and my guess is they are thus hard-coded to 1.\nIf anyone knows otherwise, please open an issue or, better yet, a pull-request.\nUpdate: user @waldol1 has an idea in [this issue](https://github.com/lucasb-eyer/pydensecrf/issues/37). Feel free to try it out!\n\nInference\n---------\n\nThe easiest way to do inference with 5 iterations is to simply call:\n\n```python\nQ = d.inference(5)\n```\n\nAnd the MAP prediction is then:\n\n```python\nmap = np.argmax(Q, axis=0).reshape((640,480))\n```\n\nIf you're interested in the class-probabilities `Q`, you'll notice `Q` is a\nwrapped Eigen matrix. The Eigen wrappers of this project implement the buffer\ninterface and can be simply cast to numpy arrays like so:\n\n```python\nproba = np.array(Q)\n```\n\nStep-by-step inference\n----------------------\n\nIf for some reason you want to run the inference loop manually, you can do so:\n\n```python\nQ, tmp1, tmp2 = d.startInference()\nfor i in range(5):\n    print(\"KL-divergence at {}: {}\".format(i, d.klDivergence(Q)))\n    d.stepInference(Q, tmp1, tmp2)\n```\n\nGeneric non-2D\n--------------\n\nThe `DenseCRF` class can be used for generic (non-2D) dense CRFs.\nIts usage is exactly the same as above, except that the 2D-specific pairwise\npotentials `addPairwiseGaussian` and `addPairwiseBilateral` are missing.\n\nInstead, you need to use the generic `addPairwiseEnergy` method like this:\n\n```python\nd = dcrf.DenseCRF(100, 5)  # npoints, nlabels\n\nfeats = np.array(...)  # Get the pairwise features from somewhere.\nprint(feats.shape)     # -\u003e (7, 100) = (feature dimensionality, npoints)\nprint(feats.dtype)     # -\u003e dtype('float32')\n\ndcrf.addPairwiseEnergy(feats)\n```\n\nIn addition, you can pass `compatibility`, `kernel` and `normalization`\narguments just like in the 2D gaussian and bilateral cases.\n\nThe potential will be computed as `w*exp(-0.5 * |f_i - f_j|^2)`.\n\n### Pairwise potentials for N-D\n\nUser @markusnagel has written a couple numpy-functions generalizing the two\nclassic 2-D image pairwise potentials (gaussian and bilateral) to an arbitrary\nnumber of dimensions: `create_pairwise_gaussian` and `create_pairwise_bilateral`.\nYou can access them as `from pydensecrf.utils import create_pairwise_gaussian`\nand then have a look at their docstring to see how to use them.\n\nLearning\n--------\n\nThe learning has not been fully wrapped. If you need it, get in touch or better\nyet, wrap it and submit a pull-request!\n\nHere's a pointer for starters: issue#24. We need to wrap the gradients and getting/setting parameters.\nBut then, we also need to do something with these, most likely call [minimizeLBFGS from optimization.cpp](https://github.com/lucasb-eyer/pydensecrf/blob/d824b89ee3867bca3e90b9f04c448f1b41821524/pydensecrf/densecrf/src/optimization.cpp).\nIt should be relatively straightforward to just follow the learning examples included in the [original code](http://graphics.stanford.edu/projects/drf/densecrf_v_2_2.zip).\n\nCommon Problems\n===============\n\n`undefined symbol` when importing\n---------------------------------\n\nIf while importing pydensecrf you get an error about some undefined symbols (for example `.../pydensecrf/densecrf.so: undefined symbol: _ZTINSt8ios_base7failureB5cxx11E`), you most likely are inadvertently mixing different compilers or toolchains. Try to see what's going on using tools like `ldd`. If you're using Anaconda, [running `conda install libgcc` might be a solution](https://github.com/lucasb-eyer/pydensecrf/issues/28).\n\nValueError: Buffer dtype mismatch, expected 'float' but got 'double'\n--------------------------------------------------------------------\n\nThis is a pretty [co](https://github.com/lucasb-eyer/pydensecrf/issues/52)mm[on](https://github.com/lucasb-eyer/pydensecrf/issues/49) user error.\nIt means exactly what it says: you are passing a `double` but it wants a `float`.\nSolve it by, for example, calling `d.setUnaryEnergy(U.astype(np.float32))` instead of just `d.setUnaryEnergy(U)`, or using `float32` in your code in the first place.\n\nMy results are all pixelated like [MS Paint's airbrush tool](http://lmgtfy.com/?q=MS+Paint+Airbrush+tool)!\n----------------------------------------------------------\n\nYou screwed up reshaping somewhere and treated the class/label dimension as spatial dimension.\nThis is you misunderstanding NumPy's memory layout and nothing that PyDenseCRF can detect or help with.\n\nThis mistake often happens for the Unary, see the [**Note** in that section of the README](https://github.com/lucasb-eyer/pydensecrf#unary-potential).\n\n\nMaintaining\n===========\n\nThese are instructions for maintainers about how to release new versions. (Mainly instructions for myself.)\n\n```\n# Go increment the version in setup.py\n\u003e python setup.py build_ext\n\u003e python setup.py sdist\n\u003e twine upload dist/pydensecrf-VERSION_NUM.tar.gz\n```\n\nAnd that's it. At some point, it would be cool to automate this on [TravisCI](https://docs.travis-ci.com/user/deployment/pypi/), but not worth it yet.\nAt that point, looking into [creating \"manylinux\" wheels](https://github.com/pypa/python-manylinux-demo) might be nice, too.\n\nTesting\n=======\n\nThanks to @MarvinTeichmann we now have proper tests, install the package and run `py.test`.\nMaybe there's a better way to run them, but both of us don't know :smile:\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flucasb-eyer%2Fpydensecrf","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flucasb-eyer%2Fpydensecrf","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flucasb-eyer%2Fpydensecrf/lists"}