{"id":19386199,"url":"https://github.com/davidstutz/superpixel-benchmark","last_synced_at":"2025-04-05T14:08:40.485Z","repository":{"id":45771440,"uuid":"75552029","full_name":"davidstutz/superpixel-benchmark","owner":"davidstutz","description":"An extensive evaluation and comparison of 28 state-of-the-art superpixel algorithms on 5 datasets.","archived":false,"fork":false,"pushed_at":"2024-01-06T15:23:41.000Z","size":28799,"stargazers_count":405,"open_issues_count":3,"forks_count":109,"subscribers_count":20,"default_branch":"master","last_synced_at":"2025-03-29T13:11:19.734Z","etag":null,"topics":["benchmark","computer-vision","evaluation","image-procesing","opencv","superpixel-algorithms","superpixels"],"latest_commit_sha":null,"homepage":null,"language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/davidstutz.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-12-04T17:16:09.000Z","updated_at":"2025-03-19T14:07:37.000Z","dependencies_parsed_at":"2024-11-17T23:33:25.746Z","dependency_job_id":null,"html_url":"https://github.com/davidstutz/superpixel-benchmark","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/davidstutz%2Fsuperpixel-benchmark","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/davidstutz%2Fsuperpixel-benchmark/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/davidstutz%2Fsuperpixel-benchmark/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/davidstutz%2Fsuperpixel-benchmark/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/davidstutz","download_url":"https://codeload.github.com/davidstutz/superpixel-benchmark/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247345854,"owners_count":20924102,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["benchmark","computer-vision","evaluation","image-procesing","opencv","superpixel-algorithms","superpixels"],"created_at":"2024-11-10T10:04:39.404Z","updated_at":"2025-04-05T14:08:40.452Z","avatar_url":"https://github.com/davidstutz.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Superpixels: An Evaluation of the State-of-the-Art\n\n[![Build Status](https://travis-ci.org/davidstutz/superpixel-benchmark.svg?branch=master)](https://travis-ci.org/davidstutz/superpixel-benchmark)\n\nThis repository contains the source code used for evaluation in [1], a large-scale \ncomparison of state-of-the-art superpixel algorithms.\n\n**[ArXiv](https://arxiv.org/abs/1612.01601) | \n[Project Page](http://davidstutz.de/projects/superpixel-benchmark/) | \n[Datasets](https://github.com/davidstutz/superpixel-benchmark-data) | \n[Doxygen Documentation](https://davidstutz.github.io/superpixel-benchmark/)**\n\nThis repository subsumes earlier work on comparing superpixel algorithms:\n[davidstutz/gcpr2015-superpixels](https://github.com/davidstutz/gcpr2015-superpixels), \n[davidstutz/superpixels-revisited](https://github.com/davidstutz/superpixels-revisited).\n\n**Please cite the following work if you use this benchmark or the provided tools or implementations:**\n\n    [1] D. Stutz, A. Hermans, B. Leibe.\n        Superpixels: An Evaluation of the State-of-the-Art.\n        Computer Vision and Image Understanding, 2018.\n\nAlso make also sure to cite additional papers when using datasets or superpixel algorithms.\n\n**Updates:**\n\n* A docker implementation containing many of the algorithms was added to [./docker](./docker/).\n* An implementation of the average metrics, i.e. Average Boundary Recall (called\nAverage Miss Rate in the updated paper), Average Undersegmentation Error\nand Average Explained Variation (called Average Unexplained Variation in the updated paper)\nis provided in `lib_eval/evaluation.h` and an easy-to-use command line tool is provided,\nsee `eval_average_cli` and the corresponding documentation and examples in\n[Executables](docs/EXECUTABLES.md) and [Examples](docs/EXAMPLE.md) respectively.\n* As of Mar 29, 2017 the paper was accepted for publication at \n[CVIU](https://www.journals.elsevier.com/computer-vision-and-image-understanding/).\n* The converted (i.e. pre-processed) NYUV2, SBD and SUNRGBD datasets are now available\nin the [data repository](https://github.com/davidstutz/superpixel-benchmark-data).\n* The source code of MSS has been added.\n* The source code of PF and SEAW has been added.\n* Doxygen documentation is now available [here](http://davidstutz.github.io/superpixel-benchmark/).\n* The presented paper was in preparation for a longer period of time — \n  some recent superpixel algorithms are not included in the comparison. These include \n  [SCSP](https://github.com/freifeld/fastSCSP) and [LRW](https://github.com/shenjianbing/lrw14).\n\n## Table of Contents\n\n* [Introduction](#introduction)\n* [Algorithms](#algorithms)\n    * [Submission](#submission)\n* [Documentation](docs/README.md)\n    * [Datasets](docs/DATASETS.md)\n    * [Algorithms](docs/ALGORITHMS.md)\n        * [Submission](docs/SUBMISSION.md)\n    * [Benchmark](docs/BENCHMARK.md)\n    * [Building](docs/BUILDING.md)\n        * [Building CIS](docs/BUILDING_CIS.md)\n    * [Executables](docs/EXECUTABLES.md)\n    * [Examples](docs/EXAMPLES.md)\n    * [Parameters](docs/PARAMETERS.md)\n    * [Results](docs/RESULTS.md)\n        * [Data](docs/DATA.md)\n* [License](#license)\n\n## Introduction\n\nSuperpixels group pixels similar in color and other low-level properties.\nIn this respect, superpixels address two problems inherent to the processing of \ndigital images: firstly, pixels are merely a result of discretization; \nand secondly, the high number of pixels in large images prevents many algorithms\nfrom being computationally feasible. Superpixels were introduced as more natural \nentities - grouping pixels which perceptually belong together while heavily reducing\nthe number of primitives.\n\nThis repository can be understood as supplementary material for an extensive \nevaluation of 28 algorithms on 5 datasets regarding visual quality, performance,\nruntime, implementation details and robustness - as presented in [1]. To ensure \na fair comparison, parameters have been optimized on separate training sets; as \nthe number of generated superpixels heavily influences parameter optimization, \nwe additionally enforced connectivity. Furthermore, to evaluate superpixel algorithms \nindependent of the number of superpixels, we propose to integrate over commonly \nused metrics such as Boundary Recall, Undersegmentation Error and Explained Variation. \nFinally, we present a ranking of the superpixel algorithms considering multiple \nmetrics and independent of the number of generated superpixels, as shown below.\n\n![Algorithm ranking.](RANKING.png?raw=true \"Algorithm ranking.\")\n\nThe table shows the average ranks across the 5 datasets, taking into account Average\nBoundary Recall (ARec) and Average Undersegmentation Error (AUE) - lower is better \nin both cases, see [Benchmark](docs/BENCHMARK.md).\nThe confusion matrix shows the rank distribution of the algorithms across the datasets.\n\n## Algorithms\n\nThe following algorithms were evaluated in [1], and most of them are included in\nthis repository:\n\nIncluded                                   | Algorithm    | Reference\n-------------------------------------------|--------------|-----------\n:ballot_box_with_check:                    | CCS          | [Ref. \u0026 Web](http://www.emrahtasli.com/research/spextraction/)\n[Instructions](docs/BUILDING_CIS.md)       | CIS          | [Ref.](http://www.csd.uwo.ca/~olga/Papers/eccv2010final.pdf) \u0026 [Web](http://www.csd.uwo.ca/faculty/olga/)\n:ballot_box_with_check:                    | CRS          | [Ref.](http://link.springer.com/chapter/10.1007%2F978-3-642-40395-8_21#page-1) \u0026 [Web](http://www.vsi.cs.uni-frankfurt.de/research/superpixel-segmentation/)\n:ballot_box_with_check:                    | CW           | [Ref.](https://www.tu-chemnitz.de/etit/proaut/rsrc/cws_pSLIC_ICPR.pdf) \u0026 [Web](https://www.tu-chemnitz.de/etit/proaut/forschung/cv/segmentation.html.en)\n:ballot_box_with_check:                    | DASP         | [Ref.](http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=6460572) \u0026 [Web](https://github.com/Danvil/dasp)\n:ballot_box_with_check:                    | EAMS         | [Ref.](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.8.5341\u0026rep=rep1\u0026type=pdf), [Ref.](http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=977560), [Ref.](https://courses.csail.mit.edu/6.869/handouts/PAMIMeanshift.pdf) \u0026 [Web](http://coewww.rutgers.edu/riul/research/code/EDISON/)\n:ballot_box_with_check:                    | ERS          | [Ref.](http://www.merl.com/publications/docs/TR2011-035.pdf) \u0026 [Web](http://mingyuliu.net/)\n:ballot_box_with_check:                    | FH           | [Ref.](http://www.cs.cornell.edu/~dph/papers/seg-ijcv.pdf) \u0026 [Web](https://cs.brown.edu/~pff/segment/index.html)\n:ballot_box_with_check:                    | MSS          | [Ref.](http://avestia.com/MVML2014_Proceedings/papers/67.pdf)\n:ballot_box_with_check:                    | PB           | [Ref.](http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=6126393\u0026tag=1) \u0026 [Web](http://yuhang.rsise.anu.edu.au/yuhang/misc.html)\n:ballot_box_with_check:                    | preSLIC      | [Ref.](https://www.tu-chemnitz.de/etit/proaut/rsrc/cws_pSLIC_ICPR.pdf) \u0026 [Web](https://www.tu-chemnitz.de/etit/proaut/forschung/cv/segmentation.html.en)\n:ballot_box_with_check:                    | reSEEDS      | [Web](http://davidstutz.de/projects/superpixelsseeds/)\n:ballot_box_with_check:                    | SEAW         | [Ref.](http://patrec.cs.tu-dortmund.de/pubs/papers/Strassburg2015-OIS) \u0026 [Web](https://github.com/JohannStrassburg/InfluenceSegImageParsingCode)\n:ballot_box_with_check:                    | SEEDS        | [Ref.](http://arxiv.org/pdf/1309.3848v1.pdf) \u0026 [Web](http://www.mvdblive.org/seeds/)\n:ballot_box_with_check:                    | SLIC         | [Ref.](http://www.kev-smith.com/papers/SLIC_Superpixels.pdf) \u0026 [Web](http://ivrl.epfl.ch/research/superpixels)\n:ballot_box_with_check:                    | TP           | [Ref.](http://www.cs.toronto.edu/~babalex/09.pami.turbopixels.pdf) \u0026 [Web](http://www.cs.toronto.edu/~babalex/research.html)\n:ballot_box_with_check:                    | TPS          | [Ref.](http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=\u0026arnumber=6298495) \u0026 [Web](http://hzfu.github.io/subpage/codes.html)\n:ballot_box_with_check:                    | vlSLIC       | [Web](http://www.vlfeat.org/overview/slic.html)\n:ballot_box_with_check:                    | W            | [Web](http://docs.opencv.org/2.4/modules/imgproc/doc/miscellaneous_transformations.html?highlight=watershed#watershed)\n:ballot_box_with_check:                    | WP           | [Ref.](http://cmm.ensmp.fr/~machairas/waterpixels.html) \u0026 [Web](http://cmm.ensmp.fr/~machairas/waterpixels.html)\n:ballot_box_with_check:                    | PF           | [Ref.](http://users.dickinson.edu/~jmac/publications/fast-superpixels-WMVC09.pdf) \u0026 [Web](http://users.dickinson.edu/~jmac/publications/PathFinder.zip)\n:ballot_box_with_check:                    | LSC          | [Ref.](http://www.cv-foundation.org/openaccess/content_cvpr_2015/papers/Li_Superpixel_Segmentation_Using_2015_CVPR_paper.pdf) \u0026 [Web](http://jschenthu.weebly.com/projects.html)\n:ballot_box_with_check:                    | RW           | [Ref.](http://cns.bu.edu/~lgrady/grady2004multilabel.pdf) \u0026 [Web](http://cns.bu.edu/~lgrady/software.html)\n:ballot_box_with_check:                    | QS           | [Ref.](http://vision.cs.ucla.edu/papers/vedaldiS08quick.pdf) \u0026 [Web](http://www.vlfeat.org/overview/quickshift.html)\n:ballot_box_with_check:                    | NC           | [Ref.](http://ttic.uchicago.edu/~xren/publication/xren_iccv03_discrim.pdf) \u0026 [Web](http://www.cs.sfu.ca/~mori/research/superpixels)\n:ballot_box_with_check:                    | VCCS         | [Ref.](http://www.cv-foundation.org/openaccess/content_cvpr_2013/papers/Papon_Voxel_Cloud_Connectivity_2013_CVPR_paper.pdf) \u0026 [Web](http://pointclouds.org/documentation/tutorials/supervoxel_clustering.php)\n:ballot_box_with_check:                    | POISE        | [Ref.](http://web.engr.oregonstate.edu/~lif/Middle_Child_ICCV15.pdf) \u0026 [Web](http://rehg.org/poise/)\n:ballot_box_with_check:                    | VC           | [Ref.](http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=\u0026arnumber=6186738) \u0026 [Web](http://www-personal.umich.edu/~jwangumi/software.html)\n:ballot_box_with_check:                    | ETPS         | [Ref.](http://www.cs.toronto.edu/~yaojian/cvpr15.pdf) \u0026 [Web](https://bitbucket.org/mboben/spixel)\n:ballot_box_with_check:                    | ERGC         | [Ref.](https://hal.archives-ouvertes.fr/hal-00945893/document), [Ref.](http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=7025886) \u0026 [Web](https://sites.google.com/site/pierrebuyssens/code/ergc)\n\n### Submission\n\nTo keep the benchmark alive, we encourage authors to make their implementations\npublicly available and integrate them into this benchmark. We are happy to help with the\nintegration and update the results published in [1] and on the \n[project page](http://davidstutz.de/projects/superpixel-benchmark/).\nAlso see the [Documentation](docs/README.md) for details.\n\n## License\n\nLicenses for source code corresponding to:\n\nD. Stutz, A. Hermans, B. Leibe. **Superpixels: An Evaluation of the State-of-the-Art.** Computer Vision and Image Understanding, 2018.\n\t\t\nNote that the source code/data is based on other projects for which separate licenses apply, see:\n\n* [Algorithms](docs/ALGORITHMS.md)\n* [Datasets](docs/DATASETS.md)\n\nCopyright (c) 2016-2018 David Stutz, RWTH Aachen University\n\n**Please read carefully the following terms and conditions and any accompanying documentation before you download and/or use this software and associated documentation files (the \"Software\").**\n\nThe authors hereby grant you a non-exclusive, non-transferable, free of charge right to copy, modify, merge, publish, distribute, and sublicense the Software for the sole purpose of performing non-commercial scientific research, non-commercial education, or non-commercial artistic projects.\n\nAny other use, in particular any use for commercial purposes, is prohibited. This includes, without limitation, incorporation in a commercial product, use in a commercial service, or production of other artefacts for commercial purposes.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.\n\nYou understand and agree that the authors are under no obligation to provide either maintenance services, update services, notices of latent defects, or corrections of defects with regard to the Software. The authors nevertheless reserve the right to update, modify, or discontinue the Software at any time.\n\nThe above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. You agree to cite the corresponding papers (see above) in documents and papers that report on research using the Software.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdavidstutz%2Fsuperpixel-benchmark","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdavidstutz%2Fsuperpixel-benchmark","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdavidstutz%2Fsuperpixel-benchmark/lists"}