{"id":20962225,"url":"https://github.com/lartpang/pysodmetrics","last_synced_at":"2025-04-07T15:10:45.116Z","repository":{"id":45333206,"uuid":"314843932","full_name":"lartpang/PySODMetrics","owner":"lartpang","description":"PySODMetrics: A Simple and Efficient Implementation of Grayscale/Binary Segmentation Metrcis","archived":false,"fork":false,"pushed_at":"2024-05-13T16:07:55.000Z","size":208,"stargazers_count":124,"open_issues_count":1,"forks_count":18,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-05-29T16:43:34.014Z","etag":null,"topics":["metrics","metrics-evaluation","metrics-library","metrics-reported","python3","saliency","saliency-detection","saliency-map","saliency-maps","saliency-methods","saliency-model","saliency-prediction","salient-object-detection","salient-regions"],"latest_commit_sha":null,"homepage":"https://pypi.org/project/pysodmetrics/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lartpang.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-11-21T15:35:51.000Z","updated_at":"2024-05-29T11:03:21.000Z","dependencies_parsed_at":"2023-02-09T03:16:14.604Z","dependency_job_id":"2c97cd2a-15e2-4963-bfa7-5ee009632e40","html_url":"https://github.com/lartpang/PySODMetrics","commit_stats":{"total_commits":49,"total_committers":2,"mean_commits":24.5,"dds":"0.020408163265306145","last_synced_commit":"e4a4e8202b9f1e4cdca6a328e089bfb768e6a04b"},"previous_names":[],"tags_count":11,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lartpang%2FPySODMetrics","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lartpang%2FPySODMetrics/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lartpang%2FPySODMetrics/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lartpang%2FPySODMetrics/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lartpang","download_url":"https://codeload.github.com/lartpang/PySODMetrics/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247675607,"owners_count":20977378,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["metrics","metrics-evaluation","metrics-library","metrics-reported","python3","saliency","saliency-detection","saliency-map","saliency-maps","saliency-methods","saliency-model","saliency-prediction","salient-object-detection","salient-regions"],"created_at":"2024-11-19T02:25:00.263Z","updated_at":"2025-04-07T15:10:45.062Z","avatar_url":"https://github.com/lartpang.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"https://github.com/lartpang/PySODMetrics/blob/main/images/logo.png?raw=true\" alt=\"Logo\" width=\"320\" height=\"auto\"\u003e\n  \u003c/br\u003e\n  \u003ch2\u003ePySODMetrics: A simple and efficient implementation of SOD metrics\u003c/h2\u003e\n  \u003ca href=\"./readme_zh.md\"\u003e\u003cimg src=\"https://img.shields.io/badge/README-%E4%B8%AD%E6%96%87-blue\"\u003e\u003c/a\u003e\n  \u003cimg src=\"https://img.shields.io/pypi/v/pysodmetrics\"\u003e\n  \u003cimg src=\"https://img.shields.io/pypi/dm/pysodmetrics?label=pypi%20downloads\u0026logo=PyPI\u0026logoColor=white\"\u003e\n  \u003cimg src=\"https://img.shields.io/github/last-commit/lartpang/PySODMetrics\"\u003e\n  \u003cimg src=\"https://img.shields.io/github/last-commit/lartpang/PySODMetrics\"\u003e\n  \u003cimg src=\"https://img.shields.io/github/release-date/lartpang/PySODMetrics\"\u003e\n  \u003cimg src=\"https://api.star-history.com/svg?repos=lartpang/PySODMetrics\u0026type=Date\" alt=\"Star History Chart\" width=\"600\" height=\"auto\"\u003e\n\u003c/div\u003e\n\n## Introduction\n\nA simple and efficient implementation of SOD metrics.\n\n- Based on `numpy` and `scipy`\n- Verification based on Fan's matlab code \u003chttps://github.com/DengPingFan/CODToolbox\u003e\n- The code structure is simple and easy to extend\n- The code is lightweight and fast\n\nYour improvements and suggestions are welcome.\n\n### Related Projects\n\n- [PySODEvalToolkit](https://github.com/lartpang/PySODEvalToolkit): A Python-based Evaluation Toolbox for Salient Object Detection and Camouflaged Object Detection\n\n### Supported Metrics\n\n| Metric                                    | Sample-based    | Whole-based | Related Class                         |\n| ----------------------------------------- | --------------- | ----------- | ------------------------------------- |\n| MAE                                       | soft            |             | `MAE`                                 |\n| S-measure $S_{m}$                         | soft            |             | `Smeasure`                            |\n| weighted F-measure ($F^{\\omega}_{\\beta}$) | soft            |             | `WeightedFmeasure`                    |\n| Multi-Scale IoU                           | bin             |             | `MSIoU`                               |\n| E-measure ($E_{m}$)                       | max,avg,adp     |             | `Emeasure`                            |\n| F-measure (old) ($F_{beta}$)              | max,avg,adp     |             | `Fmeasure` (Will be removed!)         |\n| F-measure (new) ($F_{beta}$, $F_{1}$)     | max,avg,adp,bin | bin         | `FmeasureV2`+`FmeasureHandler`        |\n| BER                                       | max,avg,adp,bin | bin         | `FmeasureV2`+`BERHandler`             |\n| Dice                                      | max,avg,adp,bin | bin         | `FmeasureV2`+`DICEHandler`            |\n| FPR                                       | max,avg,adp,bin | bin         | `FmeasureV2`+`FPRHandler`             |\n| IoU                                       | max,avg,adp,bin | bin         | `FmeasureV2`+`IOUHandler`             |\n| Kappa                                     | max,avg,adp,bin | bin         | `FmeasureV2`+`KappaHandler`           |\n| Overall Accuracy                          | max,avg,adp,bin | bin         | `FmeasureV2`+`OverallAccuracyHandler` |\n| Precision                                 | max,avg,adp,bin | bin         | `FmeasureV2`+`PrecisionHandler`       |\n| Recall                                    | max,avg,adp,bin | bin         | `FmeasureV2`+`RecallHandler`          |\n| Sensitivity                               | max,avg,adp,bin | bin         | `FmeasureV2`+`SensitivityHandler`     |\n| Specificity                               | max,avg,adp,bin | bin         | `FmeasureV2`+`SpecificityHandler`     |\n| TNR                                       | max,avg,adp,bin | bin         | `FmeasureV2`+`TNRHandler`             |\n| TPR                                       | max,avg,adp,bin | bin         | `FmeasureV2`+`TPRHandler`             |\n\n## Usage\n\nThe core files are in the folder `py_sod_metrics`.\n\n- **[Latest, but may be unstable]** Install from the source code: `pip install git+https://github.com/lartpang/PySODMetrics.git`\n- **[More stable]** Install from PyPI: `pip install pysodmetrics`\n\n### Examples\n\n- [examples/test_metrics.py](./examples/test_metrics.py)\n- [examples/metric_recorder.py](./examples/metric_recorder.py)\n\n## Reference\n\n- [Matlab Code](https://github.com/DengPingFan/CODToolbox) by DengPingFan(\u003chttps://github.com/DengPingFan\u003e): In our comparison (the test code can be seen under the `test` folder), the result is consistent with the code.\n  - The matlab code needs to change `Bi_sal(sal\u003ethreshold)=1;` to `Bi_sal(sal\u003e=threshold)=1;` in \u003chttps://github.com/DengPingFan/CODToolbox/blob/910358910c7824a4237b0ea689ac9d19d1958d11/Onekey_Evaluation_Code/OnekeyEvaluationCode/main.m#L102\u003e. For related discussion, please see [the issue](https://github.com/DengPingFan/CODToolbox/issues/1).\n  - 2021-12-20 (version `1.3.0`): Due to the difference between numpy and matlab, in version `1.2.x`, there are very slight differences on some metrics between the results of the matlab code and ours. The [recent PR](https://github.com/lartpang/PySODMetrics/pull/3) alleviated this problem. However, there are still very small differences on E-measure. The results in most papers are rounded off to three or four significant figures, so, there is no obvious difference between the new version and the version `1.2.x` for them.\n- \u003chttps://en.wikipedia.org/wiki/Precision_and_recall\u003e\n\n```text\n@inproceedings{Fmeasure,\n    title={Frequency-tuned salient region detection},\n    author={Achanta, Radhakrishna and Hemami, Sheila and Estrada, Francisco and S{\\\"u}sstrunk, Sabine},\n    booktitle=CVPR,\n    number={CONF},\n    pages={1597--1604},\n    year={2009}\n}\n\n@inproceedings{MAE,\n    title={Saliency filters: Contrast based filtering for salient region detection},\n    author={Perazzi, Federico and Kr{\\\"a}henb{\\\"u}hl, Philipp and Pritch, Yael and Hornung, Alexander},\n    booktitle=CVPR,\n    pages={733--740},\n    year={2012}\n}\n\n@inproceedings{Smeasure,\n    title={Structure-measure: A new way to evaluate foreground maps},\n    author={Fan, Deng-Ping and Cheng, Ming-Ming and Liu, Yun and Li, Tao and Borji, Ali},\n    booktitle=ICCV,\n    pages={4548--4557},\n    year={2017}\n}\n\n@inproceedings{Emeasure,\n    title=\"Enhanced-alignment Measure for Binary Foreground Map Evaluation\",\n    author=\"Deng-Ping {Fan} and Cheng {Gong} and Yang {Cao} and Bo {Ren} and Ming-Ming {Cheng} and Ali {Borji}\",\n    booktitle=IJCAI,\n    pages=\"698--704\",\n    year={2018}\n}\n\n@inproceedings{wFmeasure,\n  title={How to evaluate foreground maps?},\n  author={Margolin, Ran and Zelnik-Manor, Lihi and Tal, Ayellet},\n  booktitle=CVPR,\n  pages={248--255},\n  year={2014}\n}\n\n@inproceedings{MSIoU,\n    title = {Multiscale IOU: A Metric for Evaluation of Salient Object Detection with Fine Structures},\n    author = {Ahmadzadeh, Azim and Kempton, Dustin J. and Chen, Yang and Angryk, Rafal A.},\n    booktitle = ICIP,\n    year = {2021},\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flartpang%2Fpysodmetrics","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flartpang%2Fpysodmetrics","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flartpang%2Fpysodmetrics/lists"}