{"id":13689014,"url":"https://github.com/manuelaguadomtz/pyeer","last_synced_at":"2026-01-14T09:11:32.001Z","repository":{"id":46000436,"uuid":"146351537","full_name":"manuelaguadomtz/pyeer","owner":"manuelaguadomtz","description":"PyEER is a python package for biometric systems performance evaluation. Includes ROC, DET, FNMR, FMR and CMC curves plotting, scores distribution plotting, EER and operating points estimation. It can be also used to evaluate binary classification systems.","archived":false,"fork":false,"pushed_at":"2024-01-23T12:20:32.000Z","size":4417,"stargazers_count":61,"open_issues_count":5,"forks_count":17,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-09-18T05:27:53.059Z","etag":null,"topics":["biometrics","cmc","det","eer","equal-error-rate","fmr","fnmr","identification","roc","verification"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/manuelaguadomtz.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-08-27T20:41:06.000Z","updated_at":"2025-06-20T13:22:13.000Z","dependencies_parsed_at":"2023-12-16T18:32:17.375Z","dependency_job_id":"f6171c6f-095e-495d-8ddd-f1a24672eb89","html_url":"https://github.com/manuelaguadomtz/pyeer","commit_stats":{"total_commits":230,"total_committers":5,"mean_commits":46.0,"dds":"0.23043478260869565","last_synced_commit":"fae8f3f8732ca3c9b4ebcabce72117907e34c6b5"},"previous_names":[],"tags_count":24,"template":false,"template_full_name":null,"purl":"pkg:github/manuelaguadomtz/pyeer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/manuelaguadomtz%2Fpyeer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/manuelaguadomtz%2Fpyeer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/manuelaguadomtz%2Fpyeer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/manuelaguadomtz%2Fpyeer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/manuelaguadomtz","download_url":"https://codeload.github.com/manuelaguadomtz/pyeer/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/manuelaguadomtz%2Fpyeer/sbom","scorecard":{"id":615842,"data":{"date":"2025-08-11","repo":{"name":"github.com/manuelaguadomtz/pyeer","commit":"fae8f3f8732ca3c9b4ebcabce72117907e34c6b5"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":3,"checks":[{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Code-Review","score":0,"reason":"Found 2/28 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Pinned-Dependencies","score":-1,"reason":"no dependencies found","details":null,"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: MIT License: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 4 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-21T03:55:02.018Z","repository_id":46000436,"created_at":"2025-08-21T03:55:02.018Z","updated_at":"2025-08-21T03:55:02.018Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28414879,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-14T08:38:59.149Z","status":"ssl_error","status_checked_at":"2026-01-14T08:38:43.588Z","response_time":107,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["biometrics","cmc","det","eer","equal-error-rate","fmr","fnmr","identification","roc","verification"],"created_at":"2024-08-02T15:01:30.535Z","updated_at":"2026-01-14T09:11:31.996Z","avatar_url":"https://github.com/manuelaguadomtz.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# PyEER\n\n**PyEER** is a python package intended for biometric systems performance evaluation but it can be used to evaluate binary\nclassification systems also. It has been developed with the idea of providing researchers and the scientific community in\ngeneral with a tool to correctly evaluate and report the performance of their systems.\n\nWithin this package, two command line programs are provided:\n\n* **geteerinf:** Useful to evaluate biometrics systems in verification scenarios as well as binary classification systems.\n\n* **getcmcinf:** Useful to evaluate biometrics systems in identification scenarios.\n\nUtilities provided within this package can also be used to develop other scripts by importing the module **pyeer**.\n\n## Installing\n\nTo install the **PyEER** package you only need to type into a terminal the following line:\n\n    pip install pyeer\n\n## Usage\n\nUsing **PyEER** is very easy and straightforward. Don't you believe? Let's see if we can convince you.\n\n### geteerinf\n\nWhen evaluating biometric systems in verification scenarios some researchers will only report Equal Error Rate (EER) values\nto show the accuracy of their proposals. However, this is wrong. The behavior of biometric systems cannot be fully assessed in this way. In other to achieve that, Receiver Operating Characteristic (ROC) or Detection Error Tradeoff (DET) graph must\nbe reported. It is also very common to report other operating points besides EER (i.e. MR100, FMR1000, ZeroFMR, and ZeroFNMR).\n**geteerinf** provides these and more. You will only need to provide genuine match scores and impostor match scores [1]. Genuine\nmatch scores are obtained by matching feature sets of the same class (same person), while impostor matching scores are obtained\nby matching feature sets of different classes (different persons).\n\n#### Input file formats\n\nGenuine match scores and impostor match scores must be provided in separated files one score per line. Each line can have any\nnumber of columns but the scores must be in the last column. Additionally, impostor match scores can be provided in a different\nformat which explained next\n\n###### Histogram format \n\nAlthough the vast majority of the systems report scores normalized between 0 and 1, there are some that\nreport integer scores [3]. When computing a lot of impostor scores, millions of them, it can be computationally expensive to read\nall those scores from a file. Therefore, in those cases may be worth it to use this format.\n\nRestrictions: Only integer scores are supported\n\nFile format: Each line contains the number of scores equals to the index of the line in the file (starting from zero).\n\nRecommendations: Use this format for very large experiments (millions of scores).\n\nIf you have any doubts left, you should check the example files on [GitHub](https://github.com/manuelaguadomtz/pyeer/tree/master/pyeer/example_files).\n\n#### Usage examples\n\n##### To print the help\n\n    geteerinf -h\n\n##### One experiment (Non-histogram format):\n\n    geteerinf -p \"example_files/non_hist/\" -i \"exp3_false.txt\" -g \"exp3_true.txt\" -e \"exp3\"\n\n##### More than one experiment (Non-histogram format):\n\n    geteerinf -p \"example_files/non_hist/\" -i \"exp1_false.txt,exp2_false.txt\" -g \"exp1_true.txt,exp2_true.txt\" -e \"exp1,exp2\"\n\n##### One experiment (Histogram format):\n\n    geteerinf -p \"example_files/hist/\" -i \"exp1_false.txt\" -g \"exp1_true.txt\" -e \"exp1\" -ht\n\n#### Output\n\nAll of the above examples will generate the following information:\n\n* Graphs: \n    * Receiver operating characteristic (ROC)\n    * Detection error tradeoff (DET)\n    * False Match Rate (FMR) and False Non-Match Rate(FNMR)\n    * Genuine and impostor score histograms\n* pyeer_report.csv: \n    * Genuine scores mean\n    * Genuine sores standard deviation\n    * Impostor scores mean\n    * Impostor sores standard deviation\n    * Sensitivity index (d') (See [NICE:II](http://nice2.di.ubi.pt/) protocol evaluation)\n    * Equal Error Rate (EER) (EER values are reported as specified in [2]).\n    * Area under the ROC curve\n    * Youden's J statistic (Youden's Index)\n    * Youden's Index threshold\n    * Matthews Correlation Coefficient (MCC)\n    * Matthews Correlation Coefficient threshold\n    * Equal Error Rate (EER) (EER values are reported as specified in [2]).\n    * ZeroFMR, FMR1000, FMR100, FMR20, FMR10, and ZeroFNMR\n    * Equal Error Rate threshold\n    * ZeroFMR, FMR1000, FMR100, FMR20, FMR10, and ZeroFNMR thresholds\n* rates.csv:\n    * False Match Rates\n    * False Non-Match Rates\n\n#### To use from your own scripts\n\n    from pyeer.eer_info import get_eer_stats\n    from pyeer.report import generate_eer_report, export_error_rates\n    from pyeer.plot import plot_eer_stats\n\n    # Calculating stats for classifier A\n    stats_a = get_eer_stats(gscores_a, iscores_a)\n\n    # Calculating stats for classifier B\n    stats_b = get_eer_stats(gscores_b, iscores_b)\n\n    # Generating CSV report\n    generate_eer_report([stats_a, stats_b], ['A', 'B'], 'pyeer_report.csv')\n\n    # Generating HTML report\n    generate_eer_report([stats_a, stats_b], ['A', 'B'], 'pyeer_report.html')\n\n    # Generating Latex report\n    generate_eer_report([stats_a, stats_b], ['A', 'B'], 'pyeer_report.tex')\n\n    # Generating JSON report\n    generate_eer_report([stats_a, stats_b], ['A', 'B'], 'pyeer_report.json')\n\n    # Exporting error rates (Exporting FMR and FNMR to a CSV file)\n    # This is the DET curve, the ROC curve is a plot of FMR against 1 - FNMR\n    export_error_rates(stats_a.fmr, stats_a.fnmr, 'A_DET.csv')\n    export_error_rates(stats_b.fmr, stats_b.fnmr, 'B_DET.csv')\n\n    # Plotting\n    plot_eer_stats([stats_a, stats_b], ['A', 'B'])\n\n### getcmcinf\n\nIn identification experiments in closed sets sometimes only rank values are reported [1]. To obtain rank values and the Cumulative match curve (CMC) **getcmcinf** is provided. It receives the match scores and genuine correspondences. The input format will be described\nnext.\n\n#### Input files format\n\nInput files must have the following formats:\n\n* **Match scores:** Each line must have the following format: (query template score)\n\n* **Genuine query-template pairs:** Each line must have the following format: (query corresponding_template)\n\nFor more clarification, you should check the example files on [GitHub](https://github.com/manuelaguadomtz/pyeer/tree/master/pyeer/example_files).\n\n#### Usage examples\n\n##### To print the script help\n\n    getcmcinf -h\n\n##### One experiment\n\n    getcmcinf -p \"example_files/cmc/\" -ms \"exp1_scores.txt\" -t \"exp1_tp.txt\" -e \"Exp1\"\n\n##### More than one experiment\n\n    getcmcinf -p \"example_files/cmc/\" -ms \"exp1_scores.txt,exp2_scores.txt\" -t \"exp1_tp.txt,exp2_tp.txt\" -e \"Exp1,Exp2\"\n\n\n#### Output\n\nAll of the above examples will generate the following information:\n\n* Graphs: \n    * Cumulative match curve (CMC)\n    \n* pyeer_report.csv: \n    * Rank identification rates\n\n#### To use from your own scripts\n\n    from pyeer.cmc_stats import load_scores_from_file, get_cmc_curve, CMCstats\n    from pyeer.report import generate_cmc_report\n    from pyeer.plot import plot_cmc_stats\n\n    # CMC maximum rank\n    r = 20\n\n    # Loading scores\n    sfile = 'cmc/exp1_scores.txt'  # The scores file\n    tp_file = 'cmc/exp1_tp.txt'  # The genuine pairs relationship in \"sfile\"\n    scores = load_scores_from_file(sfile, tp_file)\n\n    # Calculating CMC curve\n    ranks = get_cmc_curve(scores, r)\n\n    # Creating stats\n    stats = [CMCstats(exp_id='A', ranks=ranks)]\n\n    # Generating CSV report\n    generate_cmc_report(stats, r, 'pyeer_report.csv')\n\n    # Generating HTML report\n    generate_cmc_report(stats, r, 'pyeer_report.html')\n\n    # Generating Latex report\n    generate_cmc_report(stats, r, 'pyeer_report.tex')\n\n    # Generating Latex report\n    generate_cmc_report(stats, r, 'pyeer_report.json')\n\n    # Plotting\n    plot_cmc_stats(stats, r)\n\n\n## Contributing\n\nDo you find **PyEER** useful? You can collaborate with us:\n\n[GitHub](https://github.com/manuelaguadomtz/pyeer\u003e)\n\nPlease follow our contributing guidelines.\n\n### Contributors\n\nWe would like to thanks to those who has collaborated with the project:\n\n* [ljsoler](https://github.com/ljsoler)\n    * Added support in CMC stats for multiple templates for a single query\n* [jpalancar](https://github.com/jpalancar)\n    * Testing and guidance\n\n\n## References\n\n[1] D. Maltoni et al., Handbook of Fingerprint Recognition, Springer-Verlag London Limited 2009\n\n[2] Maio D., Maltoni D., Cappelli R., Wayman J.L., and Jain A.K., \"FVC2000: Fingerprint verification\ncompetition,\"\" IEEE Transactions on Pattern Analysis Machine Intelligence, vol. 24, no. 3, pp. 402–412,\n2002\n\n[3] Hernandez-Palancar, J., Munoz-Briseno, A., \u0026 Gago-Alonso, A. (2014). Using a triangular matching\napproach for latent fingerprint and palmprint identification. International Journal of Pattern \nRecognition and Artificial Intelligence, 28, 1460004.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmanuelaguadomtz%2Fpyeer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmanuelaguadomtz%2Fpyeer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmanuelaguadomtz%2Fpyeer/lists"}