{"id":20780740,"url":"https://github.com/farrajota/human-pose-benchmark","last_synced_at":"2026-03-05T23:43:29.089Z","repository":{"id":202488771,"uuid":"91471861","full_name":"farrajota/human-pose-benchmark","owner":"farrajota","description":"Evaluation tool for benchmarking human pose prediction algorithms on the FLIC and LSP datasets.","archived":false,"fork":false,"pushed_at":"2017-05-16T15:15:34.000Z","size":9168,"stargazers_count":11,"open_issues_count":0,"forks_count":2,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-30T20:33:51.580Z","etag":null,"topics":["benchmark","flic","lsp","matlab","pose-estimation"],"latest_commit_sha":null,"homepage":null,"language":"Matlab","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/farrajota.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2017-05-16T15:05:23.000Z","updated_at":"2022-09-19T08:54:33.000Z","dependencies_parsed_at":null,"dependency_job_id":"7c9f8c63-0a60-4eeb-972e-49f4918dab22","html_url":"https://github.com/farrajota/human-pose-benchmark","commit_stats":null,"previous_names":["farrajota/human-pose-benchmark"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/farrajota%2Fhuman-pose-benchmark","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/farrajota%2Fhuman-pose-benchmark/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/farrajota%2Fhuman-pose-benchmark/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/farrajota%2Fhuman-pose-benchmark/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/farrajota","download_url":"https://codeload.github.com/farrajota/human-pose-benchmark/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251777792,"owners_count":21642226,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["benchmark","flic","lsp","matlab","pose-estimation"],"created_at":"2024-11-17T13:39:01.532Z","updated_at":"2026-03-05T23:43:24.064Z","avatar_url":"https://github.com/farrajota.png","language":"Matlab","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Articulated Human pose evaluation/benchmark\n\nBenchmark your method against several other methods on the popular [FLIC](http://bensapp.github.io/flic-dataset.html) and [LSP](http://www.comp.leeds.ac.uk/mat4saj/lspet.html) datasets.\n\n\n# How to use\n\nTo benchmark the algorithms simply run the scripts files `benchmark_flic.m` and `benchmark_lsp.m` to evaluate the algorithms on the FLIC and LSP datasets, respectively.\n\n\n## Adding a new algorithm to the list\n\nAdding predictions of a new algorithm is fairly simple:\n\n1. Create a folder with the name of the algorithm in `algorithms/`.\n2. Create a file called `algorithm.txt` and assign a label/alias name for the algorithm to be used to identify the algorithms name in the plot's legend.\n3. Add the predictions files with the keypoints/sticks coordinates with the names `pred_keypoints_lsp_oc.mat`, `pred_keypoints_lsp_pc.mat`, `pred_sticks_lsp_oc.mat` and `pred_sticks_lsp_pc.mat`.\n\n\u003e Note: The scripts will skip the missing files when benchmarking a method.\n\n\n## Options\n\nSeveral options are available for configuration. These, however, require the user to change the file manually.\n\nPlot options:\n\n- `list`: specify which algorithms to plot. If empty, plots all algorithms.\n- `bSave`: save plot images to `plots/` folder (if set to `true`).\n- `printLegend`: prints a legend in every plot (if set to `true`).\n- `pcp_threshold`: PCP evaluation threshold.\n- `pck_threshold`: PCK evaluation threshold.\n\n\n## Available datasets\n\nFor now, the available datasets for PCK and PCP evaluation are the [FLIC](http://bensapp.github.io/flic-dataset.html) and [LSP](http://www.comp.leeds.ac.uk/mat4saj/lspet.html). Other datasets may be introduced if it is justifiable for inclusion.\n\n```\n@inproceedings{modec13,\n    title={MODEC: Multimodal Decomposable Models for Human Pose Estimation},\n    author={Sapp, Benjamin and Taskar, Ben},\n    booktitle={In Proc. CVPR},\n    year={2013},\n}\n```\n\n```\n@inproceedings{Johnson11,\n\ttitle = {Learning Effective Human Pose Estimation from Inaccurate Annotation},\n\tauthor = {Johnson, Sam and Everingham, Mark},\n\tyear = {2011},\n\tbooktitle = {IEEE Proc. CVPR}\n}\n```\n\n\n## Benchmark results\n\nAll available methods for the LSP benchmark were downloaded from [MPII's website](http://human-pose.mpi-inf.mpg.de/#related_benchmarks). \n\nFLIC methods were gather from some authors's predictions available online.\n\nResults of the algorithms are shown bellow.\n\n\n### Frames Labeled In Cinema (FLIC)\n\n#### PCK(0.2) - Observer Centric\n\n| Method | Elbow | Wrist |\n| --- | --- | --- | \n| Sapp et al., CVPR'13  | 72.5 | 54.5 |\n| Yang et al., CVPR'16  | 91.6 | 88.8 |\n| Chen et al., NIPS'14  | 89.8 | 86.8 |\n| Wei et al., CVPR'16  | 92.5 | 90.0 |\n| Newell et al., arXiv'16  | 98.0 | 95.5 |\n\n| | | |\n| --- | --- | --- | \n| ![legends](plots/FLIC-pck-total-legend-OC.png \"Legends\") | ![hip](plots/FLIC-pck-hip-OC.png \"Hip\") | |\n| ![wrist](plots/FLIC-pck-wrist-OC.png \"Wrist\") | ![elbow](plots/FLIC-pck-elbow-OC.png \"Elbow\")  | ![shoulder](plots/FLIC-pck-shoulder-OC.png \"Shoulder\") |\n\n\n### Leeds Sport Pose (LSP)\n\n#### PCP(0.5) - Person Centric\n\n| Method | Torso | Upper leg | Lower leg | Upper arm | Forearm | Head | PCP |\n| --- | --- | --- | --- | --- | --- | --- | --- |\n| Wang et al., CVPR'13  | 87.5 | 56.0 | 55.8 | 43.1 | 32.1 | 79.1 | 54.1 |\n| Pishchulin et al., ICCV' 13  | 88.7 | 63.6 | 58.4 | 46.0 | 35.2 | 85.1 | 58.0 |\n| Tompson et al., NIPS'14  | 90.3 | 70.4 | 61.1 | 63.0 | 51.2 | 83.7 | 66.6 |\n| Fan et al., CVPR'15  | 95.4 | 77.7 | 69.8 | 62.8 | 49.1 | 86.6 | 70.1 |\n| Chen et al., NIPS'14  | 96.0 | 77.2 | 72.2 | 69.7 | 58.1 | 85.6 | 73.6 |\n| Yang et al., CVPR'16  | 95.6 | 78.5 | 71.8 | 72.2 | 61.8 | 83.9 | 74.8 |\n| Rafi et al., BMVC'16  | 97.6 | 87.3 | 80.2 | 76.8 | 66.2 | 93.3 | 81.2 |\n| Belagiannis et al., arXiv'16  | 96.0 | 86.7 | 82.2 | 79.4 | 69.4 | 89.4 | 82.1 |\n| Lifshitz et al., ECCV'16  | 97.3 | 88.8 | 84.4 | 80.6 | 71.4 | 94.8 | 84.3 |\n| Pishchulin et al., CVPR'16  | 97.0 | 88.8 | 82.0 | 82.4 | 71.8 | 95.8 | 84.3 |\n| Yu et al., ECCV'16  | 98.0 | 93.1 | 88.1 | 82.9 | 72.6 | 83.0 | 85.4 |\n| Insafutdinov et al., ECCV'16  | 97.0 | 90.6 | 86.9 | 86.1 | 79.5 | 95.4 | 87.8 |\n| Wei et al., CVPR'16  | 98.0 | 92.2 | 89.1 | 85.8 | 77.9 | 95.0 | 88.3 |\n| Bulat et al., ECCV'16  | 97.7 | 92.4 | 89.3 | 86.7 | 79.7 | 95.2 | 88.9 |\n\n\n| | | |\n| --- | --- | --- |\n| ![total](plots/LSP-pcp-total-legend-PC.png \"Legends\") | ![torso](plots/LSP-pcp-torso-PC.png \"Torso\") | ![head](plots/LSP-pcp-head-PC.png \"Head\")\n| ![upper leg](plots/LSP-pcp-upper_leg-PC.png \"Upper leg\") | ![lower leg](plots/LSP-pcp-lower_leg-PC.png \"Lower leg\") | |\n| ![upper arm](plots/LSP-pcp-upper_arm-PC.png \"Upper arm\") | ![forearm](plots/LSP-pcp-forearm-PC.png \"Forearm\") | |\n\n\n#### PCK(0.2) - Person Centric\n\n| Method | Head | Shoulder | Elbow | Wrist | Hip | Knee  | Ankle | Total |\n| --- | --- | --- | --- | --- | --- | ---  | --- | --- |\n| Wang et al., CVPR'13  | 84.7 | 57.1  | 43.7  | 36.7  | 56.7  | 52.4 | 50.8 | 54.6 |\n| Pishchulin et al., ICCV' 13  | 87.2 | 56.7  | 46.7  | 38.0  | 61.0  | 57.5 | 52.7 | 57.1 |\n| Tompson et al., NIPS'14  | 90.6 | 79.2  | 67.9  | 63.4  | 69.5  | 71.0 | 64.2 | 72.3 |\n| Fan et al., CVPR'15  | 92.4 | 75.2  | 65.3  | 64.0  | 75.7  | 68.3 | 70.4 | 73.0 |\n| Chen et al., NIPS'14  | 91.8 | 78.2  | 71.8  | 65.5  | 73.3  | 70.2 | 63.4 | 73.4 |\n| Yang et al., CVPR'16  | 90.6 | 78.1  | 73.8  | 68.8  | 74.8  | 69.9 | 58.9 | 73.6 |\n| Rafi et al., BMVC'16  | 95.8 | 86.2  | 79.3  | 75.0  | 86.6  | 83.8 | 79.8 | 83.8 |\n| Yu et al., ECCV'16  | 87.2 | 88.2  | 82.4  | 76.3  | 91.4  | 85.8 | 78.7 | 84.3 |\n| Belagiannis et al., arXiv'16  | 95.2 | 89.0  | 81.5  | 77.0  | 83.7  | 87.0 | 82.8 | 85.2 |\n| Lifshitz et al., ECCV'16  | 96.8 | 89.0  | 82.7  | 79.1  | 90.9  | 86.0 | 82.5 | 86.7 |\n| Pishchulin et al., CVPR'16  | 97.0 | 91.0  | 83.8  | 78.1  | 91.0  | 86.7 | 82.0 | 87.1 |\n| Insafutdinov et al., ECCV'16  | 97.4 | 92.7  | 87.5  | 84.4  | 91.5  | 89.9 | 87.2 | 90.1 |\n| Wei et al., CVPR'16  | 97.8 | 92.5  | 87.0  | 83.9  | 91.5  | 90.8 | 89.9 | 90.5 |\n| Bulat et al., ECCV'16  | 97.2 | 92.1  | 88.1  | 85.2  | 92.2  | 91.4 | 88.7 | 90.7 |\n\n| | | |\n| --- | --- | --- |\n| ![legends](plots/LSP-pck-total-legend-PC.png \"Legends\") | ![head](plots/LSP-pck-head-PC.png \"Head\") | |\n| ![ankle](plots/LSP-pck-ankle-PC.png \"Ankle\") | ![knee](plots/LSP-pck-knee-PC.png \"Knee\") | ![hip](plots/LSP-pck-hip-PC.png \"Hip\") |\n| ![wrist](plots/LSP-pck-wrist-PC.png \"Wrist\") | ![elbow](plots/LSP-pck-elbow-PC.png \"Elbow\") | ![shoulder](plots/LSP-pck-shoulder-PC.png \"Shoulder\") | \n\n\n# Acknowledgements\n\nThis code is a modified version of the original code made available by [MPII](http://human-pose.mpi-inf.mpg.de/results/lsp/evalLSP.zip).\n\n\n# License\n\nThe available code is released under the MIT license.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffarrajota%2Fhuman-pose-benchmark","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffarrajota%2Fhuman-pose-benchmark","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffarrajota%2Fhuman-pose-benchmark/lists"}