{"id":34039795,"url":"https://github.com/nmalkin/kupper_hafner","last_synced_at":"2026-04-02T01:01:46.054Z","repository":{"id":62574818,"uuid":"115854700","full_name":"nmalkin/kupper_hafner","owner":"nmalkin","description":"Kupper-Hafner inter-rater agreement calculation library","archived":false,"fork":false,"pushed_at":"2022-04-03T05:30:57.000Z","size":7,"stargazers_count":4,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-12-15T14:18:48.910Z","etag":null,"topics":["agreement","inter-rater-agreement","interrater-reliability","irr","kappa","python","reliability","statistics"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nmalkin.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-12-31T09:36:29.000Z","updated_at":"2025-04-03T18:09:47.000Z","dependencies_parsed_at":"2022-11-03T18:49:19.996Z","dependency_job_id":null,"html_url":"https://github.com/nmalkin/kupper_hafner","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/nmalkin/kupper_hafner","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nmalkin%2Fkupper_hafner","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nmalkin%2Fkupper_hafner/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nmalkin%2Fkupper_hafner/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nmalkin%2Fkupper_hafner/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nmalkin","download_url":"https://codeload.github.com/nmalkin/kupper_hafner/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nmalkin%2Fkupper_hafner/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31293631,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-01T21:15:39.731Z","status":"ssl_error","status_checked_at":"2026-04-01T21:15:34.046Z","response_time":53,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agreement","inter-rater-agreement","interrater-reliability","irr","kappa","python","reliability","statistics"],"created_at":"2025-12-13T21:50:04.306Z","updated_at":"2026-04-02T01:01:46.035Z","avatar_url":"https://github.com/nmalkin.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"Kupper-Hafner inter-rater agreement calculation\n================================================\n\nThis repository contains code to calculate inter-rater agreement for multiple attribute responses, as described by Lawrence L. Kupper and Kerry B. Hafner in [their article](https://dx.doi.org/10.2307/2531695):\n\n    On Assessing Interrater Agreement for Multiple Attribute Responses \n    ------------------------------------------------------------------\n    Authors: Lawrence L. Kupper and Kerry B. Hafner\n    Source: Biometrics, Vol. 45, No. 3 (Sep., 1989), pp. 957-967\n    Published by: International Biometric Society\n    Stable URL: https://www.jstor.org/stable/2531695\n\n\nWhy Kupper-Hafner agreement?\n----------------------------\nA common technique in scientific studies is for two or more raters to independently assign attributes to each item in a dataset (such as a response or specimen).\nInter-rater agreement, or [inter-rater reliability](https://en.wikipedia.org/wiki/Inter-rater_reliability), is used to measure the degree to which these raters end up agreeing with each other. Commonly used statistics for this include [Cohen's kappa](https://en.wikipedia.org/wiki/Cohen%27s_kappa) and [Fleiss' kappa](https://en.wikipedia.org/wiki/Fleiss'_kappa).\n\nOne assumption of these statistics, however, is that items are assigned to only one of several mutually exclusive categories. But, in certain coding situations, we want to be able to assign multiple codes to the same item. If we do that, the assumptions in the kappa statistics no longer hold.\n\nThe paper by Kupper and Hafner provides a statistic that allows the calculation of inter-rater agreement in these situations as well.\n\nInstallation\n------------\nThis code implements Kupper-Hafner agreement in Python. It should support both Python 2 and 3.\nYou can install it using pip:\n\n    pip install kupper-hafner\n\nUsage\n-----\nLet's say we have three items, and two raters (A and B) assigned them the following categories:\n\n- Item 0: A rated it 10; B rated it 10, 20\n- Item 1: A rated it 20, 30; B rated it 30\n- Item 2: A rated it 40; B rated it 40\n\nWe would calculate Kupper-Hafner agreement as follows:\n```python\nfrom kupper_hafner import kupper_hafner\n\nA_ratings = [[10], [20, 30], [40]]\nB_ratings = [[10, 20], [30], [40]]\nkupper_hafner(A_ratings, B_ratings)\n```\n\nIf the codebook included categories that were not used by either rater, it can be provided as follows:\n\n```python\ncodebook = [10, 20, 30, 40, 50]\nkupper_hafner(A_ratings, B_ratings, codebook)\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnmalkin%2Fkupper_hafner","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnmalkin%2Fkupper_hafner","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnmalkin%2Fkupper_hafner/lists"}