{"id":18777190,"url":"https://github.com/scikit-activeml/annotlib","last_synced_at":"2025-07-10T16:12:22.985Z","repository":{"id":74699540,"uuid":"227332499","full_name":"scikit-activeml/annotlib","owner":"scikit-activeml","description":"Documentation:","archived":false,"fork":false,"pushed_at":"2020-03-09T17:12:49.000Z","size":6025,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-06-24T01:16:48.127Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://annotlib.readthedocs.io/en/latest/index.html","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/scikit-activeml.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2019-12-11T09:57:43.000Z","updated_at":"2022-01-24T12:41:34.000Z","dependencies_parsed_at":"2024-01-09T04:12:31.941Z","dependency_job_id":null,"html_url":"https://github.com/scikit-activeml/annotlib","commit_stats":null,"previous_names":["acalma/annotlib"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/scikit-activeml/annotlib","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scikit-activeml%2Fannotlib","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scikit-activeml%2Fannotlib/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scikit-activeml%2Fannotlib/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scikit-activeml%2Fannotlib/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/scikit-activeml","download_url":"https://codeload.github.com/scikit-activeml/annotlib/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scikit-activeml%2Fannotlib/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":264605281,"owners_count":23636050,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-07T20:09:04.026Z","updated_at":"2025-07-10T16:12:22.963Z","avatar_url":"https://github.com/scikit-activeml.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# annotlib: Simulation of Annotators\n\nauthors: Marek Herde and Adrian Calma\n\n![](https://github.com/acalma/annotlib/workflows/annotlib/badge.svg)\n\n# Introduction\n\n*annotlib* is a Python package for simulating annotators in an active learning setting.\nSolving classification problems by using supervised machine learning models requires samples assigned to class labels.\nHowever, labeling these samples causes costs (e.g. workload, time, etc.), so that active learning strategies aim at reducing these costs by selecting samples, which are the most useful for training a classifier.\n\nIn real-world scenarios, human annotators are often responsible for providing the class labels.\nUnfortunately, there is no guaranty for the omniscience of such annotators.\nHence, annotators are prone to error, respectively uncertain, so that the class labels assigned to samples may be false.\nThe labeling performance of an annotator is affected by many factors, e.g. expertise, experience, concentration, level of fatigue and so on.\nMoreover, the difficulty of a sample influences the outcome of a labeling process.\n\nTo evaluate an active learning strategy in the setting of uncertain annotators, class labels of these uncertain\nannotators are required to be available, but there is a lack of publicly accessible real-world data sets labeled by error-prone annotators.\nAs a result, recently published active learning strategies are evaluated on simulated annotators where the used simulation techniques are diverse.\nOur developed *annotlib* Python package represents a of these techniques and implements additional methods, which simulate realistic characteristics of uncertain annotators.\nThis way, we establish a library simplifying and standardising the evaluation of active learning strategies coping with uncertain annotators.\n\nFor more information go to the [documentation](https://annotlib.readthedocs.io).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscikit-activeml%2Fannotlib","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fscikit-activeml%2Fannotlib","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscikit-activeml%2Fannotlib/lists"}