{"id":15360527,"url":"https://github.com/devmotion/calibrationpaper","last_synced_at":"2025-04-15T08:37:07.316Z","repository":{"id":114023656,"uuid":"217291458","full_name":"devmotion/CalibrationPaper","owner":"devmotion","description":"Repository of NeurIPS 2019 paper \"Calibration tests in multi-class classification: A unifying framework\"","archived":false,"fork":false,"pushed_at":"2021-05-06T10:13:39.000Z","size":160988,"stargazers_count":17,"open_issues_count":1,"forks_count":5,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-28T18:11:09.350Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/devmotion.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-10-24T12:16:38.000Z","updated_at":"2024-12-12T23:08:43.000Z","dependencies_parsed_at":"2023-05-22T04:45:18.141Z","dependency_job_id":null,"html_url":"https://github.com/devmotion/CalibrationPaper","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devmotion%2FCalibrationPaper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devmotion%2FCalibrationPaper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devmotion%2FCalibrationPaper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devmotion%2FCalibrationPaper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/devmotion","download_url":"https://codeload.github.com/devmotion/CalibrationPaper/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249038419,"owners_count":21202696,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-01T12:50:29.354Z","updated_at":"2025-04-15T08:37:07.298Z","avatar_url":"https://github.com/devmotion.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# CalibrationPaper\n\nThis repository accompanies the paper [\"Calibration tests in multi-class\nclassification: A unifying framework\" by Widmann, Lindsten, and Zachariah](http://papers.nips.cc/paper/9392-calibration-tests-in-multi-class-classification-a-unifying-framework),\nwhich was presented at NeurIPS 2019.\n\n2021-05-04: [**We extended the calibration errors and tests to general probabilistic predictive models in our paper \"Calibration tests beyond classification\" presented at ICLR 2021**](https://openreview.net/forum?id=-bxf89v3Nx)\n\n## Structure\n\nThe folder `paper` contains the LaTeX source code of the paper.\n\nThe folder `experiments` contains the source code and the results of our\nexperiments.\n\nThe folder `src` contains common implementations such as the definition of the\ngenerative models, which are used for generating the figures in our paper and\nfor some experiments.\n\n## Reproducibility\n\nYou can rerun our experiments and recompile our paper. Every folder contains\ninstructions for how to build and run the files therein.\n\n## Software\n\nWe published software packages for the proposed calibration errors and calibration tests.\n\n### Julia packages\n\n- [CalibrationErrors.jl](https://github.com/devmotion/CalibrationErrors.jl) and [CalibrationErrorsDistributions.jl](https://github.com/devmotion/CalibrationErrorsDistributions.jl) for estimating calibration errors from data sets of predictions and targets, including general probabilistic predictive models.\n- [CalibrationTests.jl](https://github.com/devmotion/CalibrationTests.jl) for statistical hypothesis tests of calibration.\n\n### Python and R interface\n\n- [pycalibration](https://github.com/devmotion/pycalibration) is a Python interface for CalibrationErrors.jl, CalibrationErrorsDistributions.jl, and CalibrationTests.jl.\n- [rcalibration](https://github.com/devmotion/rcalibration) is an R interface for CalibrationErrors.jl, CalibrationErrorsDistributions.jl, and CalibrationTests.jl.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdevmotion%2Fcalibrationpaper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdevmotion%2Fcalibrationpaper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdevmotion%2Fcalibrationpaper/lists"}