{"id":16275352,"url":"https://github.com/ja-thomas/pbmohpo","last_synced_at":"2025-03-20T01:31:00.481Z","repository":{"id":65940713,"uuid":"575860583","full_name":"ja-thomas/pbmohpo","owner":"ja-thomas","description":"Preferential Bayesian Multi-Objective Hyperparameter Optimization","archived":false,"fork":false,"pushed_at":"2023-10-04T17:13:59.000Z","size":3174,"stargazers_count":5,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2023-10-05T03:40:28.683Z","etag":null,"topics":["bayesian-optimization","machine-learning","multi-objective-optimization","python"],"latest_commit_sha":null,"homepage":"https://ja-thomas.github.io/pbmohpo/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"lgpl-2.1","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ja-thomas.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2022-12-08T13:16:06.000Z","updated_at":"2023-10-04T17:10:16.000Z","dependencies_parsed_at":null,"dependency_job_id":"fc50e12a-99e6-453c-8efa-d7ad82e80bc5","html_url":"https://github.com/ja-thomas/pbmohpo","commit_stats":null,"previous_names":[],"tags_count":0,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ja-thomas%2Fpbmohpo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ja-thomas%2Fpbmohpo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ja-thomas%2Fpbmohpo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ja-thomas%2Fpbmohpo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ja-thomas","download_url":"https://codeload.github.com/ja-thomas/pbmohpo/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":219865974,"owners_count":16555917,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bayesian-optimization","machine-learning","multi-objective-optimization","python"],"created_at":"2024-10-10T18:33:10.289Z","updated_at":"2024-10-10T18:33:10.347Z","avatar_url":"https://github.com/ja-thomas.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Preferential Bayesian Multi-Objective Hyperparameter Optimization\n[![Unittests](https://github.com/ja-thomas/pbmohpo/actions/workflows/unittests.yml/badge.svg?branch=main)](https://github.com/ja-thomas/pbmohpo/actions/workflows/unittests.yml)\n[![Linting](https://github.com/ja-thomas/pbmohpo/actions/workflows/black.yml/badge.svg?branch=main)](https://github.com/ja-thomas/pbmohpo/actions/workflows/black.yml)\n[![Docs](https://github.com/ja-thomas/pbmohpo/actions/workflows/docs.yml/badge.svg?branch=main)](https://github.com/ja-thomas/pbmohpo/actions/workflows/docs.yml)\n[![Module Handbook](https://img.shields.io/badge/Website-Documentation-blue)](https://ja-thomas.github.io/pbmohpo/)\n[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)\n[![License: LGPL v2](https://img.shields.io/badge/License-LGPL_v2-blue.svg)](https://github.com/ja-thomas/pbmohpo/blob/main/LICENSE)\n\n## Documentation\n\nhttps://ja-thomas.github.io/pbmohpo/\n\n## Summary\n\nWhile hyperparameter optimization has been accepted as an important component of a machine learning task, it is often conducted in an unrealistic setting.\n\nWhile research often presents Machine Learning as a one dimensional problem with a single evaluation criterion like Accuracy, real-world applications seldom present in that way:\nMultiple - often conflicting - performance metrics are of interest to a decision maker (DM), thus making the decision for a fully configured model often more challenging as a suitable trade-off needs to be identified.\n\nWhile this problem can be solved via expensive multi-objective black-box optimization, a DM might in reality not be able to specify all of their evaluation criteria, but simply state their preference of one model over another or produce a ranking of models given a list.\n\nOther applications like A/B testing or recommender systems similarly only provide feedback this way.\n\nThis scenario of optimizing only through pairwise preferences has been explored in K-armed duelling bandit problems and Preferential Bayesian Optimization (PBO).\nWhile PBO gives a direct answer to expensive black-box optimization based on pairwise preferences, the methods introduce a second bottleneck in the iterative process of Bayesian Optimization (BO).\n\nIn addition to the expensive evaluation(s) of selected models, the optimization now needs to wait for a preference expression from the DM before selecting new models to evaluate.\nTo avoid unnecessary idle time and create a drawn-out optimization process, mechanisms need to be developed in order to use computation resources when available as well as use DM time to rank models when the DM is available.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fja-thomas%2Fpbmohpo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fja-thomas%2Fpbmohpo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fja-thomas%2Fpbmohpo/lists"}