{"id":13486316,"url":"https://github.com/carperai/cheese","last_synced_at":"2025-04-07T05:15:46.428Z","repository":{"id":38390329,"uuid":"453764663","full_name":"CarperAI/cheese","owner":"CarperAI","description":"Used for adaptive human in the loop evaluation of language and embedding models.","archived":false,"fork":false,"pushed_at":"2023-03-01T13:22:37.000Z","size":512,"stargazers_count":307,"open_issues_count":3,"forks_count":25,"subscribers_count":11,"default_branch":"main","last_synced_at":"2025-03-31T04:06:25.830Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/CarperAI.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-01-30T18:24:51.000Z","updated_at":"2025-03-27T06:25:31.000Z","dependencies_parsed_at":"2024-10-30T21:31:35.109Z","dependency_job_id":null,"html_url":"https://github.com/CarperAI/cheese","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CarperAI%2Fcheese","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CarperAI%2Fcheese/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CarperAI%2Fcheese/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CarperAI%2Fcheese/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/CarperAI","download_url":"https://codeload.github.com/CarperAI/cheese/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247595335,"owners_count":20963943,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T18:00:43.956Z","updated_at":"2025-04-07T05:15:46.398Z","avatar_url":"https://github.com/CarperAI.png","language":"Python","funding_links":[],"categories":["Uncategorized"],"sub_categories":["Uncategorized"],"readme":"[![DOI](https://zenodo.org/badge/453764663.svg)](https://zenodo.org/badge/latestdoi/453764663)\n\n[docs-image]: https://readthedocs.org/projects/cheese1/badge/?version=latest\n[docs-url]: https://cheese1.readthedocs.io/en/latest/?badge=latest\n\n# Coadaptive Harness for Effective Evaluation, Steering, \u0026 Enhancement\nUsed for adaptive human in the loop evaluation of language and embedding models.\n\n---------------------------------------------------------------------------------------\n\n[![Docs Status][docs-image]][docs-url]\n\n**[Documentation](https://cheese1.readthedocs.io)**\n\n# Getting Started\n\nFirst install the required packages:\n```\ngit clone https://github.com/carperai/cheese\ncd cheese\npip install -r requirements.txt\n```\n\n\nYou need rabbitmq for the messaging system in the backend. It can be installed easily with conda as follows:\n```\nconda install -c conda-forge rabbitmq-server\n```\nThen start the server (should do this on a different screen/window)\n```\nrabbitmq-server\n```\nNote that because of how rabbitmq works, it is possible several items may remain in the queue after your application using CHEESE has terminated. If the server is not shut down and restarted, then subsequent uses of CHEESE may end up drawing from items that were placed in the queue in a previous run.  \n  \nAs an example, the following script runs an image selection task, where two images from LAION-art dataset are presented and the labeller is asked to choose one.\n```\npython -m examples.image_selection\n```\n\n# Custom tasks in Gradio\nRefer to examples/image_selection.py for a comprehensive example on how to create your own custom task with a Gradio UI.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcarperai%2Fcheese","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcarperai%2Fcheese","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcarperai%2Fcheese/lists"}