{"id":30844476,"url":"https://github.com/seblemaguer/replikant","last_synced_at":"2025-09-06T22:42:59.891Z","repository":{"id":312108491,"uuid":"1039522123","full_name":"seblemaguer/replikant","owner":"seblemaguer","description":"A flexible evaluation platform to enable researchers to conduct replicable subjective evaluation","archived":false,"fork":false,"pushed_at":"2025-08-28T14:56:58.000Z","size":79,"stargazers_count":6,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-08-28T21:55:17.165Z","etag":null,"topics":["evaluation","evaluation-framework","listening-test","replicability"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/seblemaguer.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-08-17T12:25:18.000Z","updated_at":"2025-08-28T17:35:21.000Z","dependencies_parsed_at":"2025-08-28T21:55:20.841Z","dependency_job_id":"3a0c276c-7b3e-4049-a4a3-35b9ae4c1a2c","html_url":"https://github.com/seblemaguer/replikant","commit_stats":null,"previous_names":["seblemaguer/replikant"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/seblemaguer/replikant","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/seblemaguer%2Freplikant","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/seblemaguer%2Freplikant/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/seblemaguer%2Freplikant/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/seblemaguer%2Freplikant/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/seblemaguer","download_url":"https://codeload.github.com/seblemaguer/replikant/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/seblemaguer%2Freplikant/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":273973998,"owners_count":25200579,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-06T02:00:13.247Z","response_time":2576,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["evaluation","evaluation-framework","listening-test","replicability"],"created_at":"2025-09-06T22:42:49.962Z","updated_at":"2025-09-06T22:42:59.878Z","avatar_url":"https://github.com/seblemaguer.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Replikant\n\n\u003cp align=\"center\"\u003e\n**The toolkit is useable, but still requires a bit of cleaning.** This is will be the priority after Interspeech, SSW \u0026 Blizzard.\n\u003c/p\u003e\n\nReplikant is a python toolkit which aims to enable researcher to develop their own subjective evaluation recipes.\nIt relies on Flask, Jinja2 and SQLite, and is a refactoring of FlexEval.\n\nI am currently doing some final important cleaning part which I thought would be done by now, but requires a bit more time.\n\n\n## How to install\n\nReplikant has been tested on python 3.11.\n\nTo install simply run the following command:\n\n```sh\npip install git+https://github.com/seblemaguer/replikant.git\n```\n\nWhen the toolkit will be fully cleaned, it will be submitted to pypi.\n\n## Developping and running a recipe\n\n### Defining an recipe\n\nThe easiest is to start with one of the available recipes and adapt it to your needs.\n\nHere is a list of already available recipes of published studies:\n  - https://github.com/sigmedia/bc_2013_extension/tree/master/evaluation/2013-EH2_EXT (MOS - naturalness, similarity; Intelligibility)\n  - https://github.com/sigmedia/bc_2013_extension/tree/master/evaluation/mushra (MUSHRA test)\n  - https://github.com/sigmedia/bc_2013_extension/tree/master/evaluation/ranked-choice-voting (Ranked Choice Voting)\n\n\n\n### Launching a recipe\n\nSimply run :\n\n```sh\nreplikant \u003cpath_configuration_recipe.yaml\u003e\n```\n\nBy default, the evaluation will be available at: http://127.0.0.1:8080.\n\nThe overall behaviour of replikant can be controlled from the command call. Here are the options:\n```\nusage: replikant [-h] [-d] [-i IP] [-p PORT] [-P] [-t] [-u URL] [-l LOG_FILE] [-v] RECIPE_CONFIGURATION\n\nReplikant\n\npositional arguments:\n  RECIPE_CONFIGURATION  Recipe's configuration file\n\noptions:\n  -h, --help            show this help message and exit\n  -d, --debug           Start the server in debugging mode\n  -i IP, --ip IP        IP's server\n  -p PORT, --port PORT  port\n  -P, --production      Start the server in production mode\n  -t, --threaded        Enable threads.\n  -u URL, --url URL     URL of the server (needed for flask redirections!) if different from http://\u003cip\u003e:\u003cport\u003e/\n  -l LOG_FILE, --log_file LOG_FILE\n                        Logger file\n  -v, --verbosity       increase output verbosity\n```\n\n## Contributing\n\n\nIf you want to participate to the development, you can install the necessary packages using:\n\n```sh\npip install -e .[dev]\n```\n\nYou then need to activate the pre-commit git hooks:\n\n```sh\npre-commit install\n```\n\n## Citing\n\n```bibtex\n@inproceedings{lemaguer25_interspeech,\n  title     = {Enabling the replicability of speech synthesis perceptual evaluations},\n  author    = {Sébastien {Le Maguer} and Gwénolé Lecorvé and Damien Lolive and Naomi Harte and Juraj Šimko},\n  year      = {2025},\n  booktitle = {Interspeech 2025},\n  pages     = {2545--2549},\n  doi       = {10.21437/Interspeech.2025-401},\n  issn      = {2958-1796},\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fseblemaguer%2Freplikant","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fseblemaguer%2Freplikant","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fseblemaguer%2Freplikant/lists"}