{"id":19287812,"url":"https://github.com/deeppavlov/convai-testing-system","last_synced_at":"2026-07-02T09:30:15.917Z","repository":{"id":83450907,"uuid":"96209330","full_name":"deeppavlov/convai-testing-system","owner":"deeppavlov","description":null,"archived":false,"fork":false,"pushed_at":"2017-11-29T08:22:32.000Z","size":35265,"stargazers_count":2,"open_issues_count":0,"forks_count":2,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-01-05T19:22:10.761Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/deeppavlov.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-07-04T11:08:33.000Z","updated_at":"2022-09-13T18:28:40.000Z","dependencies_parsed_at":"2023-07-02T04:46:02.674Z","dependency_job_id":null,"html_url":"https://github.com/deeppavlov/convai-testing-system","commit_stats":null,"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deeppavlov%2Fconvai-testing-system","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deeppavlov%2Fconvai-testing-system/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deeppavlov%2Fconvai-testing-system/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deeppavlov%2Fconvai-testing-system/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/deeppavlov","download_url":"https://codeload.github.com/deeppavlov/convai-testing-system/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240392596,"owners_count":19794123,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-09T22:07:18.132Z","updated_at":"2026-07-02T09:30:15.861Z","avatar_url":"https://github.com/deeppavlov.png","language":"Scala","funding_links":[],"categories":[],"sub_categories":[],"readme":"convai-testing-system\n=========================\n\nThis software is proposed to serve as host for some specific kind of Turing test.\n\nPeople are connected to other people or bots randomly through this system. While having a conversation a person is supposed to evaluate utterances of his/her interlocutor. After finishing the conversation the person is need to evaluate overall dialog quality.\n\nThe system has following components:\n* Telegram bot API connector - for people to connect to it through Telegram IM system (to use this feature you need [Telegram bot](https://core.telegram.org/bots) token);\n* server-side API - for bots to connect to system;\n* Facebook Messenger API connector (to use this feature you need to register your [Facebook bot] (https://developers.facebook.com/docs/messenger-platform)).\n\n## Installation\n\n### From Ubuntu package\nYou can download latest Ubuntu package [here](https://github.com/deepmipt/convai-testing-system/releases). \nTo install this package execute: ```dpkg -i convai-testing-system.deb```.\n\n### From source\nTo install from source please checkout this repository and run ```sbt run``` command inside project root folder.\n\n## Configuration\n\nThe default config is placed at `/etc/convai-testing-system/reference.conf`. You could see the sample config [here](./src/main/resources/reference.conf).\n\nThe variables represented in such way `${?VARIABLE}` could be set as environment variable, e.g. by `export $VARIABLE` command in command line.\n\n### Config blocks\n\n#### Telegram\n\n```\ntelegram {\n  token = ${?TOKEN}\n  webhook = ${?WEBHOOK}\n}\n```\n\nHere you should place a token and a URL for webhook to connect to Telegram server. These values could be obtained [here](https://core.telegram.org/bots).\n\n#### Facebook\n\n```\nfbmessenger {\n  secret = ${?FB_SECRET}\n  token = ${?FB_TOKEN}\n  pageAccessToken = ${?FB_PAGE_ACCESS_TOKEN}\n}\n```\n\nHere you should place a token, a secret and page access token for your [Facebook bot](https://developers.facebook.com/docs/messenger-platform).\n\n#### Bot UUIDs\n\n```\nbot {\n  registered = [\n    { token: \"5319E57A-F165-4BEC-94E6-413C38B4ACF9\", max_connections: 1000, delayOn: true },\n    { token: \"0A36119D-E6C0-4022-962F-5B5BDF21FD97\", max_connections: 1000, delayOn: true }\n  ]\n}\n```\n\n`token` here is UUID identifier for a bot. \n\n`max_connections` - max number of simultaneous connections. \n\n`delayOn` - is system need to add random delay for a bot responce. In our experiments some bots need this and some don't. So try for yoursef.\n\n#### Conversation preperties\n\n```\ntalk {\n  talk_timeout = 10 minutes\n  talk_length_max = 1000\n  bot {\n    human_bot_coefficient = 0.2\n    delay {\n      mean_k = 0.5\n      variance = 5\n    }\n  }\n ```\n  \n`talk_timeout` - max duration of a conversation.\n\n`talk_length_max` - max number of references.\n\n`human_bot_coefficient` - percentage of humans which a picked for conversations. Example: `human_bot_coefficient = 0.2` like in the sample config, then for 10 people starting dialog 2 of them will have conversation with human, while others will be connected to bots.\n\n`mean_k` \u0026 `variance` - parameters of normal distribution which is used to generate utterance delay. Used only if `delayOn` is set `true` for the bot.\n\n#### Logging \n\n```\n  logger {\n    connection_string = ${?MONGODB_URI}\n  }\n```\n  \n`MONGODB_URI` - mongo DB URI used to store logs. The dialogs should be in collection `dialogs` inside MongoDB, the assessments will be inside `assessments`.\n\n#### Context (or seed text)\n\n```\n  context {\n    type = \"wikinews\"\n  }\n```\n\nFor now we have only one option - parsed SQuAD dataset, which is included in the bundle.\n\n\n\n## Creating a package of your own\n\nYou could create a package by yourself if you want. You will need [sbt](http://www.scala-sbt.org/) first. After that, run in root folder of the project:\n `sbt debian:packageBin`\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeeppavlov%2Fconvai-testing-system","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdeeppavlov%2Fconvai-testing-system","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeeppavlov%2Fconvai-testing-system/lists"}