{"id":19287866,"url":"https://github.com/deeppavlov/explainable_qa","last_synced_at":"2025-04-22T04:32:38.593Z","repository":{"id":83450957,"uuid":"574849723","full_name":"deeppavlov/explainable_qa","owner":"deeppavlov","description":"Services for explainable QA","archived":false,"fork":false,"pushed_at":"2023-12-18T11:25:33.000Z","size":520,"stargazers_count":4,"open_issues_count":0,"forks_count":3,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-03-20T03:01:04.282Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/deeppavlov.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2022-12-06T07:57:13.000Z","updated_at":"2024-01-12T18:52:00.000Z","dependencies_parsed_at":"2023-12-18T02:26:45.732Z","dependency_job_id":"376d00e5-e9b9-4602-b044-9fcdd828ebbf","html_url":"https://github.com/deeppavlov/explainable_qa","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deeppavlov%2Fexplainable_qa","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deeppavlov%2Fexplainable_qa/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deeppavlov%2Fexplainable_qa/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deeppavlov%2Fexplainable_qa/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/deeppavlov","download_url":"https://codeload.github.com/deeppavlov/explainable_qa/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223888419,"owners_count":17220083,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-09T22:07:27.093Z","updated_at":"2024-11-09T22:07:27.602Z","avatar_url":"https://github.com/deeppavlov.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Services for question answering over knowledge graphs and text and answer explanation\n\nModels description\n------------------\n\nKBQA is the system for question answering over knowledge graphs.\n\n\u003cp align=\"center\"\u003e\n\u003cimg\n  src=\"img/kbqa.png\"\n  alt=\"Alt text\"\n  height=\"90%\"\n  width=\"90%\"\n  title=\"System for question answering over knowledge graphs\"\n  style=\"display: inline-block; margin: 0 auto; max-width: 200px\"\u003e\n\u003c/p\u003e\n\nThe system outputs the answer and answer explanation (triplets which connect the entities from the question and the answer entities).\n\nGenerative model based on JointGT approach generates answer explanation in the form of text from the triplets.\n\n\u003cp align=\"center\"\u003e\n\u003cimg\n  src=\"img/kbqa_plus_jointgt.png\"\n  alt=\"Alt text\"\n  height=\"50%\"\n  width=\"50%\"\n  title=\"System for generation of answer explanation from the triplets\"\n  style=\"display: inline-block; margin: 0 auto; max-width: 200px\"\u003e\n\u003c/p\u003e\n \nThe text-based generative model (T5) takes as input the question, the paragraph with the answer and generates detailed answer.\n\n\u003cp align=\"center\"\u003e\n\u003cimg\n  src=\"img/t5_generation.png\"\n  alt=\"Alt text\"\n  height=\"60%\"\n  width=\"60%\"\n  title=\"T5 for generation of detailed answer\"\n  style=\"display: inline-block; margin: 0 auto; max-width: 200px\"\u003e\n\u003c/p\u003e\n\nQuestion Answering system takes as input the question, finds top N relevant paragraphs, re-ranks them with BERT-based model and outputs the short answer and the detailed answer (explanation of the short answer).\n\n\u003cp align=\"center\"\u003e\n\u003cimg\n  src=\"img/text_qa_system.png\"\n  alt=\"Alt text\"\n  height=\"60%\"\n  width=\"60%\"\n  title=\"Question Answering system\"\n  style=\"display: inline-block; margin: 0 auto; max-width: 200px\"\u003e\n\u003c/p\u003e\n\nLaunching the services\n----------------------\n\nLauch the service for text-based question answering and answer explanation:\n\n```shell\ndocker-compose up --build retrieve text-based-generation\n```\n    \nLauch the service which performs question answering over knowledge graphs and outputs the answer explanation (triplets, containing the entities from the question and the answer entities):\n\n```shell\ndocker-compose up --build kbqa\n```\n\nLaunch the service which generates text explaining the answer from triplets:\n\n```shell\ndocker-compose up --build graph-based-generation\n```\n\nLaunch the services to get answer to the question, triplets, which explain the answer, and generated text with answer explanation:\n\n```shell\ndocker-compose up --build kbqa graph-based-generation\n```\n\nServices parameters\n-------------------\n\nTo choose the language of KBQA, you should change the line 58 in docker-compose.yml file: RU for Russian, EN for English.\n\nMapping of volumes\n------------------\n\nIn docker-compose.yml the default mapping of the volume with model checkpoints and databases in the following:\n\n ~/.deeppavlov:/root/.deeppavlov\n\nYou can change this mapping to your custom:\n\n \u003cyour_custom_local_directory\u003e:/root/.deeppavlov\n\nModels metrics\n--------------\n\nAccuracy of Russian version of KBQA on RuBQ2.0 dataset:\n\n**acc = 53.1.**\n\nAccuracy of finding explanations for answers:\n\n**acc = 29.3.**\n\nSacreBLEU of T5 model which generates answer explanation from question on DSberQuAD dataset:\n\n**SacreBLEU = 64.7.**\n\nBLEU scores of JointGT model which generates answer explanation from triplets on subset of LC-QUAD dataset:\n\nBLEU-1 | BLEU-2\n--- | ---\n66.5 | 52.4\n\nServices usage\n--------------\n\n### Generate detailed answer explanation from the question and paragraph:\n\n```python\nimport requests\n\nres = requests.post(\"http://0.0.0.0:8006/generate\",\n                    json={\"questions\": [\"Кто был первым человеком в космосе?\"],\n                          \"paragraphs\": [[\"12 апреля 1961 года Юрий Гагарин стал первым человеком в мировой истории, совершившим полёт в \"\n                                          \"космическое пространство.\"]]}).json()\nprint(res)\n\nres = requests.post(\"http://0.0.0.0:8006/generate\",\n                    json={\"questions\": [\"Где живут кенгуру?\"],\n                          \"paragraphs\": [[\"Водятся кенгуру в Австралии, в Тасмании, на Новой Гвинее и на архипелаге Бисмарка. \"\n                                          \"Завезены в Новую Зеландию. Большинство видов — наземные, обитают на равнинах, поросших густой \"\n                                          \"высокой травой и кустарником.\"]]}).json()\n\nres = requests.post(\"http://0.0.0.0:8006/generate\",\n                    json={\"questions\": [\"Какое самое глубокое озеро в мире?\"],\n                          \"paragraphs\": [[\"Байкал — озеро тектонического происхождения в южной части Восточной Сибири, самое глубокое \"\n                                          \"озеро на планете, крупнейший природный резервуар пресной воды и самое большое по площади \"\n                                          \"пресноводное озеро на континенте.\"]]}).json()\n```\n\n#### The script with these examples can be found in ./text_based_generation/examples.py\n\nExample of the output:\n```json\n    [{\"long_explanation\": \"Байкал - это самое глубокое озеро в мире.\"}]\n```\n\n### Find the answer, the answer paragraph, and generate answer explanation:\n\n```python\nimport requests\n\nres = requests.post(\n    \"http://0.0.0.0:8006/ans_expl\",\n    json={\"questions\": [\"Кто был первым человеком в космосе?\"]}\n).json()\n\nprint(res)\n\nres = requests.post(\n    \"http://0.0.0.0:8006/ans_expl\",\n    json={\"questions\": [\"Где живут кенгуру?\"]}\n).json()\n\nres = requests.post(\n    \"http://0.0.0.0:8006/ans_expl\",\n    json={\"questions\": [\"Какое самое глубокое озеро в мире?\"]}\n).json()\n```\n\n#### The script with these examples can be found in ./text_based_generation/examples.py\n\nExample of the output:\n\n```json\n    [{\n       \"answer\": \"Байка́л\",\n       \"answer_place\": 0,\n       \"answer_score\": 1.0,\n       \"long_explanation\": \"Байка́л - это самое глубокое озеро в мире.\"\n     }\n    ]\n```\n\n### Get metrics of text-based answer explanation model:\n\n```python\nimport requests\n\nres = requests.post(\"http://0.0.0.0:8006/get_metrics_expl\", json={\"num_samples\": 100}).json()\nprint(res)\n```\n\n### Get metrics of text-based question answering and subsequent answer explanation model:\n\n```python\nimport requests\n\nres = requests.post(\"http://0.0.0.0:8006/get_metrics_ans_expl\", json={\"num_samples\": 100}).json()\nprint(res)\n```\n\n### Answer the question over knowledge graph and get an explanation of the answer (triplets from the KG which connect the entities from the question and answer entities)\n\n#### Russian version of Knowledge Base Question Answering\n\n```python\nimport requests\n\nres = requests.post(\n    \"http://0.0.0.0:8008/respond\",\n    json={\"questions\": [\"Какой документ, подписанный 7 февраля 1992 года, положил начало Европейскому Союзу?\"]}\n).json()\n\nprint(res)\n\nres = requests.post(\n    \"http://0.0.0.0:8008/respond\",\n    json={\"questions\": [\"Какая столица России?\"]}\n).json()\n\nres = requests.post(\n    \"http://0.0.0.0:8008/respond\",\n    json={\"questions\": ['Кто сыграл Бормана в народном телесериале \"Семнадцать мгновений весны\"?']}\n).json()\n\nres = requests.post(\n    \"http://0.0.0.0:8008/respond\",\n    json={\"questions\": [\"Кто режиссер фильма Властелин колец: Две крепости?\"]}\n).json()\n\nres = requests.post(\n    \"http://0.0.0.0:8008/respond\",\n    json={\"questions\": ['Какой футболист в 2004 году получил \"Золотой мяч\"?']}\n).json()\n```\n\n#### English version of Knowledge Base Question Answering\n\n```python\nimport requests\n\nres = requests.post(\n    \"http://0.0.0.0:8008/respond\",\n    json={\"questions\": [\"Who directed Forrest Gump?\"]}\n).json()\n\nprint(res)\n\nres = requests.post(\n    \"http://0.0.0.0:8008/respond\",\n    json={\"questions\": [\"When did Jean-Paul Sartre move to Le Havre?\"]}\n).json()\n\nres = requests.post(\n    \"http://0.0.0.0:8008/respond\",\n    json={\"questions\": [\"What is the capital of Belarus?\"]}\n).json()\n\nres = requests.post(\n    \"http://0.0.0.0:8008/respond\",\n    json={\"questions\": [\"What team did Lionel Messi play for in 2004?\"]}\n).json()\n\nres = requests.post(\n    \"http://0.0.0.0:8008/respond\",\n    json={\"questions\": [\"Who is the painter of Mona Lisa?\"]}\n).json()\n\nres = requests.post(\n    \"http://0.0.0.0:8008/respond\",\n    json={\"questions\": [\"What position was held by Harry S. Truman on 1/3/1935?\"]}\n).json()\n\n```\n\n#### The scripts with these examples can be found in ./kbqa/examples_ru.py and ./kbqa/examples_en.py\n\nExample of the output:\n\n```json\n    [\n      {\"answer\": \"Юрий Иосифович Визбор\",\n       \"answer_ids\": [\"Q582152\"],\n       \"confidence\": 1.0,\n       \"entities_and_rels\": [[\"Q472018\", \"Q76367\"], [\"P161\", \"P453\"]],\n       \"sparql_query\": \"SELECT ?answer WHERE { wd:Q472018 p:P161 ?s . ?s ps:P161 ?answer . ?s ?p wd:Q76367. }\",\n       \"triplets\": [[[\"Семнадцать мгновений весны\", \"в ролях\", \"Юрий Иосифович Визбор\"],\n                     [\"Юрий Иосифович Визбор\", \"играет роль персонажа\", \"Мартин Борман\"]]]\n      }\n    ]\n```\n\n### Get metrics of knowledge base question answering and answer explanation (finding triplets which connect entities from answer with answers):\n\n```python\nimport requests\n\nres = requests.post(\"http://0.0.0.0:8008/get_metrics\", json={\"num_samples\": 100}).json()\nprint(res)\n```\n\n### Generate sentence explaining the answer from Knowledge Base triplets:\n\n```python\nimport requests\n\nres = requests.post(\n    \"http://0.0.0.0:8007/generate\",\n    json={\"triplets\": [[[\"Belarus\", \"capital\", \"Minsk\"]]]}\n).json()\n\nprint(res)\n\nres = requests.post(\n    \"http://0.0.0.0:8007/generate\",\n    json={\"triplets\": [[[\"Forrest Gump\", \"director\", \"Robert Zemeckis\"]]]}\n).json()\n\nres = requests.post(\n    \"http://0.0.0.0:8007/generate\",\n    json={\"triplets\": [[[\"Jean-Paul Sartre\", \"residence\", \"Le Havre\"],\n                        [\"Le Havre\", \"start time\", \"1931\"]]]}\n).json()\n\nres = requests.post(\n    \"http://0.0.0.0:8007/generate\",\n    json={\"triplets\": [[[\"Lionel Messi\", \"member of sports team\", \"FC Barcelona\"],\n                        [\"FC Barcelona\", \"start time\", \"2004\"]]]}\n).json()\n\nres = requests.post(\n    \"http://0.0.0.0:8007/generate\",\n    json={\"triplets\": [[[\"Mona Lisa\", \"creator\", \"Leonardo da Vinci\"]]]}\n).json()\n\nres = requests.post(\n    \"http://0.0.0.0:8007/generate\",\n    json={\"triplets\": [[[\"Harry S. Truman\", \"position held\", \"United States senator\"],\n                        [\"United States senator\", \"start time\", \"1935\"]]]}\n).json()\n\n```\n\n### Answer the question over the knowledge graph and generate sentence explaining the answer:\n\n\n```python\nimport requests\n\nres = requests.post(\n    \"http://0.0.0.0:8007/ans_expl\",\n    json={\"questions\": [\"Who directed Forrest Gump?\"]}\n).json()\n\nprint(res)\n\nres = requests.post(\n    \"http://0.0.0.0:8007/ans_expl\",\n    json={\"questions\": [\"When did Jean-Paul Sartre move to Le Havre?\"]}\n).json()\n\nres = requests.post(\n    \"http://0.0.0.0:8007/ans_expl\",\n    json={\"questions\": [\"What is the capital of Belarus?\"]}\n).json()\n\nres = requests.post(\n    \"http://0.0.0.0:8007/ans_expl\",\n    json={\"questions\": [\"What team did Lionel Messi play for in 2004?\"]}\n).json()\n\nres = requests.post(\n    \"http://0.0.0.0:8007/ans_expl\",\n    json={\"questions\": [\"Who is the painter of Mona Lisa?\"]}\n).json()\n\nres = requests.post(\n    \"http://0.0.0.0:8007/ans_expl\",\n    json={\"questions\": [\"What position was held by Harry S. Truman on 1/3/1935?\"]}\n).json()\n\n```\n\n### Get metrics of the model, which generates answer explanation from triplets:\n\n\n```python\nimport requests\n\nres = requests.post(\"http://0.0.0.0:8007/get_metrics\", json={\"num_samples\": 100}).json()\nprint(res)\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeeppavlov%2Fexplainable_qa","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdeeppavlov%2Fexplainable_qa","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeeppavlov%2Fexplainable_qa/lists"}