{"id":19457315,"url":"https://github.com/aaronbriel/jugglechat","last_synced_at":"2025-02-25T10:53:32.585Z","repository":{"id":71835220,"uuid":"314376333","full_name":"aaronbriel/jugglechat","owner":"aaronbriel","description":"An Eclectic and Malleable Multi-Chatbot Framework","archived":false,"fork":false,"pushed_at":"2021-12-22T19:38:17.000Z","size":8178,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-01-08T01:46:51.373Z","etag":null,"topics":["absum","chatbot","multiagent","nlp","question-answering","transformers"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/aaronbriel.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-11-19T21:39:43.000Z","updated_at":"2021-12-22T19:38:20.000Z","dependencies_parsed_at":"2023-09-11T00:30:38.049Z","dependency_job_id":null,"html_url":"https://github.com/aaronbriel/jugglechat","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aaronbriel%2Fjugglechat","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aaronbriel%2Fjugglechat/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aaronbriel%2Fjugglechat/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aaronbriel%2Fjugglechat/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/aaronbriel","download_url":"https://codeload.github.com/aaronbriel/jugglechat/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240654183,"owners_count":19835857,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["absum","chatbot","multiagent","nlp","question-answering","transformers"],"created_at":"2024-11-10T17:21:03.689Z","updated_at":"2025-02-25T10:53:32.514Z","avatar_url":"https://github.com/aaronbriel.png","language":"Python","readme":"![JuggleChat](https://github.com/aaronbriel/jugglechat/blob/master/logo.png?raw=true)\n\nAn Eclectic and Malleable Multi-Chatbot Framework. [Link](http://doi.org/10.1002/cae.22449) to publication.\n\n![JuggleChat](https://github.com/aaronbriel/jugglechat/blob/master/architecture.png?raw=true)\n\n## Installation\nUbuntu:\n\n\tsudo apt install python3\n\tsudo apt install python3-pip\n\tsudo apt install virtualenv (or sudo apt install python3-virtualenv)\n\nClone repo and install dependencies:\n\n    git clone https://github.com/aaronbriel/jugglechat.git\n    cd jugglechat\n    make setup\n    \nInstall redis.\n\n    sudo apt install redis\n\nInstall docker.\n\n    sudo apt install docker\n\nUpdate .env BASE_PATH ie \"/home/aaron/jugglechat/\"\n\n## Configuration\n\n### Configure Celery for Redis instance:\nSee config.py: BROKER_URL, BACKEND_URL\n\n### Start Celery:\n    make start_celery\n    \nView redis (celery task) data in browser:\n\n    npm install -g redis-commander\n    redis-commander\n\n### Starting rasa server: \n    make start_rasa\n    \n### Starting haystack (QA and FAQ chatbots):\nStart docker:\n\n    sudo systemctl start docker\n    \nStart docker container:\n\n    make elastic_start\n\nStore FAQ and QA retrieval content into ElasticSearch:\n    \n    make store_faq \n    make store_faq_experiment       \n    make store_qa\n    make store_qa_experiment\n    \nYou may need to run the following to resolve a Permission Denied issue on Ubuntu:\n\n    sudo chmod 777 /tmp/tika*\n    \nIf you run into the following error you will need to either increase disc size or free up the index:\n\n    elasticsearch.exceptions.AuthorizationException: AuthorizationException(403, 'cluster_block_exception', 'index [qa_\n    default] blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];')\n    \nOn Ubuntu, you also may need to do the following, then log out and back in:\n\n\tsudo groupadd docker\n\tsudo usermod -aG docker ${USER} \n\t\nA viable solution can be found here: \nhttps://stackoverflow.com/questions/50609417/elasticsearch-error-cluster-block-exception-forbidden-12-index-read-only-all\n\nFor example: \n\n    curl -XPUT -H \"Content-Type: application/json\" https://localhost:9200/_all/_settings -d '{\"index.blocks.read_only_allow_delete\": null}'\n\n\nView content of ElasticSearch (default port: 9200, default index_names: faq_default, qa_default):\n\n    curl https://localhost:PORT/INDEX_NAME/_search?pretty\nEx:\n\n    curl https://localhost:9200/faq/_search?pretty\n    \nStart Haystack REST API:\n\n    make start_haystack\n\nDo sanity check of haystack:\n    \n    curl --request POST --url 'http://127.0.0.1:8000/models/1/doc-faq' --data '{\"questions\": [\"can covid19 pass to pets?\"]}' \n\nView API documentation: http://127.0.0.1:8000/docs\n\n### Calling celery task that passes data to rasa core:\n    from celery_app.tasks import call_rasa\n    result = call_rasa_core.delay(\"hello\")\n    result.get()\n    \n### Calling celery task that passes data to rasa nlp:\n    from celery_app.tasks import call_rasa\n    result = call_rasa_nlp.delay(\"hello\")\n    result.get()\n\n### Calling celery task that passes data to haystack faq:\n    from celery_app.tasks import call_faq\n    result = call_faq.delay(\"How is the virus spreading?\")\n    result.get()\n    \n### Calling celery task that passes data to haystack qa:\n    from celery_app.tasks import call_qa\n    result = call_qa.delay(\"Where was COVID19 first discovered?\")\n    result.get()\n    \n### Calling celery task that does single chat with dialogpt:\n    from celery_app.tasks import call_convai\n    result = call_convai.delay(\"Who is mort?\")\n    result.get()\n\n### Calling celery task that does Abstractive Summarization:\n    from celery_app.tasks import call_summarizer\n    result = call_summarizer.delay(\"Lets do a summary of this overly long sentence please.\")\n    result.get()\n    \n### Calling celery tasks via flower API calls (expects rasa, haystack, elasticsearch, celery, and flower to be running):\n    curl -X POST -u user1:password1 -d '{\"args\":[\"can covid19 pass to pets?\"]}' http://localhost:5555/api/task/apply/celery_app.tasks.call_rasa_nlp\n    curl -X POST -u user1:password1 -d '{\"args\":[\"can covid19 pass to pets?\"]}' http://localhost:5555/api/task/apply/celery_app.tasks.call_qa\n    curl -X POST -u user1:password1 -d '{\"args\":[\"how long does the coronavirus survive on surfaces\"]}' http://localhost:5555/api/task/apply/celery_app.tasks.call_rasa_nlp\n    curl -X POST -u user1:password1 -d '{\"args\":[\"how long does the coronavirus survive on surfaces\"]}' http://localhost:5555/api/task/apply/celery_app.tasks.call_faq\n    curl -X POST -u user1:password1 -d '{\"args\":[\"can I get a summary of the lesson?\"]}' http://localhost:5555/api/task/apply/celery_app.tasks.call_rasa_nlp\n    curl -X POST -u user1:password1 -d '{\"args\":[\"can I get a summary of the lesson?\"]}' http://localhost:5555/api/task/apply/celery_app.tasks.call_summarizer\n\n    \n## Configuring production instance\n\n### Setting up supervisor\n\nFollow: https://www.digitalocean.com/community/tutorials/how-to-install-and-manage-supervisor-on-ubuntu-and-debian-vps\n\nFirst confirm redis is installed and running for the celery command.\n\nInstall supervisor:\n\n    sudo apt install supervisor\n\nCopy supervisord conf file to local installation:\n\n    sudo cp supervisor/supervisord.conf /etc/supervisor\n\nStart supervisor:\n\n    sudo supervisord\n\nRestart ctl to reflect changes:\n\n    sudo supervisorctl update\n    sudo supervisorctl reload\n\nCheck that running:\n\n    sudo supervisorctl\n\nview log for celery\nvi /home/aaron/jugglechat/celery_app/celery_app.log\n\nkill all celery workers:\n\n    sudo pkill -f \"celery_app worker\"\n\n### Exposing Flower API on GCP\n\n1. Reserve a static IP and assign to VM. \n\n2. Allow HTTP traffic\n\n3. Whitelist jugglechat-experiment GCP cloud app URL\n\n### Running a Control Group for the Experiment\n\n## Experiment Configuration and Deployment\n\nFor the non-chat control group, simply complete step 4 with experimentalGroup set to \"control\". For the rest, \nfollow all listed steps.\n\n1. Modify celery_app \u003e tasks.py call_rasa_nlp to pass in control_group (ie \"qa\" or \"faq\") to Allocator instantiation. This \nforces allocator to return that control group every time. There is no need for control_group with JuggleChat, as it is \nthe default.\n\n2. For QA or FAQ, modify supervisord.conf haystack variables to RUN_TYPE=experiment, READER_MODEL_PATH=\"models/qa_experiment\" . \nThe former  results in the elastic_search index used for retrieval to either be the faq_experiment (full FAQ dataset) or the \nqa_experiment and the latter points the qa chatbot's model path to the full QA set. For JuggleChat use RUN_TYPE=default,\nREADER_MODEL_PATH=\"models/qa_default\"\n\nNOTE: For the added experimental run to get results with deepset's extractive-QA model, RUN_TYPE=deepset, \nREADER_MODEL_PATH=\"deepset/roberta-base-squad2-covid\", with control_group=\"qa\" for (1)\n\n3. Restart all supervisor processes:\n\n\n    ```\n    sudo supervisorctl\n    restart all\n    ```\n\n4. In jugglechat-experiment \u003e client \u003e src \u003e index.tsx change experimentalGroup to the experimental group of:\njugglechat (JuggleChat framework), qa (QA bot only), faq (FAQ bot only), or control (no chatbot). Update the \ncompletionCode if needed.\n\n5. Rebuild the jugglechat-experiment app by deleting all files under build then rebuilding:\n\n\n    ```\n    cd client\n    npm run build\n    cd ..\n    npm run build\n    ```\n    \n6. Redeploy the jugglechat-experiment app:\n\n\n    ```\n    gcloud app deploy\n    ```\n\nNOTE: The Worker IDs extracted by jugglechat-experiment \u003e client/src/pages/GetId.tsx are stored in the worker_ids \ndatabase table and can be exported to use for exclusion lists in subsequent experiment runs.\n\n## Experiment Results Data Processing and Visualization\n\n    source .venv/bin/activate \n    cd experiment\n\nGenerates results.json:\n\n    python data_processing.py\n\nGenerates graphs comparing mean values of test scores, sentiment, percieved usefulness and accuracy, and (optionally) \nwordcloud images:\n\n    python data_visualization.py\n\n## Running Statistical Tests on Results Data\n\nRuns One-Way ANOVA test along with Tukey Honestly Significant Difference (HSD) post-hoc tests, generating sentiment, \nperceived accuracy, and perceived usefulness confidence interval comparison graphs for the latter:\n\n    source .venv/bin/activate \n    cd experiment\n    python statistical_tests.py\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faaronbriel%2Fjugglechat","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faaronbriel%2Fjugglechat","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faaronbriel%2Fjugglechat/lists"}