{"id":15524429,"url":"https://github.com/codeforequity-at/botium-samples-nlpanalytics","last_synced_at":"2026-02-11T03:03:55.881Z","repository":{"id":91176498,"uuid":"230244610","full_name":"codeforequity-at/botium-samples-nlpanalytics","owner":"codeforequity-at","description":"Botium Sample Repository for doing NLP Analytics","archived":false,"fork":false,"pushed_at":"2021-01-18T11:40:20.000Z","size":117,"stargazers_count":5,"open_issues_count":1,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-02-03T06:35:29.674Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/codeforequity-at.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-12-26T10:27:56.000Z","updated_at":"2021-09-13T10:57:39.000Z","dependencies_parsed_at":null,"dependency_job_id":"b9e21e97-c8b7-4632-9bfd-03bdac3c746f","html_url":"https://github.com/codeforequity-at/botium-samples-nlpanalytics","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/codeforequity-at%2Fbotium-samples-nlpanalytics","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/codeforequity-at%2Fbotium-samples-nlpanalytics/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/codeforequity-at%2Fbotium-samples-nlpanalytics/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/codeforequity-at%2Fbotium-samples-nlpanalytics/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/codeforequity-at","download_url":"https://codeload.github.com/codeforequity-at/botium-samples-nlpanalytics/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246100488,"owners_count":20723469,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-02T10:51:19.307Z","updated_at":"2026-02-11T03:03:50.845Z","avatar_url":"https://github.com/codeforequity-at.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# Botium NLP Analytics Sample\n\nThis repository is accompanying [this blog article](https://chatbotslife.com/tutorial-benchmark-your-chatbot-on-watson-dialogflow-wit-ai-and-more-92885b4fbd48) and showcases the NLP analytics capabilities of [Botium](https://www.botium.at).\n\n## Prerequisites\n\n* Node.js (\u003e 8.13)\n* Clone this repository (_git clone ..._)\n* Run _npm install_ to install dependencies\n\n## Prepare Botium Connectors\n\nIn the _config_ directory you will find several _*.botium.json_ files for the currently supported Botium connectors. Add the required information there. For most of the connectors this means setting up an account with the provider and add access keys or secrets to those files.\n\n**You are free to only configure the Botium connectors the results are of interest to you.**\n\n### IBM Watson\nSee https://github.com/codeforequity-at/botium-connector-watson\n\n### Wit.ai\nSee https://github.com/codeforequity-at/botium-connector-witai\n\n_Trained Wit.ai projects cannot be removed automatically, have to be done manually_\n\n### Dialogflow\nSee https://github.com/codeforequity-at/botium-connector-dialogflow\n\n_Create a separated Google Project for doing this analytics. The attached Dialogflow agent will be overwritten._\n\n### NLP.js\nSee https://github.com/codeforequity-at/botium-connector-nlpjs\n\n### Lex\nSee https://github.com/codeforequity-at/botium-connector-lex\n\n## Prepare Data\n\nThe _convos/smalltalk_ directory already contains a dataset for a simple _Smalltalk_ chatbot in plain text files.\n\n**See [Botium Wiki](https://botium.atlassian.net/wiki/spaces/BOTIUM/pages/491664/Botium+Scripting+-+BotiumScript) for details about other supported file formats like YAML, Excel, CSV, JSON**\n\n* Filename has to end in _.utterances.txt*_, otherwise name does not matter\n* First line of the file contains the intent name\n* Each other line contains a user example for this intent\n\nIt makes sense to use your own data to make this evaluation. If you already have the data available in one of the supported platforms, you can use this tool to extract the data and save it for further evaluation - one of the following commands will extract it and save the intents and user examples in the _convos_ directory:\n\n    \u003e npm run nlpextract:watson\n    \u003e npm run nlpextract:lex\n    \u003e npm run nlpextract:witai\n    \u003e npm run nlpextract:dialogflow\n\n**You are free to use any other tool to prepare the data, you can even write it manually (if you have the time)**\n\n## Split utterances into a training and a test set\n\nAn easy way to separate training data from test data is to split an existing dataset into two separate datasets. In package.json, there is a sample task splitting the _Smalltalk_ into 80% training data and 20% test data:\n\n    \u003e npm run nlpsplit\n\n## Run Test Set Validation on Training Set\n\nWhen you separated your test data into a training and a test set, either manually or by using the _nlpsplit_ task, you can now train a model with the training set and validate with the test set. In package.json, there are tasks defined to run the validation for the included _Smalltalk_ dataset.\n\n    \u003e npm run validate:watson\n    \u003e npm run validate:lex\n    \u003e npm run validate:witai\n    \u003e npm run validate:dialogflow\n    \u003e npm run validate:nlpjs\n\nOr you can run it all in parallel:\n\n    \u003e npm run validate\n\nAfter a while, you will find the results for each platform in the _results_ directory. For each platform:\n\n* validate-_platform_.txt to hold the validation summary\n* validate-_platform_.err.txt for processing error output\n* validate-_platform_.csv for listing the score details\n* validate-_platform_-predictions.csv for listing the predicted intents for all utterances\n\n## Run K-Fold Cross Validation\n\nIn package.json, there are tasks defined to run the K-Fold Cross Validation for the included _Smalltalk_ dataset.\n\n    \u003e npm run kfold:watson\n    \u003e npm run kfold:lex\n    \u003e npm run kfold:witai\n    \u003e npm run kfold:dialogflow\n    \u003e npm run kfold:nlpjs\n\nOr you can run it all in parallel:\n\n    \u003e npm run kfold\n\nAfter a while, you will find the results for each platform in the _results_ directory. For each platform:\n\n* k-fold-_platform_.txt to hold the validation summary\n* k-fold-_platform_.err.txt for processing error output\n* k-fold-_platform_.csv for listing the score details\n* k-fold-_platform_-predictions.csv for listing the predicted intents for all folds and utterances\n\n**See [here](https://medium.com/analytics-vidhya/quality-metrics-for-nlu-chatbot-training-data-part-1-confusion-matrix-91ac71b90270) for an introduction how to interpret the results**\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcodeforequity-at%2Fbotium-samples-nlpanalytics","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcodeforequity-at%2Fbotium-samples-nlpanalytics","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcodeforequity-at%2Fbotium-samples-nlpanalytics/lists"}