Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/syedaanif/pii-detection-bot
Steps to build a PII Detection Bot using Microsoft Bot Framework SDK and Azure Language Service
https://github.com/syedaanif/pii-detection-bot
artificial-intelligence machine-learning microsoft-azure microsoftbotframework natural-language-understanding python3
Last synced: about 6 hours ago
JSON representation
Steps to build a PII Detection Bot using Microsoft Bot Framework SDK and Azure Language Service
- Host: GitHub
- URL: https://github.com/syedaanif/pii-detection-bot
- Owner: SyedAanif
- Created: 2023-05-20T04:30:26.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2023-05-20T09:56:04.000Z (over 1 year ago)
- Last Synced: 2023-10-05T02:07:08.953Z (about 1 year ago)
- Topics: artificial-intelligence, machine-learning, microsoft-azure, microsoftbotframework, natural-language-understanding, python3
- Homepage:
- Size: 8.79 KB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# PII-Detection-Bot
Steps to build a PII Detection Bot using Microsoft Bot Framework SDK and Azure Cognitive Service for Language.
## Introduction
**Personally Identifiable Information (PII)** detection is one of the features offered by [Azure Cognitive Service for Language](https://learn.microsoft.com/en-us/azure/cognitive-services/language-service/overview) which can identify, categorize, and even redact sensitive information in natural unstructured text. For example: phone numbers, email addresses, and forms of identification.
#### Redaction example:
#### Categorization example:
Here, we will be building this bot using:
- [Azure Cognitive Service for Language](https://learn.microsoft.com/en-us/azure/cognitive-services/language-service/overview) which is a cloud-based service for Natural Language Processing(NLP) helping in understanding and analyzing text.
- [Azure Bot Service](https://learn.microsoft.com/en-us/azure/bot-service/bot-builder-basics?view=azure-bot-service-4.0) which is a cloud platform to host bots and make them available to channels like Facebook, MS Teams etc.
- [Microsoft Bot Framework SDK](https://github.com/microsoft/botframework-sdk) which is used to develop conversational applications in Python, Java, JavaScript and C#, that can be hosted on Azure Bot Service.
## Pre-requisites
- Python 3.X version:
+ [Installation using Anaconda distribution](https://docs.anaconda.com/free/anaconda/#installation)
+ [Python Website](https://www.python.org/downloads/)
- [Microsoft Azure Account](https://azure.microsoft.com/en-in/free/)
- Editor of choice:
+ [VSCode](https://code.visualstudio.com/download)
- Terminal of choice with [Azure CLI](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli) installed.
## Steps to create the PII-Detection-Bot
1. Sign-in to your [Microsoft Azure account](https://portal.azure.com/#home).
2. Create a [Resource Group](https://learn.microsoft.com/en-us/azure/azure-resource-manager/management/manage-resource-groups-portal#what-is-a-resource-group) (in a region closest to your location) to logically contain your Azure Resources.
3. Create an [Azure Cognitive Service for Language](https://learn.microsoft.com/en-us/azure/cognitive-services/language-service/overview) inside the Resource Group created above.
+ After the Language Service is successfully created, navigate to the language resource to make a note of the `Keys` and `Endpoint` which will be used for accessing this resource.
4. Open a Python project([like the one explained in echo-bot](https://github.com/microsoft/BotBuilder-Samples/tree/main/samples/python/02.echo-bot)) in editor of choice(like VSCode).
5. Install the required dependencies as shown in the `requirements.txt` file using the command:
```
pip install -r requirements.txt
```
This will install the dependencies like [`botbuilder-core`](https://pypi.org/project/botbuilder-core/), [`asyncio`](https://pypi.org/project/asyncio/), [`aiohttp`](https://pypi.org/project/aiohttp/), [`cookiecutter`](https://pypi.org/project/cookiecutter/)
6. Login to Microsoft Azure Account and set your subscription:
```
az login
az account set --subscription "<< subscription-id >>"
```
7. Create an [App Registration](https://learn.microsoft.com/en-us/power-apps/developer/data-platform/walkthrough-register-app-azure-active-directory) in the Azure portal and make a note of its `Application (client) ID` and `appSecret`.
8. Update the `config.py` file in VSCode with the copied `appId` and `appSecret` in the `MicrosoftAppId` and `MicrosoftAppPassword` respectively.
9. Update `bot.py` with the `Keys` and `Endpoint` of the `Azure Language Service` which are used to authenticate and connect to the `text-analytics-client`. The `text-analytics-client` is used to recognize PII entities in the input user text. The bot contains mainly two functions:
+ `on_members_added_activity`: this function gets called whenever a user accesses the bot and greets the user with "Hello and welcome to the PII detection bot".
+ `on_message_activity`: this function gets called whenever a user interacts with the bot. This function takes the user input text and calls the text-analytics-client for PII recognition. This function prints the redacted text and the PII categories of the redacted text.
10. Create an [App Service](https://learn.microsoft.com/en-us/azure/app-service/overview) and [App Service Plan](https://learn.microsoft.com/en-us/azure/app-service/overview-hosting-plans?source=recommendations) with [Azure Resource Manager(ARM)](https://learn.microsoft.com/en-us/azure/azure-resource-manager/templates/overview) template:
+ Modify the `parameters-for-template-BotApp-with-rg.json` with values for `appServiceName`, `newAppServicePlanName`, `newAppServicePlanLocation`, `appId`, `appSecret`
+ Change directory to `deploymentTemplates\deployUseExistResourceGroup\`
```
cd .\deploymentTemplates\deployUseExistResourceGroup\
```
+ Create a deployment for `App Service` and `App Service Plan` with ARM template-file `template-BotApp-with-rg.json`:
```
az deployment group create --resource-group "<>" --template-file "<>" --parameters "@<>"
```
11. Create an [Azure Bot Service](https://azure.microsoft.com/en-in/products/bot-services/?ef_id=_k_CjwKCAjw36GjBhAkEiwAKwIWyU8eyHxp5IuTW7RCGqs8J43g1lEkqUtbyjuGaas-tBdrIB6pkX8_DxoCpYUQAvD_BwE_k_&OCID=AIDcmmf1elj9v5_SEM__k_CjwKCAjw36GjBhAkEiwAKwIWyU8eyHxp5IuTW7RCGqs8J43g1lEkqUtbyjuGaas-tBdrIB6pkX8_DxoCpYUQAvD_BwE_k_&gclid=CjwKCAjw36GjBhAkEiwAKwIWyU8eyHxp5IuTW7RCGqs8J43g1lEkqUtbyjuGaas-tBdrIB6pkX8_DxoCpYUQAvD_BwE) with [Azure Resource Manager(ARM)](https://learn.microsoft.com/en-us/azure/azure-resource-manager/templates/overview) template:
+ Modify the `parameters-for-template-AzureBot-with-rg.json` with values for `azureBotId`, `botEndpoint`, `appId`.
+ Create a deployment for `Azure Bot Service` with ARM template-file `template-AzureBot-with-rg.json`:
```
az deployment group create --resource-group "<>" --template-file "<>" --parameters "@<>"
```
12. The creation of `Azure Bot Service`, `App Service`, `App Service Plan` can be verified by navigating to the Azure portal and the respective Resource Group.
13. Package the Microsoft Bot Framework SDK(python code) into a `.zip` file.
14. Deploy the packaged SDK as a `web-app`
+ Change the working directory to the root of the project.
+ Run the command:
```
az webapp deployment source config-zip --resource-group "<>" --name "<>" --src "<>"
```
+ If the command runs successfully, it will deploy the Bot framework SDK as a web-application and the deployment endpoint responds with an HTTP status code 202.
15. The deployed `Azure Bot PII service` can be tested in the Azure portal with any example as shown below:
+ Example 1:
```
My name is John Doe and my phone number is 1234567890. I can be contacted over [email protected]. I live in Seattle.
```
+ Example 2:
```
Barack Obama is an American politician who served as the 44th President of the United States from 2009 to 2017. He is the first African American to have served as president, as well as the first born outside the contiguous United States.
```
+ Example 3:
```
You can even pre-order from their online menu at www.contososteakhouse.com, call 312-555-0176 or send email to [email protected]!
```
