{"id":21357532,"url":"https://github.com/deepbiolab/custom-copilot-with-azure","last_synced_at":"2026-05-19T01:31:46.062Z","repository":{"id":263969537,"uuid":"891828530","full_name":"deepbiolab/custom-copilot-with-azure","owner":"deepbiolab","description":"Tutorial about building, testing, and deployment of a copilot capable of retrieving relevant information from an indexed dataset and delivering accurate, real-time responses to customer inquiries","archived":false,"fork":false,"pushed_at":"2024-11-21T08:51:47.000Z","size":15467,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-22T18:29:23.539Z","etag":null,"topics":["azure","copilot","prompt-flow","rag"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/deepbiolab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-11-21T02:55:45.000Z","updated_at":"2024-11-21T10:58:02.000Z","dependencies_parsed_at":"2024-11-21T09:42:58.138Z","dependency_job_id":null,"html_url":"https://github.com/deepbiolab/custom-copilot-with-azure","commit_stats":null,"previous_names":["deepbiolab/custom_copilot_with_azure","deepbiolab/custom-copilot-with-azure"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deepbiolab%2Fcustom-copilot-with-azure","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deepbiolab%2Fcustom-copilot-with-azure/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deepbiolab%2Fcustom-copilot-with-azure/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deepbiolab%2Fcustom-copilot-with-azure/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/deepbiolab","download_url":"https://codeload.github.com/deepbiolab/custom-copilot-with-azure/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243830955,"owners_count":20354855,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["azure","copilot","prompt-flow","rag"],"created_at":"2024-11-22T05:07:40.170Z","updated_at":"2026-05-19T01:31:41.035Z","avatar_url":"https://github.com/deepbiolab.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# Custom Copilot with Azure\n\n### Project Overview\n\nThese project instructions will guide you through creating and deploying a custom AI copilot using Azure AI Studio. Follow the steps below carefully, and be sure to complete each deliverable for submission.\n\nIn this project, you will:\n\n1. Create an AI Studio project and hub.\n2. Deploy an AI model and upload product data for indexing.\n3. Build and test a custom copilot app using Prompt Flow.\n4. Evaluate the app with both automated and manual prompt evaluation.\n5. Deploy and test the copilot application.\n\n### Step-by-Step Guide\n\n#### 1. Create an AI Studio Hub and Project\n\n- Log in to [Azure AI Studio](https://ai.azure.com/).\n\n  ![image-20241121113508200](assets/image-20241121113508200.png)\n\n- Click **Create Project**, name your project (`outlander-ai-project`), and create a new hub (`outlander-ai-hub`).\n\n  \u003cimg src=\"assets/image-20241121114026889.png\" alt=\"image-20241121114026889\"  /\u003e\n\n- When we first time create this project and not create hub yet, after click **Customize**, we will customize below settings. For  key settings in below you can custom by the instruction.\n\n  ![image-20241121114734878](assets/image-20241121114734878.png)\n\n- Leave default configurations unless necessary adjustments are needed(below is my settings), and click **Next** in below.\n\n  ![image-20241121120802182](assets/image-20241121120802182.png)\n\n\n\n- Ensure you have an **Azure AI Search** service for indexing data. And finally review the information and click **Create**\n\n  ![image-20241121114944714](assets/image-20241121114944714.png)\n\n  ![image-20241121115535949](assets/image-20241121115535949.png)\n\n#### 2. Deploy an AI Model\n\n- With your project selected, go to **Models + endpoints**.\n\n  ![image-20241121121410538](assets/image-20241121121410538.png)\n\n- Click **+Deploy Model**, choose a base model like `gpt-4o`, and confirm the deployment.\n\n  ![image-20241121121625966](assets/image-20241121121625966.png)\n\n  ![image-20241121121722867](assets/image-20241121121722867.png)\n\n  ![image-20241121121906334](assets/image-20241121121906334.png)\n\n  ![image-20241121122431637](assets/image-20241121122431637.png)\n\n  ![image-20241121122651808](assets/image-20241121122651808.png)\n\n#### 3. Upload Product Data to Azure AI Studio\n\n- Download and unzip the `product-info.zip` from [this GitHub link](https://github.com/Azure-Samples/rag-data-openai-python-promptflow/tree/main/tutorial/data).\n\n- In AI Studio, select the **Data + indexes** blade and click **+New Data**.\n\n  ![image-20241121122847180](assets/image-20241121122847180.png)\n\n- Upload the unzipped product data files or connect to your storage account if preferred.\n\n  ![image-20241121123005228](assets/image-20241121123005228.png)\n\n  ![image-20241121123215337](assets/image-20241121123215337.png)\n\n  ![image-20241121123325596](assets/image-20241121123325596.png)\n\n- After upload files, you can see the files structure in **Data + indexes** panel.\n\n  ![image-20241121123456693](assets/image-20241121123456693.png)\n\n#### 4. Create an AI Search Index\n\n- Create a Embedding model, we can transform the document (uploaded above) to dense embedding which we can use it in RAG system later. Like procedure created for `gpt-4o`, we choose `Models + endpoints` \u003e\u003e\u003e `Deploy model` \u003e\u003e\u003e `Deploy base model` in sequentially, and choose `text-embedding-ada-002` model to create.\n\n  ![image-20241121124142845](assets/image-20241121124142845.png)\n\n- Still you can custom below settings, but i leave here by default and click **Deploy**\n\n  ![image-20241121124343536](assets/image-20241121124343536.png)\n\n- After deployed, we can see the model in the service we created before.\n\n  ![image-20241121124541818](assets/image-20241121124541818.png)\n\n- Go to the **Data + indexes** blade and click **+New Index** after **Indexes**\n\n  ![image-20241121124755977](assets/image-20241121124755977.png)\n\n  ![image-20241121124924496](assets/image-20241121124924496.png)\n\n- Select **Data in Azure AI Studio** as the data source, choose the uploaded data, and proceed.\n\n  ![image-20241121125015098](assets/image-20241121125015098.png)\n\n  ![image-20241121125303308](assets/image-20241121125303308.png)\n\n  ![image-20241121125542058](assets/image-20241121125542058.png)\n\n- Choose your deployed model for text embeddings and create the index.\n\n  ![image-20241121125757914](assets/image-20241121125757914.png)\n\n  ![image-20241121125851637](assets/image-20241121125851637.png)\n\n  ![image-20241121133104319](assets/image-20241121133104319.png)\n\n  \n\n#### 5. Build the Copilot App\n\n- Navigate to the **Chat** blade under Project Playground.\n\n  ![image-20241121133250989](assets/image-20241121133250989.png)\n\n- Select your deployed model and add your data by choosing the created index.\n\n  ![image-20241121133509986](assets/image-20241121133509986.png)\n\n  ![image-20241121133644108](assets/image-20241121133644108.png)\n\n- Click **Prompt Flow**, name your flow (e.g., `Outlander AI Copilot`), and open it.\n\n  ![image-20241121133858092](assets/image-20241121133858092.png)\n\n  ![image-20241121133930169](assets/image-20241121133930169.png)\n\n- Review and understand the default components, such as data retrieval and response generation.\n\n  ![image-20241121135138303](assets/image-20241121135138303.png)\n\n#### 6. Test the Copilot\n\n- Start a compute session by clicking **Start compute session**.\n\n  ![image-20241121140303726](assets/image-20241121140303726.png)\n\n- Test the copilot by entering sample questions such as:\n\n  \u003e Below questions basically stick with the documents we uploaded, so ideally it will reply the relevant response based the document that is purpose of RAG.\n\n  - `How much do the TrailWalker Hiking Shoes cost?`\n\n    ![test01](./assets/outlander_ai_copilot_test01.png)\n\n  - `Which tent is the most waterproof?`\n\n    ![test02](./assets/outlander_ai_copilot_test02.png)\n\n  - `Can the warranty for TrailBlaze pants be transferred?`\n\n    ![test03](./assets/outlander_ai_copilot_test03.png)\n\n- After test, click **Save** for this copilot.\n\n#### 7. Perform Automated Prompt Evaluation\n\n- Create a JSONL or CSV file with evaluation questions and answers (see below for format).\n  \u003e [Sample Evaluation Data (JSONL format)](./data/qa_evaluation_data.jsonl)\n  \u003e\n  \u003e ```\n  \u003e {\"chat_input\": \"Which tent is the most waterproof?\", \n  \u003e \"truth\": \"The Alpine Explorer Tent has the highest rainfly waterproof rating at 3000m\", \n  \u003e \"chat_history\": []}\n  \u003e {\"chat_input\": \"How much do the TrailWalker Hiking Shoes cost?\", \n  \u003e \"truth\": \"The TrailWalker Hiking Shoes are priced at $110\", \n  \u003e \"chat_history\": []}\n  \u003e ```\n  \u003e\n  \u003e [Sample Evaluation Data (CSV format)](./data/qa_evaluation_data.csv)\n  \u003e\n  \u003e | chat_input                                      | truth                                                        | chat_history |\n  \u003e | ----------------------------------------------- | ------------------------------------------------------------ | ------------ |\n  \u003e | Which  tent is the most waterproof?             | The Alpine Explorer Tent has the  highest rainfly waterproof rating at 3000m | []           |\n  \u003e | How  much do the TrailWalker Hiking Shoes cost? | The TrailWalker Hiking Shoes are  priced at $110             | []           |\n\n\n- In **Prompt Flow**, click **Evaluate**, choose **Automated evaluation**, and then and use your dataset.\n\n  ![image-20241121142553882](assets/image-20241121142553882.png)\n\n  ![image-20241121142724140](assets/image-20241121142724140.png)\n\n  ![image-20241121143218630](assets/image-20241121143218630.png)\n\n- Map inputs and outputs and **Submit** on the next page and then run the evaluation.\n\n  ![image-20241121143617316](assets/image-20241121143617316.png)\n\n  ![image-20241121144300055](assets/image-20241121144300055.png)\n\n#### 8. Manual Prompt Evaluation\n\n- Go to **Evaluation** in AI Studio and Create a **Manual evaluations**\n\n  ![image-20241121144618653](assets/image-20241121144618653.png)\n\n- Add questions and expected responses and run evaluations, right here I only add one.\n\n  - Input: `How much do the TrailWalker Hiking Shoes cost?`\n  - Expected response: `The TrailWalker Hiking Shoes are priced at $110.`\n\n  ![image-20241121144810236](assets/image-20241121144810236.png)\n\n-  After run, you can provide feedback using thumbs up or down for the response.\n  ![image-20241121145131253](assets/image-20241121145131253.png)\n\n#### 9. Deploy the Copilot\n\n- In **Prompt Flow**, click **Deploy** and name the deployment.\n\n  ![image-20241121145320570](assets/image-20241121145320570.png)\n\n  ![image-20241121145504522](assets/image-20241121145504522.png)\n\n- Verify the deployment details and create, finally you will see the created endpoint in below image.\n\n  ![image-20241121150106569](assets/image-20241121150106569.png)\n\n- Finally, click the created endpoint `outlander-ai-project-znsmp-1` and you will see ready-to-use deployment info\n\n  ![image-20241121151529198](assets/image-20241121151529198.png)\n\n- And we can test some question.\n\n  ![image-20241121152034536](assets/image-20241121152034536.png)\n\n🎉Good luck, and enjoy building your custom Azure AI copilot✨!\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeepbiolab%2Fcustom-copilot-with-azure","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdeepbiolab%2Fcustom-copilot-with-azure","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeepbiolab%2Fcustom-copilot-with-azure/lists"}