{"id":14531280,"url":"https://github.com/sshkhr/safeguarding-llms","last_synced_at":"2025-07-01T11:07:23.908Z","repository":{"id":246007685,"uuid":"819819209","full_name":"sshkhr/safeguarding-llms","owner":"sshkhr","description":"TMLS 2024 Workshop: A Practitioner's Guide To Safeguarding Your LLM Applications","archived":false,"fork":false,"pushed_at":"2024-07-11T16:46:22.000Z","size":96,"stargazers_count":4,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-05-29T11:14:29.017Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sshkhr.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-06-25T09:02:18.000Z","updated_at":"2024-08-15T18:53:11.000Z","dependencies_parsed_at":null,"dependency_job_id":"bc3e1fea-309a-43f9-a1c8-1a107835c413","html_url":"https://github.com/sshkhr/safeguarding-llms","commit_stats":null,"previous_names":["sshkhr/guardrails-demo","sshkhr/safeguarding-llms"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/sshkhr/safeguarding-llms","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sshkhr%2Fsafeguarding-llms","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sshkhr%2Fsafeguarding-llms/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sshkhr%2Fsafeguarding-llms/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sshkhr%2Fsafeguarding-llms/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sshkhr","download_url":"https://codeload.github.com/sshkhr/safeguarding-llms/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sshkhr%2Fsafeguarding-llms/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262950348,"owners_count":23389643,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-09-05T00:01:13.550Z","updated_at":"2025-07-01T11:07:23.858Z","avatar_url":"https://github.com/sshkhr.png","language":"Jupyter Notebook","funding_links":[],"categories":["Study resource"],"sub_categories":[],"readme":"# Workshop: A Practitioner's Guide To Safeguarding Your LLM Applications\n\n![Workshop](https://img.shields.io/badge/Workshop-TMLS%202024-blue) \n![Python](https://img.shields.io/badge/Python-3.8+-blue)\n[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://githubtocolab.com/sshkhr/safeguarding-llms/blob/main/workshop.ipynb)\n![License](https://img.shields.io/github/license/sshkhr/safeguarding-llms)\n\nWelcome to the official repository for the workshop **\"A Practitioner's Guide To Safeguarding Your LLM Applications\"** at the Toronto Machine Learning Society conference on July 11, 2024. This repository contains all the code and resources you'll need to follow along with the workshop.\n\n## Table of Contents\n\n- [Introduction](#introduction)\n- [Prior Setup](#prior-setup)\n- [Installation](#installation)\n  - [Local Machine](#local-machine)\n  - [Google Colab](#google-colab)\n- [Usage](#usage)\n- [Slides](#slides)\n- [Contributing](#contributing)\n- [Contact](#contact)\n\n## Introduction\n\nIn this workshop, we will explore safeguarding Large Language Models (LLMs) and discuss strategies for generating structured outputs, ensuring topical relevance, preventing hallucinations, avoiding data leakage, and installing safety guardrails on third-party applications accessed by LLMs. We will be using the excellent open source library, [Nemo Guardrails](https://github.com/NVIDIA/NeMo-Guardrails) for this purpose.\n\n## Prior Setup\n\nBefore you begin with the installation, please set up the following:\n\n1. **LLM Configuration:** We use gpt-3.5-turbo-instruct as our LLM for experimentation. Ensure you have set the `OPENAI_API_KEY` environment variable. This can be done in a `.env` file or directly in the code where required.\n   \n2. **Guardrails using Hosted Models:** For the guardrails, we utilized hosted HuggingFace inference endpoints. Post the TMLS workshop, these will not be available anymore:\n   - **Topic Extraction Tool:** In `04_hallucination_tools_rails`, we use the function `utils.py/def_extract_key_topic()` on Line 10 uses a question answering model from [HuggingFace T5-base model fine-tuned on QASC](https://huggingface.co/mrm8488/t5-base-finetuned-qasc). You will need to host this model yourself as an endpoint to recreate the same functionality.\n   - **Toxicity and Implicit Output Rails:** In `06_toxicity_implicit_output_rails/actions.py`, we call the Llama Guard 7B model on Line 14 from [HuggingFace Llama Guard 7B](https://huggingface.co/meta-llama/LlamaGuard-7b). You will need to apply for access to Meta and HF, and then host an inference endpoint to use this model.\n\n## Installation\n\n### Local Machine\n\nTo run the code on your local machine, follow these steps:\n\n1. **Clone the repository:**\n\n    ```bash\n    git clone https://github.com/your-repo/workshop-llm-safeguarding.git\n    cd workshop-llm-safeguarding\n    ```\n\n2. **(Optional) Create and activate a virtual environment:**\n\n    ```bash\n    python -m venv safeguarding-venv\n    source safeguarding-venv/bin/activate  \n    # On Windows use `safeguarding-venv\\Scripts\\activate`\n    ```\n\n3. **Install the required packages:**\n\n    Ensure you have Python 3.8+ installed. Then, run:\n\n    ```bash\n    pip install -r requirements.txt\n    ```\n\n    A C++ runtime is required for the `annoy` library used by `nemoguardrails`. Most computers have an instance installed, but if needed, you can follow instructions on how to install it for your platform [here](https://docs.nvidia.com/nemo/guardrails/getting_started/installation-guide.html#prerequisites).\n\n4. **Run the demo notebook:**\n\n    Start Jupyter Notebook and open `workshop.ipynb`:\n\n    ```bash\n    jupyter notebook\n    ```\n\n    Open `workshop.ipynb` and run the cells to follow along with the workshop.\n\n### Google Colab\n\nYou can also run the code in Google Colab. Follow these steps:\n\n1. **Open the repository in Colab:**\n\n    [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://githubtocolab.com/sshkhr/safeguarding-llms/blob/main/workshop.ipynb)\n\n2. **Make a copy of the notebook:**\n\n    Go to `File \u003e Save a copy in Drive` to create your own copy of the notebook.\n\n3. **Run the notebook:**\n\n    Follow the instructions in the notebook to run the code and explore the examples provided.\n\n## Usage\n\nThe primary notebook `workshop.ipynb` contains examples and exercises that will be covered during the workshop. You can modify and experiment with the code to better understand the concepts discussed.\n\n## Slides\n\nThe slides for the workshop can be accessed [here](https://docs.google.com/presentation/d/1mpyrzLCw1aqfBZtVxVJhBcnDp4iCgL0Woo4YcNa1yaM/edit?usp=sharing).\n\n## Contributing\n\nContributions to improve this repository are always welcome! If you have suggestions or find any issues, please feel free to create a pull request or open an issue.\n\n## Contact\n\nFor any questions or further information, please contact me at [shashank@dice.health](mailto:shashank@dice.health).\n\n[![X (formerly Twitter) Follow](https://img.shields.io/twitter/follow/sshkhr16)](https://twitter.com/sshkhr16)\n[![LinkedIn](https://img.shields.io/badge/LinkedIn-0077B5)](https://linkedin.com/in/sshkhr)\n[![GitHub followers](https://img.shields.io/github/followers/sshkhr)](https://github.com/sshkhr)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsshkhr%2Fsafeguarding-llms","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsshkhr%2Fsafeguarding-llms","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsshkhr%2Fsafeguarding-llms/lists"}