https://github.com/saketh1702/data-leakage-detection-in-llms

A research repository exploring potential data leakage vulnerabilities in Large Language Models (LLMs). This work analyzes existing literature, methodologies, and privacy implications in modern LLM architectures, providing comprehensive summaries and insights from various research papers.
https://github.com/saketh1702/data-leakage-detection-in-llms

data-leakage llama2 llms mistral-7b nlp

Last synced: 4 months ago
JSON representation

A research repository exploring potential data leakage vulnerabilities in Large Language Models (LLMs). This work analyzes existing literature, methodologies, and privacy implications in modern LLM architectures, providing comprehensive summaries and insights from various research papers.

• Host: GitHub
• URL: https://github.com/saketh1702/data-leakage-detection-in-llms
• Owner: Saketh1702
• Created: 2024-12-08T01:27:32.000Z (6 months ago)
• Default Branch: main
• Last Pushed: 2024-12-08T09:03:55.000Z (6 months ago)
• Last Synced: 2025-02-14T08:24:11.620Z (4 months ago)
• Topics: data-leakage, llama2, llms, mistral-7b, nlp
• Language: Jupyter Notebook
• Homepage:
• Size: 8.17 MB
• Stars: 0
• Watchers: 1
• Forks: 2
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Data Leakage Detection in LLMs

A framework for detecting data leakage and bias in LLMs (e.g., Llama-2, Mistral) using n-gram metrics and one-shot prompting. BLEURT and ROUGE-L models are used to evaluate similarity between reference and model outputs for guided and general prompts. The framework analyzes model behavior on MMLU and TruthfulQA benchmarks to identify training data memorization and gender stereotyping patterns.