{"id":47516111,"url":"https://github.com/aatakansalar/llm-prompt-semantic-diff","last_synced_at":"2026-04-01T07:00:40.646Z","repository":{"id":302336970,"uuid":"1009837552","full_name":"aatakansalar/llm-prompt-semantic-diff","owner":"aatakansalar","description":"A CLI tool for managing and comparing LLM prompts using semantic diffing instead of traditional text-based comparison.","archived":false,"fork":false,"pushed_at":"2025-07-01T20:32:15.000Z","size":16,"stargazers_count":6,"open_issues_count":0,"forks_count":2,"subscribers_count":1,"default_branch":"master","last_synced_at":"2026-03-28T03:47:03.135Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/aatakansalar.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-06-27T19:56:58.000Z","updated_at":"2026-02-01T22:40:44.000Z","dependencies_parsed_at":"2025-07-01T20:53:12.124Z","dependency_job_id":"d4d50f3e-8aec-4fb9-8afc-7221ac489bd5","html_url":"https://github.com/aatakansalar/llm-prompt-semantic-diff","commit_stats":null,"previous_names":["aatakansalar/llm-prompt-semantic-diff"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/aatakansalar/llm-prompt-semantic-diff","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aatakansalar%2Fllm-prompt-semantic-diff","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aatakansalar%2Fllm-prompt-semantic-diff/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aatakansalar%2Fllm-prompt-semantic-diff/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aatakansalar%2Fllm-prompt-semantic-diff/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/aatakansalar","download_url":"https://codeload.github.com/aatakansalar/llm-prompt-semantic-diff/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aatakansalar%2Fllm-prompt-semantic-diff/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31269176,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-01T06:57:45.811Z","status":"ssl_error","status_checked_at":"2026-04-01T06:57:42.389Z","response_time":53,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-03-27T13:00:30.245Z","updated_at":"2026-04-01T07:00:40.640Z","avatar_url":"https://github.com/aatakansalar.png","language":"Python","funding_links":[],"categories":["\u003ca name=\"diff\"\u003e\u003c/a\u003eDiff"],"sub_categories":[],"readme":"# LLM Prompt Semantic Diff\n\nA CLI tool for managing and comparing LLM prompts using semantic diffing instead of traditional text-based comparison.\n\n[![License](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)\n[![Python](https://img.shields.io/badge/python-3.10+-blue.svg)](https://python.org)\n\n## Overview\n\nLLM Prompt Semantic Diff delivers a lightweight command‑line workflow for managing, packaging, and *semantic* diffing of Large Language Model prompts. It addresses the blind spot where ordinary text‑based `git diff` fails to reveal meaning‑level changes that materially affect model behaviour.\n\nRead More: https://medium.com/@aatakansalar/catching-prompt-regressions-before-they-ship-semantic-diffing-for-llm-workflows-feb3014ccac3\n\n## Key Features\n\n- **F-1**: `prompt init` - Generates skeletal prompt files and default manifests\n- **F-2**: `prompt pack` - Embeds prompts into `.pp.json` with semantic versioning\n- **F-3**: `prompt diff` - Semantic comparison with percentage scores and exit codes\n- **F-4**: Dual embedding providers (OpenAI cloud + SentenceTransformers local)\n- **F-5**: JSON output for CI/CD integration\n- **F-6**: Schema validation for all manifests\n- **F-7**: Comprehensive test suite with \u003e75% coverage\n\n## Installation\n\n### Install from PyPI (Recommended)\n\n```bash\npip install llm-prompt-semantic-diff\n```\n\n### Install from source\n\n```bash\ngit clone https://github.com/aatakansalar/llm-prompt-semantic-diff\ncd llm-prompt-semantic-diff\npython -m venv venv\nsource venv/bin/activate  # On Windows: venv\\Scripts\\activate\npip install -e .\n```\n\n## Quick Start\n\n### 1. Initialize a New Prompt\n\n```bash\nprompt init my-greeting\n```\n\nCreates `my-greeting.prompt` and `my-greeting.pp.json` with default structure.\n\n### 2. Package an Existing Prompt\n\n```bash\nprompt pack my-prompt.prompt\n```\n\nGenerates embeddings and creates a versioned manifest.\n\n### 3. Compare Prompt Versions\n\n```bash\n# Human-readable output\nprompt diff v1.pp.json v2.pp.json --threshold 0.8\n\n# JSON output for CI/CD\nprompt diff v1.pp.json v2.pp.json --json --threshold 0.8\n```\n\nReturns exit code 1 if similarity below threshold.\n\n### 4. Validate Manifest Schema\n\n```bash\nprompt validate my-prompt.pp.json\n```\n\n## Embedding Providers\n\n### Local (Default)\nUses SentenceTransformers with `all-MiniLM-L6-v2` model:\n```bash\nprompt pack my-prompt.prompt --provider sentence-transformers\n```\n\n### Cloud (OpenAI)\nRequires `OPENAI_API_KEY` environment variable:\n```bash\nexport OPENAI_API_KEY=\"your-api-key\"\nprompt pack my-prompt.prompt --provider openai\n```\n\n## CI/CD Integration\n\nUse `--json` flag for machine-readable output:\n\n```yaml\n- name: Check prompt changes\n  run: |\n    prompt diff main.pp.json feature.pp.json --json --threshold 0.8\n    if [ $? -eq 1 ]; then\n      echo \"Prompt changes exceed threshold - review required\"\n      exit 1\n    fi\n```\n\n## Manifest Format\n\nPrompts are packaged into `.pp.json` files:\n\n```json\n{\n  \"content\": \"Your prompt text here...\",\n  \"version\": \"0.1.0\",\n  \"embeddings\": [0.1, -0.2, 0.3, ...],\n  \"description\": \"Optional description\",\n  \"tags\": [\"category\", \"type\"],\n  \"model\": \"gpt-4\"\n}\n```\n\n## Example Workflow\n\n```bash\n# Create new prompt\nprompt init greeting\n\n# Edit greeting.prompt file\n# ... make changes ...\n\n# Package with embeddings\nprompt pack greeting.prompt\n\n# Create modified version\ncp greeting.prompt greeting-v2.prompt\n# ... make more changes ...\nprompt pack greeting-v2.prompt\n\n# Compare versions\nprompt diff greeting.pp.json greeting-v2.pp.json\n\n# Output:\n# Semantic similarity: 85.2%\n# Threshold: 80.0%\n# Above threshold: Yes\n# Version A: 0.1.0\n# Version B: 0.1.0\n```\n\n## Security \u0026 Privacy\n\n- **Local-first**: No data leaves your machine unless OpenAI provider is explicitly selected\n- **API keys**: Only read from environment variables (`OPENAI_API_KEY`)\n- **No telemetry**: No analytics, tracking, or hidden network calls\n\n## Development\n\n```bash\ngit clone https://github.com/aatakansalar/llm-prompt-semantic-diff\ncd llm-prompt-semantic-diff\npython -m venv venv\nsource venv/bin/activate  # On Windows: venv\\Scripts\\activate\npip install -e \".[dev]\"\npytest tests/ -v\n```\n\n## License\n\nLicensed under the MIT License. See [LICENSE](LICENSE) for details. \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faatakansalar%2Fllm-prompt-semantic-diff","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faatakansalar%2Fllm-prompt-semantic-diff","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faatakansalar%2Fllm-prompt-semantic-diff/lists"}