{"id":50032855,"url":"https://github.com/veerajsai/semshift","last_synced_at":"2026-05-25T02:01:55.996Z","repository":{"id":358745717,"uuid":"1242721755","full_name":"VeerajSai/SemShift","owner":"VeerajSai","description":"Git diff for meaning. Detect semantic shifts, claim changes, tone drift, and risk changes in text, docs, and prompts.","archived":false,"fork":false,"pushed_at":"2026-05-21T21:28:14.000Z","size":276,"stargazers_count":0,"open_issues_count":7,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-23T00:02:55.144Z","etag":null,"topics":["cli","document-review","github-actions","nlp","policy","python","semantic-diff","semantic-similarity","text-analysis","tfidf"],"latest_commit_sha":null,"homepage":"https://pypi.org/project/semshift/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/VeerajSai.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":"ROADMAP.md","authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-05-18T17:33:19.000Z","updated_at":"2026-05-21T21:26:22.000Z","dependencies_parsed_at":"2026-05-24T01:00:31.687Z","dependency_job_id":null,"html_url":"https://github.com/VeerajSai/SemShift","commit_stats":null,"previous_names":["veerajsai/semshift"],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/VeerajSai/SemShift","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VeerajSai%2FSemShift","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VeerajSai%2FSemShift/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VeerajSai%2FSemShift/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VeerajSai%2FSemShift/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/VeerajSai","download_url":"https://codeload.github.com/VeerajSai/SemShift/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VeerajSai%2FSemShift/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33417489,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-23T22:14:44.296Z","status":"ssl_error","status_checked_at":"2026-05-23T22:14:43.778Z","response_time":53,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cli","document-review","github-actions","nlp","policy","python","semantic-diff","semantic-similarity","text-analysis","tfidf"],"created_at":"2026-05-20T22:18:02.628Z","updated_at":"2026-05-24T01:00:49.042Z","avatar_url":"https://github.com/VeerajSai.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# SemShift\n\n[![PyPI](https://img.shields.io/pypi/v/semshift.svg)](https://pypi.org/project/semshift/)\n[![Python](https://img.shields.io/pypi/pyversions/semshift.svg)](https://pypi.org/project/semshift/)\n[![CI](https://github.com/VeerajSai/SemShift/actions/workflows/ci.yml/badge.svg)](https://github.com/VeerajSai/SemShift/actions/workflows/ci.yml)\n[![Security](https://github.com/VeerajSai/SemShift/actions/workflows/security.yml/badge.svg)](https://github.com/VeerajSai/SemShift/actions/workflows/security.yml)\n[![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)\n\nCatch risky meaning changes Git diff misses.\n\nSemShift is a local-first review assistant for AI-rewritten and human-edited docs, prompts, policies, resumes, and research drafts. It flags likely semantic drift before you merge, publish, or submit text.\n\nCurrent release line: `v0.2.x` alpha. The default backend is lexical + heuristic (`tfidf`). Optional SentenceTransformers embeddings are local semantic embeddings, not a claim of legal, factual, or scientific authority.\n\n## 5-Second Demo\n\nBefore:\n\n```text\nWe do not share personal data with third parties.\n```\n\nAfter:\n\n```text\nWe may share personal data with trusted partners.\n```\n\nSemShift:\n\n```text\nCRITICAL: privacy commitment weakened.\nRisk flag: third-party sharing.\nRecommendation: hold approval until a human reviews the change.\n```\n\n## Install\n\n```bash\npip install semshift\n```\n\nOptional local embedding backend:\n\n```bash\npip install \"semshift[models]\"\n```\n\nDevelopment:\n\n```bash\npip install -e \".[dev]\"\n```\n\n## Quick Start\n\n```bash\nsemshift compare examples/old_policy.md examples/new_policy.md --mode policy\nsemshift compare examples/old_policy.md examples/new_policy.md --mode policy --json\nsemshift compare examples/old_policy.md examples/new_policy.md --mode policy --report semshift-report.md\n```\n\nUse limits for large or generated files:\n\n```bash\nsemshift compare old.md new.md --max-file-size 5242880 --max-chunks 2000\n```\n\n## GitHub Action\n\n```yaml\nname: SemShift Check\n\non:\n  pull_request:\n    paths:\n      - \"**/*.md\"\n      - \"**/*.txt\"\n      - \"**/*.yml\"\n\npermissions:\n  contents: read\n  pull-requests: write\n\njobs:\n  semshift:\n    runs-on: ubuntu-latest\n    steps:\n      - uses: actions/checkout@v5\n        with:\n          fetch-depth: 0\n\n      - uses: VeerajSai/SemShift@v0.2.0\n        with:\n          mode: policy\n          fail_on: high\n          pr_comment: \"true\"\n          paths: \"docs/**,prompts/**,**/*.md,**/*.txt\"\n          exclude_paths: \".github/workflows/**\"\n          model: tfidf\n          report: semshift-report.md\n          artifact_name: semshift-policy-report\n```\n\nInputs include `files`, `paths`, `exclude_paths`, `mode`, `fail_on`, `model`, `report`, `artifact_name`, `base_ref`, `pr_comment`, `github_token`, `max_file_size`, and `max_chunks`.\n\n\u003e **Note:** `fail_on` defaults to `high`. Use `fail_on: none` for warn-only mode.\n\n## Python API\n\n```python\nfrom semshift import compare_files, compare_text\n\nresult = compare_text(\n    old=\"We do not share personal data.\",\n    new=\"We may share personal data with partners.\",\n    mode=\"policy\",\n)\n\nprint(result.drift_label)\nprint(result.summary)\nprint(result.risk_flags)\nprint(result.to_markdown())\n\nfile_result = compare_files(\"old_policy.md\", \"new_policy.md\", mode=\"policy\")\nreport = file_result.to_markdown()\n```\n\nCanonical fields include `drift_label`, `overall_score`, `drift_score`, `summary`, `matched_chunks`, `chunk_matches`, `claim_changes`, `tone_shift`, `risk_flags`, `warnings`, `metadata`, `to_dict()`, `to_json()`, and `to_markdown()`.\n\n## Modes\n\n| Mode | Maturity | Best for | Main signals |\n| --- | --- | --- | --- |\n| `policy` | stable | privacy policies, terms, consent language | sharing, retention, rights, obligations |\n| `prompt` | stable | system prompts and instruction files | safety rules, hidden instructions, scope |\n| `research` | experimental | research drafts and reports | metrics, datasets, baselines, limitations |\n| `resume` | experimental | resumes and bios | titles, metrics, company/project names |\n| `readme` | experimental | README and support docs | install requirements, guarantees, scope |\n| `default` | stable | general text review | drift score, claims, tone, generic risk |\n\n## How It Works\n\nSemShift combines transparent signals:\n\n1. Chunk alignment by headings and text structure.\n2. Lexical TF-IDF similarity by default, or optional local SentenceTransformers embeddings.\n3. Claim extraction, tone signals, and mode-specific risk rules.\n\nTF-IDF is a lexical backend, not a true semantic model. Optional embedding models may download weights on first use; document text is processed locally unless you explicitly integrate external services.\n\n## Benchmarks\n\nSemShift includes a starter self-evaluation benchmark for regression tracking. See [docs/benchmarks.md](docs/benchmarks.md).\n\nDo not treat starter benchmark numbers as independent validation. Human-labeled outside evaluation is still needed.\n\n## Compared To\n\n| Tool | What it catches | What it misses |\n| --- | --- | --- |\n| Git diff | exact text edits | risk, claims, weakened obligations |\n| diff-match-patch | text similarity | domain-specific meaning changes |\n| LLM judge | broad qualitative review | local determinism, reproducibility, privacy by default |\n| Grammar checker | style and grammar | policy, prompt, research, and factual drift |\n| SemShift | likely risky semantic drift | subtle context, truth verification, legal authority |\n\n## Limitations\n\nSemShift is:\n\n- not legal advice\n- not a fact-checker\n- not scientific authority\n- not a replacement for human review\n- likely to miss subtle context-dependent changes\n- likely to false-positive on harmless paraphrases\n- lexical + heuristic by default\n\n## Troubleshooting\n\n`semshift: command not found`: Confirm the active environment is the one where you installed `semshift`.\n\nModel import error: Install optional dependencies with `pip install \"semshift[models]\"`, or use `--model tfidf`.\n\nSlow first model run: SentenceTransformers may download weights and initialize on first use.\n\nWindows path issues: Quote paths with spaces and prefer PowerShell-compatible quoting.\n\nGitHub Action fork PRs: PR comments can be unavailable for forks with restricted permissions; the report artifact is still written.\n\nNo files matched: Pass `files` or `paths`, use `actions/checkout@v5` with `fetch-depth: 0`, or check supported extensions. Use `exclude_paths` for generated files or workflow YAML.\n\nReport too long: GitHub comments are truncated and link to the workflow run where the configured report artifact is uploaded.\n\n## Roadmap\n\n- stronger external benchmark\n- NLI-based deep mode for contradiction/entailment checks\n- VS Code extension\n- web demo\n- docs site\n- more file formats\n\n## Author\n\nBuilt by Veeraj Sai.\n\n## Citation\n\nPlease cite SemShift using [CITATION.cff](CITATION.cff).\n\n## License\n\nMIT. See [LICENSE](LICENSE).\n\n## Security\n\nReport vulnerabilities through [GitHub Security Advisories](https://github.com/VeerajSai/SemShift/security/advisories/new). SemShift is local-first by default, but optional model downloads and external CI integrations should be reviewed in your environment.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fveerajsai%2Fsemshift","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fveerajsai%2Fsemshift","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fveerajsai%2Fsemshift/lists"}