{"id":31889068,"url":"https://github.com/happyhackingspace/privacy-policy-analyzer","last_synced_at":"2025-10-13T06:59:31.460Z","repository":{"id":279492508,"uuid":"923720774","full_name":"HappyHackingSpace/privacy-policy-analyzer","owner":"HappyHackingSpace","description":"A command-line tool that automatically fetches, analyzes, and scores website privacy policies using AI to highlight strengths, risks, and recommendations.","archived":false,"fork":false,"pushed_at":"2025-09-11T20:43:41.000Z","size":811,"stargazers_count":5,"open_issues_count":3,"forks_count":2,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-09-11T23:01:25.629Z","etag":null,"topics":["privacy","security"],"latest_commit_sha":null,"homepage":"https://happyhackingspace.github.io/privacy-policy/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/HappyHackingSpace.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-01-28T18:23:27.000Z","updated_at":"2025-09-11T20:43:28.000Z","dependencies_parsed_at":null,"dependency_job_id":"dad603a9-c674-464b-97a4-56b1c51878a1","html_url":"https://github.com/HappyHackingSpace/privacy-policy-analyzer","commit_stats":null,"previous_names":["happyhackingspace/privacy-policy","happyhackingspace/privacy-policy-analyzer"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/HappyHackingSpace/privacy-policy-analyzer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HappyHackingSpace%2Fprivacy-policy-analyzer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HappyHackingSpace%2Fprivacy-policy-analyzer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HappyHackingSpace%2Fprivacy-policy-analyzer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HappyHackingSpace%2Fprivacy-policy-analyzer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/HappyHackingSpace","download_url":"https://codeload.github.com/HappyHackingSpace/privacy-policy-analyzer/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HappyHackingSpace%2Fprivacy-policy-analyzer/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279013997,"owners_count":26085430,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-13T02:00:06.723Z","response_time":61,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["privacy","security"],"created_at":"2025-10-13T06:59:30.598Z","updated_at":"2025-10-13T06:59:31.452Z","avatar_url":"https://github.com/HappyHackingSpace.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Privacy Policy Analyzer\n\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n[![Python 3.13+](https://img.shields.io/badge/python-3.13+-blue.svg)](https://www.python.org/downloads/)\n[![GitHub stars](https://img.shields.io/github/stars/HappyHackingSpace/privacy-policy-analyzer.svg)](\n  https://github.com/HappyHackingSpace/privacy-policy-analyzer/stargazers\n)\n[![GitHub forks](https://img.shields.io/github/forks/HappyHackingSpace/privacy-policy-analyzer.svg)](\n  https://github.com/HappyHackingSpace/privacy-policy-analyzer/network\n)\n\nAnalyze a website's privacy policy end-to-end: **auto-discover** the policy URL, **fetch** clean text\n(HTTP first, Selenium fallback), **chunk** the content, **evaluate** it via an LLM with a structured rubric,\nand **aggregate** category scores into an overall score with strengths, risks, red flags, and recommendations.\n\n\u003e **Part of [Happy Hacking Space](https://github.com/HappyHackingSpace) - A community-driven\n\u003e organization focused on security, AI, and software development.**\n\n## Features\n\n- **Auto-discovery**: Common paths → robots.txt/sitemaps → footer links.\n- **HTTP-first extraction**: `trafilatura` (clean text) or `BeautifulSoup` fallback; **Selenium** for dynamic pages.\n- **Structured scoring (JSON)**: Per-category (0–10) scores + rationales; aggregated to 0–100 overall in `scoring.py`.\n- **Configurable chunking**: Paragraph-aware recursive splitting; `--max-chunks` hard cap to control cost/latency.\n- **Simple CLI**: Choose `summary`, `detailed`, or `full` reports.\n\n## Project Layout\n\n```\nprivacy-policy-analyzer/\n├── src/\n│   ├── __init__.py\n│   ├── main.py                    # Main CLI application\n│   └── analyzer/\n│       ├── __init__.py\n│       ├── prompts.py             # LLM prompts for analysis\n│       └── scoring.py             # Scoring algorithms\n├── docs/                          # Documentation\n│   ├── index.md\n│   ├── user-guide.md\n│   ├── api.md\n│   ├── contributing.md\n│   └── changelog.md\n├── .github/\n│   └── workflows/                 # CI/CD pipelines\n│       ├── ci.yaml\n│       └── release.yml\n├── pyproject.toml                 # Project configuration\n├── requirements.txt               # Legacy requirements\n├── .env.example                   # Environment template\n├── .gitignore\n├── LICENSE\n└── README.md\n```\n\n## Requirements\n\n- Python **3.13+**\n- An **OpenAI API key**\n- (Optional) **Chrome/Chromium** on the machine (Selenium fallback; driver auto-installs)\n\n## Installation\n\n### Using uv (Recommended)\n\n```bash\n# Clone the repository\ngit clone https://github.com/HappyHackingSpace/privacy-policy-analyzer.git\ncd privacy-policy-analyzer\n\n# Install dependencies\nuv sync\n\n# Activate the virtual environment\nsource .venv/bin/activate  # On macOS/Linux\n# or\n.venv\\Scripts\\activate     # On Windows\n```\n\n### Using pip\n\n```bash\n# Clone the repository\ngit clone https://github.com/HappyHackingSpace/privacy-policy-analyzer.git\ncd privacy-policy-analyzer\n\n# Create virtual environment\npython -m venv venv\nsource venv/bin/activate  # On macOS/Linux\n# or\nvenv\\Scripts\\activate     # On Windows\n\n# Install dependencies\npip install -e .\n```\n\n## Configuration\n\nCopy `.env.example` → `.env` and set your credentials:\n\n```\nOPENAI_API_KEY=sk-************************\n# Optional (overrides default):\nOPENAI_MODEL=gpt-4o\n```\n\n## Usage\n\n### Quick start (auto-discovery, summary report)\n\n```bash\n# Using uv\nuv run python src/main.py --url https://www.example.com/ --report summary\n\n# Using pip\npython src/main.py --url https://www.example.com/ --report summary\n```\n\n### Detailed JSON (category scores, rationales, red flags)\n\n```bash\nuv run python src/main.py --url https://www.example.com/ --report detailed\n```\n\n### Force a specific fetch method\n\n- HTTP only (faster/cleaner when available):\n\n```bash\nuv run python src/main.py --url https://www.example.com/ --fetch http --report detailed\n```\n\n- Selenium only (for heavily dynamic pages):\n\n```bash\nuv run python src/main.py --url https://www.example.com/ --fetch selenium --report detailed\n```\n\n### Skip auto-discovery (analyze a known policy URL as-is)\n\n```bash\nuv run python src/main.py --url https://www.example.com/legal/privacy --no-discover --report detailed\n```\n\n### Tuning chunking and cost/latency\n\n```bash\n# Larger chunks = fewer requests (cheaper/faster), but slightly coarser analysis\nuv run python src/main.py --url https://example.com --chunk-size 3500 --chunk-overlap 350 --max-chunks 30 --report summary\n```\n\n## CLI Options (summary)\n\n- `--url` **(required)**: Site homepage or direct privacy policy URL.\n- `--model` *(default: env `OPENAI_MODEL` or `gpt-4o`)*: OpenAI chat model name.\n- `--fetch` *(default: `auto`)*: `auto` | `http` | `selenium`.\n- `--no-discover`: Analyze the given URL without discovery.\n- `--chunk-size` *(default: 3500)* and `--chunk-overlap` *(default: 350)*.\n- `--max-chunks` *(default: 30)*: Hard cap; tail chunks are merged to keep requests bounded.\n- `--report` *(default: `summary`)*: `summary` | `detailed` | `full`.\n\n## Output\n\n- **summary**: overall score, confidence, top strengths/risks, red-flags count.\n- **detailed**: adds per-category scores (0–10), rationales, deduped red flags,\n  recommendations.\n- **full**: includes all per-chunk JSON items along with the aggregated report.\n\n## Notes \u0026 Tips\n\n- **Determinism**: For consistent runs, pin `--fetch http` or `--fetch selenium` and/or use\n  `--no-discover` with a fixed policy URL.\n- **International sites**: The HTTP client sets `Accept-Language: en-US,en;q=0.9` to reduce\n  locale variance.\n- **Selenium**: Ensure Chrome/Chromium exists; `chromedriver-autoinstaller` will fetch a matching\n  driver automatically.\n\n## Troubleshooting\n\n- **`ImportError: lxml.html.clean ...`**\n  Ensure `lxml[html_clean]` is installed (it’s included in `requirements.txt`).\n- **Very low or inconsistent scores**\n  Try `--fetch selenium` or analyze the explicit policy URL with `--no-discover`. Some sites\n  serve different content per region/session.\n\n## Security \u0026 Ethics\n\nThis tool provides **automated analysis heuristics** and LLM-generated assessments. Treat results\nas decision support, not legal advice. Always review the original policy and consult qualified\ncounsel for compliance-critical use cases.\n\n## Documentation\n\n- 📖 **[Full Documentation](https://happyhackingspace.github.io/privacy-policy-analyzer/)** - Complete user\n  guide and API reference\n- 🚀 **[Quick Start Guide](https://happyhackingspace.github.io/privacy-policy-analyzer/user-guide/)** - Get\n  up and running quickly\n- 🔧 **[API Reference](https://happyhackingspace.github.io/privacy-policy-analyzer/api/)** - Detailed API\n  documentation\n- 🤝 **[Contributing](https://happyhackingspace.github.io/privacy-policy-analyzer/contributing/)** - How to\n  contribute to the project\n\n## About Happy Hacking Space\n\nThis project is part of [Happy Hacking Space](https://github.com/HappyHackingSpace), a\ncommunity-driven organization focused on:\n\n- 🔒 **Security Research** - Tools and techniques for security professionals\n- 🤖 **AI \u0026 Machine Learning** - Practical AI applications and research\n- 💻 **Software Development** - Open source tools and libraries\n- 🌐 **Community Building** - Bringing together developers, researchers, and security experts\n\n### Other Projects\n\n- [vulnerable-target](https://github.com/HappyHackingSpace/vulnerable-target) - Intentionally\n  vulnerable environments for security training\n- [events](https://github.com/HappyHackingSpace/events) - Community events directory\n- [site](https://github.com/HappyHackingSpace/site) - Happy Hacking Space website\n\n## Contributing\n\nWe welcome contributions! Please see our\n[Contributing Guide](https://happyhackingspace.github.io/privacy-policy-analyzer/contributing/) for details.\n\n1. Fork the repository\n2. Create a feature branch (`git checkout -b feature/amazing-feature`)\n3. Commit your changes (`git commit -m 'Add some amazing feature'`)\n4. Push to the branch (`git push origin feature/amazing-feature`)\n5. Open a Pull Request\n\n## Support\n\n- 🐛 **Bug Reports**: [GitHub Issues](https://github.com/HappyHackingSpace/privacy-policy-analyzer/issues)\n- 💬 **Discussions**: [GitHub Discussions](https://github.com/HappyHackingSpace/privacy-policy-analyzer/discussions)\n- 📧 **Contact**: [Happy Hacking Space](https://github.com/HappyHackingSpace)\n\n## License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n\n## Acknowledgments\n\n- Built with ❤️ by the [Happy Hacking Space](https://github.com/HappyHackingSpace) community\n- Powered by OpenAI's GPT models\n- Uses [trafilatura](https://github.com/adbar/trafilatura) for web content extraction\n- Inspired by the need for better privacy policy transparency\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhappyhackingspace%2Fprivacy-policy-analyzer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhappyhackingspace%2Fprivacy-policy-analyzer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhappyhackingspace%2Fprivacy-policy-analyzer/lists"}