{"id":30512911,"url":"https://github.com/evalops/bandit_dspy","last_synced_at":"2026-04-15T16:31:23.324Z","repository":{"id":310744470,"uuid":"1041062633","full_name":"evalops/bandit_dspy","owner":"evalops","description":"A DSPy library for security-aware LLM development using Bandit.","archived":false,"fork":false,"pushed_at":"2025-08-20T00:59:37.000Z","size":54,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-08-20T01:23:38.901Z","etag":null,"topics":["bandit","dspy","security"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/evalops.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-08-19T23:37:54.000Z","updated_at":"2025-08-20T00:59:41.000Z","dependencies_parsed_at":"2025-08-20T01:23:40.291Z","dependency_job_id":"766431ef-8edf-447b-8515-3990ad816cc4","html_url":"https://github.com/evalops/bandit_dspy","commit_stats":null,"previous_names":["evalops/bandit_dspy"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/evalops/bandit_dspy","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/evalops%2Fbandit_dspy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/evalops%2Fbandit_dspy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/evalops%2Fbandit_dspy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/evalops%2Fbandit_dspy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/evalops","download_url":"https://codeload.github.com/evalops/bandit_dspy/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/evalops%2Fbandit_dspy/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31849689,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-15T15:24:51.572Z","status":"ssl_error","status_checked_at":"2026-04-15T15:24:39.138Z","response_time":63,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bandit","dspy","security"],"created_at":"2025-08-26T06:00:21.849Z","updated_at":"2026-04-15T16:31:23.318Z","avatar_url":"https://github.com/evalops.png","language":"Python","readme":"# bandit_dspy\n\nA production-ready library that integrates Bandit static analysis with DSPy for security-aware LLM development and optimization.\n\n## 🚀 FEATURES\n\n### Core Security Integration\n- **Security-Aware LLM Development:** Integrate static analysis directly into your DSPy optimization loops\n- **Advanced Bandit Integration:** Leverage Bandit to identify Python security vulnerabilities in generated code with caching and performance optimizations\n- **Configurable Security Metrics:** Customizable security scoring with severity/confidence weighting and penalty thresholds\n\n### Optimization Algorithms\n- **Multi-Algorithm Support:** Choose from random search, genetic algorithms, Bayesian optimization, or MIPRO\n- **GEPA Optimizer:** State-of-the-art Reflective Prompt Evolution using LLM feedback for iterative security improvements\n- **Multi-Objective Optimization:** Balance security, performance, and functionality using Pareto frontiers\n\n### Production Features\n- **Performance Optimizations:** Bounded LRU caching for faster repeated analysis\n- **Robust Error Handling:** Graceful fallbacks and a comprehensive test suite\n- **Type Hints:** Typed core APIs and mypy-friendly usage\n- **Configurable Pipeline:** Flexible train/validation splits and optimization parameters\n\n## 🔧 PREREQUISITES\n\n- Python 3.8+\n- A configured DSPy Language Model (e.g., `dspy.OllamaLocal`, `dspy.OpenAI`)\n\n## 📦 INSTALLATION\n\n### From Source\n```bash\ngit clone https://github.com/evalops/bandit_dspy.git\ncd bandit_dspy\npip install -e .\n```\n\n### Development Installation\n```bash\npip install -e .[dev]  # Includes pytest, mypy, ruff, etc.\n```\n\n## 🚦 QUICK START\n\n### Basic Security Optimization\n\n```python\nimport dspy\nfrom bandit_dspy import BanditTeleprompter, create_bandit_metric\n\nclass CodeGen(dspy.Signature):\n    \"\"\"Generate secure Python code for the given task.\"\"\"\n    description = dspy.InputField()\n    code = dspy.OutputField()\n\nclass SecureCodeGenerator(dspy.Module):\n    def __init__(self):\n        super().__init__()\n        self.predictor = dspy.Predict(CodeGen)\n    def forward(self, description):\n        return self.predictor(description=description)\n\ndspy.settings.configure(lm=dspy.OpenAI(model=\"gpt-3.5-turbo\"))\n\ntrainset = [\n    dspy.Example(description=\"hash a password securely\", code=\"import bcrypt\ndef hash_password(password): return bcrypt.hashpw(password.encode(), bcrypt.gensalt())\").with_inputs('description'),\n    dspy.Example(description=\"validate user input\", code=\"import re\ndef validate_email(email): return bool(re.match(r'^[^@]+@[^@]+\\.[^@]+$', email))\").with_inputs('description'),\n]\n\nstudent = SecureCodeGenerator()\nteleprompter = BanditTeleprompter(metric=create_bandit_metric(), optimization_method=\"genetic\", k=3, num_candidates=10)\ncompiled = teleprompter.compile(student, trainset=trainset)\nres = compiled(description=\"create a secure file upload function\")\nprint(res.code)\n```\n\n### GEPA Optimization\n\n```python\nfrom bandit_dspy import GEPATeleprompter, create_bandit_metric\n\ngepa = GEPATeleprompter(metric=create_bandit_metric(), max_iterations=4, population_size=3)\ncompiled = gepa.compile(student, trainset=trainset)\n```\n\n### Performance Monitoring\n\n```python\nimport time\nfrom bandit_dspy import BanditRunner\n\nrunner = BanditRunner(cache_maxsize=2000)\n\n# Measure cache performance\ncode = \"import os\\npassword = 'hardcoded'\"\n\n# Cold cache\nstart = time.time()\nresult1 = runner.analyze_code(code)\ncold_time = time.time() - start\n\n# Warm cache  \nstart = time.time()\nresult2 = runner.analyze_code(code)\nwarm_time = time.time() - start\n\nprint(f\"Speedup: {cold_time/warm_time:.1f}x\")\n```\n\n## 🧪 TESTING\n\n### Run All Tests\n```bash\nexport DSPY_CACHEDIR=\"$(pwd)/.dspy_cache\"  # ensure writable cache\\npytest tests/ -v\n```\n\n### Run with Coverage\n```bash\npytest tests/ --cov=bandit_dspy --cov-report=html\n```\n\n### Run Performance Tests\n```bash\npytest tests/test_performance.py -v\n```\n\n### Run GEPA Tests (requires dependencies)\n```bash\npytest tests/test_gepa_optimizer.py -v\n```\n\n## 🐛 TROUBLESHOOTING\n\n### Common Issues\n\n**Import Errors:**\n```bash\n# Install all dependencies\npip install -e .[dev]\n\n# Or install individual packages\npip install dspy-ai bandit\n```\n\n**sqlite3 OperationalError: attempt to write a readonly database**\n```bash\n# Set DSPy cache directory to a writable location\nexport DSPY_CACHEDIR=\"$(pwd)/.dspy_cache\"\n```\n\n**Memory Issues with Large Codebases:**\n```python\n# Reduce cache size\nrunner = BanditRunner(cache_maxsize=500)\n\n# Use streaming evaluation\nfor code_chunk in large_codebase:\n    result = runner.analyze_code(code_chunk)\n```\n\n**Slow Optimization:**\n```python\n# Reduce candidates for faster iteration\nteleprompter = BanditTeleprompter(\n    num_candidates=5,          # Reduce from default 10\n    optimization_method=\"random\"  # Fastest method\n)\n\n# Use smaller population for genetic algorithm\nteleprompter = BanditTeleprompter(\n    optimization_method=\"genetic\",\n    genetic_config={\"population_size\": 10, \"generations\": 5}\n)\n```\n\n## 🔬 DEVELOPMENT\n\n### Project Structure\n\n```\nbandit_dspy/\n├── src/bandit_dspy/\n│   ├── __init__.py           # Package exports and compatibility shims\n│   ├── core.py              # BanditRunner with caching and AST analysis  \n│   ├── metric.py            # SecurityMetric and create_bandit_metric factory\n│   ├── teleprompter.py      # BanditTeleprompter with multiple algorithms\n│   └── gepa_optimizer.py    # GEPA implementation for reflective optimization\n├── tests/                   # Comprehensive test suite\n│   ├── conftest.py         # Shared fixtures and test configuration\n│   ├── test_*.py           # Individual test modules\n│   └── test_gepa_*.py      # GEPA-specific tests\n├── example.py              # Quick start example\n├── pyproject.toml          # Project metadata and dependencies\n└── README.md               # This file\n```\n\n### Contributing\n\n1. Fork the repository\n2. Create a feature branch: `git checkout -b feature/amazing-feature`\n3. Install dev dependencies: `pip install -e .[dev]`\n4. Make changes and add tests\n5. Run the test suite: `export DSPY_CACHEDIR=\"$(pwd)/.dspy_cache\"  # ensure writable cache\\npytest tests/ -v`\n6. Run type checking: `mypy src/bandit_dspy`\n7. Run linting: `ruff check src/ tests/`\n8. Format code: `black src/ tests/`\n9. Commit with conventional messages: `git commit -m \"feat: add amazing feature\"`\n10. Push and create a pull request\n\n### Release Process\n\n```bash\n# Update version in pyproject.toml\n# Create release notes\ngit tag v0.1.0\ngit push origin v0.1.0\n\n# Build and publish\npython -m build\ntwine upload dist/*\n```\n\n## 📄 LICENSE\n\nThis project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.\n\n## 🔗 DEPENDENCIES\n\n### Required\n- **dspy-ai**: The DSPy framework for language model programming\n- **bandit**: Python security linter for static analysis\n\n### Optional Development\n- **pytest**: Testing framework with coverage and fixtures\n- **mypy**: Static type checking\n- **ruff**: Fast Python linter and formatter  \n- **black**: Code formatting\n- **pre-commit**: Git hooks for code quality\n\n## 📚 RESOURCES\n\n- [DSPy Documentation](https://dspy.ai/docs/) - DSPy framework guide\n- [Bandit Documentation](https://bandit.readthedocs.io/) - Security scanning tool\n- [GEPA Paper](https://arxiv.org/abs/2404.00757) - Reflective Prompt Evolution research\n- [OWASP Guidelines](https://owasp.org/www-project-top-ten/) - Security best practices\n\n## 🏆 ACKNOWLEDGMENTS\n\n- **DSPy Team** for the innovative language model programming framework\n- **Bandit Contributors** for the robust security analysis tool\n- **GEPA Researchers** for the reflective prompt evolution methodology\n\n---\n\n**Built with ❤️ for secure AI development**","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fevalops%2Fbandit_dspy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fevalops%2Fbandit_dspy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fevalops%2Fbandit_dspy/lists"}