{"id":21302189,"url":"https://github.com/riveryio/beeperpurge","last_synced_at":"2025-09-08T16:42:24.515Z","repository":{"id":260791903,"uuid":"882288961","full_name":"RiveryIO/BeeperPurge","owner":"RiveryIO","description":"A robust, high-performance tool for safely cleaning old files from directories. Designed for high-throughput environments and sensitive file systems.","archived":false,"fork":false,"pushed_at":"2025-04-04T13:18:01.000Z","size":84,"stargazers_count":5,"open_issues_count":0,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-05-07T14:06:07.165Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/RiveryIO.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":"CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-11-02T12:22:09.000Z","updated_at":"2024-12-10T12:06:59.000Z","dependencies_parsed_at":"2024-11-02T16:20:03.418Z","dependency_job_id":"7cd4defe-6aa6-470b-9c83-19b68185aa3a","html_url":"https://github.com/RiveryIO/BeeperPurge","commit_stats":null,"previous_names":["riveryio/beeperpurge"],"tags_count":20,"template":false,"template_full_name":null,"purl":"pkg:github/RiveryIO/BeeperPurge","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RiveryIO%2FBeeperPurge","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RiveryIO%2FBeeperPurge/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RiveryIO%2FBeeperPurge/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RiveryIO%2FBeeperPurge/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/RiveryIO","download_url":"https://codeload.github.com/RiveryIO/BeeperPurge/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RiveryIO%2FBeeperPurge/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":264892282,"owners_count":23679264,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-21T15:55:14.732Z","updated_at":"2025-07-11T20:31:30.256Z","avatar_url":"https://github.com/RiveryIO.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# BeeperPurge 🧹\n\n[![Python Tests on Release branch (main)](https://github.com/RiveryIO/BeeperPurge/actions/workflows/python-ci-main.yaml/badge.svg?branch=main)](https://github.com/RiveryIO/BeeperPurge/actions/workflows/python-ci-main.yaml)\n[![codecov](https://codecov.io/gh/RiveryIO/BeeperPurge/branch/main/graph/badge.svg?token=YOUR_TOKEN)](https://codecov.io/gh/RiveryIO/BeeperPurge)\n\nHigh-throughput parallel file system cleaner designed for efficiently eliminating millions of old files as close to simultaneously as possible.\n\n## Features\n\n- 🚀 Parallel processing with multi-threading\n- 🎯 Precision targeting of files by age\n- 🔍 Dry-run mode for operation verification\n- 📝 Kubernetes-friendly JSON logging\n- 🔒 Safe handling of sensitive file systems\n- ⚙️ Configurable age thresholds\n- 🐳 Production-ready container with security best practices\n\n## Installation and Running\n\n### Running as a Kubernetes Cron Job (Recommended)\nTo automate regular cleanups using beeper-purge in Kubernetes, you can configure a Kubernetes CronJob that runs at a specified interval. This example mounts an existing PersistentVolumeClaim (PVC) to the cron job container.\n\nCreate a CronJob Manifest: Replace /data with your target path in the volume and adjust schedule and other parameters as needed.\n\n```yaml\napiVersion: batch/v1\nkind: CronJob\nmetadata:\n  name: beeper-purge-cron\nspec:\n  schedule: \"0 0 * * *\"  # Run daily at midnight\n  jobTemplate:\n    spec:\n      template:\n        spec:\n          containers:\n            - name: beeper-purge\n              image: ghcr.io/RiveryIO/beeper-purge:latest\n              args: \n                - \"/data\"\n                - \"--max-age-hours\"\n                - \"36\"\n              volumeMounts:\n                - name: data-volume\n                  mountPath: /data\n          restartPolicy: OnFailure\n          volumes:\n            - name: data-volume\n              persistentVolumeClaim:\n                claimName: your-existing-pvc-name  # Replace with your PVC name\n\n```\n\n### Using Docker\n\n```bash\ndocker pull ghcr.io/RiveryIO/beeper-purge:latest\n\n# Always verify targets first with dry run\ndocker run -v /path/to/clean:/data ghcr.io/RiveryIO/beeper-purge:latest \\\n    /data --dry-run --max-age-hours 36\n\n# Execute purge operation\ndocker run -v /path/to/clean:/data ghcr.io/RiveryIO/beeper-purge:latest \\\n    /data --max-age-hours 36\n```\n\n### Using pip\n\n```bash\npip install beeper-purge\n```\n\n## Usage\n\n```bash\n# Show help\nbeeperpurge --help\n\n# Reconnaissance (dry run)\nbeeperpurge /path/to/clean --dry-run --max-age-hours 36\n\n# Execute purge\nbeeperpurge /path/to/clean --max-age-hours 36 --workers 16\n\n# Show version\nbeeperpurge --version\n```\n\n## Operational Metrics\n\n```bash\n$ beeperpurge /data --dry-run\n{\n    \"timestamp\": \"2024-11-02T10:15:30,123\",\n    \"level\": \"INFO\",\n    \"message\": \"Starting purge operation\",\n    \"extra_fields\": {\n        \"root_path\": \"/data\",\n        \"dry_run\": true,\n        \"max_workers\": 16\n    }\n}\n...\n{\n    \"timestamp\": \"2024-11-02T10:15:35,456\",\n    \"level\": \"INFO\",\n    \"message\": \"Operation completed\",\n    \"extra_fields\": {\n        \"files_processed\": 1000000,\n        \"files_targeted\": 150000,\n        \"duration_seconds\": 5.33,\n        \"elimination_rate\": 187617\n    }\n}\n```\n\n## Safety Protocols\n\n- 🛡️ Dry-run mode for target verification\n- 🔗 No symlink following\n- 🚨 Comprehensive error handling\n- 👤 Non-root container execution\n- ✅ Extensive test coverage\n\n## Performance Specifications\n\n### Scalability\n- Efficiently handles millions of files\n- Memory usage scales linearly with worker count\n- I/O optimized operations\n\n### Recommended Configurations\n- Standard systems: 8-16 workers\n- High-performance systems: 16-32 workers\n- Adjust based on:\n  - Available CPU cores\n  - I/O capabilities\n  - File system response times\n\n## Development\n\n### Setup\n\n```bash\n# Clone repository\ngit clone https://github.com/RiveryIO/BeeperPurge.git\ncd beeperpurge\n\n# Create virtual environment\npython -m venv venv\nsource venv/bin/activate  # or `venv\\Scripts\\activate` on Windows\n\n# Install development dependencies\npip install -e \".[dev]\"\n```\n\n### Testing\n\n```bash\n# Full test suite\npytest\n\n# Coverage analysis\npytest --cov=beeper_purge\n\n# Specific test execution\npytest tests/test_cleaner.py\n```\n\n### Container Build\n\n```bash\ndocker build -t beeper-purge .\n```\n\n## Contributing\n\n1. Fork the repository\n2. Create your feature branch (`git checkout -b feat/enhancement`). Valid branch prefixes are feat,fix,chore.\n3. Commit your changes (`git commit -m 'Add enhancement'`)\n4. Push to the branch (`git push origin feat/enhancement`)\n5. Open a Pull Request\n\n## License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Friveryio%2Fbeeperpurge","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Friveryio%2Fbeeperpurge","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Friveryio%2Fbeeperpurge/lists"}