{"id":31637622,"url":"https://github.com/griffincancode/scantron","last_synced_at":"2026-04-05T21:01:40.818Z","repository":{"id":317798874,"uuid":"1065940565","full_name":"GriffinCanCode/scantron","owner":"GriffinCanCode","description":"A file scanner for secret management and more -- faster than the speed of light.","archived":false,"fork":false,"pushed_at":"2025-09-29T01:56:17.000Z","size":2479,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-10-03T06:08:54.421Z","etag":null,"topics":["devops","elm","git","leak-detection","password-protection","protection","rust","scan","scss","secrets","zig"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/GriffinCanCode.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-09-28T18:27:19.000Z","updated_at":"2025-09-29T01:56:20.000Z","dependencies_parsed_at":"2025-10-03T06:09:02.285Z","dependency_job_id":"048b621b-0993-44ae-8653-bda424f5f8ae","html_url":"https://github.com/GriffinCanCode/scantron","commit_stats":null,"previous_names":["griffincancode/scantron"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/GriffinCanCode/scantron","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GriffinCanCode%2Fscantron","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GriffinCanCode%2Fscantron/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GriffinCanCode%2Fscantron/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GriffinCanCode%2Fscantron/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/GriffinCanCode","download_url":"https://codeload.github.com/GriffinCanCode/scantron/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GriffinCanCode%2Fscantron/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278708074,"owners_count":26031932,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-06T02:00:05.630Z","response_time":65,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["devops","elm","git","leak-detection","password-protection","protection","rust","scan","scss","secrets","zig"],"created_at":"2025-10-07T01:38:28.130Z","updated_at":"2025-10-07T01:38:32.006Z","avatar_url":"https://github.com/GriffinCanCode.png","language":"Rust","readme":"# Scantron - Universal Code \u0026 Configuration Scanner\n\n**Grep on steroids**\n\nScantron is a sophisticated scanning engine that crawls ANY text-based files on your system and extracts structured information based on customizable patterns. Perfect for DevOps engineers managing multiple projects with different tech stacks.\n\n## Core Use Cases\n\n- **Security Audits**: Find hardcoded passwords, API keys, and secrets across all projects\n- **Migration Planning**: Identify all references to deprecated APIs, old database servers, or legacy systems\n- **Dependency Management**: Track which versions of libraries are in use across your infrastructure\n- **Compliance Reporting**: Generate audit reports for security and compliance requirements\n- **Technical Debt Analysis**: Identify code smells and problematic patterns across codebases\n\n## Architecture\n\n### Backend: Perl (Mojolicious)\n- **High-performance text processing** with parallel scanning\n- **RESTful API** for all operations\n- **WebSocket support** for real-time updates\n- **Type-safe data structures** using Type::Tiny\n- **Extensible scanner engine** with plugin architecture\n- **Job queue system** with Minion for background processing\n\n### Frontend: Elm Web UI\n- **Functional reactive programming** for reliable UI\n- **Strong typing** eliminates runtime errors  \n- **Real-time dashboard** with live scan progress\n- **Interactive result visualization** with filtering and export\n- **Responsive design** works on desktop and mobile\n\n### CLI: Rust\n- **Fast startup** and cross-platform binaries\n- **Modern argument parsing** with rich help\n- **Auto-server management** starts backend as needed\n- **Multiple output formats** (table, JSON, CSV, plain)\n- **CI/CD integration** with exit codes and quiet mode\n\n## Features\n\n### Scanning Capabilities\n- **Pattern Matching**: Regex patterns with customizable flags\n- **Data Extraction**: Parse YAML, JSON, ENV files, and more\n- **File Filtering**: Include/exclude by extension, path patterns, file size\n- **Multi-target Support**: Local files, Git repos, Docker containers, SSH remotes, S3 buckets\n- **Parallel Processing**: Configurable worker threads for fast scanning\n- **Background Job Queue**: Reliable async processing with retry and monitoring\n\n### Scan Profiles\n- **Reusable Configurations**: Save and share scanning profiles\n- **Pattern Libraries**: Built-in patterns for common security issues\n- **Custom Extractors**: Define structured data extraction rules\n- **Profile Inheritance**: Extend existing profiles with new patterns\n\n### Results \u0026 Reporting  \n- **Timeline View**: See when patterns were introduced\n- **Heatmap Visualization**: Identify problem areas at a glance\n- **Dependency Graphs**: Understand relationships between findings\n- **Export Options**: JSON, CSV, HTML reports\n- **Trend Analysis**: Track technical debt over time\n\n## Project Structure\n\n```\nscantron/\n├── backend/           # Perl/Mojolicious API server\n│   ├── lib/Scantron/\n│   │   ├── API/       # REST API controllers\n│   │   ├── Scanner/   # Core scanning engine\n│   │   ├── Types/     # Type definitions\n│   │   └── Utils/     # Utilities and storage\n│   ├── t/             # Perl tests\n│   ├── script/        # Startup scripts\n│   └── cpanfile       # Perl dependencies\n├── frontend/          # Elm web application\n│   ├── src/\n│   │   ├── Types/     # Elm type definitions\n│   │   ├── Pages/     # UI pages/views\n│   │   ├── Components/# Reusable UI components\n│   │   └── API.elm    # Backend communication\n│   ├── public/        # Static assets\n│   ├── tests/         # Elm tests\n│   └── elm.json       # Elm configuration\n├── cli/               # Rust CLI application\n│   ├── src/\n│   │   ├── cli/       # Command implementations\n│   │   ├── client.rs  # HTTP client for backend\n│   │   ├── output.rs  # Output formatters\n│   │   ├── server.rs  # Server management\n│   │   └── types.rs   # Data structures\n│   ├── Cargo.toml     # Rust dependencies\n│   └── README.md      # CLI documentation\n│   ├── security-audit.json\n│   ├── aws-migration.json\n│   └── dependency-audit.json\n└── README.md\n```\n\n## Installation \u0026 Setup\n\n### Quick Start (Recommended)\n\n```bash\n# Install all dependencies and build everything\nmake install\n\n# Build CLI and web app\nmake build\n\n# Start development servers (web UI)\nmake dev\n\n# Or use the fast CLI\n./cli/target/release/scantron --help\n```\n\n### CLI Setup (Fast Path)\n\n```bash\ncd cli\n\n# Build CLI\ncargo build --release\n\n# Install to system (optional)\ncargo install --path .\n\n# Quick security scan\n./target/release/scantron scan --profile security-audit ./\n```\n\n### Web App Setup (Full Experience)\n\n```bash\n# Backend setup\ncd backend\ncpanm --installdeps .\n./script/scantron daemon -l http://localhost:3000\n\n# Frontend setup  \ncd frontend\nnpm install -g elm\nelm make src/Main.elm --output=public/elm.js --debug\n\n# Open http://localhost:3000 in browser\n```\n\n### Production Deployment\n\n```bash\n# Build everything\nmake build\n\n# Start server and background workers\nmake start\n\n# Or manually:\n# Backend with Hypnotoad (production PSGI server)\ncd backend\n./script/scantron prefork \u0026\n\n# Start background job workers\n./script/scantron-worker --queues default \u0026\n\n# CLI is ready to use\n./cli/target/release/scantron scan --profile security-audit ./\n```\n\n### Worker Management\n\n```bash\n# Start/stop everything\nmake start    # Start server + workers\nmake stop     # Stop server + workers\n\n# Individual control\nmake start-workers  # Start job queue workers only\nmake stop-workers   # Stop job queue workers only\nmake worker-logs    # View worker logs\n```\n\n## Configuration\n\n### Environment Variables\n\n```bash\n# Optional: Custom data directory\nSCANTRON_DATA_DIR=/var/lib/scantron\n\n# Optional: Log level\nMOJO_LOG_LEVEL=info\n```\n\n### Scan Profile Example\n\n```json\n{\n  \"name\": \"Security Audit\",\n  \"description\": \"Find security issues in code\",\n  \"patterns\": [\n    {\n      \"name\": \"hardcoded-password\",\n      \"regex\": \"password\\\\s*=\\\\s*[\\\"']([^\\\"']+)[\\\"']\",\n      \"description\": \"Hardcoded passwords\"\n    },\n    {\n      \"name\": \"api-keys\",\n      \"regex\": \"api[_-]?key\\\\s*[=:]\\\\s*[\\\"']?([a-zA-Z0-9]{20,})\",\n      \"description\": \"API keys and tokens\"\n    }\n  ],\n  \"extractors\": [\n    {\n      \"type\": \"yaml\",\n      \"path\": \"database.password\",\n      \"description\": \"Extract DB passwords from YAML\"\n    }\n  ],\n  \"file_filter\": {\n    \"extensions\": [\"py\", \"js\", \"yaml\", \"json\"],\n    \"exclude_patterns\": [\"node_modules\", \"*.min.js\"],\n    \"max_size\": 1048576\n  }\n}\n```\n\n## Testing\n\n### Backend Tests\n```bash\ncd backend\nprove -l t/\n```\n\n### Frontend Tests  \n```bash\ncd frontend\nelm-test\n```\n\n### Integration Tests\n```bash\n# Start backend in test mode\ncd backend\n./script/scantron test\n\n# Run API tests\ncurl -X GET http://localhost:3000/api/v1/scans\n```\n\n## Usage Examples\n\n### CLI Quick Start\n\n```bash\n# 1. Security audit with built-in profile\nscantron scan --profile security-audit ./my-project\n\n# 2. Find AWS references for migration \nscantron scan --profile aws-migration ./infrastructure\n\n# 3. Track dependencies across projects\nscantron scan --profile dependency-audit ./\n\n# 4. Custom pattern search\nscantron scan --pattern \"TODO|FIXME\" --ext js,py,ts ./src\n\n# 5. CI/CD integration (fail if secrets found)\nscantron scan --profile security-audit --fail-on-match --quiet ./\n```\n\n### Web UI Workflows\n\n1. **Open http://localhost:3000** in your browser\n2. **Create scan profiles** or use built-in ones\n3. **Start scans** and watch real-time progress\n4. **Explore results** with interactive filtering\n5. **Export reports** for compliance and analysis\n\n### Profile Examples\n\nBuilt-in profile templates available in `backend/profile-templates/`:\n\n**Security Audit** (`security-audit.json`)\n- Hardcoded passwords and API keys\n- Weak cryptographic algorithms  \n- SQL injection patterns\n- Debug code and TODOs\n\n**AWS Migration** (`aws-migration.json`)  \n- AWS service references\n- EC2 instance IDs and S3 buckets\n- Lambda functions and IAM roles\n- CloudFormation templates\n\n**Dependency Audit** (`dependency-audit.json`)\n- NPM, pip, maven, gradle dependencies\n- Version constraints and outdated packages\n- Security vulnerability patterns\n\n## Contributing\n\nWe welcome contributions! Please see our guidelines:\n\n1. **Code Style**: Follow existing patterns and naming conventions\n2. **Testing**: Add tests for new features\n3. **Documentation**: Update README and inline docs\n4. **Type Safety**: Maintain strong typing in both Perl and Elm\n\n### Development Principles\n\n- **Minimize Tech Debt**: Every file and function should be concise but sophisticated\n- **Strong Typing**: Use Type::Tiny in Perl and native types in Elm\n- **Testability**: Design for easy testing and mocking\n- **Readability**: One-word, memorable, relevant names for all files\n- **Extensibility**: Plugin architecture for custom scanners and exporters\n\n## License\n\nMIT License - see LICENSE file for details.\n\n## Roadmap\n\n- [ ] **Plugin System**: Custom scanner plugins for specialized file types\n- [ ] **Cloud Integration**: Native support for AWS, Azure, GCP scanning  \n- [ ] **AI-Powered Patterns**: Machine learning to suggest scan patterns\n- [ ] **Continuous Monitoring**: Git hooks and CI/CD integration\n- [ ] **Team Features**: User management and shared scan profiles\n- [ ] **Advanced Visualizations**: Graph analysis and trend charts\n- [ ] **Mobile App**: Native iOS/Android apps for monitoring\n\n---\n\n**Scantron** - Making large-scale code analysis simple, fast, and beautiful.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgriffincancode%2Fscantron","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgriffincancode%2Fscantron","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgriffincancode%2Fscantron/lists"}