{"id":29864567,"url":"https://github.com/savinrazvan/hydra-logger","last_synced_at":"2025-07-30T08:36:01.349Z","repository":{"id":302395845,"uuid":"1012303987","full_name":"SavinRazvan/hydra-logger","owner":"SavinRazvan","description":"Enterprise-ready Python logging library with modular architecture, zero-configuration, and exceptional performance (~108K messages/sec).","archived":false,"fork":false,"pushed_at":"2025-07-09T03:09:38.000Z","size":840,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-07-09T19:46:50.686Z","etag":null,"topics":["async","documentation","enterprise","examples","hydra-logger","library","logging","microservices","modular","multi-layer","performance","plugins","production","python","security","testing","zero-configuration"],"latest_commit_sha":null,"homepage":"https://pypi.org/project/hydra-logger/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/SavinRazvan.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-07-02T06:12:57.000Z","updated_at":"2025-07-09T03:14:29.000Z","dependencies_parsed_at":"2025-07-02T07:38:04.352Z","dependency_job_id":null,"html_url":"https://github.com/SavinRazvan/hydra-logger","commit_stats":null,"previous_names":["savinrazvan/hydra-logger"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/SavinRazvan/hydra-logger","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SavinRazvan%2Fhydra-logger","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SavinRazvan%2Fhydra-logger/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SavinRazvan%2Fhydra-logger/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SavinRazvan%2Fhydra-logger/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/SavinRazvan","download_url":"https://codeload.github.com/SavinRazvan/hydra-logger/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SavinRazvan%2Fhydra-logger/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":267836667,"owners_count":24152095,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-30T02:00:09.044Z","response_time":70,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["async","documentation","enterprise","examples","hydra-logger","library","logging","microservices","modular","multi-layer","performance","plugins","production","python","security","testing","zero-configuration"],"created_at":"2025-07-30T08:35:58.389Z","updated_at":"2025-07-30T08:36:01.340Z","avatar_url":"https://github.com/SavinRazvan.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Hydra-Logger\n\n**A modular, Python logging library with zero-configuration, comprehensive sync logging, plugins, and advanced formatting capabilities.**\n\n[![Python 3.8+](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/downloads/)\n[![License: MIT](https://img.shields.io/badge/License-MIT-white.svg)](https://opensource.org/licenses/MIT)\n[![Code Style: Black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)\n[![PyPI Downloads](https://static.pepy.tech/badge/hydra-logger)](https://pepy.tech/projects/hydra-logger)\n\n## Performance Summary\n\n**Benchmark results from comprehensive testing:**\n\n- **Development/Background Worker**: 101,236 messages/sec\n- **Production/Microservice/Web App**: 99,248-98,670 messages/sec\n- **API Service**: 87,045 messages/sec\n- **Zero memory leaks** across all tested configurations\n- **6.9x performance improvement** from default to optimized configurations\n\n**Comprehensive Benchmark Suite:**\n- **Format Testing**: Plain-text, JSON, CSV, syslog, GELF formats\n- **Security Testing**: PII detection, data sanitization, security features\n- **Plugin Testing**: Analytics, formatters, security plugins\n- **Magic Config Testing**: Production, development, microservice configurations\n- **Multi-Layer Testing**: Multiple functional layers (frontend, backend, database, security)\n- **Centralized Testing**: Single logger with multiple destinations\n- **Mixed Sync/Async Testing**: Both implementations with feature comparison\n\n*See [benchmarks/README.md](benchmarks/README.md) for detailed performance analysis.*\n\n## Core Features\n\n### **Zero Configuration**\n```python\nfrom hydra_logger import HydraLogger\n\n# Works immediately without configuration\nlogger = HydraLogger()\nlogger.info(\"Application started\")\n```\n\n### **Layer-Based Logging**\n```python\n# Organize logs by functional areas\nlogger.info(\"FRONTEND\", \"User interface updated\")\nlogger.info(\"BACKEND\", \"API endpoint called\")\nlogger.info(\"DATABASE\", \"Query executed\")\n\n# Or use centralized logging\nlogger.info(\"Application started\")\nlogger.debug(\"Configuration loaded\")\nlogger.warning(\"High memory usage detected\")\n```\n\n### **Optimized Configurations**\n```python\n# Pre-configured for specific use cases\nlogger = HydraLogger.for_production()         # 99K messages/sec\nlogger = HydraLogger.for_development()        # 101K messages/sec\nlogger = HydraLogger.for_background_worker()  # 101K messages/sec\nlogger = HydraLogger.for_microservice()       # 99K messages/sec\nlogger = HydraLogger.for_web_app()           # 98K messages/sec\nlogger = HydraLogger.for_api_service()        # 87K messages/sec\n```\n\n### **Format Customization**\n```python\n# Complete control over log format\nlogger = HydraLogger(\n    date_format=\"%Y-%m-%d\",\n    time_format=\"%H:%M:%S\",\n    logger_name_format=\"[{name}]\",\n    message_format=\"{level}: {message}\"\n)\n\nlogger.info(\"APP\", \"Custom format message\")\n# Output: [APP] INFO: Custom format message\n```\n\n### **Color Mode Control**\n```python\nconfig = {\n    \"layers\": {\n        \"APP\": {\n            \"destinations\": [\n                {\"type\": \"console\", \"color_mode\": \"always\"},  # Colored console\n                {\"type\": \"file\", \"path\": \"logs/app.log\", \"color_mode\": \"never\"}  # Plain file\n            ]\n        }\n    }\n}\n\nlogger = HydraLogger(config=config)\n```\n\n### **Async Logging (Experimental)**\n```python\nfrom hydra_logger.async_hydra import AsyncHydraLogger\nimport asyncio\n\nasync def main():\n    logger = AsyncHydraLogger()\n    await logger.initialize()\n    \n    await logger.info(\"ASYNC\", \"Async message\")\n    await logger.close()\n\nasyncio.run(main())\n```\n\n**Note**: Async logging is currently in development. File writing and some features may not work as expected. For production use, we recommend the sync HydraLogger.\n\n### **Plugin System**\n```python\nfrom hydra_logger import HydraLogger, register_plugin, AnalyticsPlugin\n\nclass CustomAnalytics(AnalyticsPlugin):\n    def process_event(self, event):\n        # Custom analytics logic\n        return {\"processed\": True}\n\nregister_plugin(\"analytics\", CustomAnalytics)\nlogger = HydraLogger(enable_plugins=True)\n```\n\n### **Security \u0026 Data Protection**\n```python\nlogger = HydraLogger(enable_security=True, enable_sanitization=True)\n\n# Automatic sensitive data masking\nlogger.info(\"AUTH\", \"Login attempt\", \n           extra={\"email\": \"user@example.com\", \"password\": \"secret123\"})\n# Output: email=***@***.com password=***\n```\n\n### **Environment Variable Support**\n```bash\nexport HYDRA_LOG_DATE_FORMAT=\"%Y-%m-%d\"\nexport HYDRA_LOG_MESSAGE_FORMAT=\"[{level}] {message}\"\nexport HYDRA_LOG_LEVEL=DEBUG\n```\n\n### **Custom Magic Config System**\n```python\nfrom hydra_logger import HydraLogger, LoggingConfig\n\n@HydraLogger.register_magic(\"my_app\")\ndef my_app_config():\n    return LoggingConfig(layers={\"APP\": LogLayer(...)})\n\nlogger = HydraLogger.for_my_app()\n```\n\n### **Performance Modes**\n```python\n# Minimal features mode for optimized performance (~14K msgs/sec)\nlogger = HydraLogger.for_minimal_features()\nlogger.info(\"PERFORMANCE\", \"Fast log message\")\n\n# Bare metal mode for maximum performance (~14K msgs/sec)\nlogger = HydraLogger.for_bare_metal()\nlogger.info(\"PERFORMANCE\", \"Bare metal log message\")\n```\n\n## Quick Start\n\n### **Installation**\n```bash\npip install hydra-logger\n```\n\n### **Basic Usage**\n```python\nfrom hydra_logger import HydraLogger\n\n# Zero configuration - works immediately\nlogger = HydraLogger()\n\n# Centralized logging (no layers)\nlogger.info(\"Application started\")\nlogger.debug(\"Configuration loaded\")\nlogger.warning(\"High memory usage detected\")\nlogger.error(\"Authentication failed\")\n\n# Or use custom layer names\nlogger.info(\"FRONTEND\", \"User interface updated\")\nlogger.info(\"BACKEND\", \"API endpoint called\")\nlogger.info(\"DATABASE\", \"Query executed\")\n```\n\n### **High-Performance Usage**\n```python\nfrom hydra_logger import HydraLogger\n\n# Choose configuration based on use case\nif is_development:\n    logger = HydraLogger.for_development()        # 101K messages/sec\nelif is_background_worker:\n    logger = HydraLogger.for_background_worker()  # 101K messages/sec\nelif is_production:\n    logger = HydraLogger.for_production()         # 99K messages/sec\nelif is_microservice:\n    logger = HydraLogger.for_microservice()       # 99K messages/sec\nelif is_web_app:\n    logger = HydraLogger.for_web_app()           # 98K messages/sec\nelse:\n    logger = HydraLogger.for_api_service()        # 87K messages/sec\n\nlogger.info(\"APP\", \"Application message\")\n```\n\n### **Advanced Configuration**\n```python\nfrom hydra_logger import HydraLogger\n\nconfig = {\n    \"layers\": {\n        \"FRONTEND\": {\n            \"level\": \"INFO\",\n            \"destinations\": [\n                {\n                    \"type\": \"console\",\n                    \"format\": \"plain-text\",\n                    \"level\": \"INFO\",\n                    \"color_mode\": \"always\"\n                },\n                {\n                    \"type\": \"file\",\n                    \"path\": \"logs/frontend.log\",\n                    \"format\": \"json\",\n                    \"level\": \"INFO\"\n                }\n            ]\n        },\n        \"BACKEND\": {\n            \"level\": \"DEBUG\",\n            \"destinations\": [\n                {\n                    \"type\": \"file\",\n                    \"path\": \"logs/backend.json\",\n                    \"format\": \"json\"\n                }\n            ]\n        }\n    }\n}\n\nlogger = HydraLogger(config=config)\nlogger.info(\"FRONTEND\", \"User interface updated\")\nlogger.debug(\"BACKEND\", \"API endpoint called\")\n```\n\n### **Async Logging (Experimental)**\n```python\nfrom hydra_logger.async_hydra import AsyncHydraLogger\nimport asyncio\n\nasync def main():\n    # Create async logger\n    logger = AsyncHydraLogger()\n    await logger.initialize()\n    \n    # Log messages asynchronously\n    await logger.info(\"ASYNC\", \"Async message\")\n    await logger.error(\"ERROR\", \"Async error\")\n    \n    # Close logger\n    await logger.close()\n\nasyncio.run(main())\n```\n\n**Note**: Async logging is currently in development. For production use, we recommend the sync HydraLogger.\n\n### **Security Features**\n```python\nfrom hydra_logger import HydraLogger\n\n# Enable security and sanitization\nlogger = HydraLogger(\n    enable_security=True,\n    enable_sanitization=True,\n    redact_sensitive=True\n)\n\n# Sensitive data is automatically redacted\nlogger.info(\"AUTH\", \"Login attempt\", \n           extra={\"email\": \"user@example.com\", \"password\": \"secret123\"})\n\n# Security features are enabled\n# Note: Security-specific logging methods are available in the async logger\n```\n\n### **Plugin System**\n```python\nfrom hydra_logger import HydraLogger, AnalyticsPlugin\n\nclass CustomAnalytics(AnalyticsPlugin):\n    def process_event(self, event):\n        # Process log events\n        return {\"processed\": True}\n    \n    def get_insights(self):\n        # Return analytics insights\n        return {\"total_events\": 100}\n\n# Register and use plugin\nlogger = HydraLogger(enable_plugins=True)\nlogger.add_plugin(\"custom_analytics\", CustomAnalytics())\n\n# Get plugin insights\ninsights = logger.get_plugin_insights()\n```\n\n### **Magic Configs**\n```python\nfrom hydra_logger import HydraLogger\n\n# Use built-in magic configs\nlogger = HydraLogger.for_production()\nlogger = HydraLogger.for_development()\nlogger = HydraLogger.for_testing()\n\n# Create custom magic config\n@HydraLogger.register_magic(\"my_app\")\ndef my_app_config():\n    return {\n    \"layers\": {\n        \"APP\": {\n                \"level\": \"INFO\",\n            \"destinations\": [\n                    {\"type\": \"console\", \"format\": \"plain-text\"}\n            ]\n        }\n    }\n}\n\n# Use custom magic config\nlogger = HydraLogger.for_my_app()\n```\n\n## Detailed Features\n\n### **Modular Architecture**\n- **Core Module**: Main logging functionality with exceptions and constants\n- **Config Module**: Configuration loaders and Pydantic models\n- **Async Module**: Async logging system (experimental)\n- **Plugin Module**: Extensible plugin architecture with registry and base classes\n- **Data Protection**: Security features and fallback mechanisms\n\n### **Format Support**\n- **Plain-Text Format**: Human-readable text with color control via color_mode\n- **JSON Format**: Structured logging for machine processing\n- **CSV Format**: Tabular data for analysis\n- **Syslog Format**: System logging compatibility\n- **GELF Format**: Graylog integration\n\n### **Color System**\n- **Auto Detection**: Automatic color detection for TTY/Jupyter\n- **Manual Control**: Force colors on/off per destination\n- **Per-Destination**: Different color settings for console vs file\n- **Smart Selection**: Intelligent formatter selection based on environment\n\n### **Security Features**\n- **PII Detection**: Email, password, API key, credit card, SSN, phone number patterns\n- **Automatic Redaction**: Configurable sensitive data masking\n- **Data Sanitization**: Input validation and output sanitization\n- **Security Validation**: Built-in security checks and validation\n- **Thread Safety**: All security operations are thread-safe\n\n### **Performance Optimization**\n- **High-Performance Mode**: Disabled expensive features for speed\n- **Bare Metal Mode**: Maximum performance optimization\n- **Buffered Operations**: High-performance file writing\n- **Memory Optimization**: Object pooling and efficient data structures\n- **Zero-Copy Logging**: Minimized data copying where possible\n\n### **Async Capabilities (Experimental)**\n- **AsyncHydraLogger**: Async logging implementation (in development)\n- **Async Handlers**: Async-compatible handlers\n- **Async Queues**: Message queue system\n- **Async Sinks**: HTTP, database, queue, and cloud destinations\n- **Async Context**: Context propagation for async applications\n\n### **Real-Time Communication (Planned)**\n- **WebSocket Bridge**: Real-time frontend-backend communication\n- **TypeScript/JavaScript SDK**: Frontend SDK for real-time logging\n- **Python Client SDK**: Backend SDK for bridge integration\n- **Real-time Log Streaming**: Live log streaming to frontend\n- **Bi-directional Communication**: Frontend-to-backend log submission\n- **Connection Management**: Automatic reconnection and health monitoring\n\n### **Plugin Architecture**\n- **AnalyticsPlugin**: Custom analytics processing\n- **FormatterPlugin**: Custom formatters\n- **SecurityPlugin**: Security features\n- **Registry System**: Plugin registration and management\n- **Lifecycle Management**: Plugin initialization and cleanup\n\n### **Magic Config System**\n- **Built-in Configs**: Production, development, testing, microservice, web app, API service, background worker\n- **Custom Registration**: `@HydraLogger.register_magic()` decorator\n- **Configuration Validation**: Error handling and validation\n- **Documentation**: Built-in help and examples\n\n### **Environment Integration**\n- **Environment Variables**: All format parameters configurable via environment\n- **Auto-Detection**: Automatic environment detection\n- **Fallback Chain**: Intelligent layer fallback mechanism\n- **Backward Compatibility**: Maintains compatibility with existing code\n\n## Development Plan\n\n### **Completed Features**\n- ✅ **Performance Optimization**: Comprehensive benchmarks and optimization\n- ✅ **Security Features**: PII detection, data sanitization, and compliance logging\n- ✅ **Magic Config System**: Extensible configuration system with built-in presets\n- ✅ **Plugin Architecture**: Extensible plugin system with registry and base classes\n- ✅ **Format Customization**: Complete control over log formats and color modes\n\n### **In Progress**\n- 🔄 **Async Logging**: Complete async implementation with data loss protection\n- 🔄 **Enhanced Color System**: Colored formatters for JSON, CSV, and syslog formats\n- 🔄 **Smart Formatter Selection**: Intelligent formatter selection based on environment\n\n### **Planned Features**\n- 📋 **Dynamic Modular Bridge**: Real-time frontend-backend communication with WebSocket bridge\n- 📋 **Plugin Marketplace**: Community plugin repository with built-in integrations\n- 📋 **Cloud Integrations**: Auto-detection for AWS, GCP, Azure environments\n- 📋 **Advanced Analytics**: Log analytics and business intelligence integration\n- 📋 **Enterprise Features**: Multi-tenant support and role-based access control\n- 📋 **Production Enhancements**: Advanced monitoring and enterprise support\n\n\n## Performance Targets\n\n**Current Performance Metrics:**\n- **Throughput**: 101,236 messages/sec (best configuration)\n- **Latency**: \u003c0.1ms average\n- **Memory**: Zero memory leaks across all configurations\n- **Startup**: ~50ms initialization time\n- **Async**: In development\n\n## Documentation\n\n- [Usage Guide](USAGE_GUIDE.md) - Comprehensive usage examples\n- [Performance Summary](PERFORMANCE_SUMMARY.md) - Latest benchmark results and performance data\n- [Strategic Plan](STRATEGIC_PLAN.md) - Development roadmap\n- [Current Status](CURRENT_STATUS.md) - Project status and progress\n- [Error Handling](ERROR_HANDLING_SUMMARY.md) - Error handling guide\n- [Roadmap](ROADMAP.md) - Future development plans\n\n## Contributing\n\nWe welcome contributions! Please see our [Contributing Guide](CONTRIBUTING.md) for details.\n\n## License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n\n---\n\n**Hydra-Logger**: A modular, Python logging library with zero-configuration, comprehensive sync logging, and advanced formatting capabilities.\n\n## Performance Recommendations\n\nTo ensure good performance with Hydra-Logger:\n\n1. **Console Logging**\n   - Console logs are written immediately with no buffering or delay. No further tuning is needed for good performance.\n\n2. **File Logging**\n   - File logging uses high-performance buffered operations with automatic flushing\n   - No additional dependencies required for good performance\n\n3. **Async Logging (Experimental)**\n   - Async logging is currently in development\n   - For production use, we recommend the sync HydraLogger\n   - When async logging is stable, it will support non-blocking operations\n\n4. **No Blocking Calls**\n   - The sync logging pipeline is fully optimized. Avoid adding any unnecessary I/O operations.\n\n5. **Error Handling**\n   - All exceptions in handlers are caught and logged. Logging failures will not crash your application.\n\n6. **Initialization and Shutdown**\n   - Handlers start and stop cleanly. No resource leaks or unawaited coroutines.\n\n7. **Performance Monitoring**\n   - Optional performance monitoring is available and adds minimal overhead.\n\n**Summary:**\n- Hydra-Logger's sync logging is production-ready, robust, and performant. Async logging is in development and should be used with caution in production environments.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsavinrazvan%2Fhydra-logger","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsavinrazvan%2Fhydra-logger","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsavinrazvan%2Fhydra-logger/lists"}