{"id":24201553,"url":"https://github.com/mstgnz/cdn","last_synced_at":"2025-09-22T00:31:38.031Z","repository":{"id":133800148,"uuid":"461873364","full_name":"mstgnz/cdn","owner":"mstgnz","description":"CDN Service is a high-performance, cloud-native content delivery solution built with Go.","archived":false,"fork":false,"pushed_at":"2025-06-16T17:51:22.000Z","size":7935,"stargazers_count":43,"open_issues_count":0,"forks_count":8,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-06-16T17:54:32.018Z","etag":null,"topics":["aws","cdn","docker","fiber","go","imagick","minio","s3"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mstgnz.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null},"funding":{"github":"mstgnz"}},"created_at":"2022-02-21T13:26:58.000Z","updated_at":"2025-06-16T17:51:26.000Z","dependencies_parsed_at":"2023-09-22T08:40:39.253Z","dependency_job_id":"0fa490a1-939a-426a-9d5b-b25e533ad2b7","html_url":"https://github.com/mstgnz/cdn","commit_stats":null,"previous_names":["mstgnz/cdn","mstgnz/go-minio-cdn"],"tags_count":17,"template":false,"template_full_name":null,"purl":"pkg:github/mstgnz/cdn","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mstgnz%2Fcdn","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mstgnz%2Fcdn/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mstgnz%2Fcdn/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mstgnz%2Fcdn/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mstgnz","download_url":"https://codeload.github.com/mstgnz/cdn/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mstgnz%2Fcdn/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":276328904,"owners_count":25623323,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-21T02:00:07.055Z","response_time":72,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","cdn","docker","fiber","go","imagick","minio","s3"],"created_at":"2025-01-13T21:16:20.758Z","updated_at":"2025-09-22T00:31:37.997Z","avatar_url":"https://github.com/mstgnz.png","language":"Go","funding_links":["https://github.com/sponsors/mstgnz"],"categories":[],"sub_categories":[],"readme":"# CDN Service\n\n## About\n\nCDN Service is a high-performance, cloud-native content delivery solution built with Go. It provides a robust and scalable platform for managing and delivering digital assets across multiple cloud providers. The service combines modern architectural patterns with enterprise-grade features to offer:\n\n- **Multi-Cloud Storage**: Seamless integration with MinIO and AWS S3, including Glacier support for cost-effective archival\n- **Advanced Image Processing**: Real-time image resizing and optimization with support for various formats\n- **Enterprise Features**: Circuit breaker pattern, rate limiting, and batch operations\n- **High Performance**: Redis caching, worker pools, and optimized file handling\n- **Real-Time Monitoring**: WebSocket-based live monitoring, Prometheus metrics, and comprehensive health checks\n- **Developer-Friendly**: Scalar documentation, standardized API responses, and easy deployment options\n\nPerfect for organizations needing a reliable, scalable, and feature-rich content delivery solution with support for multiple cloud providers and advanced monitoring capabilities.\n\n## Features\n\n### Storage\n\n- Multi-cloud storage support (MinIO, AWS S3)\n- Glacier archive support\n- Bucket management\n- Automatic file type detection\n- Secure file handling\n\n### Image Processing\n\n- Real-time image resizing\n- Batch processing capabilities\n- Worker pool for concurrent operations\n- Support for multiple image formats\n- URL-based image processing\n\n### Performance\n\n- Redis caching layer with optimized storage\n- Batch processing with configurable sizes\n- Worker pool for parallel processing\n- Rate limiting and request throttling\n- Performance metrics and monitoring\n- Dynamic ImageMagick version management\n- Circuit breaker pattern for fault tolerance\n  - Automatic failure detection\n  - Graceful service degradation\n  - Self-healing capabilities\n  - Configurable thresholds and timeouts\n  - Real-time state monitoring\n\n### Security\n\n- Token-based authentication\n- CORS configuration\n- Rate limiting per endpoint with bypass protection\n- Redis-based rate limit storage\n- Request size limitations\n- Trusted proxy support\n\n### Monitoring \u0026 Observability\n\n- Prometheus metrics\n- Jaeger tracing integration\n- Structured logging with zerolog\n- Health check endpoints\n- Detailed error tracking\n- Real-time system monitoring via WebSocket\n- Live performance metrics\n  - Active uploads count\n  - Upload speed\n  - Cache hit rate\n  - CPU usage\n  - Memory usage\n  - Disk usage by mount point\n  - Recent error logs\n\n### Additional Features\n\n- Environment variable configuration\n- Hot reload for configuration changes\n- Scalar documentation\n- Docker support\n- Graceful shutdown\n\n### API Standardization\n\n- Consistent response formats across all endpoints\n- Detailed error messages and codes\n- Standardized success/error patterns\n- Comprehensive request validation\n\n## Quick Start\n\n### Prerequisites\n\n- Go 1.22+\n- Docker and Docker Compose\n- MinIO Server (or AWS S3 access)\n- Redis Server\n\n### Installation\n\n1. Clone the repository:\n\n```bash\ngit clone https://github.com/mstgnz/cdn.git\ncd cdn\n```\n\n2. Copy the example environment file:\n\n```bash\ncp .env.example .env\n```\n\n3. Start the services using Docker Compose:\n\n```bash\ndocker-compose up -d\n```\n\n### Configuration\n\nEdit the `.env` file with your settings:\n\n```env\nAPP_PORT=9090\nAPP_NAME=cdn\nTOKEN=your-secure-token\n\n# MinIO Configuration\nMINIO_ENDPOINT=localhost:9000\nMINIO_ROOT_USER=minioadmin\nMINIO_ROOT_PASSWORD=minioadmin\nMINIO_USE_SSL=false\n\n# AWS Configuration (optional)\nAWS_ACCESS_KEY_ID=your-access-key\nAWS_SECRET_ACCESS_KEY=your-secret-key\nAWS_REGION=your-region\n\n# Redis Configuration\nREDIS_URL=redis://localhost:6379\n\n# Feature Flags\nDISABLE_DELETE=false\nDISABLE_UPLOAD=false\nDISABLE_GET=false\n```\n\n### API Usage\n\n#### Image Operations\n\n1. Upload an image:\n\n```bash\ncurl -X POST http://localhost:9090/upload \\\n  -H \"Authorization: your-token\" \\\n  -F \"file=@image.jpg\" \\\n  -F \"bucket=your-bucket\" \\\n  -F \"path=your/path\"\n```\n\n2. Get an image with resizing:\n\n```bash\n# Original size\nhttp://localhost:9090/your-bucket/image.jpg\n\n# Resize with width\nhttp://localhost:9090/your-bucket/w:300/image.jpg\n\n# Resize with height\nhttp://localhost:9090/your-bucket/h:200/image.jpg\n\n# Resize with both\nhttp://localhost:9090/your-bucket/w:300/h:200/image.jpg\n```\n\n3. Delete an image:\n\n```bash\ncurl -X DELETE http://localhost:9090/your-bucket/image.jpg \\\n  -H \"Authorization: your-token\"\n```\n\n#### Bucket Operations\n\n1. List buckets:\n\n```bash\ncurl http://localhost:9090/minio/bucket-list \\\n  -H \"Authorization: your-token\"\n```\n\n2. Create bucket:\n\n```bash\ncurl http://localhost:9090/minio/your-bucket/create \\\n  -H \"Authorization: your-token\"\n```\n\n### Monitoring\n\n1. Connect to WebSocket for real-time updates:\n\n```javascript\nconst ws = new WebSocket(\"ws://localhost:9090/ws\");\nws.onmessage = (event) =\u003e {\n  const stats = JSON.parse(event.data);\n  console.log(\"System stats:\", stats);\n  // Example stats data:\n  // {\n  //   \"timestamp\": \"2024-01-15T10:30:00Z\",\n  //   \"active_uploads\": 5,\n  //   \"upload_speed\": 1048576, // bytes/sec\n  //   \"cache_hit_rate\": 85.5,  // percentage\n  //   \"cpu_usage\": 45.2,       // percentage\n  //   \"memory_usage\": 60.8,    // percentage\n  //   \"disk_usage\": {\n  //     \"/data\": 75,           // percentage\n  //     \"/uploads\": 45\n  //   },\n  //   \"errors\": [\n  //     \"Failed to process image: invalid format\"\n  //   ]\n  // }\n};\n```\n\n2. Get current monitoring stats:\n\n```bash\ncurl -H \"Authorization: your-token\" http://localhost:9090/monitor\n```\n\n## Kubernetes Deployment\n\nFor production deployments, we provide comprehensive Kubernetes configurations with:\n\n- Horizontal Pod Autoscaling (3-10 pods)\n- Resource quotas and limits\n- Health monitoring and readiness probes\n- Load balancing strategies\n- Secrets management\n- Persistent volume claims\n\nFor detailed instructions, see [Kubernetes Deployment Guide](k8s/README.md)\n\n## Documentation\n\nFor detailed information, please refer to:\n\n- [Testing Guide](docs/testing.md)\n- [Troubleshooting Guide](docs/troubleshooting.md)\n- [Migration Guide](docs/migration.md)\n- [Changelog](CHANGELOG.md)\n- [Kubernetes Deployment Guide](k8s/README.md)\n- [Contributing](CONTRIBUTING.md)\n\n## License\n\nThis project is licensed under the Apache License - see the [LICENSE](LICENSE) file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmstgnz%2Fcdn","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmstgnz%2Fcdn","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmstgnz%2Fcdn/lists"}