{"id":30068506,"url":"https://github.com/kuberocketci/krci-cache","last_synced_at":"2026-05-18T02:33:18.661Z","repository":{"id":297417030,"uuid":"996610475","full_name":"KubeRocketCI/krci-cache","owner":"KubeRocketCI","description":"Secure Go-based caching service for KubeRocketCI pipeline artifacts with authentication, tar.gz extraction, and size limitations.","archived":false,"fork":false,"pushed_at":"2025-07-29T11:18:28.000Z","size":6064,"stargazers_count":0,"open_issues_count":0,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-07-29T11:50:45.372Z","etag":null,"topics":["cache","ci-cd","containerized","devops","echo-framework","file-upload","golang","kubernetes","kuberocketci","pipeline-artifacts"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":"chmouel/go-simple-uploader","license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/KubeRocketCI.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":"CODEOWNERS","security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-06-05T07:41:14.000Z","updated_at":"2025-07-29T11:18:31.000Z","dependencies_parsed_at":"2025-07-29T11:23:27.671Z","dependency_job_id":null,"html_url":"https://github.com/KubeRocketCI/krci-cache","commit_stats":null,"previous_names":["kuberocketci/krci-cache"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/KubeRocketCI/krci-cache","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KubeRocketCI%2Fkrci-cache","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KubeRocketCI%2Fkrci-cache/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KubeRocketCI%2Fkrci-cache/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KubeRocketCI%2Fkrci-cache/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/KubeRocketCI","download_url":"https://codeload.github.com/KubeRocketCI/krci-cache/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KubeRocketCI%2Fkrci-cache/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":269403952,"owners_count":24411317,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-08T02:00:09.200Z","response_time":72,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cache","ci-cd","containerized","devops","echo-framework","file-upload","golang","kubernetes","kuberocketci","pipeline-artifacts"],"created_at":"2025-08-08T10:13:28.252Z","updated_at":"2026-05-18T02:33:13.630Z","avatar_url":"https://github.com/KubeRocketCI.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![License](https://img.shields.io/github/license/KubeRocketCI/krci-cache)](/LICENSE)\n\n# KubeRocketCI Cache (krci-cache)\n\nA lightweight caching component for KubeRocketCI pipeline artifacts, built in Go with Echo framework.\nSimplified for memory efficiency and stability in resource-constrained environments.\n\n**Note**: This is a simplified version optimized for memory efficiency. Some advanced features have been removed to resolve OOM issues in Kubernetes environments with limited memory (512MB).\n\n## Table of Contents\n\n- [Install](#install)\n- [Configuration](#configuration)\n  - [Basic Configuration](#basic-configuration)\n  - [Production Example](#production-example)\n- [Core Features](#core-features)\n  - [Memory-Optimized Architecture](#memory-optimized-architecture)\n  - [Security Features](#security-features)\n- [Limitations](#limitations)\n  - [Tar.gz Archive Limits](#targz-archive-limits)\n  - [Removed Features](#removed-features)\n- [Usage](#usage)\n  - [API Endpoints](#api-endpoints)\n  - [Upload](#upload)\n  - [Response Format](#response-format)\n- [Container Deployment](#container-deployment)\n  - [Build](#build)\n  - [Simplified Deployment](#simplified-deployment)\n- [Setup](#setup)\n  - [Run directly](#run-directly)\n  - [Run with authentication](#run-with-authentication)\n- [API](#api)\n  - [Upload File](#upload-file)\n  - [Delete File](#delete-file)\n  - [Delete Old Files](#delete-old-files)\n- [Use Cases](#use-cases)\n- [LICENSE](#license)\n\n## Install\n\n```shell\ngo install github.com/KubeRocketCI/krci-cache\n```\n\n### Configuration\n\nConfiguration is done via environment variables:\n\n#### Basic Configuration\n\n- **UPLOADER_HOST** -- hostname to bind to (default: localhost)\n- **UPLOADER_PORT** -- port to bind to (default: 8080)\n- **UPLOADER_DIRECTORY** -- Directory where to upload (default: ./pub)\n- **UPLOADER_UPLOAD_CREDENTIALS** -- Protect upload/delete endpoints with username:password (e.g: `username:password`)\n\n#### Production Example\n\n```shell\nexport UPLOADER_HOST=\"0.0.0.0\"\nexport UPLOADER_PORT=\"8080\"\nexport UPLOADER_DIRECTORY=\"/var/cache/artifacts\"\nexport UPLOADER_UPLOAD_CREDENTIALS=\"cache-user:secure-password\"\n```\n\nThe service should be deployed behind proper authentication and access controls.\nDo not expose this directly to the internet without protection.\n\n## Core Features\n\n### Memory-Optimized Architecture\n\n- **Simplified Design**: Reduced complexity to minimize memory footprint\n- **Essential Middleware Only**: Only recovery and basic logging middleware\n- **Direct File Operations**: Streamlined upload/download without complex buffering\n- **Built-in Tar.gz Limits**: File extraction with safety limits (2GB per file, 8GB total)\n\n### Security Features\n\n- **Path Traversal Protection**: Prevents uploads outside designated directory\n- **Basic Authentication**: Optional username/password protection for sensitive endpoints\n- **Tar.gz Safety**: Built-in protection against zip bombs and malicious archives\n- **Directory Isolation**: All operations confined to the configured upload directory\n\n## Limitations\n\nThe service has been simplified for memory efficiency, with some trade-offs in functionality:\n\n### Tar.gz Archive Limits\n\n- **Individual File Size**: Maximum 2GB per file within tar.gz archives\n- **Archive Total Size**: Maximum 8GB total uncompressed size for tar.gz uploads\n- **Regular File Uploads**: No configurable size limits (limited by available disk space)\n- **Security**: Built-in protection against zip bombs, path traversal, and malicious archives\n\n### Features\n\n- Basic file upload/download\n- Tar.gz extraction with built-in size limits\n- Basic authentication (`UPLOADER_UPLOAD_CREDENTIALS`)\n- Health check endpoint (`/health`)\n- File deletion (single and batch by age)\n- Path traversal protection\n- Static file serving\n\n## Usage\n\n### API Endpoints\n\n#### Health Check\n\n- **method**: GET\n- **path**: */health*\n- **description**: Health check endpoint for load balancers and monitoring\n- **response**: JSON with status, timestamp, and version\n\n```shell\ncurl http://localhost:8080/health\n```\n\n#### Upload\n\nThe service accepts HTTP form fields:\n\n- **file**: The file stream of the upload\n- **path**: The target path for the file\n- **targz**: Set to extract tar.gz archives automatically on the filesystem\n\n#### Response Format\n\nAll endpoints return JSON responses for better integration:\n\n```json\n{\n  \"message\": \"File has been uploaded to example.txt\",\n  \"path\": \"example.txt\",\n  \"size\": 1024\n}\n```\n\n## Container Deployment\n\nThe application is containerized using a multi-architecture approach with pre-built binaries.\nThe container runs as a non-root user for security.\n\n### Build\n\nUse the provided Dockerfile which supports both amd64 and arm64 architectures:\n\n```shell\ndocker build --build-arg TARGETARCH=amd64 -t krci-cache .\n```\n\n### Simplified Deployment\n\n```yaml\n# docker-compose.yml example\nversion: '3.8'\nservices:\n  krci-cache:\n    image: krci-cache:latest\n    ports:\n      - \"8080:8080\"\n    environment:\n      - UPLOADER_HOST=0.0.0.0\n      - UPLOADER_PORT=8080\n      - UPLOADER_DIRECTORY=/var/cache/artifacts\n      - UPLOADER_UPLOAD_CREDENTIALS=cache-user:secure-password\n    volumes:\n      - ./cache:/var/cache/artifacts\n    restart: unless-stopped\n    healthcheck:\n      test: [\"CMD\", \"curl\", \"-f\", \"http://localhost:8080/health\"]\n      interval: 30s\n      timeout: 10s\n      retries: 3\n```\n\n## Setup\n\n### Run directly\n\nYou can run the service directly or use containerization. The default upload credentials are `username:password` for the `/upload` and `/delete` endpoints.\n\nFor production deployment, set proper credentials via the `UPLOADER_UPLOAD_CREDENTIALS` environment variable.\n\n### Run with authentication\n\nSet up authentication credentials:\n\n```shell\nexport UPLOADER_UPLOAD_CREDENTIALS=\"username:password\"\n./krci-cache\n```\n\nTest the deployment:\n\n```shell\necho \"HELLO WORLD\" \u003e /tmp/hello.txt\ncurl -u username:password -F path=hello-upload.txt -X POST -F file=@/tmp/hello.txt http://localhost:8080/upload\ncurl http://localhost:8080/hello-upload.txt\n```\n\n### API\n\n#### Upload File\n\n- **method**: POST\n- **path**: */upload*\n- **arguments**:\n  - **path**: Target path for the file (relative to upload directory, directory traversal prevented)\n  - **file**: File post data (no size limits, limited by available disk space)\n  - **targz**: Boolean flag to extract tar.gz files on filesystem (tar.gz uploads subject to built-in size limits: max 2GB per file, 8GB total)\n\n- **examples**:\n\n```shell\n# Regular file upload (no size limits)\ncurl -u username:password -F path=hello-upload.txt -X POST -F file=@/tmp/hello.txt http://localhost:8080/upload\n```\n\n```shell\n# Large file upload (limited by disk space)\ncurl -u username:password -F path=large-database.sql -X POST -F file=@/path/to/large-database.sql http://localhost:8080/upload\n```\n\n```shell\n# Extract tar.gz automatically (max 2GB per file, 8GB total uncompressed)\ntar czf - /path/to/directory|curl -u username:password -F path=hello-upload.txt -F targz=true -X POST -F file=@- http://localhost:8080/upload\n```\n\n### Delete File\n\n- **method**: DELETE\n- **path**: */upload*\n- **arguments**:\n  - **path**: Path to delete\n\n- **example**:\n\n```shell\ncurl -u username:password -F path=hello-upload.txt -X DELETE http://localhost:8080/upload\n```\n\n### Delete Old Files\n\n- **method**: DELETE\n- **path**: */delete*\n- **arguments**:\n  - **path**: Directory path to clean up\n  - **days**: Delete files older than X days\n  - **recursive**: Recursively delete in subdirectories (defaults to `false`)\n\n- **example**:\n\n```shell\ncurl -u username:password -F path=/path/to/directory -F days=1 -F recursive=true -X DELETE http://localhost:8080/delete\n```\n\n## Use Cases\n\n### 1. Simple CI/CD Artifact Storage\n\nStore and manage build artifacts with basic cleanup functionality.\n\n**Upload build artifacts with extraction:**\n\n```bash\n# Upload and extract build artifacts to versioned directory\ncurl -u cache-user:secure-password \\\n  -X POST \\\n  -F \"file=@app-v1.2.3.tar.gz\" \\\n  -F \"targz=true\" \\\n  -F \"path=builds/v1.2.3/\" \\\n  http://cache-server:8080/upload\n```\n\n**Download specific artifacts:**\n\n```bash\n# Access extracted binary\ncurl http://cache-server:8080/builds/v1.2.3/bin/application\n\n# Check if artifact exists\ncurl -I http://cache-server:8080/builds/v1.2.3/config.json\n```\n\n**Cleanup old build artifacts:**\n\n```bash\n# Remove builds older than 30 days\ncurl -u cache-user:secure-password \\\n  -X DELETE \\\n  \"http://cache-server:8080/delete?path=builds\u0026days=30\u0026recursive=true\"\n```\n\n### 2. Basic File Storage\n\nSimple file upload and retrieval for small to medium-scale applications.\n\n**Upload files:**\n\n```bash\n# Upload application logs\ncurl -u cache-user:secure-password \\\n  -X POST \\\n  -F \"file=@application.log\" \\\n  -F \"path=logs/application-2024-01-15.log\" \\\n  http://cache-server:8080/upload\n```\n\n**Basic retention:**\n\n```bash\n# Remove old files after 7 days\ncurl -u cache-user:secure-password \\\n  -X DELETE \\\n  \"http://cache-server:8080/delete?path=logs\u0026days=7\u0026recursive=true\"\n```\n\n### 3. Static Content Hosting\n\nHost simple static websites and documentation.\n\n**Deploy documentation:**\n\n```bash\n# Upload and extract documentation site\ntar czf docs.tar.gz -C /path/to/docs .\ncurl -u cache-user:secure-password \\\n  -X POST \\\n  -F \"file=@docs.tar.gz\" \\\n  -F \"targz=true\" \\\n  -F \"path=docs/v2.1/\" \\\n  http://cache-server:8080/upload\n```\n\n**Access content:**\n\n```bash\n# Browse documentation\ncurl http://cache-server:8080/docs/v2.1/index.html\n```\n\n## [LICENSE](LICENSE)\n\n[Apache 2.0](LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkuberocketci%2Fkrci-cache","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkuberocketci%2Fkrci-cache","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkuberocketci%2Fkrci-cache/lists"}