{"id":15588269,"url":"https://github.com/orlangure/gompress","last_synced_at":"2026-05-17T02:33:57.073Z","repository":{"id":64303383,"uuid":"239214209","full_name":"orlangure/gompress","owner":"orlangure","description":"CLI tool to compress the contents of AWS S3 bucket","archived":false,"fork":false,"pushed_at":"2020-06-23T17:31:18.000Z","size":240,"stargazers_count":2,"open_issues_count":2,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-29T09:24:06.858Z","etag":null,"topics":["aws","go","golang","gzip","s3"],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/orlangure.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-02-08T22:35:52.000Z","updated_at":"2022-05-10T01:52:56.000Z","dependencies_parsed_at":"2023-01-15T10:00:35.718Z","dependency_job_id":null,"html_url":"https://github.com/orlangure/gompress","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/orlangure/gompress","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/orlangure%2Fgompress","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/orlangure%2Fgompress/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/orlangure%2Fgompress/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/orlangure%2Fgompress/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/orlangure","download_url":"https://codeload.github.com/orlangure/gompress/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/orlangure%2Fgompress/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":281114834,"owners_count":26446040,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-26T02:00:06.575Z","response_time":61,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","go","golang","gzip","s3"],"created_at":"2024-10-02T22:22:16.683Z","updated_at":"2025-10-26T13:43:10.579Z","avatar_url":"https://github.com/orlangure.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# gompress\nCLI tool to compress the contents of AWS S3 bucket.\n\nGompress takes files from the original location, compresses them on the fly\nusing gzip, and uploads to their destination, keeping the original names (only\nadding `.gz`).\n\nGompress never keeps large files completely in memory, and never writes them to\ndisk. It streams data from source into destination, adding a compression layer\nin the middle.\n\nIt is recommended to become familiar with [AWS S3\npricing](https://aws.amazon.com/s3/pricing/) before using this utility. Moving\nfiles around costs money. To reduce the cost, use EC2 instances in the same\nregion as the buckets!\n\n## Usage\n\n### Running in a docker container:\n\n```\ndocker run --rm \\\n    -e AWS_ACCESS_KEY_ID \\\n    -e AWS_SECRET_ACCESS_KEY \\\n    orlangure/gompress \\\n    -src-region us-east-1 \\\n    -src-bucket uncompressed-files \\\n    -src-prefix some-folder/and-sub-folder \\\n    -dst-region us-east-1 \\\n    -dst-bucket compressed-files \\\n    -dst-prefix a-folder \\\n    -keep\n```\n\nAWS credentials with sufficient permissions to list, get, create and delete\nobjects are required. AWS key and secret should be provided using [environment\nvariables](https://docs.docker.com/engine/reference/commandline/run/#set-environment-variables--e---env---env-file).\n\nWhen `-keep` flag is provided, original files remain in their original\nlocation. Otherwise, when `-keep` flag is not set, original files are removed\nonce their compressed versions are successfully uploaded to the destination\npath.\n\n### Building from source (Go 1.13 or above is required):\n\n```\n$ git clone https://github.com/orlangure/gompress\n$ cd gompress\n$ go install .\n\n$ gompress\nUsage of gompress:\n  -dst-bucket string\n        target s3 bucket name\n  -dst-prefix string\n        new files will be prefixed with this value\n  -dst-region string\n        target region (default \"us-east-1\")\n  -keep\n        set to keep original files (remove by default)\n  -src-bucket string\n        source s3 bucket name\n  -src-prefix string\n        source file prefix\n  -src-region string\n        source region (default \"us-east-1\")\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Forlangure%2Fgompress","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Forlangure%2Fgompress","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Forlangure%2Fgompress/lists"}