https://github.com/lucrnz/ripvex
CLI program to download & verify files (plus extracts archives!)
https://github.com/lucrnz/ripvex
cli curl downloader golang
Last synced: about 1 month ago
JSON representation
CLI program to download & verify files (plus extracts archives!)
- Host: GitHub
- URL: https://github.com/lucrnz/ripvex
- Owner: lucrnz
- License: mit
- Created: 2025-12-05T21:50:47.000Z (about 2 months ago)
- Default Branch: main
- Last Pushed: 2025-12-11T14:47:14.000Z (about 2 months ago)
- Last Synced: 2025-12-11T20:45:21.128Z (about 2 months ago)
- Topics: cli, curl, downloader, golang
- Language: Go
- Homepage:
- Size: 3.18 MB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
- Agents: AGENTS.md
Awesome Lists containing this project
README
# ripvex
A lightweight Go program for downloading files from URLs with optional hash integrity verification and archive extraction.
Designed for simplicity and easy embedding in Docker containers or CI/CD pipelines.
It supports progress reporting, human-readable byte sizes, and quiet mode for non-interactive environments.
## Disclaimer
This project is a work in progress, even though the developer makes the best effort to keep it production-ready, it does not come with a warranty on its reliability, stability and/or security.
## Features
- **Download with Progress**: Real-time progress bar showing percentage and human-readable bytes (e.g., "1.2 MB / 5.0 GB"), with configurable update intervals to prevent output spam.
- **Hash Verification**: Optional hash check against the downloaded file using SHA-256 or SHA-512—exits with code 1 on mismatch for easy CI integration. Hash values must be prefixed with the algorithm (e.g., `sha256:xxxxx...` or `sha512:xxxxx...`). When outputting to stdout (`--output -`) with hash verification, the file is stored in a temporary location, verified, and only written to stdout if the hash matches.
- **Archive Extraction**: Extract downloaded archives automatically. Supports zip, tar, tar.gz, tar.bz2, tar.xz, and tar.zstd formats.
- **Magic Byte Detection**: Archive format detection uses file magic bytes, not extensions, for reliable format identification.
- **Zip Slip Protection**: Production-ready security against path traversal attacks in archives.
- **Redirect Handling**: Automatically follows HTTP redirects up to a configurable limit (default: 30).
- **HTTP Safety**: Rejects plain HTTP unless a hash is provided or `--allow-unsafe-http` is set.
- **Quiet Mode**: Suppress all non-error output for scripts or logs.
- **Flexible Output**: Write to file (default: URL basename) or stdout (`--output -`).
- **Clean Piping**: All status messages (progress, hash verification, final messages) are written to stderr, keeping stdout clean for data piping.
- **Working Directory**: Change to a specific directory before any operation with `--chdir`.
## Usage
```sh
ripvex [flags]
```
Run `ripvex --help` for full options.
### Flags
#### Global
| Flag | Short | Description | Default |
|------|-------|-------------|---------|
| `--chdir` | `-C` | Change working directory before any operation. Panics if directory doesn't exist. | None |
| `--chdir-create` | | Create directory if it doesn't exist. Requires `--chdir`. | `false` |
| `--quiet` | `-q` | Suppress progress and final messages (ideal for CI/CD). Errors still printed to stderr. | `false` |
#### Downloader
| Flag | Short | Description | Default |
|------|-------|-------------|---------|
| `--url` | `-U` | **Required**: The URL to download (e.g., `https://example.com/file.zip`). | None |
| `--output` | `-O` | Output file path. Use `-` for stdout. Defaults to the URL's basename (or `download` if none). | URL basename |
| `--hash` | `-H` | Expected hash with algorithm prefix (e.g., `sha256:xxxxx...` or `sha512:xxxxx...`). Supported algorithms: `sha256` (64 hex chars), `sha512` (128 hex chars). Case-insensitive. Verifies file integrity; exits 1 on mismatch. In quiet mode, no success message. When used with `--output -`, the file is buffered in memory and only written to stdout after successful verification. | None |
| `--connect-timeout` | | Maximum time for connection establishment. Supports human-readable formats (e.g., `"5m"`, `"1h30m"`, `"2d"`). | `300s` |
| `--download-max-time` | `-m` | Maximum time for the download operation. Supports human-readable formats (e.g., `"1h"`, `"2d"`, `"1w"`). | `1h` |
| `--max-redirs` | | Maximum number of redirects to follow. | `30` |
| `--max-bytes` | `-M` | Maximum bytes to download (supports `k/K/KB/KiB`, `m/M/MB/MiB`, `g/G/GB/GiB`). | `4GiB` |
| `--progress-interval` | | Interval between progress updates (supports human-readable formats like `"500ms"`, `"1s"`, `"2s"`). | `400ms` |
| `--log-level` | | Log level: `debug`, `info`, `warn`, `error`. Quiet mode forces `error`. | `info` |
| `--log-format` | | Log format: `text` or `json`. JSON mode disables the visual progress bar but keeps milestone logs. | `text` |
| `--log-progress-step` | | Percent interval for milestone progress logs (1-50). | `5` |
| `--log-progress-step-unknown` | | Byte interval for progress logs when size is unknown (supports human-readable sizes like `"25MB"`, `"50MiB"`, `"100k"`). | `25MB` |
| `--allow-insecure-tls` | | Allow insecure TLS versions (1.0/1.1) with known vulnerabilities. | `false` |
| `--allow-unsafe-http` | | Allow plain HTTP without hash verification (unsafe). By default, plain HTTP requires `--hash`. | `false` |
#### Archive Extractor
| Flag | Short | Description | Default |
|------|-------|-------------|---------|
| `--extract-archive` | `-x` | Extract the downloaded archive. Format auto-detected via magic bytes. | `false` |
| `--remove-archive` | | Delete archive file after successful extraction. | `true` |
| `--extract-strip-components` | | Strip N leading components from file names during extraction. | `0` |
| `--extract-max-bytes` | | Maximum total bytes to extract from the archive. Supports the same units as `--max-bytes`. | `8GiB` |
| `--extract-timeout` | | Maximum time for archive extraction. Supports human-readable formats (e.g., `"30m"`, `"1h"`, `"2d"`). | `30m` |
#### Authorization Flags
| Flag | Short | Description | Default |
|------|-------|-------------|---------|
| `--header` | | Custom header in "Key: Value" format. Can be specified multiple times. | None |
| `--auth` | `-A` | Set Authorization header to the provided value | None |
| `--auth-bearer` | `-B` | Set Authorization header to "Bearer {value}" | None |
| `--auth-basic-user` | | Username for HTTP Basic authentication (requires `--auth-basic-pass`) | None |
| `--auth-basic-pass` | | Password for HTTP Basic authentication (requires `--auth-basic-user`) | None |
| `--auth-basic` | | Custom base64 value for Basic auth (cannot be used with `--auth-basic-user/pass`) | None |
**Note**: Only one authentication method (`--auth`, `--auth-bearer`, `--auth-basic-user/pass`, or `--auth-basic`) can be specified at a time. They are mutually exclusive.
### Supported Archive Formats
- ZIP
- TAR
- GZIP (tar.gz)
- BZIP2 (tar.bz2)
- XZ (tar.xz)
- ZSTD (tar.zstd)
### Examples
Download and extract a tarball:
```sh
ripvex -U https://example.com/archive.tar.gz -x
```
Download to a specific directory and extract:
```sh
ripvex -U https://example.com/release.zip -C /opt/app -x
```
Download with hash verification and quiet mode:
```sh
ripvex -U https://example.com/file.tar.xz -H sha256:abc123... -x -q
```
Download with SHA-512 hash verification:
```sh
ripvex -U https://example.com/file.tar.xz -H sha512:def456... -x
```
Download with an explicit limit (recommended for CI/CD):
```sh
ripvex -U https://example.com/file.bin -M 2GiB
```
Keep the archive after extraction:
```sh
ripvex -U https://example.com/data.tar.gz -x --remove-archive=false
```
Download to stdout with hash verification (buffered):
```sh
ripvex -U https://example.com/file.bin -O - -H sha256:abc123... | process-file
```
Download with custom header:
```sh
ripvex -U https://example.com/file.tar.gz --header "X-Custom: value" -x
```
Download with Bearer token authentication:
```sh
ripvex -U https://registry.example.com/file.tar.gz -B "$TOKEN" -x
```
Download with Basic authentication using credentials:
```sh
ripvex -U https://private.example.com/file.tar.gz --auth-basic-user myuser --auth-basic-pass mypass -x
```
Download with Basic authentication using pre-encoded value:
```sh
ripvex -U https://private.example.com/file.tar.gz --auth-basic "dXNlcjpwYXNz" -x
```
## Output Behavior
### Stdout vs Stderr
- **stdout**: Contains only the downloaded file data (when using `--output -`)
- **stderr**: Contains all status messages (progress, hash verification results, final messages, archive extraction logs)
This design ensures clean piping: `ripvex -U url -O - | other-tool` will only pass file data to the next command.
### Hash Algorithm Prefix
Hash values must be prefixed with the algorithm name followed by a colon:
- `sha256:` for SHA-256 (64 hex characters)
- `sha512:` for SHA-512 (128 hex characters)
Examples:
- `sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855`
- `sha512:cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83f4a921d36ce9ce47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e`
## TLS Security
By default, ripvex enforces TLS 1.2 as the minimum version for HTTPS connections. The TLS handshake will negotiate the highest mutually supported version (preferring TLS 1.3 when available, falling back to TLS 1.2).
TLS 1.0 and 1.1 have known security vulnerabilities (BEAST, POODLE, etc.) and are deprecated by major browsers and security standards.
### Connecting to Legacy Servers
If you need to connect to a server that only supports TLS 1.0/1.1:
```sh
ripvex -U https://legacy-server.example.com/file.bin --allow-insecure-tls
```
**Warning**: Only use `--allow-insecure-tls` when absolutely necessary and you understand the security implications.
## Proxy Support
ripvex respects standard proxy environment variables for HTTP and HTTPS requests. This allows seamless integration with corporate proxies or network configurations.
### Environment Variables
- `HTTP_PROXY` / `http_proxy` - Proxy server for HTTP requests (e.g., `http://proxy.example.com:8080`)
- `HTTPS_PROXY` / `https_proxy` - Proxy server for HTTPS requests (e.g., `https://proxy.example.com:8080`)
- `NO_PROXY` / `no_proxy` - Comma-separated list of hosts to exclude from proxying (e.g., `localhost,127.0.0.1,.internal`)
### Usage Examples
Using an HTTPS proxy:
```sh
export HTTPS_PROXY=http://proxy.example.com:8080
ripvex -U https://example.com/file.tar.gz -x
```
Excluding internal hosts from the proxy:
```sh
export HTTPS_PROXY=http://proxy.example.com:8080
export NO_PROXY=localhost,127.0.0.1,.internal
ripvex -U https://example.com/file.tar.gz -x
```
## License
MIT License. See [LICENSE](./LICENSE) for details.