{"id":37142591,"url":"https://github.com/techiecaro/remblob","last_synced_at":"2026-01-14T16:44:08.982Z","repository":{"id":45965605,"uuid":"421585412","full_name":"techiecaro/remblob","owner":"techiecaro","description":"Edit remote files locally","archived":false,"fork":false,"pushed_at":"2025-12-06T17:07:40.000Z","size":219,"stargazers_count":3,"open_issues_count":7,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-12-08T17:29:27.397Z","etag":null,"topics":["cli","s3"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/techiecaro.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2021-10-26T21:11:45.000Z","updated_at":"2025-08-14T23:18:41.000Z","dependencies_parsed_at":"2025-08-12T01:06:11.902Z","dependency_job_id":"1b0e2064-0942-4970-95da-fd73906df07f","html_url":"https://github.com/techiecaro/remblob","commit_stats":{"total_commits":35,"total_committers":1,"mean_commits":35.0,"dds":0.0,"last_synced_commit":"90d4e874a3b701614fbfa74adb1d14ddfdf7af27"},"previous_names":[],"tags_count":9,"template":false,"template_full_name":null,"purl":"pkg:github/techiecaro/remblob","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/techiecaro%2Fremblob","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/techiecaro%2Fremblob/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/techiecaro%2Fremblob/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/techiecaro%2Fremblob/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/techiecaro","download_url":"https://codeload.github.com/techiecaro/remblob/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/techiecaro%2Fremblob/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28426274,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-14T16:38:47.836Z","status":"ssl_error","status_checked_at":"2026-01-14T16:34:59.695Z","response_time":107,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cli","s3"],"created_at":"2026-01-14T16:44:08.404Z","updated_at":"2026-01-14T16:44:08.977Z","avatar_url":"https://github.com/techiecaro.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Remblob\n\n**Edit remote files locally with your favorite editor**\n\nRemblob is a command-line tool that makes working with remote files effortless. Download, edit in your local editor, and automatically upload changes back - all in one seamless workflow.\n\n## ✨ Features\n\n- 🎯 **Edit any remote file locally** using your preferred editor (`$EDITOR`)\n- 📊 **Parquet file support** - automatically converts to CSV for editing, preserves schema\n- 🗜️ **Smart compression handling** - automatically compress/decompress gzip files\n- ☁️ **Multiple storage backends** - AWS S3, local files, custom S3 endpoints\n- 🔄 **Round-trip integrity** - preserves data types and structure\n- 📝 **View-only mode** - inspect files without making changes\n\n## 🚀 Quick Start\n\n```bash\n# Edit a JSON file from S3\nremblob edit s3://my-bucket/config.json\n\n# Edit a parquet file (automatically converts to CSV for editing)\nremblob edit s3://data-bucket/analytics.parquet\n\n# Edit with compression\nremblob edit s3://logs/app.log.gz\n\n# Save to different location\nremblob edit local-file.json s3://backup-bucket/file.json.gz\n\n# View without editing\nremblob view s3://my-bucket/readonly.json\n```\n\n## 📋 Supported File Formats\n\n| Format | Extension | Notes |\n|--------|-----------|-------|\n| **Parquet** | `.parquet` | Converted to CSV for editing, schema preserved |\n| **JSON** | `.json` | Direct editing |\n| **Text files** | `.txt`, `.log`, etc. | Direct editing |\n| **Compressed** | `.gz` | Automatic compression/decompression |\n\n### Parquet Files\nWhen editing parquet files, remblob:\n- Converts to CSV format for easy editing in any text editor\n- Preserves the original schema and data types\n- Handles type inference and validation on save\n- Provides clear error messages for type conversion issues\n\n```bash\n# Edit parquet data as CSV\nremblob edit s3://analytics/user_events.parquet\n# Opens CSV in your editor, saves back as parquet with original schema\n```\n\n## 🛠 Installation\n\n### macOS\n```bash\nbrew install techiecaro/tap/remblob\n```\n\n### Linux/Windows\nDownload the latest release for your platform:\n```bash\n# Download and install latest release\ncurl -sf https://gobinaries.com/techiecaro/remblob | sh\n```\n\nOr download directly from [GitHub Releases](https://github.com/techiecaro/remblob/releases)\n\n### Go Install\n```bash\ngo install github.com/techiecaro/remblob@latest\n```\n\n### Build from Source\n```bash\ngit clone https://github.com/techiecaro/remblob\ncd remblob\ngo build -o remblob .\n```\n\n## ⚙️ Configuration\n\n### Environment Variables\n\n| Variable | Description | Example |\n|----------|-------------|---------|\n| `EDITOR` | Your preferred text editor \u003cbr\u003e(Needs `Wait for the files to be closed before returning` option) | `vim`, `code --wait`, `nano` |\n| `AWS_ENDPOINT_URL` | Custom S3 endpoint URL | `https://s3.us-west-2.amazonaws.com` |\n| `AWS_NO_SIGN_REQUEST` | Enable anonymous S3 access | `true` |\n\n### AWS Authentication\n\nRemblob uses standard AWS credential chain:\n\n1. **Environment variables**: `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`\n2. **AWS credentials file**: `~/.aws/credentials`\n3. **IAM roles** (when running on EC2)\n4. **AWS profiles**: Use `AWS_PROFILE=myprofile`\n\n```bash\n# Using environment variables\nexport AWS_ACCESS_KEY_ID=your-key-id\nexport AWS_SECRET_ACCESS_KEY=your-secret-key\nexport AWS_REGION=us-west-2\nremblob edit s3://my-bucket/file.json\n\n# Using AWS profiles\nAWS_PROFILE=production remblob edit s3://prod-bucket/config.json\n\n# Anonymous access (public buckets)\nAWS_NO_SIGN_REQUEST=true remblob view s3://public-bucket/data.json\n```\n\n## 📖 Usage Examples\n\n### Basic Operations\n```bash\n# Edit and save to same location\nremblob edit s3://bucket/config.json\n\n# Edit and save to different location\nremblob edit input.json s3://bucket/output.json\n\n# View without editing\nremblob view s3://bucket/readonly.json\n```\n\n### Working with Parquet Files\n```bash\n# Edit parquet file - opens as CSV in your editor\nremblob edit s3://analytics/events.parquet\n\n# Convert local CSV to parquet\nremblob edit data.csv s3://bucket/data.parquet\n\n# View parquet data as CSV\nremblob view s3://warehouse/sales.parquet\n```\n\n### Compression Examples\n```bash\n# Edit compressed file\nremblob edit s3://logs/app.log.gz\n\n# Compress while saving\nremblob edit config.json s3://backup/config.json.gz\n\n# Decompress while saving\nremblob edit s3://data/file.json.gz local-file.json\n```\n\n### Custom S3 Endpoints\n```bash\n# MinIO or other S3-compatible storage\nAWS_ENDPOINT_URL=https://minio.example.com remblob edit s3://bucket/file.json\n\n# DigitalOcean Spaces\nAWS_ENDPOINT_URL=https://nyc3.digitaloceanspaces.com remblob edit s3://space/file.json\n```\n\n## 🔧 Command Reference\n\n```\nUsage: remblob \u003ccommand\u003e\n\nCommands:\n  edit \u003csource_path\u003e [\u003cdestination_path\u003e]\n    Edit a remote file locally. If destination_path is omitted,\n    saves back to source_path.\n\n  view \u003csource_path\u003e\n    Open a remote file in read-only mode.\n\n  version [--json]\n    Show version information. Use --json for machine-readable output.\n\n  install-completions\n    Install shell completions for your shell.\n\nFlags:\n  -h, --help    Show help information\n```\n\n## 🔍 Troubleshooting\n\n### Common Issues\n\n**\"cannot convert 'value' to type\"**\n- When editing parquet files, ensure data types match the original schema\n- Check the error message for the specific row and column with issues\n\n**\"AWS authentication failed\"**\n- Verify your AWS credentials are configured correctly\n- Check that your IAM user/role has S3 permissions for the bucket\n\n**\"Editor not found\"**\n- Set the `EDITOR` environment variable: `export EDITOR=vim`\n- Or use full path: `export EDITOR=/usr/bin/nano`\n\n**\"Permission denied\"**\n- Ensure your AWS credentials have read/write access to the S3 bucket\n- For public buckets, try: `AWS_NO_SIGN_REQUEST=true remblob view s3://bucket/file`\n\n### Getting Help\n- Run `remblob --help` for usage information\n- Run `remblob version` to check your installed version\n- Check [GitHub Issues](https://github.com/techiecaro/remblob/issues) for known problems\n- Create a new issue for bugs or feature requests\n\n## 🤝 Contributing\n\nContributions are welcome! Please feel free to submit a Pull Request.\n\n## 📄 License\n\n[MIT](https://choosealicense.com/licenses/mit/)\n\n---\n\n**Made with ❤️ by [techiecaro](https://github.com/techiecaro)**\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftechiecaro%2Fremblob","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftechiecaro%2Fremblob","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftechiecaro%2Fremblob/lists"}