https://github.com/andrewgross/s3browser

CLI Based Browser for S3 Buckets
https://github.com/andrewgross/s3browser

Last synced: about 1 month ago
JSON representation

CLI Based Browser for S3 Buckets

• Host: GitHub
• URL: https://github.com/andrewgross/s3browser
• Owner: andrewgross
• Created: 2016-06-26T18:51:14.000Z (about 8 years ago)
• Default Branch: master
• Last Pushed: 2016-08-12T18:09:24.000Z (about 8 years ago)
• Last Synced: 2024-07-24T18:22:28.439Z (about 2 months ago)
• Language: Python
• Size: 52.7 KB
• Stars: 14
• Watchers: 2
• Forks: 0
• Open Issues: 7
• Metadata Files:
  • Readme: README.md
  • Changelog: CHANGELOG.md

Awesome Lists containing this project

• starred-awesome - s3browser - CLI Based Browser for S3 Buckets (Python)

README

        
## S3 Browser [![Build Status](https://travis-ci.org/andrewgross/s3browser.svg?branch=master)](https://travis-ci.org/andrewgross/s3browser) [![Coverage Status](https://coveralls.io/repos/github/andrewgross/s3browser/badge.svg?branch=master)](https://coveralls.io/github/andrewgross/s3browser?branch=master)

S3Browser is a tool to help you browse your S3 Buckets like a local filesystem.  It features `cd`, `ls`, and `pwd` for now, with some added bonuses around showing rollups for directory sizes and the most recently modified file.  For now it is read only, though if you want any features feel free to suggest them.

![](https://s3.amazonaws.com/captured-pics/tty.gif)

### Installation

```

pip install s3browser

```

### Usage

You can pass access keys to `s3browser` directly, or just let it pick them up from your environment.  It uses `boto` under the hood so you can use an existing configurations for that.

```

usage: s3browser [-h] [--access-key-id ACCESS_KEY_ID]

                 [--secret-access-key SECRET_ACCESS_KEY]

Run S3Browser

optional arguments:

  -h, --help            show this help message and exit

  --access-key-id ACCESS_KEY_ID

                        AWS_ACCESS_KEY_ID used by Boto

  --secret-access-key SECRET_ACCESS_KEY

                        AWS_SECRET_ACCESS_KEY used by Boto

```

Once you are in the CLI, it will automatically load a list of all of your available S3 buckets.  You can use the `help` command to get detailed information for each of the commands.

### Gotchas

`s3browser` is written in Python, so it is not the most efficient.  For really large buckets be prepared to wait a while for it to complete.  S3 requires us to page through all the files to retrieve them, and currently that is done serially in chunks of 1000.  Additionally, the internal representation of each S3 Key is ~800 Bytes, once you use `refresh` on a bucket with millions of keys, expect some memory pressure.

I have successfully browsed ~15mm keys on my dev machine with 16GB of RAM, of which python used ~12GB.  If key retrieval speed or memory usage are big issues for you, feel free to open a ticket and we can spend the time to find better ways to implement the internal structures so they are more compact!