https://github.com/chanzuckerberg/gs

A small, user-friendly Google Cloud Storage CLI client
https://github.com/chanzuckerberg/gs

cli gcp gcp-storage google-cloud google-storage

Last synced: 12 months ago
JSON representation

A small, user-friendly Google Cloud Storage CLI client

• Host: GitHub
• URL: https://github.com/chanzuckerberg/gs
• Owner: chanzuckerberg
• License: mit
• Created: 2018-08-22T03:34:47.000Z (over 7 years ago)
• Default Branch: main
• Last Pushed: 2020-07-15T21:34:52.000Z (over 5 years ago)
• Last Synced: 2025-04-10T23:43:48.879Z (12 months ago)
• Topics: cli, gcp, gcp-storage, google-cloud, google-storage
• Language: Python
• Homepage:
• Size: 140 KB
• Stars: 6
• Watchers: 2
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.rst
  • Changelog: Changes.rst
  • License: LICENSE

Awesome Lists containing this project

README

          
GS: A minimalistic Google Storage client

========================================

*gs* is a command line interface (CLI) and Python library that provides a set of essential commands for

`Google Cloud Storage `_. It is modeled after the AWS CLI's ``aws s3`` command. Its

features are:

* Python 3 compatibility

* A minimalistic set of dependencies

* A tiny footprint

* Intuitive convention-driven configuration of API credentials without browser login prompts

* Checksum validation to ensure end-to-end data integrity in uploads and downloads

* Progress bars for long-running upload and download operations

* Resumable uploads and downloads

* Multithreaded directory sync and batch delete, capable of handling large numbers of objects

* An attractive paging and table layout interface

* A JSON object metadata output mode for feeding data to other utilities

Installation

~~~~~~~~~~~~

::

   pip install gs

Synopsis

~~~~~~~~

Usage:

  ``gs [OPTIONS] COMMAND [ARGS]...``

Options:

  --version  Show the version and exit.

  --help     Show this message and exit.

Commands:

   +------------------+--------------------------------------------------+

   | ``gs configure`` | Set gs config options, including the API key.    |

   +------------------+--------------------------------------------------+

   | ``gs ls``        | List buckets or objects in a bucket/prefix.      |

   +------------------+--------------------------------------------------+

   | ``gs cp``        | Copy files to, from, or between buckets.         |

   +------------------+--------------------------------------------------+

   | ``gs mv``        | Move files to, from, or between buckets.         |

   +------------------+--------------------------------------------------+

   | ``gs mb``        | Create a new Google Storage bucket.              |

   +------------------+--------------------------------------------------+

   | ``gs rb``        | Permanently delete an empty bucket.              |

   +------------------+--------------------------------------------------+

   | ``gs rm``        | Delete objects (files) from buckets.             |

   +------------------+--------------------------------------------------+

   | ``gs sync``      | Sync a directory of files with bucket/prefix.    |

   +------------------+--------------------------------------------------+

   | ``gs api``       | Use httpie to perform a raw HTTP API request.    |

   +------------------+--------------------------------------------------+

   | ``gs presign``   | Get a pre-signed URL for accessing an object.    |

   +------------------+--------------------------------------------------+

Run ``gs configure`` to configure Google service account access credentials that will be used by the

``gs`` command. You can create a new service account key at https://console.cloud.google.com/iam-admin/serviceaccounts.

Credentials

~~~~~~~~~~~

Before making API calls, *gs* ingests API credentials in the following order of priority:

- First, *gs* checks if a ``GOOGLE_APPLICATION_CREDENTIALS`` environment variable is set. If so, it attempts to load and use

  credentials from the service account credentials filename referenced by the variable.

- If that varible is not set, *gs* attempts to load service account credentials previously configured with ``gs configure``

  (stored in ``~/.config/gs/config.json``).

- If that fails, *gs* attempts to load a service account API token from

  `Google instance metadata `_.

- If that fails, *gs* prints a warning and attempts to make API requests

  `anonymously `_.

Using the Python library interface

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code-block:: python

    from gs import GSClient

    client = GSClient()

    object_meta = client.get("b/my-bucket/o/my-object")

    with client.get("b/my-bucket/o/my-object", params=dict(alt="media"), stream=True) as res:

        object_bytes = res.raw.read()

    presigned_url = client.get_presigned_url("my-bucket", "my-object", expires_at=time.time()+3600)

Authors

-------

* Andrey Kislyuk

Links

-----

* `Project home page (GitHub) `_

* `Package distribution (PyPI) `_

* `Change log `_

* `GCB builds `_

* `Google Cloud Storage `_

Bugs

~~~~

Please report bugs, issues, feature requests, etc. on `GitHub `_.

License

-------

Licensed under the terms of the MIT License.

.. image:: https://travis-ci.com/chanzuckerberg/gs.png

   :target: https://travis-ci.com/chanzuckerberg/gs

.. image:: https://img.shields.io/pypi/v/gs.svg

   :target: https://pypi.python.org/pypi/gs

.. image:: https://img.shields.io/pypi/l/gs.svg

   :target: https://pypi.python.org/pypi/gs

.. image:: https://codecov.io/gh/chanzuckerberg/gs/branch/master/graph/badge.svg

   :target: https://codecov.io/gh/chanzuckerberg/gs