https://github.com/kislyuk/tractorbeam
File I/O staging for JSON documents with Amazon S3 URLs
https://github.com/kislyuk/tractorbeam
Last synced: 3 months ago
JSON representation
File I/O staging for JSON documents with Amazon S3 URLs
- Host: GitHub
- URL: https://github.com/kislyuk/tractorbeam
- Owner: kislyuk
- License: apache-2.0
- Created: 2017-01-12T22:22:11.000Z (over 8 years ago)
- Default Branch: master
- Last Pushed: 2017-01-31T01:24:05.000Z (over 8 years ago)
- Last Synced: 2025-01-17T17:59:44.020Z (5 months ago)
- Language: Python
- Homepage:
- Size: 18.6 KB
- Stars: 2
- Watchers: 3
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.rst
- Changelog: Changes.rst
- License: LICENSE
Awesome Lists containing this project
README
Tractor Beam: File I/O staging for JSON documents with Amazon S3 URLs
=====================================================================
Tractor Beam is a utility to download S3 URLs using the AWS CLI. If you have structured input (JSON) for your application, and
that input contains references to files on S3, and your application expects these files to be on a local POSIX path, then you
can pipe your JSON through ``tractor pull`` to download these URLs and rewrite the JSON to contain ``file://`` URLs instead.
You can also do the reverse with ``tractor push``.
Installation
------------
::
pip install tractorbeam
Synopsis
--------
Use ``aws configure`` to set up your AWS command line environment.
.. code-block:: bash
$ echo '{"input1": "s3://mybucket/path/to/myfile.bam"}' | tractor pull --strip-components 1
{"input1": "file:///cwd/path/to/myfile.bam"}
$ echo '{"input2": "file:///path/to/myfile.bam"}' | tractor push s3://mybucket/prefix/ --strip-components 0
{"input2": "s3://mybucket/prefix/path/to/myfile.bam"}
Authors
-------
* Andrey Kislyuk
Links
-----
* `Project home page (GitHub) `_
* `Documentation (Read the Docs) `_
* `Package distribution (PyPI) `_
* `Change log `_
Bugs
~~~~
Please report bugs, issues, feature requests, etc. on `GitHub `_.
License
-------
Licensed under the terms of the `Apache License, Version 2.0 `_.
.. image:: https://img.shields.io/travis/kislyuk/tractorbeam.svg
:target: https://travis-ci.org/kislyuk/tractorbeam
.. image:: https://codecov.io/github/kislyuk/tractorbeam/coverage.svg?branch=master
:target: https://codecov.io/github/kislyuk/tractorbeam?branch=master
.. image:: https://img.shields.io/pypi/v/tractorbeam.svg
:target: https://pypi.python.org/pypi/tractorbeam
.. image:: https://img.shields.io/pypi/l/tractorbeam.svg
:target: https://pypi.python.org/pypi/tractorbeam
.. image:: https://readthedocs.org/projects/tractorbeam/badge/?version=latest
:target: https://tractorbeam.readthedocs.io/