https://github.com/lukebakken/riak-cs-loader

Script/stuff to load up data into Riak CS
https://github.com/lukebakken/riak-cs-loader

Last synced: 8 days ago
JSON representation

Script/stuff to load up data into Riak CS

• Host: GitHub
• URL: https://github.com/lukebakken/riak-cs-loader
• Owner: lukebakken
• License: unlicense
• Created: 2014-04-24T20:54:27.000Z (over 10 years ago)
• Default Branch: master
• Last Pushed: 2014-11-12T16:10:09.000Z (about 10 years ago)
• Last Synced: 2024-04-23T16:14:20.420Z (7 months ago)
• Language: Perl
• Size: 250 KB
• Stars: 0
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
#### riak-cs-loader

To use:

* Clone including submodules: `git clone --recursive git://github.com/lukebakken/riak-cs-loader.git`

* Set up local Python virtualenv: `virtualenv s3cmd-env && source s3cmd-env/bin/activate`

* Install modules: `pip install python-magic; pip install python-dateutil`

* Ensure correct auth tokens in `.s3cfg` and `.s3curl`

* Tweak data load settings in `load-s3` if necessary

* Fire it up `./load-s3`. It will communicate using `localhost:8080` as the proxy, edit `.s3cfg` to change proxy IP

#### Other utils

* `clean-s3` - lists `test-bucket` and deletes all files

* `del-s3` reads `*.log` files in the `load-s3-data` directory and deletes uploaded S3 files

* `del-pipe-s3` reads uploaded files from `load-s3.fifo` and deletes them