https://github.com/m3dev/thunderbolt

gokart file manager
https://github.com/m3dev/thunderbolt

Last synced: 2 days ago
JSON representation

gokart file manager

• Host: GitHub
• URL: https://github.com/m3dev/thunderbolt
• Owner: m3dev
• License: mit
• Created: 2019-08-18T00:49:39.000Z (about 5 years ago)
• Default Branch: master
• Last Pushed: 2024-02-26T12:44:51.000Z (9 months ago)
• Last Synced: 2024-07-27T04:24:26.220Z (4 months ago)
• Language: Python
• Size: 240 KB
• Stars: 23
• Watchers: 6
• Forks: 10
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Thunderbolt

[![Test](https://github.com/m3dev/thunderbolt/workflows/Test/badge.svg)](https://github.com/m3dev/thunderbolt/actions?query=workflow%3ATest)

[![Python Versions](https://img.shields.io/pypi/pyversions/thunderbolt.svg)](https://pypi.org/project/thunderbolt/)

[![](https://img.shields.io/pypi/v/thunderbolt)](https://pypi.org/project/thunderbolt/)

![](https://img.shields.io/pypi/l/thunderbolt)

Thunderbolt is data manager for gokart.

1. Auto loading gokart task logs

1. Check task params using pandas

1. Download data from python

# Usage

### install

```shell

pip install thunderbolt

```

### Example

If you specify `TASK_WORKSPACE_DIRECTORY`, thunderbolt reads the log.

So making tasks pandas.DataFrame, and load dumped data.

This is also possible from S3 or GCS. (s3://, gs://)

Example:

```python

from thunderbolt import Thunderbolt

tb = Thunderbolt()

print(tb.get_task_df())

print(tb.get_data('TASK_NAME'))

```

Please look here too: https://github.com/m3dev/thunderbolt/blob/master/examples/example.ipynb

# Thanks

- `gokart`: https://github.com/m3dev/gokart