Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/snek-at/pit
Snek Pit allows users to create and manage parquet files using DuckDB and Pandas.
https://github.com/snek-at/pit
Last synced: about 1 month ago
JSON representation
Snek Pit allows users to create and manage parquet files using DuckDB and Pandas.
- Host: GitHub
- URL: https://github.com/snek-at/pit
- Owner: snek-at
- License: other
- Created: 2023-01-20T10:28:52.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2024-04-27T12:22:40.000Z (8 months ago)
- Last Synced: 2024-04-27T13:59:22.682Z (8 months ago)
- Language: Python
- Size: 26.4 KB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: COPYING
- Code of conduct: CODE_OF_CONDUCT.md
Awesome Lists containing this project
README
Snek Toolbox
This is a collection of handy scripts used in snek-at/functions provided by snek-at.
# Snek Pit
This application allows users to create and manage parquet files using DuckDB and Pandas. The application has the following functionality:
- `dump` command: Takes json data and saves it to a parquet file
- `retrieve` command: Loads a parquet file and returns the data as a json object
- `search` command: Search the specific column and value in the parquet file and returns the data as a json object
### Requirements
- Python 3
- DuckDB
- Pandas
### Installation
```bash
pip install duckdb pandas
```
### Usage
```bash
python script.py [command] [path/to/file.parquet] [data.json] [column_name1=value1] [column_name2=value2]
```
### Example
```bash
# Loads a parquet file and returns the data as a json object
python script.py retrieve path/to/file.parquet
# Saves json data to a parquet file
python script.py dump path/to/file.parquet data.json
# Search the specific column and value in the parquet file and returns the data as a json object
python script.py search path/to/file.parquet column_name1=value1 column_name2=value2
```
SPDX-License-Identifier: (EUPL-1.2)
Copyright © 2019-2022 snek.at