https://github.com/haozeke/bless

A commandline helper to bless development runs
https://github.com/haozeke/bless

Last synced: 22 days ago
JSON representation

A commandline helper to bless development runs

• Host: GitHub
• URL: https://github.com/haozeke/bless
• Owner: HaoZeke
• License: mit
• Created: 2024-02-22T09:48:30.000Z (9 months ago)
• Default Branch: main
• Last Pushed: 2024-07-17T04:58:00.000Z (4 months ago)
• Last Synced: 2024-07-17T07:03:47.909Z (4 months ago)
• Language: Rust
• Size: 246 KB
• Stars: 0
• Watchers: 2
• Forks: 0
• Open Issues: 16
• Metadata Files:
  • Readme: readme.md
  • License: LICENSE
  • Code of conduct: CODE_OF_CONDUCT.md
  • Codeowners: CODEOWNERS

Awesome Lists containing this project

README

        
![img](branding/logo/bless_logo.png)

# Table of Contents

-   [About](#org3bf058b)

    -   [Why?](#org37bc08e)

    -   [Design](#orgfc61a1a)

-   [Usage](#org2b345d0)

-   [Development](#org603159f)

    -   [Component Rationale](#org89150b7)

    -   [Local MongoDB](#orgb4d73ad)

    -   [Documentation](#org7940300)

-   [License](#org654b5a6)



# About

A simple command line wrapper for repeated runs, with metadata and lightweight

tracking.



## Why?

During development, a full interface to HPC oriented workflow engines like

AiiDA, Fireworks, Jobflow, and the like is typically too heavy, and more

importantly, the API is often not stable. That being said, this could also be

used in conjunction with `pychum` and workflow runners like Snakemake to store



## Design

### File (gzip) Writer

This overrides the `Log` levels of Rust, so:

-   `TRACE` is for additional information for the command, as written by `bless`

-   `INFO` corresponds to `stdout` of the command

-   `ERROR` corresponds to a `bless` error

-   `WARN` corresponds to `stderr` of the command

### MongoDB Writer

Only `stderr` and `stdout` of the command are stored in a `.gz` file which is

added to the database as a binary blob, with additional metadata.



# Usage

    cargo build --release

    MONGODB_URI="mongodb://localhost:27017/" ./target/release/bless --use-mongodb -- echo "bye"

    # Then view it in mongosh

    # or

    ./target/release/bless -- echo "bye"

    zcat default_label*.gz



# Development



## Component Rationale

-   **Duct:** For [the gotchas](https://github.com/oconnor663/duct.py/blob/master/gotchas.md)

-   **Wild:** For cross-platform globs

-   **Flate2:** For compression

-   **UUID:** For the unique IDs

-   **Fern:** For log handling



## Local MongoDB

Assuming `pixi` is used to get an instance of `mongod`.

    pixi run mongod --dbpath $(pwd)/data/database

    MONGODB_URI="mongodb://localhost:27017/" ./target/release/bless --use-mongodb -- $CMD_TO_RUN

I use `npx mongosh` for validating commands.

    npx mongsh

    use local

    # Show all entries

    db.commands.find()

    # Suppress blob data

    db.commands.find({}, { gzip_blob: 0 })

    # Dangerous, drop all entries!

    db.getCollectionNames().forEach(c=>db[c].drop())

### Extracting run output

Since the `.gzip` is stored as binary data keyed to the entry, a small helper script is provided.

    python scripts/get_db_gzip.py --db-name local --collection-name commands --query-field args --query-value orca.inp



## Documentation

### Readme

The `readme` can be constructed via:

    ./scripts/org_to_md.sh readme_src.org readme.md

metadata more generically.



# License

MIT. However, this is an academic resource, so **please cite** as much as possible

via:

-   The Zenodo DOI for general use.

-   TBD a publication