https://github.com/ryancswallace/jobman
A command line job manager with flexible support for retries, timeouts, logging, notifications, and more.
https://github.com/ryancswallace/jobman
background-jobs cli cli-app command-line command-line-tool nohup python retry supervisor
Last synced: over 1 year ago
JSON representation
A command line job manager with flexible support for retries, timeouts, logging, notifications, and more.
- Host: GitHub
- URL: https://github.com/ryancswallace/jobman
- Owner: ryancswallace
- License: mit
- Created: 2022-12-03T18:30:07.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2023-09-17T19:34:02.000Z (almost 3 years ago)
- Last Synced: 2025-03-23T17:12:53.437Z (over 1 year ago)
- Topics: background-jobs, cli, cli-app, command-line, command-line-tool, nohup, python, retry, supervisor
- Language: Python
- Homepage:
- Size: 293 KB
- Stars: 4
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Citation: CITATION.ccf
- Codeowners: CODEOWNERS
- Authors: AUTHORS.md
Awesome Lists containing this project
README
Jobman automates the process of running and monitoring jobs on the command line. Jobman supports
* running commands in the background immune to hangups
* logging command output
* retrying commands
* aborting commands after timeout
* delaying command execution for a specified time or event
* sending notifications on command success or failure
[](https://codecov.io/gh/ryancswallace/jobman)
[](https://ryancswallace.github.io/jobman/)
# Documentation
**Visit the :book: [jobman documentation site](https://ryancswallace.github.io/jobman/) :book: for complete information on using jobman.**
# Requirements
Jobman runs on UNIX-like operating systems including Linux and MacOS. Windows is not supported.
Jobman requires Python3.9+.
# Installation
Install or upgrade from PyPI with
```
$ pip install -U jobman
```
# Example
The example below uses jobman to run a Python script `train.py` in the background and immune to hangups (e.g., a SIGHUP from an SSH timeout).
Jobman will ensure 60 seconds have passed *and* that the file `data.csv` exists before starting the program. If those conditions haven't been met by 5:00PM on March 5, 2032, jobman will abort the job.
Jobman will retry the program up to five times until there's one successful run, defined as an exit code of `0` or `42`, waiting ten seconds between retries.
If the job succeeds, jobman will send a notification email. If the job fails, jobman will send an SMS message.
```bash
$ jobman \
--wait-duration 60s --wait-for-file data.csv \
--abort-time "2032-03-05T17:00:00" \
--retry-attempts 5 --retry-delay 10s -c 0 -c 42 \
--notify-on-job-success my-email --notify-on-job-failure my-cell \
train.py
12e4b604
```
After submitting the `train.py` job above, use `jobman show` to display details on job progress:
```bash
jobman show 12e4b604
```
To view a running log of the consolidated stdout and stderr streams of the latest run of the job, use `jobman logs`:
```bash
jobman logs 12e4b604 --follow
```
# Alternatives
Jobman aims to be reliable and fully-featured. It operates *without* requiring a system service/daemon for orchestration.
Alternative tools for similar use cases include:
* **cron**: for scheduling repeated executions of a job
* **Airflow**, **Prefect**, and **Dagster**: for managing dependencies between multiple jobs
* **supervisord**: for daemon-based job management
# Developing
Jobman uses pyenv for Python version management and poetry for builds. Before working on Jobman, ensure you have `pyenv` and `poetry` installed.
The `Makefile` defines targets for common operations during development, including the following:
* `make setup`: set up and install the package
* `make fmt`: run the autoformatters
* `make test`: run the type tests and unit test suite
* `make build`: build the jobman wheel
To release a new version of the package, use the `bumpver.sh` script. For example, to update to version 1.2.3, run `./bumpver.sh 1.2.3`. Before committing, be sure to update the [CHANGELOG.md](https://github.com/ryancswallace/jobman/blob/main/CHANGELOG.md) with user-facing changes described in the recent commit log printed out by `bumpver.sh`.
# Contributing
Feature requests, bug reports, and pull requests are welcome! See [CONTRIBUTING.md](https://github.com/ryancswallace/jobman/blob/main/CONTRIBUTING.md) for details on how to contribute to jobman.