https://github.com/vsoch/caliper

Caliper is a tool for measuring and assessing change in packages.
https://github.com/vsoch/caliper

Last synced: over 1 year ago
JSON representation

Caliper is a tool for measuring and assessing change in packages.

• Host: GitHub
• URL: https://github.com/vsoch/caliper
• Owner: vsoch
• License: mpl-2.0
• Created: 2020-12-18T20:52:32.000Z (over 5 years ago)
• Default Branch: main
• Last Pushed: 2024-10-27T22:09:48.000Z (over 1 year ago)
• Last Synced: 2025-04-12T12:12:02.943Z (over 1 year ago)
• Language: Python
• Homepage: https://caliper-python.readthedocs.io/
• Size: 1.11 MB
• Stars: 5
• Watchers: 2
• Forks: 3
• Open Issues: 7
• Metadata Files:
  • Readme: README.md
  • Changelog: CHANGELOG.md
  • Funding: .github/FUNDING.yml
  • License: LICENSE

Awesome Lists containing this project

README

          
![docs/assets/img/caliper-logo-sketch.png](docs/assets/img/caliper-logo-sketch.png)

[![PyPI version](https://badge.fury.io/py/caliper.svg)](https://badge.fury.io/py/caliper)

[![Documentation Status](https://readthedocs.org/projects/caliper-python/badge/?version=latest)](https://caliper-python.readthedocs.io/en/latest/?badge=latest)

[![DOI](https://zenodo.org/badge/322702177.svg)](https://zenodo.org/badge/latestdoi/322702177)

Caliper is a tool for measuring and assessing change in packages.

![docs/assets/img/spack-changes.png](docs/assets/img/spack-changes.png)

See the [documentation](https://caliper-python.readthedocs.io/en/latest/) for getting started.

## TODO

- write tests to discover and test all metrics (type, name, etc.)

- think of common functions to run metric

- create visualization of grid of errors / scripts, and mouesover to see codes (server rendered)

- stats: try to estimate types of codes (e.g., error name)

### Analysis Ideas

1. Start with a bunch of unit tests for a library (or example scripts) and build a model that can predict success (return code 0) vs fail (any other code) based on the scripts (tokens is a simple idea). Then given a new script without declared versions, predict which will work.

2. (A slightly different project) - given a set of known "optimal" containers for a library like tensorflow, parse dependencies, versions, and library versions, run across same set of unit tests / example scripts, and try to say which combos (architecture and dependencies) works for different kinds of scripts.

## License

 * Free software: MPL 2.0 License