Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/kentakoong/mtnlog
A simple multinode performance logger for Python
https://github.com/kentakoong/mtnlog
cuda lanta nvitop python slurm-cluster
Last synced: 22 days ago
JSON representation
A simple multinode performance logger for Python
- Host: GitHub
- URL: https://github.com/kentakoong/mtnlog
- Owner: Kentakoong
- License: mit
- Created: 2024-07-29T17:14:36.000Z (7 months ago)
- Default Branch: master
- Last Pushed: 2024-09-24T07:21:57.000Z (5 months ago)
- Last Synced: 2025-01-10T04:04:44.082Z (about 1 month ago)
- Topics: cuda, lanta, nvitop, python, slurm-cluster
- Language: Python
- Homepage:
- Size: 85.9 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE.md
Awesome Lists containing this project
README
# mtnlog - A simple multinode performance logger for Python
## Introduction
mtnlog is a simple multinode performance logger for Python. It is designed to be used in a similar way to Python's built-in logging module, but with a focus on performance logging. It provides a simple API for logging performance data, including start and end times, and allows for easy integration with other logging systems.
## Installation
You can install mtnlog using pip:
```bash
pip install mtnlog
```
## Usage
To use mtnlog, you have two features: `JSONLogger` and `PerformanceLogger` and `PerformancePlotter`.
### JSONLogger
The `JSONLogger` class is a simple logger that writes performance data to a JSON file. You can create a new `JSONLogger` instance by passing a file path to the constructor:
```python
from mtnlog import JSONLogger
logger = JSONLogger(log_dir='logs') # logs is the directory where the log file will be saved
```
You can then use the `log` method to log performance data:
```python
logger.log('', filename='log') # your_dict is a dictionary with the data you want to log / filename is the name of the file
```
`your_dict` is a dictionary with the data you want to log.
`filename` is the name of the file where the data will be saved
### PerformanceLogger
The `PerformanceLogger` class is a logger for system performance data. It logs the the time taken to execute the block, as well as the CPU, memory, and GPU usage. You can create a new `PerformanceLogger` instance by passing a file path to the constructor:
```python
from mtnlog import PerformanceLogger
collector = PerformanceLogger(log_dir="", log_node="")
```
`your_log_dir` is the directory where the log file will be saved.
`current_node` is the number of the node you are logging.
You can then use the `change_tag` method to change the tag of the log:
```python
collector.change_tag("")
```
`new_tag` is the new tag you want to use.
To stop logging, you can use the `stop` method:
```python
collector.stop()
```
### PerformancePlotter
The `PerformancePlotter` class is a plotter for system performance data. It plots the time taken to execute the block, as well as the CPU, memory, GPU, and network usage. You can create a new `PerformancePlotter` instance by passing a file path to the constructor:
```python
from mtnlog import PerformancePlotter
plotter = PerformancePlotter(base_dir="", log_node="")
```
`your_base_dir` is the base directory where the log file will be saved.
`current_node` is the number of the node you are logging.
You can then use the `plot` method to plot the data:
```python
plotter.plot()
```
## Example
Here is an example of how to use mtnlog:
```python
from mtnlog import JSONLogger, PerformanceLogger, PerformancePlotter
# Create a JSONLogger instance
logger = JSONLogger(log_dir='logs')
# Log some data
logger.log({'message': 'Hello, world!'}, filename='log')
# Create a PerformanceLogger instance
collector = PerformanceLogger(log_dir='logs', log_node="0")
# Change the tag
collector.change_tag('new_tag')
# Stop logging
collector.stop()
# Create a PerformancePlotter instance
plotter = PerformancePlotter(base_dir='logs', log_node="0")
# Plot the data
plotter.plot()
```