Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/beacon-biosignals/lighthouse.jl

Performance evaluation tools for multiclass, multirater classification models
https://github.com/beacon-biosignals/lighthouse.jl

Last synced: about 5 hours ago
JSON representation

Performance evaluation tools for multiclass, multirater classification models

Host: GitHub
URL: https://github.com/beacon-biosignals/lighthouse.jl
Owner: beacon-biosignals
License: mit
Created: 2020-12-07T19:48:18.000Z (almost 4 years ago)
Default Branch: main
Last Pushed: 2024-07-23T14:40:05.000Z (4 months ago)
Last Synced: 2024-07-24T16:02:12.651Z (4 months ago)
Language: Julia
Size: 25.7 MB
Stars: 18
Watchers: 24
Forks: 2
Open Issues: 29
Metadata Files:
- Readme: README.md
- License: LICENSE

Awesome Lists containing this project

README

        


  



# Lighthouse.jl

[![CI](https://github.com/beacon-biosignals/Lighthouse.jl/actions/workflows/CI.yml/badge.svg)](https://github.com/beacon-biosignals/Lighthouse.jl/actions/workflows/CI.yml)

[![codecov](https://codecov.io/gh/beacon-biosignals/Lighthouse.jl/branch/main/graph/badge.svg?token=8DnNEbLw2x)](https://codecov.io/gh/beacon-biosignals/Lighthouse.jl)

[![Docs: stable](https://img.shields.io/badge/docs-stable-blue.svg)](https://beacon-biosignals.github.io/Lighthouse.jl/stable)

[![Docs: development](https://img.shields.io/badge/docs-dev-blue.svg)](https://beacon-biosignals.github.io/Lighthouse.jl/dev)

Lighthouse.jl is a Julia package that standardizes and automates performance evaluation for multiclass, multirater classification models. By implementing a minimal interface, your classifier automagically gains a thoroughly instrumented training/testing harness (`Lighthouse.learn!`) that computes and logs tons of meaningful performance metrics to TensorBoard in real-time, including:

- test set loss

- inter-rater agreement (e.g. Cohen's Kappa)

- PR curves

- ROC curves

- calibration curves

Lighthouse itself is framework-agnostic; end-users should use whichever extension package matches their desired framework (e.g. https://github.com/beacon-biosignals/LighthouseFlux.jl).

This package follows the [YASGuide](https://github.com/jrevels/YASGuide).

## Installation

To install Lighthouse for development, run:

```

julia -e 'using Pkg; Pkg.develop(PackageSpec(url="https://github.com/beacon-biosignals/Lighthouse.jl"))'

```

This will install Lighthouse to the default package development directory, `~/.julia/dev/Lighthouse`.

### TensorBoard

Note that Lighthouse's `LearnLogger` logs metrics to a user-specified path in [TensorBoard's](https://github.com/tensorflow/tensorboard) `logdir` format. TensorBoard can be installed via `python3 -m pip install tensorboard` (note: if you have `tensorflow>=1.14`, you should already have `tensorboard`). Once TensorBoard is installed, you can view Lighthouse-generated metrics via `tensorboard --logdir path` where `path` is the path specified by `Lighthouse.LearnLogger`. From there, TensorBoard itself can be used/configured however you like; see https://github.com/tensorflow/tensorboard for more information.

You can use alternative loggers, as long as they comply with the [logging interface](https://beacon-biosignals.github.io/Lighthouse.jl/dev#The-logging-interface).