https://github.com/mit-lcp/metaann
Collaborative labelling of events in physiological waveforms
https://github.com/mit-lcp/metaann
Last synced: 11 months ago
JSON representation
Collaborative labelling of events in physiological waveforms
- Host: GitHub
- URL: https://github.com/mit-lcp/metaann
- Owner: MIT-LCP
- License: gpl-2.0
- Created: 2022-09-29T18:05:07.000Z (over 3 years ago)
- Default Branch: master
- Last Pushed: 2022-09-29T18:06:38.000Z (over 3 years ago)
- Last Synced: 2025-02-15T19:51:16.934Z (over 1 year ago)
- Language: C
- Homepage:
- Size: 152 KB
- Stars: 0
- Watchers: 6
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: NEWS
- License: COPYING
Awesome Lists containing this project
README
# metaann
metaann is a system for reviewing and labelling events in
physiological waveforms. This system was built and used from about
2013 to 2017, to annotate several large collections of waveform data
at the MIT Laboratory for Computational Physiology.
This system was built to solve a very specific problem. We have a
collection of long-duration, multi-channel waveform recordings (ECG,
PPG, blood pressure, etc.) And for each of those recordings, we have
a list of events that were detected by some automated classifier. We
have thousands of these events, and we want to group the events into
buckets (e.g. "ventricular arrhythmia" versus "motion artifact") for
later study. We also wanted to have several people review each event
independently, to be sure they agreed.
As such, we needed a process that would allow a person to quickly
review each event. It needed to allow the reviewer to focus on their
task, which meant that the UI needed to load and display data without
any perceptible delay, require only one click to submit an answer, and
save the answers to a central server automatically.
The metaann client is a heavily modified, stripped-down version of
WAVE. It uses GTK+ 2.24 (because that made it easy to port WAVE's
Xlib display code to work natively on Windows and Mac OS.) Naturally,
it uses WFDB, and libcurl.
The metaann server is implemented using CGI.pm. It relies on the
PhysioNetWorks user authentication system. If you want to deploy it
yourself, you'll have to provide your own authentication system and
modify the scripts to work with it. Or write your own server; the
HTTP API is not terribly complicated.
One significant issue we discovered - obvious in retrospect - is that
not everyone works at the same speed. Some reviewers are highly
productive and motivated, and can routinely review hundreds of events
per week, while other reviewers will only review a handful of events
now and again. If you have a specific target to achieve (e.g.,
maximizing the number of events that have at least two independent
reviews), you can't simply divide up the database evenly among all the
reviewers. The metaann system has no mechanism for addressing this
situation, and required active involvement from the administrator
(manually re-partitioning and re-shuffling the record assignment
lists) in order to make effective use of everyone's time.