{"id":41832627,"url":"https://github.com/bnelair/mef_tools","last_synced_at":"2026-01-25T08:41:51.713Z","repository":{"id":57440690,"uuid":"274251654","full_name":"bnelair/mef_tools","owner":"bnelair","description":"High-level lib for reading and writing physiological data into mef file format using python wrapper pymef for meflib.","archived":false,"fork":false,"pushed_at":"2025-07-07T16:44:01.000Z","size":1752,"stargazers_count":1,"open_issues_count":4,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-09-20T09:51:39.435Z","etag":null,"topics":["electrophysiology","mef","signals","timeseries-data"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bnelair.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2020-06-22T22:04:31.000Z","updated_at":"2025-09-17T18:11:54.000Z","dependencies_parsed_at":"2025-06-29T17:28:27.017Z","dependency_job_id":"8938d805-9478-4581-848c-d2026f54d674","html_url":"https://github.com/bnelair/mef_tools","commit_stats":null,"previous_names":["xmival00/mef_tools","mselair/mef_tools"],"tags_count":16,"template":false,"template_full_name":null,"purl":"pkg:github/bnelair/mef_tools","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bnelair%2Fmef_tools","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bnelair%2Fmef_tools/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bnelair%2Fmef_tools/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bnelair%2Fmef_tools/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bnelair","download_url":"https://codeload.github.com/bnelair/mef_tools/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bnelair%2Fmef_tools/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28749778,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-25T08:31:04.260Z","status":"ssl_error","status_checked_at":"2026-01-25T08:30:28.859Z","response_time":113,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["electrophysiology","mef","signals","timeseries-data"],"created_at":"2026-01-25T08:41:51.633Z","updated_at":"2026-01-25T08:41:51.708Z","avatar_url":"https://github.com/bnelair.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":".. image:: https://github.com/mselair/mef_tools/actions/workflows/test_publish.yml/badge.svg\n    :target: https://pypi.org/project/mef-tools/\n\n.. image:: https://readthedocs.org/projects/mef-tools/badge/?version=latest\n    :target: https://mef-tools.readthedocs.io/en/latest/?badge=latest\n\n.. image:: https://img.shields.io/pypi/pyversions/Django\n    :target: https://pypi.org/project/mef-tools/\n\n.. image:: https://img.shields.io/badge/platform-windows%20%7C%20macos%20%7C%20linux-lightgrey\n    :target: https://pypi.org/project/mef-tools/\n\n\n\nMEF_Tools\n----------------\n\nThis package provides tools for easier `Multiscale Electrophysiology Format (MEF) \u003chttps://doi.org/10.1016%2Fj.jneumeth.2009.03.022\u003e`_ data saving and reading. See the example below and `documentation \u003chttps://mef-tools.readthedocs.io/en/latest/?badge=latest\u003e`_.\n\n\nMultiscale Electrophysiology Format (MEF)\n-------------------------------------------\n\n`Multiscale Electrophysiology Format (MEF) \u003chttps://doi.org/10.1016%2Fj.jneumeth.2009.03.022\u003e`_ is a specialized file format designed for storing electrophysiological data. This format is capable of storing multiple channels of data in a single file, with each channel storing a time series of data points.\n\nMEF is particularly useful for handling large volumes of electrophysiological data, as it employs a variety of techniques such as lossless and lossy compression, data encryption and data de-identification to make the storage and transmission of such data more efficient and secure.\n\nPython's pymef library provides a set of tools for working with MEF files, including reading from and writing to these files. Below are examples demonstrating the use of these tools.\n\n* BH Brinkmann et al., “Large-scale electrophysiology: acquisition, compression, encryption, and storage of big data,“ J. Neurosci Methods. 2009;180(1):185‐192. doi:10.1016/j.jneumeth.2009.03.022\n\nDependencies\n----------------\n- `meflib \u003chttps://github.com/msel-source/meflib\u003e`_ - binaries are included in the pymef package\n- `pymef \u003chttps://github.com/msel-source/pymef\u003e`_\n- `numpy \u003chttps://numpy.org/\u003e`_\n- `pandas \u003chttps://pandas.pydata.org/\u003e`_\n\n\nInstallation\n----------------\n\nSee installation instructions `INSTALL.rst \u003chttps://github.com/xmival00/MEF_Tools/blob/master/INSTALL.rst\u003e`_.\n\nLicense\n----------------\n\nThis software is licensed under the Apache-2.0 License. See `LICENSE \u003chttps://github.com/xmival00/MEF_Tools/blob/master/LICENSE\u003e`_ file in the root directory of this project.\n\n\nCite\n----------------\nThis toolbox was developed as a part of the following projects. When use whole, parts, or are inspired by, we appreciate you acknowledge and refer these journal papers:\n\n* V. Sladky et al., “Distributed brain co-processor for tracking spikes, seizures and behaviour during electrical brain stimulation,” Brain Commun., vol. 4, no. 3, May 2022, doi: 10.1093/braincomms/fcac115.\n\n* F. Mivalt et al., “Electrical brain stimulation and continuous behavioral state tracking in ambulatory humans,” J. Neural Eng., vol. 19, no. 1, p. 016019, Feb. 2022, doi: 10.1088/1741-2552/ac4bfd.\n\n\nExample 1\n----------------\n\n\n.. code-block:: python\n\n    import numpy as np\n    from tqdm import tqdm\n    from datetime import datetime\n    from mef_tools.io import MefWriter, MefReader\n    \n    path = '/mnt/some/path/mef_test.mefd' # Update this !!!\n    password_write = 'pwd_write'\n    password_read = 'pwd_read'\n    \n    \n    chnames = ['test_channel_1', 'test_channel_2']\n    fsamp = 1000 # Hz\n    start = datetime.now().timestamp()\n    x = [np.random.randn(fsamp*3600), np.random.randn(fsamp*3600)]\n    \n    Wrt = MefWriter(path, overwrite=True, password1=password_write, password2=password_read) # if overwrite is True, any file with the same name will be overwritten, otherwise the data is appended to the existing file\n    Wrt.mef_block_len = int(fsamp)\n    Wrt.max_nans_written = 0\n    \n    \n    for idx, ch in tqdm(list(enumerate(chnames))):\n        x_ = x[idx]\n        Wrt.write_data(x_, ch, start_uutc=start * 1e6, sampling_freq=fsamp, reload_metadata=False, )\n    \n    \n    Rdr = MefReader(path, password_read)\n    channels_read = Rdr.channels\n    \n    print(\"All properties\", Rdr.properties)\n    print(f\"Sampling rate for channel {channels_read[0]}\", Rdr.get_property('fsamp', channels_read[0]))\n    x_read = Rdr.get_data(channels_read[0]) # read full length length\n    x_read_1s = Rdr.get_data(channels_read[0], start*1e6, (start+1)*1e6) # read 1 second - reading limited data is useful for really huge files.\n\nSee more `examples \u003chttps://github.com/mselair/mef_tools/tree/master/examples\u003e`_.\n\nExample 2\n----------------\n\nSee more `examples \u003chttps://github.com/mselair/mef_tools/tree/master/examples\u003e`_.\n\nFirst, we need to import the necessary libraries:\n\n.. code-block:: python\n\n    import os\n    import time\n    import numpy as np\n    import pandas as pd\n    from mef_tools.io import MefWriter, MefReader, create_pink_noise\n\nNext, we define the path to our MEF file, and the amount of data (in seconds) we want to write:\n\n.. code-block:: python\n\n    session_name = 'session'\n    session_path = os.getcwd() + f'/{session_name}.mefd'\n    mef_session_path = session_path\n    secs_to_write = 30\n\nWe also need to specify the start and end times of our data in uUTC time. uUTC time is the number of microseconds since January 1, 1970, 00:00:00 UTC. We can use the `time \u003chttps://docs.python.org/3/library/time.html\u003e`_ library to convert between UTC time and other time formats. In this example, we will use the current time as the start time, and the start time plus the number of seconds we want to write as the end time:\n\n.. code-block:: python\n\n    start_time = int(time.time() * 1e6)\n    end_time = int(start_time + 1e6*secs_to_write)\n\n\nWith our file path and timing details set, we can now create our MEFWriter instance:\n\n.. code-block:: python\n    pass1 = 'pass1' # password needed for writing to file\n    pass2 = 'pass2' # password needed for every read/write operation\n    Wrt = MefWriter(session_path, overwrite=True, password1=pass1, password2=pass2)\n    Wrt.max_nans_written = 0\n    Wrt.data_units = 'mV'\n\nWe then create some test data to write to our file:\n\n.. code-block:: python\n\n    fs = 500\n    low_b = -10\n    up_b = 10\n    data_to_write = create_pink_noise(fs, secs_to_write, low_b, up_b)\n\nThis data is written to a channel in our MEF file:\n\n.. code-block:: python\n    channel = 'channel_1'\n    precision = 3\n    Wrt.write_data(data_to_write, channel, start_time, fs, precision=precision)\n\nAppending Data to an Existing MEF File\n________________________________________\n\nTo append data to an existing MEF file, we first need to create a new writer:\n\n.. code-block:: python\n\n    secs_to_append = 5\n    discont_length = 3\n    append_time = end_time + int(discont_length*1e6)\n    append_end = append_time + 1e6*secs_to_append\n    data = create_pink_noise(fs, secs_to_append, low_b, up_b)\n    Wrt2 = MefWriter(session_path, overwrite=False, password1=pass1, password2=pass2)\n    Wrt2.write_data(data, channel, append_time, fs)\n\nCreating a New Segment in the MEF File\n________________________________________\n\nTo create a new segment, we simply need to change the new_segment flag to True:\n\n.. code-block:: python\n\n    secs_to_write_seg2 = 10\n    gap_time = 3.36*1e6\n    newseg_time = append_end + int(gap_time)\n    newseg_end = newseg_time + 1e6*secs_to_write_seg2\n    data = create_pink_noise(fs, secs_to_write_seg2, low_b, up_b)\n    data[30:540] = np.nan\n    data[660:780] = np.nan\n    Writer2.write_data(data, channel, newseg_time, fs, new_segment=True)\n\nWe can also write data to a new channel with inferred precision:\n\n.. code-block:: python\n\n    channel = 'channel_2'\n    Wrt2.write_data(data, channel, newseg_time, fs, new_segment=True)\n\n\nWriting Annotations to the MEF File\n________________________________________\n\nAnnotations can also be added to the MEF file at both the session and channel levels. Here's an example of how to do this:\n\n.. code-block:: python\n\n    start_time = start_time\n    end_time = start_time + 1e6 * 300\n    offset = start_time - 1e6\n    starts = np.arange(start_time, end_time, 2e6)\n    text = ['test'] * len(starts)\n    types = ['Note'] * len(starts)\n    note_annotations = pd.DataFrame(data={'time': starts, 'text': text, 'type': types})\n    Wrt2.write_annotations(note_annotations)\n\n    starts = np.arange(start_time, end_time, 1e5)\n    text = ['test'] * len(starts)\n    types = ['EDFA'] * len(starts)\n    duration = [10025462] * len(starts)\n    note_annotations = pd.DataFrame(data={'time': starts, 'text': text, 'type': types, 'duration':duration})\n    Wrt2.write_annotations(note_annotations, channel=channel )\n\n\nReading from MEF File\n________________________________________\n\n\nIn this example, we create a MefReader instance, print out the properties of the MEF file, and then read the first 10 seconds of data from each channel. The data from each channel is appended to a list.\n\n.. code-block:: python\n\n    Reader = MefReader(session_path, password2=pass2)\n    signals = []\n\n    properties = Reader.properties\n    print(properties)\n\n    for channel in Reader.channels:\n        start_time = Reader.get_property('start_time', channel)\n        end_time = Reader.get_property('end_time', channel)\n        x = Reader.get_data(channel, start_time, start_time+10*1e6)\n        signals.append(x)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbnelair%2Fmef_tools","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbnelair%2Fmef_tools","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbnelair%2Fmef_tools/lists"}