{"id":27203594,"url":"https://github.com/benjamin-heasly/udpevents","last_synced_at":"2025-10-16T04:43:38.727Z","repository":{"id":231356005,"uuid":"781608030","full_name":"benjamin-heasly/UDPEvents","owner":"benjamin-heasly","description":"UDP service plugin to receive TTL and Text events from an external source and align them with a data stream","archived":false,"fork":false,"pushed_at":"2025-01-22T14:16:58.000Z","size":444,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-09T22:55:15.358Z","etag":null,"topics":["ephys","open","open-ephys","text","ttl","udp"],"latest_commit_sha":null,"homepage":"https://github.com/benjamin-heasly/UDPEvents","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/benjamin-heasly.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-04-03T17:50:55.000Z","updated_at":"2025-01-22T14:13:05.000Z","dependencies_parsed_at":"2024-05-13T15:38:24.512Z","dependency_job_id":"c1db3ff0-189b-49d7-be8f-add8b4091a92","html_url":"https://github.com/benjamin-heasly/UDPEvents","commit_stats":null,"previous_names":["benjamin-heasly/udpevents"],"tags_count":28,"template":false,"template_full_name":"open-ephys-plugins/processor-plugin-template","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/benjamin-heasly%2FUDPEvents","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/benjamin-heasly%2FUDPEvents/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/benjamin-heasly%2FUDPEvents/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/benjamin-heasly%2FUDPEvents/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/benjamin-heasly","download_url":"https://codeload.github.com/benjamin-heasly/UDPEvents/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248125629,"owners_count":21051766,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ephys","open","open-ephys","text","ttl","udp"],"created_at":"2025-04-09T22:55:20.595Z","updated_at":"2025-10-16T04:43:33.688Z","avatar_url":"https://github.com/benjamin-heasly.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# UDP Events\n\nThis repo is for an [Open Ephys GUI](https://github.com/open-ephys/plugin-GUI) processor plugin.\nIt's based on the Open Ephys [processor plugin template](https://github.com/open-ephys-plugins/processor-plugin-template).\nFor more info on Open Ephys and plugins in general, please see the [Open Ephys docs](https://open-ephys.github.io/gui-docs/Tutorials/How-To-Make-Your-Own-Plugin.html).\n\nThe UDP Events plugin can inject TTL and text events into an existing Open Ephys data stream via a UDP network socket.\nIt can align the \"soft\" timestamps from an external system to \"real\" sample numbers in the existing data stream.\nFor TTL messages, the timestamp alignment can have high precision (more precise than Open Ephys data blocks).\n\n![UDP Events Editor](./udp-events-editor.png)\n\nUDP Events will bind a **HOST** address and **PORT** number where it can receive UDP messages.\n\nIt will look for real, upstream TTL events on a selected **LINE**, with the selected **STATE** and use these to align soft TTL and text messages received via UDP.\n\nIt will add aligned TTL and text messages to a selected Open Ephys data stream, shown here as \"**example_...**\".\n\n## Downloading and Installing\n\nWe're building the plugin for different platforms using GitHub Actions.\nTo download a built plugin go to the latest build for your platform at this repo's [actions](https://github.com/benjamin-heasly/UDPEvents/actions) page.\n\nOn the latest build page, look for the \"Artifacts\" section.\nDownload an artifact with a name like `UDPEvents-system_version-API8.zip` and unzip it.\n\n### Linux\n\nOn Linux the `.zip` file contains a dynamic library file called `UDPEvents.so`.\nThis is the plugin.\nCopy `UDPEvents.so` into a folder where Open Ephys can find it.\n\nThe standard place would be your own user's `.config` directory, for example:\n\n```\n~/.config/open-ephys/plugins-api8\n```\n\nIf for some reason that doesn't work, you might also try the `plugins/` subdir of your Open Ephys GUI installation.\nYou might need root / `sudo` permission to copy into this directory.\nFor example:\n\n```\n/usr/local/bin/open-ephys-gui/plugins/\n```\n\nOnce the plugin is copied over you should be able to launch `open-ephys` and see \"UDP Events\" listed along with other plugins.\n\n### Windows\n\nOn Windows the `.zip` file contains a dynamic library file called `UDPEvents.dll`.\nThis is the plugin.\nCopy `UDPEvents.dll` into a folder where Open Ephys can find it.\n\nThe standard place would be the system's common app data directory, for example:\n\n```\nC:\\Documents and Settings\\All Users\\Application Data\\Open Ephys\\plugins-api8\\\n```\n\nIf for some reason that doesn't work, you might also try the `plugins/` subdir of your Open Ephys GUI installation, for example:\n\n```\nC:\\Program Files\\Open Ephys\\plugins\n```\n\nOnce the plugin is copied over, you should be able to launch the Open Ephys GUI and see \"UDP Events\" listed along with other plugins.\n\n### macOS\n\nOn macOS the `.zip` file contains a dynamic library file called `UDPEvents.bundle`.\nThis is the plugin.\nCopy `UDPEvents.bundle` into a folder where Open Ephys can find it.\n\nThe standard place would be your own users's application data directory, for example:\n\n```\n~/Library/Application Support/open-ephys/plugins-api8\n```\n\nIf for some reason that doesn't work, you might also try the `PlugIns/` subdir of the open-ephys app bundle you installed.\nTo do this you might need to use the terminal or right-click the open-ephys app and choose \"Show Package Contents\".\nFor example:\n\n```\n/Applications/open-ephys.app/Contents/PlugIns/\n```\n\nOnce the plugin is copied over, you should be able to launch the Open Ephys GUI and see \"UDP Events\" listed along with other plugins.\n\n## Integrating with Clients\n\nThe UDP Events plugin will act like a server that starts and stops whenever Open Ephys starts and stops acquisition.\nDuring acquisition UDP Events will bind its **HOST** address and UDP **PORT** and wait for messages to arrive from a client.\nAs each message arrives, UDP Events will:\n\n - take a local timestamp\n - reply to the client with this timestamp, as an acknowledgement\n - parse the message as either TTL or text\n - save the message in a queue, to be added to the selected data stream along with other signals and events\n\nThe ack timestamps are informational only.\nClients can use them to check that they are connecting to UDP Events as expected, and can expect that the timesamps will increase over time.\n\nClients should send events as a single UDP message each, with binary data in one of the two formats described below.\nFor a working example client in Python, see [test-client.py](./test-client.py) in this repo.\n\n### TTL Events\n\nTTL event messages should have exactly 11 bytes:\n\n| byte index | number of bytes | data type | description |\n| --- | --- | --- | --- |\n| 0 | 1 | uint8 | **message type** for TTL messages this is the literal value `0x01` |\n| 1 | 8 | double | **timestamp** event time in seconds (including fractions) from the client's point of view |\n| 9 | 1 | uint8 | **line number** 0-based line number for an Open Ephys TTL line (0-255) |\n| 10 | 1 | uint8 | **line state** on/off state for the Open Ephys TTL line (nonzero is \"on\") |\n\n### Text Events\n\nText event messages should start with exactly 11 header bytes, followed by a variable number of text bytes:\n\n| byte index | number of bytes | data type | description |\n| --- | --- | --- | --- |\n| 0 | 1 | uint8 | **message type** for text messages this is the literal value `0x02` |\n| 1 | 8 | double | **timestamp** event time in seconds (including fractions) from the client's point of view |\n| 9 | 2 | uint16 | **text length** byte length of text that follows (network byte order -- use [htons()](https://beej.us/guide/bgnet/html/#htonsman)) |\n| 11 | **text length** | char | **text** message text encoded as ASCII or UTF-8 |\n\n### Ack Timestamps\n\nUDP Events will reply to the sender of each message with an 8-byte acknowledgement:\n\n| byte index | number of bytes | data type | description |\n| --- | --- | --- | --- |\n| 0 | 8 | double | **timestamp** ack time in seconds from the UDP Events point of view |\n\n## Data Stream Alignment\n\nUDP Events will align soft timestamps received in UDP messages to real sample numbers in a selected Open Ephys data stream.\nFor TTL messages, the alignment preserves high timing precision -- more precise than the start of each Open Ephys data block.\n\n### TTL Event Pairs\n\nFor this alignment to work the client must send TTL Event messages via UDP with the same **LINE** number as real, upstream TTL events on the same Open Ephys data stream.\nThe soft timestamp for these TTL event messages should be the client's best estimate of when the real TTL event actually occurred, from the client's point of view.\n\nUDP Events will look for pairs of TTL events on the same **LINE** -- one from a UDP message and one from upstream in Open Ephys.\nOptionally it can filter these events by a the line **STATE**: low, high, or both.\nFor each pair it will estimate and record a conversion from client soft timestamp to data stream sample number.\n\nAs other TTL and text messages arrive via UDP, UDP Events will convert their soft timestamps to the closest sample number on the selected data stream, and add them as events to the stream.\n\n### Accuracy\n\nAlignment accuracy will be limited by how well the client can measure when real TTL events actually occur, and report these measurements via UDP.\nSo, UDP Events will make the most sense when the client has solid timing and has control over both the UDP messages and the corresponding upstream events.\nSuch a client could enrich a single DIO line with various other \"soft TTL\" and text events.\n\n![UDP Events DIO Example](./udp-events-open-ephys.png)\n\n### Reading Data Downstream / Offline\n\n#### TTL\n\nFor soft TTL messages received via UDP, no special handling should be required.  UDP Events will save these as events to the selected data stream with high-precision sample numbers.  These sample numbers should flow all the way to the Open Ephys data file (Binary, NWB, etc.) just like other TTL events.\n\n#### Text\n\nText messages are a bit different.\nAs of writing (April 2024) Open Ephys only saves text events with relatively coarse, per-block timestamps.\nFor some situations this might be sufficient.\n\nIn case it's helpful to recover high-precision timing for text events, UDP Events appends timing info to the body of each text message it receives.\n\nThe format looks like this:\n\n```\noriginal message text@\u003cclient_soft_timestamp\u003e=\u003cstream_sample_number\u003e\n```\n\nDownstream tools can looking for the delimiters `@` and `=` at the end of each message and parse out the details.  The `\u003cclient_soft_timestamp\u003e` would be the raw value in seconds sent by the client.  The `\u003cstream_sample_number\u003e` would be an aligned, integer sample number on the selected data stream.\n\n#### TTL Pair Text Events\n\nIn addition, UDP Events saves a separate text event for each TTL event pair it receives on **LINE**, as described above.\nThese record and expose the raw timing data that UDP Events uses when aligning client seconds to stream sample numbers.\n\nThese messages have a similar format:\n\n```\nUDP Events sync on line \u003cLINE\u003e@\u003cclient_soft_timestamp\u003e=\u003cstream_sample_number\u003e\n```\n\nThese always start with the same literal text: `UDP Events sync on line `.  The following `\u003cLINE\u003e` is the selected **LINE** number.  As above, `\u003cclient_soft_timestamp\u003e` is the raw value in seconds sent by the client.  Here, the `\u003cstream_sample_number\u003e` is the *actual* sample number of an upstream TTL event on the same **LINE**.\n\n## Testing\n\nYou can test UDP Events using a Python script like [test-client.py](./test-client.py) in this repo.\nThis script depends on [pyzmq](https://pypi.org/project/pyzmq/).\nYou might be able to install this with `pip install pyzmq`.\n\nThis script expects an Open Ephys signal chain with the following:\n\n - [File Reader](https://open-ephys.github.io/gui-docs/User-Manual/Plugins/File-Reader.html) -- Provide sample data and a data stream to work in.\n - [Network Events](https://open-ephys.github.io/gui-docs/User-Manual/Plugins/Network-Events.html) -- Create upstream TTL events for UDP Events to look for.\n - UDP Events -- this plugin!\n - [TTL Display Panel](https://open-ephys.github.io/gui-docs/User-Manual/Plugins/TTL-Panels.html) -- Blink as TTL events arrive.\n - [LFP Viewer](https://open-ephys.github.io/gui-docs/User-Manual/Plugins/LFP-Viewer.html) -- Show TTL events aligned with upstream sampled signals.\n - [Record Node](https://open-ephys.github.io/gui-docs/User-Manual/Plugins/Record-Node.html) -- Save data for detailed inspection.\n\n![UDP Events Test Signal Chain](./udp-events-test-signal-chain.png)\n\n### Phony Client\nIn this setup, [test-client.py](./test-client.py) will play the role of \"client\", with control over both Network Events and UDP Events.\n\nThe signal chain's File Reader and Network Events will work together to stand in for a genuine data source like an [Acquisition Board](https://open-ephys.github.io/gui-docs/User-Manual/Plugins/Acquisition-Board.html), which might record both sampled signals and TTL events with respect to a single clock.\nTiming accuracy of this test setup will be limited by the relatively coarse, best-effort timing of Network Events with respect to upstream sample numbers.\n\nChoose **LINE** 4 in the UDP Events settings editor, to match the line number used in `test-client.py`.\n\n### Running a Test\n\nTo start a test press the Open Ephys Record button.\nThis will start data streaming from the File Reader's example file.\nWaveforms should apear in the LFP Viewer window.\n\nWhile acquisition is still running, execute the client script.\n\n```\npython test-client.py\n```\n\nThis should run for about 10 seconds, then stop.  While running it will send a series of events to Open Ephys.\n\nThe script will send 10 pairs of TTL events on **LINE** 4.  For each pair:\n - a real/upstream event sent to Network Events\n - a soft UDP counterpart sent to UDP Events\n\nThe upstream events themselves should be visible as blinking lights in the TTL Display Panel, and transparent overlays in the LFP Viewer window.  UDP Events will use these event pairs to align other events, below, and instert them into the selected data stream.\n\nThe script will also send 10 pairs of soft TTL events on **LINE** 1:\n - one to turn line 1 on\n - antoher to turn line 1 off\n\nThese cycle line 1 as fast as the script can manage.\nThe cycle time will usually be too quick to see in the TTL Display Panel or LFP viewer.\nBut these events should be saved in the recorded data file with high timing precision -- perhaps receiving sample numbers that are 1 or 2 samples apart.\n\nAlong with TTL event mesages above, the script will send 10 text messages via UDP, which should also be saved in the data file.\n\nIf all this happens, then it seems UDP Events is working for you!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbenjamin-heasly%2Fudpevents","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbenjamin-heasly%2Fudpevents","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbenjamin-heasly%2Fudpevents/lists"}