{"id":47833014,"url":"https://github.com/jtcostello/bcisimulator","last_synced_at":"2026-04-03T20:16:44.964Z","repository":{"id":200160440,"uuid":"697495776","full_name":"jtcostello/bcisimulator","owner":"jtcostello","description":"A simple closed-loop BCI simulator for testing real-time neural decoding algorithms","archived":false,"fork":false,"pushed_at":"2023-12-22T21:04:33.000Z","size":47544,"stargazers_count":30,"open_issues_count":0,"forks_count":2,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-01-27T05:32:44.154Z","etag":null,"topics":["bci","brain-computer-interface","neural-data","neural-network","real-time"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jtcostello.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2023-09-27T21:08:38.000Z","updated_at":"2024-01-13T19:49:15.000Z","dependencies_parsed_at":"2023-12-22T21:35:00.146Z","dependency_job_id":"e5e1049f-f776-4486-9040-136ff0ff27b8","html_url":"https://github.com/jtcostello/bcisimulator","commit_stats":{"total_commits":19,"total_committers":2,"mean_commits":9.5,"dds":"0.052631578947368474","last_synced_commit":"6d65ff9f6011ae7f887e0d4379937f2b417848ee"},"previous_names":["jtcostello/bcisimulator"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/jtcostello/bcisimulator","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jtcostello%2Fbcisimulator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jtcostello%2Fbcisimulator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jtcostello%2Fbcisimulator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jtcostello%2Fbcisimulator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jtcostello","download_url":"https://codeload.github.com/jtcostello/bcisimulator/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jtcostello%2Fbcisimulator/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31374519,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-03T17:53:18.093Z","status":"ssl_error","status_checked_at":"2026-04-03T17:53:17.617Z","response_time":107,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bci","brain-computer-interface","neural-data","neural-network","real-time"],"created_at":"2026-04-03T20:16:44.354Z","updated_at":"2026-04-03T20:16:44.959Z","avatar_url":"https://github.com/jtcostello.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"docs/img/bci simulator logo text.png\" style=\"max-width: 500px; width: 60%;\" alt=\"BCI Simulator\"\u003e\n\u003c/p\u003e\n\nBCI simulator is a lightweight simulator for closed-loop brain-computer interfaces (BCIs), with the goal of simplicity \nand being easily modifiable. It's designed for researchers to quickly test out decoder algorithms and get a \"feel\" for \nhow they might work in a closed-loop setting. It could also be helpful tool to teach students about the implementation \nof closed-loop, real-time BCIs. Currently, we have two tasks implemented: a 2D cursor task and a 5-finger hand task.\n\n\u003cimg src=\"docs/img/bcisim task overview.png\" alt=\"simulator flow chart\"/\u003e\n\nThe simulator works as follows:\n1. The user's intended movements are recorded using either the mouse or hand tracking.\n2. Neural data is simulated that encodes the user's movements. We currently implement a simple log-linear tuning model\nusing position \u0026 velocity as input.\n3. A decoder is trained to predict the user's movements from neural data.\nWe give examples for RNN and ridge regression decoders which predict both position and velocity.\n4. In closed-loop or \"online\" control, neural data is simulated in real-time and fed into the decoder to predict the user's\nmovements. The predicted velocities are integrated to get the cursor/hand position.\n\nRelevant files:\n- Tasks are defined in the `/tasks` folder.\n- Inputs (hand tracking or the real-time decoder) are defined in the `/inputs` folder.\n- Neural simulators are defined in `neuralsim.py`.\n- Decoders are defined in the `/decoders` folder.\n- Anything starting with `main_` is a script that can be run from the command line.\n- Decoders/\"fake brains\"/movement data are saved in the `/data` folder.\n\n\nNote that in trying to keep the simulator simple and easily modifiable, we don't optimize for performance and we don't\nsimulate neural data at the spike level (instead simulating at the bin level). For a more realistic real-time simulator,\nwe highly recommend checking out the [AE Studio Neural Data Simulator](https://github.com/agencyenterprise/neural-data-simulator) and \n[BRAND/Ali et al. 2023](https://www.biorxiv.org/content/10.1101/2023.08.08.552473v1.full). \nHowever, with the current design, we get 7-10 FPS doing simultaneous hand tracking \u0026 decoding on an M1 Macbook Pro, and \nsignificantly faster for the cursor task.\n\n### Cursor Task:\nIn the cursor task, the user controls a 2D cursor and tries to move to a target. We have both center-out and \nrandom targets.\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"docs/img/cursortask_online.gif\" alt=\"cursor simulator\" style=\"max-width: 500px; width: 80%;\"/\u003e\n\u003c/p\u003e\n\n\n### Hand Task:\nIn the hand task, the user controls a 5-finger virtual hand. We use the \n[Google Mediapipe hand tracking model](https://developers.google.com/mediapipe/solutions/vision/hand_landmarker)\nto track the user's fingers with a webcam. Currently, this task is freely paced without targets.\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"docs/img/hand demo short.gif\" alt=\"cursor simulator\" style=\"max-width: 500px; width: 80%;\"/\u003e\n\u003c/p\u003e\n\n## Installation\nClone the repository:\n```\ngit clone https://github.com/jtcostello/bcisimulator.git\ncd bcisimulator\n```\n\nCreate a conda environment and activate:\n```\nconda create -n bcisimulator python=3.9\nconda activate bcisimulator\n```\n\nOption 1 - Install packages needed for just the simple cursor task:\n```\npython -m pip install -r requirements_simple.txt\n```\n\nOption 2 - Install all packages (for both the cursor and hand tasks). This involves installing requirements (including\nGoogle Mediapipe) and downloading the Mediapipe hand tracking model:\n```\npython -m pip install -r requirements_full.txt\nwget -q https://storage.googleapis.com/mediapipe-models/hand_landmarker/hand_landmarker/float16/1/hand_landmarker.task -O mediapipe_hand_landmarker.task\nmkdir inputs/models\nmv mediapipe_hand_landmarker.task inputs/models/\n```\n\n\n\n## Example Usage\n\n### 1. Collect movement data\n\nRun the 2d cursor task on center-out or random targets, using the mouse as input:\n```\npython main_run_task.py -t cursor -tt centerout\nOR\npython main_run_task.py -t cursor -tt random\n```\nRun the 5-finger hand task (uses your webcam to track finger movements):\n```\npython main_run_task.py -t hand\n```\nRecorded datasets are saved in the `/data/movedata` folder. Feel free to rename the files.\n\n### 2. Create a fake brain to simulate neural data\nCreate a fake brain with 100 channels and 0.1 neural noise for the cursor task:\n```\npython main_create_fakebrain.py -c 100 -n 0.1 -t cursor -o cursorbrain_100_01\n```\nCreate a fake brain with 100 channels and 0.1 neural noise for the hand task:\n```\npython main_create_fakebrain.py -c 100 -n 0.1 -t hand -o handbrain_100_01\n```\n\n### 3. Train a decoder\n\nTrain a ridge regression decoder with 5 history bins:\n```\npython main_train_decoder.py --decoder_type ridge --seq_len 5 -d dataset_20231012_250sec_random.pkl -fb cursorbrain_100_02 -o cursorridge1 \n```\nTrain an RNN decoder (trained for 30 epochs):\n```\npython main_train_decoder.py --decoder_type rnn --epochs 30 -d dataset_20231012_250sec_random.pkl -fb cursorbrain_100_02 -o cursorrnn1\n```\nDecoders are saved in the `/data/trained_decoders` folder.\nNote that the fake brain is also saved in the decoder file.\n\n### 4. Test the decoder in closed-loop (simulating neural data in real time)\n\nRun the 2d cursor task on random targets, using a decoder as input:\n```\npython main_run_task.py -t cursor -d cursorrnn1 -tt random\npython main_run_task.py -t cursor -d cursorridge1 -tt random\n```\nRun the 2d cursor task like above, but use 80% integrated velocity and 20% position for decoded output:\n```\npython main_run_task.py -t cursor -d rnndecoder1 -tt random -b 0.8\n```\nRun the 5-finger hand task, using a decoder as input:\n```\npython main_run_task.py -t hand -d rnndecoder1\n```\n\nView all the available command line arguments:\n```\npython main_run_task.py --help\npython main_train_decoder.py --help\n```\n\n\n\n\n## Simulator design notes\n#### Real-Time Clocking/Timing\n- For now, we don't incorporate any precise clocking/timing in the desire for simplicity. \nIn the cursor task, the update rate is determined by the `clock.tick(FPS)` command within the task loop\n(a pygame command). In the hand task, the max update rate is also set by `clock.tick(FPS)`, but the actual update rate\nis usually much slower (7-12 fps).\n\n- We simulate neural data at the bin level, rather than the spike/ephys level. \nFor example, we use 20ms timesteps, so that the neural data generated is the average firing rate over a 20ms bin.\nThis minimizes the amount of data that needs to be generated and processed.\n\n#### Neural Simulation\n- The `LogLinUnitGenerator` simulates channels using a log-linear relationship, as in \n[Trucollo et al. 2008](https://www.jneurosci.org/content/28/5/1163.short). This makes\nit easy to generate an arbitrary number of degrees-of-freedom (whereas cosine tuning typically is for 2D).\n- A new neural simulator is created for each new decoder. This means if you train two decoders with the same neural\nsettings, they will have different neural data, and could have different performance. In the future we may add the \nability to use the same neural generator for multiple decoders.\n- Neural simulators approximate the random tuning we see in real neural data - they are by no means an accurate\nphysiological model. Our simulated signals lack many real features, like a large variance related to trial timing.\n- The `neural_noise` parameter sets the std of the noise added to the average firing rate.\n- Two ways of changing the information content of the neural data: (1) change the number of channels and/or (2) change \nthe neural noise level.\n\n## Parameters to explore\nA few system/decoder parameters that may be interesting to explore:\n- **Time History**: how many previous timesteps of neural data to use as input to the decoder. \nIncreased time history typically improves offline accuracy, but usually harms closed-loop performance.\n- **Neural Noise \u0026 Number of Channels**: increasing neural noise adds more variability to the generated neural data,\nand increasing the number of channels increases the amount of information available to the decoder.\n- **Decoder Architecture**: many groups have turned toward deep learning approaches for BCI decoders\n- **Training Data**: How much training data is needed? Is random targets or center out targets better? You might notice\na tendancy of the RNN decoder to overfit if trained only on center-out targets.\n- **Simulated neural data**: We currently implement a simple log-linear tuning model, but more realistic models could be used.\n\n## Relevant research papers\n(not at all an exhaustive list, just a few relevant papers)\n\nA key challenge in the design of real-time BCI algorithms is how the offline accuracy of the decoder does not \ndirectly indicate how well it will perform in a closed-loop setting. \n\n**Importance of incorporating closed-loop feedback in BCI simulations:**\n- Koyama et al. 2010\n- Zhang and Chase 2018\n- [Cunningham et al. 2011](https://journals.physiology.org/doi/full/10.1152/jn.00503.2010)\n- [Willet et al. 2019](https://www.nature.com/articles/s41598-019-44166-7)\n\n**Simulating Neural Data:**\n- Liang et al. 2020, Deep Learning Neural Encoders for Motor Cortex\n- Wen et al. 2021, Rapid adaptation of brain–computer interfaces to new neuronal ensembles or participants via generative modelling\n- Awasthi et al. 2022, Validation of a non-invasive, real-time, human-in-the-loop model of intracortical brain-computer interfaces\n\n\n## Todo / Future work\n- Add online recalibration (ReFIT)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjtcostello%2Fbcisimulator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjtcostello%2Fbcisimulator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjtcostello%2Fbcisimulator/lists"}