{"id":31290152,"url":"https://github.com/csinva/fmri_decoding","last_synced_at":"2025-09-24T15:01:58.300Z","repository":{"id":315351063,"uuid":"1059120362","full_name":"csinva/fmri_decoding","owner":"csinva","description":null,"archived":false,"fork":false,"pushed_at":"2025-09-18T03:58:04.000Z","size":2717,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-09-18T05:43:22.463Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/csinva.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":"license.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-09-18T03:01:53.000Z","updated_at":"2025-09-18T03:58:07.000Z","dependencies_parsed_at":"2025-09-18T05:43:29.857Z","dependency_job_id":"2d83ee8e-0c18-418b-b4b8-ba69c15e17a9","html_url":"https://github.com/csinva/fmri_decoding","commit_stats":null,"previous_names":["csinva/fmri_decoding"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/csinva/fmri_decoding","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/csinva%2Ffmri_decoding","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/csinva%2Ffmri_decoding/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/csinva%2Ffmri_decoding/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/csinva%2Ffmri_decoding/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/csinva","download_url":"https://codeload.github.com/csinva/fmri_decoding/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/csinva%2Ffmri_decoding/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":276769579,"owners_count":25701569,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-24T02:00:09.776Z","response_time":97,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-09-24T15:00:29.759Z","updated_at":"2025-09-24T15:01:58.290Z","avatar_url":"https://github.com/csinva.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"This repository extends the code used in the paper \"Semantic reconstruction of continuous language from non-invasive brain recordings\" by Jerry Tang, Amanda LeBel, Shailee Jain, and Alexander G. Huth.  \n\n## Setup\n- install dependencies with `uv sync`\n- set up data\n  - set paths in `decoding.config.py` for where you want to download things and save results\n  - then run `uv run experiments/_download_data.py` to download all the data necessary in the right places. If this fails, download the data manually as follows:\n    - Download [language model data](https://utexas.box.com/shared/static/7ab8qm5e3i0vfsku0ee4dc6hzgeg7nyh.zip) and extract contents into `decoding.config.DATA_LM_DIR`\n    - Download [training data](https://utexas.box.com/shared/static/3go1g4gcdar2cntjit2knz5jwr3mvxwe.zip) and extract contents into `decoding.config.DATA_TRAIN_DIR`\n      - download stimulus data for `train_stimulus/` and response data for `train_response/[SUBJECT_ID]` from [OpenNeuro ds003020](https://openneuro.org/datasets/ds003020/). This should be specified in `decoding.config.DATA_PATH_TO_DERIVATIVE_DS003020`\n    - Download [test data](https://utexas.box.com/shared/static/ae5u0t3sh4f46nvmrd3skniq0kk2t5uh.zip) and extract contents into `decoding.config.DATA_TEST_DIR`\n      - download stimulus data for `test_stimulus/[EXPERIMENT]` and response data for `test_response/[SUBJECT_ID]` from [OpenNeuro ds004510](https://openneuro.org/datasets/ds004510/). This should be specified in `decoding.config.DATA_PATH_TO_DERIVATIVE_DS004510`\n\n## Running\n1. Estimate the encoding model. The encoding model predicts brain responses from contextual features of the stimulus extracted using GPT. The `--gpt` parameter determines the GPT checkpoint used. Use `--gpt imagined` when estimating models for imagined speech data, as this will extract features using a GPT checkpoint that was not trained on the imagined speech stories. Use `--gpt perceived` when estimating models for other data. The encoding model will be saved in `MODEL_DIR/[SUBJECT_ID]`. Alternatively, download [pre-fit encoding models](https://utexas.box.com/s/ri13t06iwpkyk17h8tfk0dtyva7qtqlz).\n\n```bash\nuv run python experiments/00_train_EM.py --subject S3 --gpt perceived\n```\n\n2. Estimate the word rate model. The word rate model predicts word times from brain responses. Two word rate models will be saved in `MODEL_DIR/[SUBJECT_ID]`. The `word_rate_model_speech` model uses brain responses in speech regions, and should be used when decoding imagined speech and perceived movie data. The `word_rate_model_auditory` model uses brain responses in auditory cortex, and should be used when decoding perceived speech data. Alternatively, download [pre-fit word rate models](https://utexas.box.com/s/ri13t06iwpkyk17h8tfk0dtyva7qtqlz).\n\n```bash\nuv run python experiments/01_train_WR.py --subject S3\n```\n\n3. Test the decoder on brain responses not used in model estimation. The decoder predictions will be saved in `RESULTS_DIR/[SUBJECT_ID]/[EXPERIMENT_NAME]`.\n\n```bash\nuv run python experiments/02_run_decoder.py --subject S3\n```\n\n4. Evaluate the decoder predictions against reference transcripts. The evaluation results will be saved in `SCORE_DIR/[SUBJECT_ID]/[EXPERIMENT_NAME]`.\n\n```bash\nuv run python experiments/03_evaluate_predictions.py --subject S3\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcsinva%2Ffmri_decoding","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcsinva%2Ffmri_decoding","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcsinva%2Ffmri_decoding/lists"}