{"id":32157541,"url":"https://github.com/harrisonritz/statespaceanalysis.jl","last_synced_at":"2026-04-08T14:32:24.860Z","repository":{"id":263526132,"uuid":"890683460","full_name":"harrisonritz/StateSpaceAnalysis.jl","owner":"harrisonritz","description":"A package for parameter estimation of Linear-Gaussian state-space models, specialized for neuroimaging analysis","archived":false,"fork":false,"pushed_at":"2025-08-12T04:24:11.000Z","size":19507,"stargazers_count":4,"open_issues_count":2,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-09-30T11:50:35.522Z","etag":null,"topics":["computational-neuroscience","expectation-maximization","latent-variable-models","linear-dynamical-systems","machine-learning","state-space-model"],"latest_commit_sha":null,"homepage":"","language":"Julia","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/harrisonritz.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-11-19T02:02:13.000Z","updated_at":"2025-08-21T22:59:40.000Z","dependencies_parsed_at":"2024-11-19T03:28:18.541Z","dependency_job_id":null,"html_url":"https://github.com/harrisonritz/StateSpaceAnalysis.jl","commit_stats":null,"previous_names":["harrisonritz/statespaceanalysis.jl"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/harrisonritz/StateSpaceAnalysis.jl","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/harrisonritz%2FStateSpaceAnalysis.jl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/harrisonritz%2FStateSpaceAnalysis.jl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/harrisonritz%2FStateSpaceAnalysis.jl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/harrisonritz%2FStateSpaceAnalysis.jl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/harrisonritz","download_url":"https://codeload.github.com/harrisonritz/StateSpaceAnalysis.jl/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/harrisonritz%2FStateSpaceAnalysis.jl/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":280264174,"owners_count":26300771,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-21T02:00:06.614Z","response_time":58,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["computational-neuroscience","expectation-maximization","latent-variable-models","linear-dynamical-systems","machine-learning","state-space-model"],"created_at":"2025-10-21T12:50:42.373Z","updated_at":"2025-10-21T12:50:45.909Z","avatar_url":"https://github.com/harrisonritz.png","language":"Julia","funding_links":[],"categories":[],"sub_categories":[],"readme":"# StateSpaceAnalysis.jl\n\n[![Build Status](https://github.com/harrisonritz/StateSpaceAnalysis.jl/actions/workflows/CI.yml/badge.svg?branch=main)](https://github.com/harrisonritz/StateSpaceAnalysis.jl/actions/workflows/CI.yml?query=branch%3Amain) [![Aqua QA](https://raw.githubusercontent.com/JuliaTesting/Aqua.jl/master/badge.svg)](https://github.com/JuliaTesting/Aqua.jl)\n\n\n## Overview\n\n*StateSpaceAnalysis.jl* is a Julia package designed for fitting linear-Gaussian state space models (SSMs) using Subspace System Identification (SSID) and Expectation Maximization (EM) algorithms. \n\nThis package provides tools for preprocessing data, fitting models, and evaluating model performance, with methods especially tailored towards neuroimaging analysis:\n\n\n### Event-related data\n\nNeuroimaging data often has epoched/batched sequences (e.g., states x timesteps x trials). *StateSpaceAnalysis.jl* handles epoched data by reusing computations across batches, and it includes spline temporal bases for flexible input modeling over the epoch.   \n\n\n### High-dimensional systems\n\nWhole-brain modelling may require a large number of latent factors. *StateSpaceAnalysis.jl* handles scaling through efficient memory allocation, robust covariance formats (via [*PDMats.jl*](https://github.com/JuliaStats/PDMats.jl)), and regularization.  \n\n\n### Data-driven initialization\n\nWe need good initialization for systems for which we don't have great domain knowledge (especially when there are many latent factors). *StateSpaceAnalysis.jl* handles parameter initialization through subspace identification methods adapted from [*ControlSystemsIdentification.jl*](https://github.com/baggepinnen/ControlSystemIdentification.jl).\n\nThis version is very preliminary, so there may be some rough edges!\n\n## Installation\n\nYou can easily install the current release of *StateSpaceAnalysis.jl* from the Julia General Registry:\n\n```julia\nusing Pkg\nPkg.add(\"StateSpaceAnalysis\")\n```\n\nYou'll probably want to work directly with the package, e.g., to run the example code, or modify custom functions for setting up your input bases.\nYou can create a local copy by cloning the github repo:\n\n1. **Clone the repository:**\n    ```sh\n    git clone https://github.com/harrisonritz/StateSpaceAnalysis.jl.git\n    cd StateSpaceAnalysis.jl\n    ```    \n\n2. **Open Julia in the folder and activate the package environment:**\n    ```julia\n    using Pkg\n    Pkg.activate(\".\")\n    Pkg.instantiate()\n    ```\n\n3. **Add the package to your Julia environment:**\n    ```julia\n    Pkg.add(path=\".\")\n    using StateSpaceAnalysis\n    ```\n\nThis will install all the necessary dependencies and set up the StateSpaceAnalysis.jl package for local use.\n\nNote: You can check which directory you are working in with `pwd()` in Julia. Opening a folder in VS code sets that folder to your path. You can specify the paths in `Pkg.activate(\"path/to/package\")` and `Pkg.add(\"path/to/package\")` even in you aren't in the right folder.\n\n\n\n## Walkthrough of the `example/fit_example.jl` script\n\n### Set up `S`, the core structure which carries the parameters and data structure\n\n```julia\nS = core_struct(\n    prm=param_struct(\n        ... # high-level parameters\n        ), \n    dat=data_struct(\n        ... # data and data description\n        ),\n    res=results_struct(\n        ... # fit metrics and model derivates\n    ),\n    est=estimates_struct(\n        ... # scratch space\n    ),\n    mdl=model_struct(\n        ... # estimated model parameters\n    ),\n    fnc=function_struct{core_struct}(\n        ... # custom functions for setup\n    )\n);\n```\nThis structure is used throughout the script, which allows for effective memory management (i.e., the complier can know the size of the data tensors).\n\n### Preprocess the data:\n\n```julia\n@reset S = StateSpaceAnalysis.preprocess_fit(S);\n```\n\nPreprocessing steps within `preprocess_fit(S)`:\n```julia\n# read in arguments, helpful for running on a cluster\nS = deepcopy(StateSpaceAnalysis.read_args(S, ARGS));\n\n# set up the paths\nStateSpaceAnalysis.setup_path(S)\n\n# load and format the data; split for cross-validation\nS = deepcopy(StateSpaceAnalysis.load_data(S));\n\n# build the input tenors (e.g., z-score and convolve with basis)\nS = deepcopy(StateSpaceAnalysis.build_inputs(S));\n\n# transform the observed data (PCA)\nS = deepcopy(StateSpaceAnalysis.project(S));\n\n# fit baseline models to the data\nStateSpaceAnalysis.null_loglik!(S);\n\n# initialize the expectations and parameters\n@reset S.est = deepcopy(set_estimates(S));\n@reset S = deepcopy(generate_rand_params(S));\n```\n\n\n### Warm-start the EM with initial parameters from Subspace Identification (SSID):\n\n```julia\nif S.prm.ssid_fit == \"fit\" # if fitting the SSID\n    @reset S = StateSpaceAnalysis.launch_SSID(S);\nelseif S.prm.ssid_fit == \"load\" # if loading a previously-fit SSID\n    @reset S = StateSpaceAnalysis.load_SSID(S);\nend\n```\n\n### Fit the parameters use EM:\n\n```julia\n@reset S = StateSpaceAnalysis.launch_EM(S);\n```\nThe basic structure of the EM script:\n```julia\nfor em_iter = 1:S.prm.max_iter_em\n\n        # ==== E-STEP ================================================================\n        @inline StateSpaceAnalysis.ESTEP!(S); # estimate the sufficient statistics\n\n        # ==== M-STEP ================================================================\n        @reset S.mdl = deepcopy(StateSpaceAnalysis.MSTEP(S)); # use the sufficient statistics to update the parameters\n\n        # ==== TOTAL LOGLIK ==========================================================\n        StateSpaceAnalysis.total_loglik!(S) # compute the total likelihood\n\n        # [...] quality \u0026 convergence checks\nend\n```\n\n### Save the fit:\n\n```julia\nStateSpaceAnalysis.save_results(S)\n```\n\n\n\n\n## Functions Overview\n\n### `setup/custom.jl`\n\n**This needs to be set by the user for the project-specific parameters**\n- `assign_arguments`: Assigns command-line arguments to the structure.\n- `select_trials`: Selects trials based on custom criteria.\n- `scale_input`: Scales the input data.\n- `create_input_basis`: Formats inputs with basis functions.\n- `transform_observations`: Transforms observations, typically using PCA.\n- `format_B_preSSID`: Formats the B matrix for SSID.\n- `format_B_postSSID`: Assigns the estimated B columns to the rest of the matrix.\n\n### `fit/launch.jl`\n\n- `preprocess_fit`: Preprocesses the data and sets up the fitting environment.\n- `launch_SSID`: Launches the SSID fitting process.\n- `launch_EM`: Launches the EM fitting process.\n- `load_SSID`: Loads a previously saved SSID model.\n- `save_SSID`: Saves the SSID model.\n- `save_results`: Saves the fitting results.\n\n### `fit/SSID.jl`\n\n**These function are modifed from [*ControlSystemsIdentification.jl*](https://github.com/baggepinnen/ControlSystemIdentification.jl)**\n- `fit_SSID`: Performs subspace identification for state space analysis.\n- `subspaceid_SSA`: modified ControlSystemsIdentification.jl for SSID\n\n### `fit/EM.jl`\n\n- `fit_EM`: Runs the EM algorithm for individual participants.\n- `ESTEP!`: Executes the E-step of the EM algorithm.\n- `MSTEP`: Executes the M-step of the EM algorithm.\n- `estimate_cov!`: Estimates the latent covariance.\n- `estimate_mean!`: Estimates the latent mean.\n- `estimate_moments!`: update the sufficient statistics.\n\n\n### `fit/posteriors.jl`\n\n- `posterior_all`: Generates all posterior estimates (mean and covariance).\n- `posterior_mean`: Generates only the posterior means.\n- `posterior_sse`: Computes the sum of squared errors for the posteriors.\n\n\n### `setup/setup.jl`\n\n- `read_args`: process command line arguments (for running on the cluster)\n- `setup_path`: Sets up the directory paths for saving results.\n- `load_data`: Loads the data from files.\n- `build_inputs`: Builds the input matrices for the model.\n- `project`: projects the observations (PCA).\n\n### `setup/generate.jl`\n\n- `gen_rand_params`: generate random SSM parameters\n- `generate_ssm_trials`: simulate trials from a set of SSM parameters \n\n### `setup/structs.jl`\n\n- `param_struct`: Defines the parameters structure.\n- `data_struct`: Defines the data structure.\n- `results_struct`: Defines the results structure.\n- `estimates_struct`: Defines the estimates structure.\n- `model_struct`: Defines the model structure.\n- `core_struct`: Combines all the structures into a core structure.\n- `post_all`: Defines the structure for all posterior estimates.\n- `post_mean`: Defines the structure for posterior means.\n- `post_sse`: Defines the structure for posterior sum of squared errors.\n\n### `utils/utils.jl`\n\n- `tol_PD`: Ensures a matrix is positive definite with a tolerance.\n- `tol_PSD`: Ensures a matrix is positive semi-definite with a tolerance.\n- `demix`: Demixes the observations using the saved PCA transformation.\n- `remix`: Remixes the observations using the saved PCA transformation.\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fharrisonritz%2Fstatespaceanalysis.jl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fharrisonritz%2Fstatespaceanalysis.jl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fharrisonritz%2Fstatespaceanalysis.jl/lists"}