{"id":19768604,"url":"https://github.com/m2ci-msp/mri-shape-framework","last_synced_at":"2026-05-09T20:48:22.313Z","repository":{"id":79178508,"uuid":"93748014","full_name":"m2ci-msp/mri-shape-framework","owner":"m2ci-msp","description":"Gradle framework for deriving a tongue model from MRI data.","archived":false,"fork":false,"pushed_at":"2019-04-28T10:11:26.000Z","size":574,"stargazers_count":1,"open_issues_count":0,"forks_count":2,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-02-28T11:01:16.510Z","etag":null,"topics":["gradle","mri","statistical-model","tongue"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/1612.05005","language":"R","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/m2ci-msp.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-06-08T12:41:23.000Z","updated_at":"2023-05-02T06:27:52.000Z","dependencies_parsed_at":"2023-05-30T11:45:15.677Z","dependency_job_id":null,"html_url":"https://github.com/m2ci-msp/mri-shape-framework","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/m2ci-msp/mri-shape-framework","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/m2ci-msp%2Fmri-shape-framework","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/m2ci-msp%2Fmri-shape-framework/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/m2ci-msp%2Fmri-shape-framework/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/m2ci-msp%2Fmri-shape-framework/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/m2ci-msp","download_url":"https://codeload.github.com/m2ci-msp/mri-shape-framework/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/m2ci-msp%2Fmri-shape-framework/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":285852003,"owners_count":27242460,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-11-22T02:00:05.934Z","response_time":64,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["gradle","mri","statistical-model","tongue"],"created_at":"2024-11-12T04:39:25.000Z","updated_at":"2025-11-22T20:04:38.595Z","avatar_url":"https://github.com/m2ci-msp.png","language":"R","funding_links":[],"categories":[],"sub_categories":[],"readme":"# MRI shape framework\n\n## Introduction\n\nOriginally, this framework derives a multilinear tongue model from a given MRI dataset by performing the steps described in [*Hewer et al.*][1] in a minimally supervised way.\nSince then, we updated the approach somewhat, have a look at the [changelog][4] to learn about changes since the original release.\n\nBasically, this means that the framework automatically takes care of the dependencies between the different steps and calls the respective tools.\nAs a user, you only have to provide the MRI data with labels and the settings you want to use for the dataset.\n\n## Requirements\n\nPlease make sure that the following tools are installed and are also available on your path:\n\n- Java SDK ( version 7 or greater )\n- [Blender](https://www.blender.org)\n- [R](https://www.r-project.org)\n- The R packages ggplot2, reshape2, plyr, and reshape2\n- All tools from [MSP MRI Shape Tools][2]\n\n## Setup\n\nThe following sections explain how to add an MRI dataset to the framework and how it is configured.\nIn the following, `${...}` are always placeholder variables.\n\n### Adding a Dataset\n\nAdding a dataset of MRI scans to the framework is straightforward:\nThe scan files have to placed in the *resources/mri* folder where the following directory hierarchy is expected:\n\n```sh\n.\n└── resources\n    └── mri\n        └── ${DATASET_NAME}\n            └── ${SPEAKER_NAME}\n                └── ${SCAN_NAME}\n                    └── scan.json\n```\n\nwhere `${DATASET_NAME}`, `${SPEAKER_NAME}`, and `${SCAN_NAME}` can be freely chosen.\nThe file *scan.json* is the actual MRI scan in [JSON format][3].\n\n### Configuring the Dataset\n\nIn this step, we create the configuration files for the dataset.\nThese are placed in the *configuration* folder:\n\n```sh\n.\n└── configuration\n    └── ${DATASET_NAME}\n```\n\nwhere `${DATASET_NAME}` is the same name chosen above.\n\nFirst, we create the file *database.json* that contains meta information about the added dataset.\nThis file contains a JSON list of JSON objects.\nEach object has the following structure:\n\n```sh\n{\n    \"prompt\": \"${LABEL}\",\n    \"speaker\": \"${SPEAKER_NAME}\",\n    \"missing\": true|false,\n    \"id\": \"${SCAN_NAME}\"\n}\n```\n\nHere, *prompt* is a label that identifies the sound that was produced during the scan.\nThe variables `${SCAN_NAME}` and `${SPEAKER_NAME}` are chosen like above.\nThe flag *missing* indicates if the scan for the respective speaker and prompt is missing in the dataset.\nIn this case, `${SCAN_NAME}` should be set to something that also indicates that the scan is missing.\n\nFurthermore, we create the file *palateDatabase.json* that contains information about scans that are used for deriving the hard palate shape of each speaker in the dataset.\nAgain, we have a list of JSON objects:\n\n```sh\n{\n    \"prompt\": \"palate\",\n    \"speaker\": \"${SPEAKER_NAME}\",\n    \"missing\": false,\n    \"id\": \"${SCAN_NAME}\"\n}\n```\n\nAfterwards, we add the file *dataset.groovy* to the folder that contains the following settings:\n\n```groovy\n\n// these are settings for bootstrapping the tongue model\nbootstrapTongue{\n\n  // iterations to perform\n  iterations = 4\n  // iteration that should be used for the deriving the final model\n  selectedIteration = 4\n\n}\n\n// these are settings for bootstrapping the palate shapes\nbootstrapPalate{\n  // is the bootstrapping active?\n  active = true\n  // iterations to perform\n  iterations = 1\n  // iteration to use for the shapes\n  selectedIteration = 1\n}\n\n\n// settings for evaluation the final model\nevaluation{\n  priorSize = 2\n  convergenceFactor = 10000000\n  projectedGradientTolerance = 0.00001\n  maxFunctionEvals = 1000\n  // sample amount for specificity\n  samples = 1000000\n  // truncated modes for specificity\n  truncatedPhoneme = 4\n  truncatedSpeaker = 5\n  // subsets to use, have a look at the resources/evaluation folder for available subsets.\n  // you can also add your own\n  subsets = [\"bladetip\", \"combined\", \"bladebackdorsum\"]\n}\n\n// settings for the final tongue model\ncreateFinalTongueModel{\n  truncatedSpeaker = 5\n  truncatedPhoneme = 4\n}\n\n// settings for the final palate model\ncreateFinalPalateModel{\n  truncatedSpeaker = 11\n}\n\n// settings for performing the Procrustes alignment of the palate shapes\nprocrustesPalate{\n  originIndex = 93\n  iter = 40\n}\n\ndataset{\n  name = \"${DATASET_NAME}\"\n  speakers = [\"${SPEAKER_NAME}\", ... ]\n}\n```\n\nFinally, we create, for each considered speaker in the dataset, a subfolder with the corresponding name. Such a folder must contain a file named *speaker.groovy* with the following settings:\n\n```groovy\nspeaker {\n  // name of the speaker\n  name = \"${SPEAKER_NAME}\"\n  // list of scans to consider\n  scans = [\"${SCAN_NAME}\", ...]\n  palateScan = \"${PALATE_SCAN_NAME}\"\n}\n```\n\nHere, **scans** is a list of names of the scans belonging to that speaker.\n**palateScan** refers to the name of the scan that is used for the palate estimation.\n\nThis folder contains also a *settings.groovy* file that may override the [default settings](resources/settings/default.groovy).\nHowever, it should at least provide for each speaker the region of interest of the associated scans:\n\n```groovy\nspeaker {\n  cropToVocalTract{\n    minX = 41\n    minY = 152\n    minZ = 0\n    maxX = 171\n    maxY = 258\n    maxZ = 43\n  }\n}\n```\n\n### Providing Landmarks\n\nIt is necessary to provide landmarks for each considered scan.\nThese should be organized as follows:\n\n```sh\n.\n└── resources\n    └── landmarksPalate\n        └── ${DATASET_NAME}\n            └── ${SPEAKER_NAME}\n                └── ${SCAN_NAME}\n                    └── landmarks.json\n.\n└── resources\n    └── landmarksTongue\n        └── ${DATASET_NAME}\n            └── ${SPEAKER_NAME}\n                └── ${SCAN_NAME}\n                    └── landmarks.json\n```\n\nFor the tongue, the following landmarks are available:\n\n- FrontBaseCenter\n- Tip\n- SurfaceCenter\n- BackCenter\n- Airway\n\nFor the palate, we have:\n\n- FrontTeethCenter\n- SlopeMiddleRight\n- SlopeMiddleCenter\n- SlopeMiddleLeft\n- SlopeEndedRight\n- SlopeEndedCenter\n- SlopeEndedLeft\n- BackLeft\n- BackCenter\n- BackRight\n\nOpen the *blend files* in [resources/template](resources/template) with Blender to see where these landmarks are located on the template meshes. (They are stored in the form of *VertexGroups*)\nBasically, you can also add your own landmarks by modifying the blend files.\nWe recommend using the *landmark-tool* of the [MSP MRI Shape Tools][2] to distribute the landmarks on the MRI scans.\n\n### Landmarks for palate reconstruction\n\nAdditionally, we need landmarks on the scan that is used for the palate estimation.\nTheir names can be arbitrary, their positions, however, should describe a bounding box of a scan region that can only undergo rigid motions.\n\nThe files should be organized as follows:\n\n```sh\n.\n└── resources\n    └── landmarksAlignment\n        └── ${DATASET_NAME}\n            └── ${SPEAKER_NAME}\n                └── ${PALATE_SCAN_NAME}\n                    └── landmarks.json\n```\n\n## Example Data\n\nPlease have a look at the [example data repository](https://github.com/m2ci-msp/mri-shape-framework-example-data) for sample MRI data and corresponding configuration and landmark files.\n\n## Running the Framework\n\nAfter the configuration is finished, you can run the following commands from the root directory.\n\n### Deriving the Models\n\n```sh\n    ./gradlew createFinalTongueModel\n    ./gradlew createFinalPalateModel\n```\n\nThese commands perform the necessary steps to derive the final tongue model (first command) and palate model (second command).\nThe results are afterwards available under\n\n```sh\n.\n└── build\n    └── ${DATASET_NAME}\n        └── createFinalTongueModel\n```\n\nand\n\n```sh\n.\n└── build\n    └── ${DATASET_NAME}\n        └── createFinalPalateModel\n```\n\nIn both cases, the model is output in YAML and JSON format.\nHave a look at the documentation of the [MSP MRI Shape Tools][2] to learn more about the data format.\nFurthermore, most results of the immediate steps are available in the subfolders of\n\n```sh\n.\n└── build\n    └── ${DATASET_NAME}\n```\n\nHere, for example the template matching or segmentation results can be inspected.\n\n### Evaluating the Tongue Model\n\nRun the command\n\n```sh\n./gradlew evaluateTongueModel\n```\n\nto evaluate the specificity, generalization, and compactness of the model.\nPlots of the results can then be found in\n\n```sh\n.\n└── build\n    └── ${DATASET_NAME}\n        └── evaluation\n```\n\n### Visualizing the Results\n\nExecute\n\n```sh\n./gradlew createTongueHTML\n```\n\nto create HTML visualizations of the achieved results after the initial matching and at each boostrap iteration.\n\nThe visualizations of the initial results are available in\n\n```sh\n.\n└── build\n    └── ${DATASET_NAME}\n        └── html\n```\n\nThe visualizations of the bootstrap results are present in\n\n\n```sh\n.\n└── build\n    └── ${DATASET_NAME}\n        └── bootstrapTongue\n            └── ${COUNT}\n                └── html\n```\nwhere `${COUNT}` is the number of the iteration.\n\nIn both cases, you can start a HTML server in the corresponding subfolder and then inspect the results in your browser.\n\nThe command\n\n```sh\n./gradlew createPalateHTML\n```\n\nproduces visualizations of the matched and reconstructed palate meshes.\n\n## License\n\nThis work is licensed under the [MIT license](./LICENSE.md).\n\nIf you are using our framework, please cite, for the time being, the following paper:\n\n```bibtex\n@article{Hewer2018CSL,\n  author = {Hewer, Alexander and Wuhrer, Stefanie and Steiner, Ingmar and Richmond, Korin},\n  doi = {10.1016/j.csl.2018.02.001},\n  eprint = {1612.05005},\n  eprinttype = {arxiv},\n  journal = {Computer Speech \\\u0026 Language},\n  month = sep,\n  pages = {68-92},\n  title = {A Multilinear Tongue Model Derived from Speech Related {MRI} Data of the Human Vocal Tract},\n  url = {https://arxiv.org/abs/1612.05005},\n  volume = {51},\n  year = {2018}\n}\n```\n\n[1]: http://arxiv.org/abs/1612.05005\n[2]: https://github.com/m2ci-msp/mri-shape-tools\n[3]: https://github.com/m2ci-msp/mri-shape-tools/blob/master/dataFormats/scan.md\n[4]:./CHANGELOG.md\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fm2ci-msp%2Fmri-shape-framework","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fm2ci-msp%2Fmri-shape-framework","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fm2ci-msp%2Fmri-shape-framework/lists"}