{"id":18750361,"url":"https://github.com/ntia/abc-mrt16","last_synced_at":"2025-06-20T20:37:49.278Z","repository":{"id":49202934,"uuid":"315691004","full_name":"NTIA/ABC-MRT16","owner":"NTIA","description":"Articulation Band Correlation Modified Rhyme Test","archived":false,"fork":false,"pushed_at":"2021-06-23T19:57:18.000Z","size":1080,"stargazers_count":5,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-05-20T11:08:32.659Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"MATLAB","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/NTIA.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-11-24T16:32:13.000Z","updated_at":"2025-02-19T21:06:09.000Z","dependencies_parsed_at":"2022-09-11T08:11:35.673Z","dependency_job_id":null,"html_url":"https://github.com/NTIA/ABC-MRT16","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/NTIA/ABC-MRT16","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NTIA%2FABC-MRT16","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NTIA%2FABC-MRT16/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NTIA%2FABC-MRT16/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NTIA%2FABC-MRT16/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/NTIA","download_url":"https://codeload.github.com/NTIA/ABC-MRT16/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NTIA%2FABC-MRT16/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":261013508,"owners_count":23097057,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-07T17:11:32.583Z","updated_at":"2025-06-20T20:37:44.262Z","avatar_url":"https://github.com/NTIA.png","language":"MATLAB","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Conducting ABC-MRT and ABC-MRT16 Tests\r\n\r\nS. Voran, October 28, 2013\r\n\r\nUpdated by S. Voran on March 2, 2017 to cover ABC-MRT16 as well.\r\n\r\n## Background\r\nThis software implements the ABC-MRT and ABC-MRT16 algorithms for\r\nobjective estimation of speech intelligibility. The algorithms are\r\ndiscussed in detail in [[1](#f1)] and [[2](#f2)]. ABC-MRT is short for\r\n“Articulation Band Correlation Modified Rhyme Test.”\r\n\r\nThe Modified Rhyme Test (MRT) [[3](#f3)] is a protocol for evaluating speech \r\nintelligibility using human subjects. The subjects are presented with the \r\ntask of identifying one of six different words that take the phonetic form \r\nCVC. The six options differ only in the leading or trailing consonant. \r\nMRT results take the form of success rates (corrected for guessing) that \r\nrange from 0 (guessing) to 1 (correct identification in every case). \r\nThese success rates form a measure of speech intelligibility in this \r\nspecific (MRT) context.\r\n\r\nArticulation Band Correlation-MRT (ABC-MRT) is a signal processing \r\nalgorithm that processes MRT audio files and produces success rates. The \r\ngoal of ABC-MRT is to produce success rates that agree with those produced \r\nby MRT. Thus ABC-MRT is an automated or objective version of MRT and no \r\nhuman subjects are required.\r\n \r\nABC-MRT performs a narrowband (nominally 4 kHz) analysis. ABC-MRT16\r\nis applicable to narrowband, wideband, superwideband, and fullband speech.\r\nABC-MRT processes the first 17 AI bands while ABC-MRT16 processes all 20 AI\r\nbands, as well as an additional \"AI Band 21\" that covers 7 kHz to 20 kHz.\r\nOf equal importance is that ABC-MRT16 incorporates a model for attention\r\nthat allows it to properly operate across the different bandwidths without\r\nany bandwidth detection or switching.\r\n\r\nUnless backwards compatibility is required, ABC-MRT16 is the recommended\r\nalgorithm, even if only narrowband conditions are to be tested. The \r\nattention model makes it superior to ABC-MRT.\r\n\r\nThe software provided here runs in the [Matlab®](http://mathworks.com) or \r\n[Octave](http://www.gnu.org/software/octave) environments.\r\n\r\nApplication of ABC-MRT(16) to a speech communication system-under-test (SUT) \r\nrequires two steps.\r\n1. Pass a set of reference recordings through the SUT to produce a set \r\nof test recordings.\r\n2. Apply ABC-MRT(16) to the test recordings to produce a success rates that \r\ndescribe the intelligibility of the SUT.\r\n\r\nFull details for each step follow. In addition, a small demonstration \r\nprovides an example for getting started.\r\n\r\n## Prepare SUT Recordings\r\n\r\n### Using `git-lfs` to Download Reference Recordings\r\nThe reference recordings in this repository are stored on GitHub using \r\n`git-lfs`. This means that the computer cloning this repo must have `git-lfs`\r\ninstalled in order to properly download the reference recordings.\r\n\r\n**Before cloning this repo, download, install, and initialize\r\n[`git-lfs`](https://git-lfs.github.com).**\r\n\r\n**For the time being, GitHub does not correctly include the reference\r\nrecordings when downloading the code as a `.zip` file.**\r\n\r\nIf you have cloned this repo before installing `git-lfs`, issuing the command\r\n`git lfs pull` may properly download the reference recordings. As a last\r\nresort, cloning the repo to a new location after installing `git-lfs` will\r\ndownload the reference recordings.\r\n\r\n### Locate Reference Recordings\r\nThe SourceAudio folder contains 1200 `.wav` files. The \r\nnaming convention is `TT_bnn_wm_orig.wav`. `TT={F1,F3,M3,M4}` to indicate the \r\ntalker, `nn` is an two digit integer 01 to 50 to indicate the MRT block or \r\nlist, and `m` is a single-digit integer 1 to 6 to indicate the word within \r\nthat list.\r\n\r\nYou now have access to 1200 `.wav` files for ABC-MRT(16) testing. Each file \r\ncontains the same carrier phase followed by a single keyword. For example \r\nin `TT_b01_w1_orig.wav` you will hear “Please select the word went.” The \r\nkeyword is “went.” In `TT_b01_w2_orig.wav` you will hear “Please select the \r\nword sent.” The `.wav` file format is mono, linear PCM, with 16 bits/sample\r\nand 48,000 samples/second.\r\n\r\nThe most thorough testing will use all 1200 files. If this is \r\nprohibitive, then somewhat less thorough testing can be accomplished using \r\na subset of these files. The best subsets are balanced with respect to \r\ntalker and are enumerated by `filelist.m` which produces a Matlab variable, \r\na `.mat` file, and a `.txt` file. The first argument to `filelist.m` is an \r\ninteger `N`, which tells how many `.wav` files will be used, `16\u003c=N\u003c=1200`. An \r\ninput of `N=16` will produce a list of 16 files and increasing `N` to 20 will \r\nadd 4 more files, but will not change the original 16.\r\n\r\nWe have characterized the deviation in ABC-MRT results caused by using\r\n`N\u003c1200` files. This deviation is measured with respect to the ABC-MRT results\r\nwith 1200 files. Note that ABC-MRT results nominally range from 0 to 1, so a\r\ndeviation of .01 is 1% of the intelligibility scale. We have calculated\r\nthese deviations across 139 different conditions:\r\n\r\n\r\n|   N  | RMS Deviation |Maximum Absolute Deviation| Total Speech File Time |\r\n|------|---------------|--------------------------|------------------------|\r\n|   16 |      0.017    |           0.053          |      28 sec            |\r\n|   32 |      0.009    |           0.033          |      57 sec            |\r\n|   64 |      0.005    |           0.014          |     115 sec            |\r\n|  128 |      0.003    |           0.008          |     231 sec            |\r\n|  256 |      0.002    |           0.004          |     462 sec ( 8 min)   |\r\n|  512 |      0.001    |           0.002          |     922 sec (15 min)   |\r\n| 1200 |      0.000    |           0.000          |    2170 sec (36 min)   |\r\n\r\n\r\n### Pass reference recordings through SUT\r\n\r\nThe next task is to pass the desired reference `.wav` files through each SUT \r\nof interest. More generally, one is often interested in a particular \r\noperating configuration for an SUT, possibly combined with \r\nenvironmental factors. To describe this we use the more general term \r\n“condition.” For example, conditions 1, 2, 3, and 4 might be SUT 1 in \r\nquiet, SUT 1 with car noise at 5 dB SNR, SUT 1 with babble noise at 0 dB \r\nSNR, and SUT 2 in quiet, respectively.\r\n\r\nThe recording of the conditions can be scripted and implemented using one \r\nor more computers with suitable sound I/O, but other implementations are \r\npossible as well. The Matlab code `playrec.m` gives one example of how one \r\nmight script these processes.\r\n\r\nABC-MRT(16) expects that reference file `TT_bnn_wm_orig.wav` will produce the \r\ntest file `testpath/Cpp/TT_bnn_wm_cpp.wav`, where `pp` is a two digit integer \r\n00 to 99 that can be used to identify the condition, and testpath is an \r\narbitrary path identifier. For example, passing `F1_b01_w1_orig.wav` \r\nthrough condition 17 should produce `mytestpath/C17/F1_b01_w1_c17.wav`.\r\n \r\nIn general, a condition will have a non-zero delay so perfect \r\nsynchronization of playing and recording may truncate the contents of the \r\nfile. Thus the timing of the recording process must be adjusted so \r\nthat each test file includes at least the entire keyword. In addition \r\neach test file must contain at least 42,000 samples (875 ms duration).\r\nNote that ABC-MRT(16) processing time increases as file length increases.\r\n\r\nFor best results, the playback and recording processes must be virtually \r\ntransparent relative to the condition. A/D and D/A conversion external to \r\nthe computer may be advantageous. The playback level must be properly \r\nadjusted, and the recording level is especially important. If it is too \r\nhigh, clipping may impair the A/D process. If it is too low quantization \r\nnoise may impair the A/D process.\r\n\r\nA higher level of automation might be achieved by concatenating the \r\ndesired reference files, playing that single long file while recording a \r\nsingle long test file, then cutting the long test file into properly named \r\nindividual test files. Or one might modify the ABC-MRT(16) software to accept \r\nand properly parse a single long test file. One might attempt to further \r\nstreamline the process by passing only keywords (and not the carrier \r\nphrase) through the condition. For some conditions this change may \r\nproduce identical results. But if the condition includes any adaptive \r\nprocess (e.g., noise reduction) then this change may alter the results. \r\nIf this is the case, then consider how the condition is used in practice. \r\nIn many cases transmitting a sequence of sentences (carrier phrases with \r\nkeywords) may be more realistic than transmitting a list of unrelated \r\nwords (keywords alone).\r\n\r\n## Apply ABC-MRT or ABC-MRT16 to the test recordings\r\nRun `ABC_MRT.m` or `ABC_MRT16.m`. It will process all `.wav` files (up to 1200\r\nof them) for a single condition. A simple script can apply `ABC_MRT.m`\r\nor `ABC_MRT16.m` to multiple conditions. Examples are provided in\r\n`ABC_MRTdemo.m` and `ABC_MRT16demo.m`.\r\n\r\nThe call to `ABC_MRT.m` is:\r\n\r\n```\r\n[phi_hat, success]=ABC_MRT(speech_path,cond_num,n_files,verbose)\r\n```\r\n\r\nand the call to `ABC_MRT16.m` is:\r\n\r\n```\r\n[phi_hat, success]=ABC_MRT16(speech_path,cond_num,n_files,verbose)\r\n```\r\n\r\n - `speech_path` is a string that gives the path to the speech files\r\n\r\n - `cond_num` is the condition number, 0 to 99. This is used to form the end \r\nof the path and the filenames as specified below.\r\n\r\n - `n_files` is the number of speech files to use, 16 \u003c= N \u003c= 1200. Using \r\nmore speech files gives a more robust result, but takes longer. One can \r\nuse `filelist.m` to generate the list of `.wav` files required for any value \r\nof `n_files`.\r\n\r\n - `verbose` is set to any nonzero value to force progress reporting\r\n\r\n - `success` is a column vector with length n_files that gives the success \r\nrate for each file involved in the test.\r\n\r\n - `phi_hat` is a scalar that gives the final intelligibility estimate for \r\nthe condition. `phi_hat` is expected to range from 0 to 1, similar to MRT\r\nresults. Larger values of `phi_hat` indicate higher levels of speech\r\nintelligibility\r\n\r\n`ABC_MRT.m` and `ABC_MRT16.m` expect a specific file naming convention. Here are some \r\nexamples:\r\n\r\n```\r\nABC_MRT(e:/soundfiles/MRT/,1,1200,*), requires 1200 .wav files of the form \r\ne:/soundfiles/MRT/C01/TT_bnn_wm_c01.wav\r\n\r\nABC_MRT(e:/soundfiles/MRT/,17,16,*), requires 16 .wav files of the form \r\ne:/soundfiles/MRT/C17/TT_bnn_wm_c17.wav\r\n\r\nABC_MRT16(e:/soundfiles/MRT/,1,1200,*), requires 1200 .wav files of the form \r\ne:/soundfiles/MRT/C01/TT_bnn_wm_c01.wav\r\n\r\nABC_MRT16(e:/soundfiles/MRT/,17,16,*), requires 16 .wav files of the form \r\ne:/soundfiles/MRT/C17/TT_bnn_wm_c17.wav\r\n```\r\n\r\nIn every case the base of the filenames takes the form TT_bnn_wm with:\r\n\r\n```\r\n  TT=F1, F3, M3, or M4 (talker specification)\r\n  nn=1 to 50 (list specification)\r\n  m=1 to 6 (word specification)\r\n```\r\n\r\n(Note that 4 x 50 x 6 =1200, so these ranges allow specification of all \r\n1200 files)\r\n\r\n## Demonstration\r\nRun `ABC_MRTdemo.m`. It applies ABC_MRT to 6 different conditions, using \r\njust 16 .wav files per condition. The required files are provided in the \r\ndemo directory. The results (phi_hat values) are plotted against actual \r\nMRT scores. Using 16 files is fast and it shows how the software works, \r\nbut it gives only rough results. In practice one should use as many files \r\nas practical.\r\n\r\nOr run `ABC_MRT16demo.m`. It applies ABC_MRT16 to 6 different conditions, using \r\njust 16 .wav files per condition. The required files are provided in the \r\ndemo directory. The results (phi_hat values) are plotted against actual \r\nMRT scores. Using 16 files is fast and it shows how the software works, \r\nbut it gives only rough results. In practice one should use as many files \r\nas practical.\r\n\r\n## References\r\n[\u003cb id=\"f1\"\u003e1\u003c/b\u003e] S. Voran \"Using articulation index band correlations to objectively \r\nestimate speech intelligibility consistent with the modified rhyme test,\" \r\nProc. 2013 IEEE Workshop on Applications of Signal Processing to Audio and\r\nAcoustics, New Paltz, NY, October 20- 23, 2013. Available at \r\nwww.its.bldrdoc.gov/audio after October 20, 2013.\r\n\r\n[\u003cb id=\"f2\"\u003e2\u003c/b\u003e] S. Voran \"A multiple bandwidth objective speech intelligibility \r\nestimator based on articulation index band correlations and attention,\"\r\nProc. 2017 IEEE International Conference on Acoustics, Speech, and \r\nSignal Processing, New Orleans, March 5-9, 2017. Available at\r\nwww.its.bldrdoc.gov/audio.\r\n\r\n[\u003cb id=\"f3\"\u003e3\u003c/b\u003e] ANSI S3.2, \"American national standard method for measuring the \r\nintelligibility of speech over communication systems,\" 1989.\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fntia%2Fabc-mrt16","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fntia%2Fabc-mrt16","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fntia%2Fabc-mrt16/lists"}