{"id":15136523,"url":"https://github.com/mariatepei/vt_thesis_mtepei","last_synced_at":"2026-01-19T04:32:22.188Z","repository":{"id":243941859,"uuid":"807526066","full_name":"mariatepei/VT_thesis_MTepei","owner":"mariatepei","description":"This repository accompanies my MSc Thesis for the degree Voice Technology, storing all referenced data and other relevant resources.","archived":false,"fork":false,"pushed_at":"2024-06-11T20:32:56.000Z","size":1190,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-08T14:34:18.075Z","etag":null,"topics":["data-augmentation","fastspeech2","speech-recognition","whisper"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mariatepei.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-05-29T09:12:22.000Z","updated_at":"2024-06-11T20:35:04.000Z","dependencies_parsed_at":"2024-06-13T19:16:28.858Z","dependency_job_id":null,"html_url":"https://github.com/mariatepei/VT_thesis_MTepei","commit_stats":null,"previous_names":["mariatepei/vt_thesis_mtepei"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mariatepei%2FVT_thesis_MTepei","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mariatepei%2FVT_thesis_MTepei/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mariatepei%2FVT_thesis_MTepei/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mariatepei%2FVT_thesis_MTepei/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mariatepei","download_url":"https://codeload.github.com/mariatepei/VT_thesis_MTepei/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247327542,"owners_count":20921058,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-augmentation","fastspeech2","speech-recognition","whisper"],"created_at":"2024-09-26T06:22:24.797Z","updated_at":"2026-01-19T04:32:22.156Z","avatar_url":"https://github.com/mariatepei.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# VT_thesis\n\nThis is the acompanying repository for my master's thesis paper in Voice Technology at the University of Groningen/Campus Fryslan: **Addressing ASR Bias Against Foreign-Accented Dutch: A Synthetic Data Approach**.\nSubmitted on June 11th, 2024.\n\nThe paper can be found at (thesis repo link).\n\n## Abstract\nDespite substantial improvements in automatic speech recognition (ASR) over the last years, the high performance achieved for ”standard speakers” does not hold across all genders, ages, or foreign ac\u0002cents. As a result, an important area of research is inclusive ASR, aimed at reducing the performance gaps such systems display across subgroups of the population. In the present thesis, I evaluate one of the most recent and robust ASR systems (OpenAI’s Whisper) to uncover and assess the level of bias it displays against foreign-accented Dutch. Additionally, I investigate whether synthetically accented speech samples obtained from a fine-tuned speech synthesis model (FastSpeech2) can act as a viable data augmentation tool to create additional training data for Whisper, in a fine-tuning transfer learning paradigm. By investigating bias, as opposed to WER reduction, I specifically pay attention to both the improvement in performance on foreign-accented Dutch and the potential decrease in performance on native Dutch. Experimental results show that fine-tuning Whisper on synthetic accented speech data does increase its performance on natural speech samples, although this comes at the cost of decreased performance on native samples after fine-tuning. Additionally, the insights from fine-tuning Whisper put into question its suitability for this learning paradigm, as its large number of parameters displays increased stability on small, low-resource datasets.\n\n## Repo structure\nVarious files I have used to reach my results can be found here, related both to the FastSpeech2 part and to the Whisper part. Most importantly, the notebooks I've used for fine-tuning whisper can be found in the whisper-related folder and contain the hyperparameters defined for training.\n\n## Data sources and fine-tuned checkpoints\nMost of the data I have used and all the fine-tuned model checkpoints are saved to my [HuggingFace](https://huggingface.co/mariatepei) for convenience. The CSS10 dataset of native Dutch speech can be found [here](https://github.com/Kyubyong/css10/tree/master).\n\n## Acknowledgments\nThe FastSpeech 2 implementation I've used is [here](https://github.com/ming024/FastSpeech2).\nIn figuring out how to fine-tune Whisper on a custom dataset, important insights came from [Sanchit's bolgpost](https://huggingface.co/blog/fine-tune-whisper#closing-remarks) and [Trelis Research YouTube videos](https://www.youtube.com/@TrelisResearch).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmariatepei%2Fvt_thesis_mtepei","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmariatepei%2Fvt_thesis_mtepei","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmariatepei%2Fvt_thesis_mtepei/lists"}