{"id":16298858,"url":"https://github.com/braden1996/audio-synthesiser","last_synced_at":"2025-03-20T05:30:45.045Z","repository":{"id":85770738,"uuid":"166553533","full_name":"Braden1996/Audio-Synthesiser","owner":"Braden1996","description":"A fourier-based audio-synthesiser wrote in MATLAB as a university project.","archived":false,"fork":false,"pushed_at":"2019-01-19T14:30:59.000Z","size":1992,"stargazers_count":12,"open_issues_count":0,"forks_count":2,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-02-28T23:16:06.645Z","etag":null,"topics":["adsr","audio-synthesis","flanger","fourier-analysis","fourier-transform","matlab","matlab-gui","phase-vocoder","spectral-analysis","spectrogram"],"latest_commit_sha":null,"homepage":"","language":"MATLAB","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Braden1996.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-01-19T14:08:01.000Z","updated_at":"2024-10-24T11:44:52.000Z","dependencies_parsed_at":null,"dependency_job_id":"d34349d8-970b-4121-b225-26bef1f1091b","html_url":"https://github.com/Braden1996/Audio-Synthesiser","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Braden1996%2FAudio-Synthesiser","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Braden1996%2FAudio-Synthesiser/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Braden1996%2FAudio-Synthesiser/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Braden1996%2FAudio-Synthesiser/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Braden1996","download_url":"https://codeload.github.com/Braden1996/Audio-Synthesiser/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244056343,"owners_count":20390719,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["adsr","audio-synthesis","flanger","fourier-analysis","fourier-transform","matlab","matlab-gui","phase-vocoder","spectral-analysis","spectrogram"],"created_at":"2024-10-10T20:45:46.816Z","updated_at":"2025-03-20T05:30:45.033Z","avatar_url":"https://github.com/Braden1996.png","language":"MATLAB","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Fourier-based Audio Synthesiser\n\n\u003ch1 align=\"center\"\u003e\n  \u003cimg src=\"/resources/images/maingui.png?raw=true\"\u003e\n\u003c/h1\u003e\n\n## Contents\n\n- [1 Basic Requirements](#1-basic-requirements)\n  - [1.1 Synthesiser Overview](#11-synthesiser-overview)\n  - [1.2 Spectrogram Editing](#12-spectrogram-editing)\n  - [1.3 Audio Playback and the Phase Vocoder](#13-audio-playback-and-the-phase-vocoder)\n  - [1.4 Audio Effects](#14-audio-effects)\n    - [1.4.1 ADSR Volume Shaping](#14.1-adsr-volume-shaping)\n    - [1.4.2 Wah-Wah](#142-wah-wah)\n    - [1.4.3 Flanger](#143-flanger)\n- [2 Additional Features](#2additional-features)\n  - [2.1 Advance User Interface](#21-advance-user-interface)\n  - [2.2 Playback Control](#22-playback-control)\n  - [2.3 Modular Effects and Editable Pipeline](#23-modular-effects-and-editable-pipeline)\n  - [2.4 Add/Subtract Spectral Energy](#24-add/subtract-spectral-energy)\n  - [2.5 Software Design](#25-software-design)\n- [References](#references)\n\n## 1 Basic Requirements\n\nThroughout this section, I shall discuss the implementation of all the basic\nrequirements I’ve included in my synthesiser. There are many places where\nwe refer to the additional features, as they intertwine quite naturally with\nthe rest of the application. For the marker’s benefit, I try to leave out the\ndetails regarding additional features so they can be discussed separately.\u003csup\u003e1\u003c/sup\u003e\n\n**\u003csup\u003e1\u003c/sup\u003e** All sources within the `+thirdparty` package is credited to their rightful owners.\n\n### 1.1 Synthesiser Overview\n\nTo launch the synthesiser, open MATLAB and add the source-code folder to\nthe search path then run the entry-point file `synthesiser.m`.\n\nOnce the synthesiser has opened, you will notice most of the GUI components are\nin a disabled state. This is because an audio file has not yet been loaded.\nTo load an audio file, click the _Open audio file_ button in the tool-bar.\nThe majority of the GUI components should now become enabled.\n\nAll the available operations should be fairly self-explanatory - keep in\nmind that you can learn more by hovering to reveal a tool-tip.\n\n### 1.2 Spectrogram Editing\n\nUpon loading some audio, you are presented with both an image of the audio\nsignal (on top) and its spectrogram (on bottom). Editing of the spectrogram\nis done by creating a mask which is then applied to the spectrogram in some\narbitrary way. You can create and modify your mask simply by brushing\nover the spectrogram image - this is done by holding your mouse down on\nits axes and dragging around.\n\nTo the left of the spectrogram axes you will find a panel titled _Brush\nSettings_. This contains a variety of configurations which alter the way in\nwhich your brush effects the mask. For example, you can change the size and\nshape (circle or square) of the brush.\n\nTo the right of the spectrogram axes, you will see a panel titled _Mask\nDisplay Alpha_. This allows you to modify the transparency of the mask;\nrevealing or covering the spectrogram beneath. It is purely aesthetic and has\nnoeffect on the operations performed. You will notice there are three types\nof masks, I shall discuss these now:\n\n- Mask: The currently buffered mask, i.e. to be used by the next mask-\n  utilising operation.\n- Drag Mask: The current brush motion, i.e. between mouse down and\n  mouse up.\n- Saved Mask: The mask which was previously used by a mask-utilising\n  operation.\n\nThe tool-bar contains four tools directly relating to the mask and/or\nspectrogram. These are described below:\n\n- Clear Mask: Erase the currentMask.\n- Inverse Mask: Inverse the selection of the mask, i.e. M ask= 1−\n  M ask.\n- Choose Colour Map: Display a modal of alternative colour-maps to\n  display the spectrogram.\n- Select Mask Colour: Display a modal of alternative colours to dis-\n  play the mask.\n\nThe operations which utilise the mask sit to the left of the spectrogram\nin a panel titled _Operations_. The only operation which was required by\nthe basic requirements is _Apply Spectrogram Mask_. This simply takes the\npointwise product between the spectrogram and its mask, i.e. so that the\nunselected areas of the spectrogram are discarded.\n\nThe other operations will be further discussed in\n[subsection 2.4: Add/Subtract Spectral Energy](#2.4-add/subtract-spectral-energy).\n\n### 1.3 Audio Playback and the Phase Vocoder\n\nThere exists two ways in which you can playback the audio you’ve developed.\nOne of which resides in the tool-bar, and integrates with an additional feature,\nwhich we discuss in [subsection 2.2: Playback Control](#2.2-playback-control).\nThe other is presented in the form of a keyboard, which resides below the spectrogram image.\n\nThe primary difference between the two is that the keyboard internally uses a\nphase vocoder to provide a greater range of sound. Middle C, that is C4, will\nplay the audio with no additional effect. However, keys to the left will play\nat a higher pitched note and conversely keys to the right play at a lower pitch\nnote.\n\n### 1.4 Audio Effects\n\nThe following subsection will discuss ADSR volume shaping along with two\nadditional audio effects which were implemented.\n\n#### 1.4.1 ADSR Volume Shaping\n\nADSR is an acronym for Attack, Decay, Sustain, Release and is a technique\nused to produce more natural sounds. We implement this by interpolating\nthe audio signal’s amplitude between five control points, whose position can\nbe customised by the user.\n\nThe ADSR can be accessed by pressing its corresponding button within\nthe tool-bar, which presents the user with a modal.\n\n#### 1.4.2 Wah-Wah\n\nThe wah-wah is essentially a band-pass filter which is modulated over time\nand then mixed back in with the direct signal. This gives the effect of changing\nfrom _wwww_ sounds through to _aaaahhhh_ sounds.\n\nSimilar to the ADSR, the wah-wah can be accessed by pressing its corresponding\nbutton within the tool-bar, which presents the user with a modal.\n\n#### 1.4.3 Flanger\n\nThe flanger is an audio effect which is produced using a comb filter - mixing\ntwo identical signals together, except applying a small delay (0ms-15ms) -\nand some modulation.\n\nOnce again, the flanger can be accessed by pressing its corresponding\nbutton within the tool-bar, which presents the user with a modal.\n\n## 2 Additional Features\n\nThroughout this section, I shall discuss the features I have implemented\nwhich go beyond the basic specifications. This includes both entirely new\nfunctionality and drastic improvements to the basic requirements.\n\n### 2.1 Advance User Interface\n\nTo make the synthesiser more of a usable application, I’ve invested a lot of\ntime into the user-interface. In particular, I shall discuss three main aspects.\n\nFirstly, the variety of brush settings allow for some fairly powerful drawing\nto the spectrogram mask. For example, the use of opacity allows you to\ncapture aspects of the spectrogram at different levels. This is due to the\nmask not being binary, but instead a floating point value between 0 and 1.\n\nI utilised this idea of opacity and implemented the ability to draw with a\nGaussian blur [[2]](#references). The user is free to control the strength of the blur as they\nmay.\n\n### 2.2 Playback Control\n\nWithin the audio signal’s amplitude/time axes, the user can select particular\nareas of the audio signal. They are then free to loop playback of this selection\nusing the controls available within the tool-bar.\n\nAlso, certain operations are only applied on the selected slice of audio.\nThis is extremely useful as it allows you to easily apply the same audio\neffect, except with different parameters, on separate, overlapping, or identical\nselections of audio. It also pairs very well the editable pipeline ([see 2.3](#2.3-modular-effects-and-editable-pipeline)).\n\n### 2.3 Modular Effects and Editable Pipeline\n\nOne of the most fundamental additional features I implemented was that of\nhaving modular effects using an editable pipeline. Conceptually, this works\nby recording each applied effect, and some effect data, as a node in a linked\n\nlist. We are then able to pipe an audio signal through each effect in this\nlinked list and use the output as we see fit.\n\nThis allows us to have full control over the history of effects that have been\napplied. For example: we can apply the same effect twice, remove effects,\nreorder effects and even insert new effects between two existing effects.\n\nI have provided a visual representation of this pipeline to the user through\nthe use of the listbox component. When a listbox item, i.e. an effect, is\nselected we set the synthesiser’s state to reflect that as the current loaded\naudio signal. This allows the user to easily analyse the changes each effect\nmakes.\n\nAlso, it is worth noting, this maps very nicely with the real-time ability\nto play audio; as when the user selects a different effect the playing audio\nsignal switches instantaneous without losing its playback position.\n\nFor performance reasons, we cache the post-effect signal at each node and\nreapply them only when they become stale, i.e. an alteration previously in\nthe pipe has occurred.\n\n### 2.4 Add/Subtract Spectral Energy\n\nFor one of the primary features, the spectrogram mask played a fairly minor\npart in the overall synthesiser. Wanting to change this, I decided to try design\nsome new operations which utilised it. I chose to try create a way in which\nit could be used to add/subtract energy to the selected areas; effectively\nallowing you to draw your own audio signal.\n\nDuring development, I wasn’t quite sure how to justify what scaling factor\nshould be applied to the mask before adding it to the spectrogram mask. I\ntried several seemingly logical values, such as: the mean, max and even a\nrange of absolute values. None of which seemed to work.\n\nUpon investigation, I could see that the changes to the STFT were visible\nin the spectrogram but not the audio signal. After talking to the lab assistant,\nZafi, he discovered that the ISTFT seems to scale the amplitude of the mask\nby the number of frequency bins.[[1]](#references) Using this knowledge, I now appear to\nget desirable results when I multiply the scaling factor byf /h- where f is\nthe STFT’s F-point FFT column count, and h is the offset.\n\n### 2.5 Software Design\n\nWhilst developing the synthesiser, I wanted to try learn some deeper\nfunctionality MATLAB has to offer. For the most part, this has the bonus of\nimproving maintainability and straightforwardness of the code. Below is a\nbrief list of what I’ve done:\n\n- Decoupled my code to help improve structure and organisation. This\n  includes routing the GUI callbacks away of the entry-point file.\n- Followed a class-base design in effort to improve maintainability and\n  re-usability. Notably, created a class (AbstractMouse) which makes it\n  very easy to detect mouse interactions for anyGUI component.\n- Tasked the GPU to compute some more computationally extensive\n  tasks, e.g. applying masks etc...\n- Utilised MATLAB’s inputParser [[3]](#references) to validate input arguments to\n  necessary functions.\n- Kept code up to a consistent standard, including naming conventions.\n\n## References\n\n[1] Zafi Syed. Add Spectral Energy - Scaling Factor. Personal communication.\nEmail and in person. 2016.\n\n[2] The MathWorks, Inc. 2-D Gaussian filtering of images - MATLAB\nimgaussfilt. 2016.url: https://uk.mathworks.com/help/images/\nref/imgaussfilt.html(visited on 12/01/2016).\n\n[3] The MathWorks, Inc. 2-D Gaussian filtering of images - MATLAB\nimgaussfilt. 2016.url: https://uk.mathworks.com/help/images/\nref/imgaussfilt.html(visited on 12/01/2016).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbraden1996%2Faudio-synthesiser","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbraden1996%2Faudio-synthesiser","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbraden1996%2Faudio-synthesiser/lists"}