{"id":14990302,"url":"https://github.com/zakaton/pink-trombone","last_synced_at":"2025-04-04T21:06:06.856Z","repository":{"id":39527003,"uuid":"175134792","full_name":"zakaton/Pink-Trombone","owner":"zakaton","description":"A programmable version of Neil Thapen's Pink Trombone","archived":false,"fork":false,"pushed_at":"2025-01-11T07:35:30.000Z","size":17903,"stargazers_count":177,"open_issues_count":29,"forks_count":30,"subscribers_count":8,"default_branch":"master","last_synced_at":"2025-04-04T21:06:01.159Z","etag":null,"topics":["api","auditory-display","pink-trombone","procedural-audio","sound-design","speech-synthesis","vocal-tract","voice","voice-design","voice-synthesis","voice-ui","web-audio","web-audio-worklet","web-component"],"latest_commit_sha":null,"homepage":"https://zakaton.github.io/Pink-Trombone/","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zakaton.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-03-12T04:22:41.000Z","updated_at":"2025-03-16T14:38:10.000Z","dependencies_parsed_at":"2025-02-13T16:12:20.552Z","dependency_job_id":"424c580a-4c21-49fd-8a31-e63a994e624f","html_url":"https://github.com/zakaton/Pink-Trombone","commit_stats":{"total_commits":53,"total_committers":1,"mean_commits":53.0,"dds":0.0,"last_synced_commit":"d80ae9076c12df9516d43adc71b329e70fe383c6"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zakaton%2FPink-Trombone","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zakaton%2FPink-Trombone/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zakaton%2FPink-Trombone/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zakaton%2FPink-Trombone/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zakaton","download_url":"https://codeload.github.com/zakaton/Pink-Trombone/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247249524,"owners_count":20908212,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api","auditory-display","pink-trombone","procedural-audio","sound-design","speech-synthesis","vocal-tract","voice","voice-design","voice-synthesis","voice-ui","web-audio","web-audio-worklet","web-component"],"created_at":"2024-09-24T14:19:51.510Z","updated_at":"2025-04-04T21:06:06.836Z","avatar_url":"https://github.com/zakaton.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ca href=\"https://twitter.com/ConcreteSciFi/status/1138555909133594624\" target=\"_blank\"\u003e![Fabien](images/pinkTrombone.gif)\u003c/a\u003e\u003cbr\u003e\n_Sound is generated in the glottis (at the bottom left), then filtered by the shape of the vocal tract. The voicebox controls the pitch and intensity of the initial sound_ - [Neil Thapen](http://venuspatrol.nfshost.com/)\n\n# 🗣️ Pink Trombone - Bare-handed Speech Synthesis\n_A programmable version of [Neil Thapen's](http://venuspatrol.nfshost.com/) famous and wonderful [Pink Trombone](https://dood.al/pinktrombone/)_\n\n## 📚 Table of Contents\n[📦 Setting Up](#-setting-up)\n\n[👄 Producing Sound](#-producing-sound)\n\n[👀 Enabling and Disabling the UI](#-enabling-and-disabling-the-ui)\n\n[🎛️ Audio Parameters](#-audio-parameters)\n\n[🎺 Manipulating Vocal Tract Constrictions](#-manipulating-vocal-tract-constrictions)\n\n[👅 Common Phonemes](#-common-phonemes)\n\n[🏆 Developer Showcase](#-developer-showcase)\n\n[🙏 Developer Wishlist](#-developer-wishlist)\n\n[📖 Bibliography](#-bibliography)\n\n## 📦 Setting Up\n1. Save a local copy of [`pink-trombone.min.js`](https://raw.githubusercontent.com/zakaton/Pink-Trombone/master/pink-trombone.min.js) and [`pink-trombone-worklet-processor.min.js`](https://raw.githubusercontent.com/zakaton/Pink-Trombone/master/pink-trombone-worklet-processor.min.js) and make sure they're both in the same relative location (the first will import the other as a [Audio Worklet Processor](https://developers.google.com/web/updates/2017/12/audio-worklet))\n\n2. In your HTML `\u003chead\u003e\u003c/head\u003e` element, insert the file in a script element as a [module](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Modules):\n```html\n\u003cscript src=\"pink-trombone.min.js\" type=\"module\"\u003e\u003c/script\u003e\n```\n\n3. In your HTML `\u003cbody\u003e\u003c/body\u003e` element, insert the following custom element:\n```html\n\u003cpink-trombone\u003e\u003c/pink-trombone\u003e\n```\n\n4. In your JavaScript code, grab the `\u003cpink-trombone\u003e\u003c/pink-trombone\u003e` element:\n```javascript\nvar pinkTromboneElement = document.querySelector(\"pink-trombone\");\n```\n\n5. Add a `load` eventListener to the `pinkTromboneElement` element:\n```javascript\npinkTromboneElement.addEventListener(\"load\", myCallback);\n```\n\n6. In the `\"load\"` callback, assign an [Audio Context](https://developer.mozilla.org/en-US/docs/Web/API/AudioContext) using `.setAudioContext(myAudioContext)` (if none is specified, an Audio Context instance is created for you):\n```javascript\nfunction myCallback(event) {\n  pinkTromboneElement.setAudioContext(myAudioContext)\n}\n```\nThis method returns a [Promise](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise) once the [AudioWorkletProcessor](https://github.com/zakaton/Pink-Trombone/blob/master/script/audio/nodes/pinkTrombone/processors/WorkletProcessor.js) module is loaded.\n\n7. In the [promise resolution](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/then), a [Pink Trombone audio node](https://github.com/zakaton/Pink-Trombone/blob/master/script/audio/nodes/pinkTrombone/AudioNode.js) is created, which you can connect to other audio nodes from the scope of the `\u003cpink-trombone\u003e\u003c/pink-trombone\u003e` element:\n```javascript\nfunction myCallback(event) {\n  pinkTromboneElement.setAudioContext(myAudioContext)\n    .then(() =\u003e {\n      const audioContext = pinkTromboneElement.audioContext\n      pinkTromboneElement.connect(audioContext.destination);\n    });\n}\n```\n\n## 👄 Producing Sound\n😃 To start generating sound, run the `.start()` method:\n```javascript\npinkTromboneElement.start();\n```\n\n🤐 To stop generating sound, run the `.stop()` method:\n```javascript\npinkTromboneElement.stop();\n```\n\n## 👀 Enabling and Disabling the UI\n🙂 To show the interactive visualization:\n```javascript\npinkTromboneElement.enableUI();\n```\n✍️ To start animating the visualization:\n```javascript\npinkTromboneElement.startUI();\n```\n\n🛑 To stop animating the visualization:\n```javascript\npinkTromboneElement.stopUI();\n```\n\n😊 To hide the interactive visualization:\n```javascript\npinkTromboneElement.disableUI();\n```\n\n\n## 🎛️ Audio Parameters\nThe [audio parameters](https://developer.mozilla.org/en-US/docs/Web/API/AudioParam) of the [Pink Trombone audio node](https://github.com/zakaton/Pink-Trombone/blob/master/script/audio/nodes/pinkTrombone/AudioNode.js) can be accessed from the `\u003cpink-trombone\u003e\u003c/pink-trombone\u003e` element's scope:\n\n🎚️ Intensity\n```javascript\npinkTromboneElement.intensity;\n```\n🎵 Frequency\n```javascript\npinkTromboneElement.frequency;\n```\n👄 Tenseness\n```javascript\npinkTromboneElement.tenseness;\n```\n📢 Loudness\n```javascript\npinkTromboneElement.loudness;\n```\n〰️ Vibrato\n```javascript\npinkTromboneElement.vibrato.frequency;\npinkTromboneElement.vibrato.gain;\npinkTromboneElement.vibrato.wobble;\n```\n👅 Tongue\n```javascript\n// 'index' and 'diameter' refer to the tongue's location in the mouth\npinkTromboneElement.tongue.index;\npinkTromboneElement.tongue.diameter;\n```\n\nTo change the [voiceness](https://en.wikipedia.org/wiki/Voice_(phonetics)) between voiced and voiceless, change the `.tenseness` and `.loudness` [audio parameters](https://developer.mozilla.org/en-US/docs/Web/API/AudioParam) as follows:\n```javascript\nfunction setVoiceness(voiceness) {\n  const tenseness = 1 - Math.cos((voiceness) * Math.PI * 0.5);\n  const loudness = Math.pow(tenseness, 0.25);\n  \n  pinkTromboneElement.tenseness.value = tenseness;\n  pinkTromboneElement.loudness.value = loudness;\n}\n\n// voiced\nsetVoiceness(1);\n\n// voiceless\nsetVoiceness(0);\n```\n_Later on I may add a `.voiceness` [audio parameter](https://developer.mozilla.org/en-US/docs/Web/API/AudioParam) that automates this - for now I'm just adopting the [original version](https://dood.al/pinktrombone/)_\n\n## 🎺 Manipulating Vocal Tract Constrictions\nVocal Tract constrictions comprise of an object containing `.index` and `.diameter` [Audio Parameter](https://developer.mozilla.org/en-US/docs/Web/API/AudioParam) properties that are implicitly connected to the [Pink Trombone audio node](https://github.com/zakaton/Pink-Trombone/blob/master/script/audio/nodes/pinkTrombone/AudioNode.js)\n\nTo add a vocal tract constriction:\n```javascript\nvar myConstriction = pinkTromboneElement.newConstriction(indexValue, diameterValue);\n```\n\nTo set a vocal tract constriction:\n```javascript\nmyConstriction.index.value = newIndexValue;\nmyConstriction.diameter.value = newDiameterValue;\n```\n\nTo remove a vocal tract constriction:\n```javascript\npinkTromboneElement.removeConstriction(myConstriction);\n```\n\n## 👅 Common Phonemes\nFor reference, here are some preset index \u0026 diameter preset values for [some phonemes](http://www.internationalphoneticalphabet.org/ipa-sounds/ipa-chart-with-sounds/):\n\n👅 Tongue phonemes:\n\næ [pat]\n  - index : 14.93\n  - diameter : 2.78\n  \nɑ [part]\n  - index : 2.3\n  - diameter : 12.75\n  \nɒ [pot]\n  - index : 12\n  - diameter : 2.05\n  \nɔ [port (rounded)]\n  - index : 17.7\n  - diameter : 2.05\n  \nɪ [pit]\n  - index : 26.11\n  - diameter : 2.87\n  \ni [peat]\n  - index : 27.2\n  - diameter : 2.2\n  \ne [pet]\n  - index : 19.4\n  - diameter : 3.43\n  \nʌ [put]\n  - index : 17.8\n  - diameter : 2.46\n  \nu [poot (rounded)]\n  - index : 22.8\n  - diameter : 2.05\n  \nə [pert]\n  - index : 20.7\n  - diameter : 2.8\n\n\n🎺 [Vocal Tract Constriction](https://en.wikipedia.org/wiki/Manner_of_articulation) phonemes:\n_voiced and voiceless consonants share the same values, differing in [voiceness](https://en.wikipedia.org/wiki/Voice_(phonetics))_\n\n- [Fricatives](https://en.wikipedia.org/wiki/Fricative_consonant)\n\n  - (ʒ, ʃ) [\"s\" in \"pleasure\"]\n    - index : 31\n    - diameter : 0.6\n    \n  - (z, s) [\"z\" in \"zoo\"]\n    - index : 36\n    - diameter : 0.6\n    \n  - (v, f) [\"v\" in \"very\"]\n    - index : 41\n    - diameter : 0.5\n    \n- [Stops](https://en.wikipedia.org/wiki/Stop_consonant)\n\n  - (g, k) [\"g\" in \"go\"]\n    - index : 20\n    - diameter : 0\n    \n  - (d, t) [\"d\" in \"den\"]\n    - index : 36\n    - diameter : 0\n    \n  - (b, p) [\"b\" in \"bad\"]\n    - index : 41\n    - diameter : 0\n\n- [Nasals](https://en.wikipedia.org/wiki/Nasal_consonant)\n\n  - (ŋ) [\"ng\" in \"hang\"]\n    - index : 20\n    - diameter : -1\n    \n  - (n) [\"n\" in \"not\"]\n    - index : 36\n    - diameter : -1\n    \n  - (m) [\"m\" in \"woman\"]\n    - index : 41\n    - diameter : -1\n\n## 🏆 Developer Showcase\n*Send us an email at zack@ukaton.com if you have a cool application made with our api!*  \n\u003ca href=\"https://twitter.com/ConcreteSciFi/status/1565439155131129857?s=20\" target=\"_target\"\u003e![Zack](developerShowcase/developerShowcase5.gif)\u003c/a\u003e\n\u003cbr\u003e\n\u003ca href=\"https://twitter.com/ConcreteSciFi/status/1212548283895177216?s=20\" target=\"_target\"\u003e![Zack](developerShowcase/developerShowcase4.gif)\u003c/a\u003e\n\u003cbr\u003e\n\u003ca href=\"https://twitter.com/ConcreteSciFi/status/1200218613669449728?s=20\" target=\"_target\"\u003e![Zack](developerShowcase/developerShowcase3.gif)\u003c/a\u003e\n\u003cbr\u003e\n\u003ca href=\"https://twitter.com/ConcreteSciFi/status/1189319140231786498?s=20\" target=\"_target\"\u003e![Zack](developerShowcase/developerShowcase2.gif)\u003c/a\u003e\n\u003cbr\u003e\n\u003ca href=\"https://twitter.com/ConcreteSciFi/status/1162501265374322688?s=20\" target=\"_target\"\u003e![Zack](developerShowcase/developerShowcase1.gif)\u003c/a\u003e\n\u003cbr\u003e\n\u003ca href=\"https://twitter.com/ConcreteSciFi/status/1138555909133594624?s=20\" target=\"_target\"\u003e![Zack](developerShowcase/developerShowcase0.gif)\u003c/a\u003e\n\u003cbr\u003e\n\n## 🙏 Developer Wishlist\n*Our time is limited, so we'd greatly appreciate it if you guys could implement some of these ideas:*\n- [ ] __IPA Speak n' See__ 🗣️💬 - Take input speech from the user using the [Media Recording API](https://developer.mozilla.org/en-US/docs/Web/API/MediaStream_Recording_API) and approximate their articulation using the Pink Trombone, allowing speakers to visualize how they speak.\n- [ ] __Phonetic Voice Editor__ 🎹👄⌨️ - Create a cross between a [Text Editor](https://en.wikipedia.org/wiki/List_of_text_editors) and a [Digita Audio Workstation](https://en.wikipedia.org/wiki/Digital_audio_workstation), where the user can type in phonemes instead of characters, with [automation](https://en.wikipedia.org/wiki/Mix_automation) to programmatically adjust the cadence, pitch, and other features over time.\n- [ ] __SSML Simulator__ 📝💬 - Implement a [Speech Synthesis Markup Language](https://en.wikipedia.org/wiki/Speech_Synthesis_Markup_Language) emulator that can take an [utterance](https://developer.mozilla.org/en-US/docs/Web/API/SpeechSynthesisUtterance) and process the speech request using Pink Trombone's audio processing\n\n## 📖 Bibliography\n- [Julius O. Smith III, \"Physical audio signal processing for virtual musical instruments and audio effects.\"](https://ccrma.stanford.edu/~jos/pasp/)\n- [Story, Brad H. \"A parametric model of the vocal tract area function for vowel and consonant simulation.\" \nThe Journal of the Acoustical Society of America 117.5 (2005): 3231-3254.](http://sal.arizona.edu/sites/default/files/story_jasa2005.pdf)\n- [Lu, Hui-Ling, and J. O. Smith. \"Glottal source modeling for singing voice synthesis.\" \nProceedings of the 2000 International Computer Music Conference. 2000.](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.142.5360\u0026rep=rep1\u0026type=pdf)\n- [Mullen, Jack. Physical modelling of the vocal tract with the 2D digital waveguide mesh. \nPhD thesis, University of York, 2006.](http://www-users.york.ac.uk/~dtm3/Download/JackThesis.pdf)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzakaton%2Fpink-trombone","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzakaton%2Fpink-trombone","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzakaton%2Fpink-trombone/lists"}