{"id":17968204,"url":"https://github.com/gre/wavegl","last_synced_at":"2025-08-13T18:34:34.537Z","repository":{"id":18982686,"uuid":"22204070","full_name":"gre/wavegl","owner":"gre","description":"Generate Audio in the GPU and pipe to the Audio Card.","archived":false,"fork":false,"pushed_at":"2014-08-09T21:10:19.000Z","size":3079,"stargazers_count":191,"open_issues_count":0,"forks_count":12,"subscribers_count":13,"default_branch":"master","last_synced_at":"2025-04-05T15:45:37.231Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"http://greweb.me/wavegl","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/gre.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2014-07-24T07:48:03.000Z","updated_at":"2024-11-07T08:13:14.000Z","dependencies_parsed_at":"2022-07-20T23:28:16.914Z","dependency_job_id":null,"html_url":"https://github.com/gre/wavegl","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/gre/wavegl","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gre%2Fwavegl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gre%2Fwavegl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gre%2Fwavegl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gre%2Fwavegl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/gre","download_url":"https://codeload.github.com/gre/wavegl/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gre%2Fwavegl/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":270288760,"owners_count":24558854,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-13T02:00:09.904Z","response_time":66,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-29T14:20:33.913Z","updated_at":"2025-08-13T18:34:34.447Z","avatar_url":"https://github.com/gre.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"wavegl\n======\n\nGenerate Audio in the GPU and stream to the Audio Card.\n\nWebGL + Web Audio API.\n\nJS Libraries:\n- [gl-shader](https://npmjs.org/package/gl-shader)\n- [lodash](https://npmjs.org/package/lodash)\n- [React](https://npmjs.org/package/react)\n\nSome code taken from [GLSL.io](https://glsl.io).\n\nThe idea\n---\n\n- We can **implement a DSP function in a GLSL Fragment**. A texture will package a buffer of DSPs calls.\n- The DSP signal must be **encoded in a scanline direction**: colors, left-right, top-bottom,\n- Then we can just use the buffer of the texture as an audio chunk:\nWe just have to **schedule this buffer with Web Audio API**.\n- Then, we just do this **loop again each second** :-)\n\n\u003e A 105 x 105 image is enough to code 1 second of audio *(1-channel, 44100Hz, 8bit-samples)*.\nRendering a 105x105 image each second is nothing :-)\n\nProof of Concept\n---\n\nWebGL uses **GLSL language**: a C-like, but with high level types and math functions.\n\nhttps://www.khronos.org/registry/webgl/specs/latest/1.0/#readpixels\n```javascript\nfunction readPixels (gl, width, height) {\n  var pixels = new Uint8Array(width * height * 4);\n  gl.readPixels(0, 0, width, height, gl.RGBA, gl.UNSIGNED_BYTE, pixels);\n  return pixels;\n}\n```\n\nhttp://www.w3.org/TR/webaudio/\n```javascript\nvar audioCtx = new AudioContext();\nvar playTime = audioCtx.currentTime;\nvar bufferSource = context.createBufferSource();\nbufferSource.buffer = pixels;\nbufferSource.start(time);\nsetInterval(function schedulingLoop () {\n  if (endOfTheScheduledAudioIsSoon()) {\n    scheduleNext();\n  }\n}, 100);\n```\nSee also https://github.com/gre/zampling/blob/master/src/utils/AudioChunker.js\n\n**Notes:**\n- The GLSL code will have to encode the image in order that it respect the readPixels order:\nRGBA, left to right, top to bottom. Then we can just plug this into the **Audio Card** !!\n- Render one frame in the GPU is very fast.\n- a texture of 105x105 is enough to generate 1 second of sound. (44100 samples – 105x105x4)\n\n\nThen we can make a wrapper to only have to implement this:\n```glsl\nconst float PI = acos(-1.0)\nfloat dsp (float t) {\n  return 0.1 * sin(2.0 * PI * t * 440); // Hello world of the audio\n}\n```\n\nAbout \"Functional Audio\"\n---\n\n**\"Functional Audio\"** is a low level way to generate music,\nquite different from **Web Audio API** paradigm (modular audio).\n\n```scala\ndsp: (t: Number) =\u003e (sample: Float)\n```\n\n\u003e **`t`: absolute time in seconds**.\n\n\u003e **`sample`: value from -1.0 to 1.0**. Amplitude of the sound.\n\nThis function is called **44'100 times per second**!\n\nIt is:\n- Generative audio.\n- Low Level.\n- Pure Math.\n\nThe team\n---\n\n- @dohzya\n- @c4m\n- @guyonvarch\n- @gre\n\nBootstraped in Hackday @zengularity\n\nLicense\n---\n\nAGPLv3\n\nBuild the project\n---\n\nFirst time:\n```\nnpm install\n```\n\nThen:\n```\nnpm run build\n```\n\nThen open `index.html`.\n\nWatch loop for development:\n---\n\n```\nnpm run watch\n```\n\nKnown limitations\n---\n\nthe main idea is to have this pipeline: `GLSL (GPU) -\u003e readPixels -\u003e WebAudioAPI (Audio Card)`. No overhead of JS, pipe an Uint8Array from the GPU to the Audio Card !\n\nThe idea is a hack at the start but they could be some real potential :-)\n\nThe GLSL have limitation for such hack usage though :-D After like 2 minutes you will start notice some noise in the sound, well that is due to float imprecision (I also had to use a float in second instead of an int sample number) + there is no support for binary logic operator so we can't make ByteBeat + Latency is an issue if you want to have user interaction.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgre%2Fwavegl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgre%2Fwavegl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgre%2Fwavegl/lists"}