{"id":16087486,"url":"https://github.com/linqlover/sonyx","last_synced_at":"2026-03-18T17:52:02.890Z","repository":{"id":43358810,"uuid":"378654366","full_name":"LinqLover/sonyx","owner":"LinqLover","description":"A toolkit to explore software systems through sonification in Squeak/Smaltalk","archived":false,"fork":false,"pushed_at":"2024-10-11T22:37:13.000Z","size":7711,"stargazers_count":0,"open_issues_count":3,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-04-05T15:27:08.623Z","etag":null,"topics":["auditory-display","programming-experience","smalltalk","squeak"],"latest_commit_sha":null,"homepage":"https://hpi.de/neurodesign/projects/sonyx.html","language":"Smalltalk","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/LinqLover.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-06-20T13:39:53.000Z","updated_at":"2024-10-11T22:37:17.000Z","dependencies_parsed_at":"2025-04-05T15:37:46.905Z","dependency_job_id":null,"html_url":"https://github.com/LinqLover/sonyx","commit_stats":null,"previous_names":[],"tags_count":33,"template":false,"template_full_name":null,"purl":"pkg:github/LinqLover/sonyx","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LinqLover%2Fsonyx","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LinqLover%2Fsonyx/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LinqLover%2Fsonyx/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LinqLover%2Fsonyx/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/LinqLover","download_url":"https://codeload.github.com/LinqLover/sonyx/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LinqLover%2Fsonyx/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28585523,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-19T20:45:59.482Z","status":"ssl_error","status_checked_at":"2026-01-19T20:45:41.500Z","response_time":67,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["auditory-display","programming-experience","smalltalk","squeak"],"created_at":"2024-10-09T13:29:53.289Z","updated_at":"2026-01-19T21:34:00.400Z","avatar_url":"https://github.com/LinqLover.png","language":"Smalltalk","funding_links":[],"categories":[],"sub_categories":[],"readme":"# sonyx\n\n[![Tests](https://github.com/LinqLover/sonyx/actions/workflows/test.yml/badge.svg)](https://github.com/LinqLover/sonyx/actions/workflows/test.yml)\n[![Coverage Status](https://coveralls.io/repos/github/LinqLover/sonyx/badge.svg?branch=prototyping)](https://coveralls.io/github/LinqLover/sonyx?branch=prototyping)\n[![Release](https://github.com/LinqLover/sonyx/actions/workflows/release.yml/badge.svg)](https://github.com/LinqLover/sonyx/actions/workflows/release.yml)\n\n**sonyx** (***S***ound-based t***O***ols for u***N***derstanding of software s***Y***stems through e***X***ploration) is a toolkit for exploring software systems through sonification in [Squeak/Smalltalk](https://squeak.org).\nIt has initially been developed within the course [\"Sonic Thinking - Methods of Working with Sound\"](https://hpi.de/studium/im-studium/lehrveranstaltungen/it-systems-engineering-ma/lehrveranstaltung/sose-21-3286-sonic-thinking-seminar-_-methods-of-working-with-sound.html) and has been evaluated and extended within the course [\"Neurodesign Lecture - Designing for Empathy in Business Contexts\"](https://hpi.de/studium/im-studium/lehrveranstaltungen/digital-health-ma/lehrveranstaltung/wise-21-22-3319-neurodesign-lecture-_-designing-for-empathy-in-business-contexts.html) offered by Julia von Thienen from the [neurodesign group @ HPI](https://hpi.de/neurodesign/home.html).\nFor more information, please refer to [theoretical background](#theoretical-background-literature--acknowledgments).\n\nThe main idea of sonyx is to empower developers to understand software systems by listening to particular interesting aspects in the source code.\nTo do this, developers can create *sound probes* on the fly for any expression in any method in the system.\nWhenever this expression is reached during the system execution, a user-defined sound is played.\nDevelopers can customize and combine these sounds or even configure them to dynamically reflect the state or result of the expression.\n\nThis project is based on [Babylonian Programming/Smalltalk](https://github.com/hpi-swa-lab/babylonian-programming-smalltalk/) and [Sandblocks](https://github.com/hpi-swa/sandblocks).\n\n## Impressions\n\n*Make sure to turn your sound on!*\n\u003cvideo src=\"https://user-images.githubusercontent.com/38782922/131224109-b474991a-5558-4a62-aff4-ed17e512e663.mp4\"\u003e\u003c/video\u003e\n\n\u003ctable\u003e\n\u003ctbody\u003e\n\u003ctr\u003e\n\u003ctd\u003e\u003ca href=\"./assets/probe.png\"\u003e\u003cimg alt=\"A sound probe\" src=\"./assets/probe.png\" width=\"55000\"\u003e\u003c/img\u003e\u003c/a\u003e\u003c/td\u003e\n\u003ctd\u003e\u003ca href=\"./assets/monitor.png\"\u003e\u003cimg alt=\"The sound monitor\" src=\"./assets/monitor.png\" width=\"45000\"\u003e\u003c/img\u003e\u003c/a\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/tbody\u003e\n\u003c/table\u003e\n\n\u003csmall\u003e[Browse all screenshots and screencasts](./assets)\u003c/small\u003e\n\n## Download and installation\n\nA ready-to-use Squeak image is available in the [latest release](https://github.com/LinqLover/sonyx/releases).\nExtract the zip file and execute the `squeak.bat` file on Windows, the `squeak.sh` on Linux, or run the app bundle on macOS.\n\nIf you wish, you can also set up sonyx yourself:\n\n1. Install the latest Trunk updates for [Squeak](https://squeak.org).\n\n2. Open a workspace and evaluate the following:\n   ```smalltalk\n   Metacello new\n   \tbaseline: 'Sonyx';\n   \trepository: 'github://LinqLover/sonyx:master';\n   \tload.\n   ```\n\n## Usage\n\n### About Squeak\n\nIf you are new to Squeak/Smalltalk, I recommend you learn a few basics about it before exploring this project.\nSqueak is a programming environment that is quite different from many other contemporary solutions.\nSome aspects of its main philosophy are that everything is an object, including the environment, the compiler, and even all tools in the system, as well as that the user can always interact with or modify all of these objects.\nAs a consequence, you will need to have a basic knowledge of the Squeak tools in order to understand this project or to browse its implementation.\n\nThe book **[Squeak by Example](https://github.com/hpi-swa-lab/SqueakByExample-english#readme)** provides a pretty good introduction to Squeak (for this project, you should refer to the Trunk version/`SBE-Trunk.pdf` from the latest preprint).\nIt is recommended that you skim through the sections in chapter 1 (\"A quick tour of Squeak\") but at the very least read section 1.6 (\"The system browser\") and section 1.7 (\"Finding classes\").\n\n### Using sonyx\n\nIn the ready-to-use image (see [download and installation](#download-and-installation) above), you will find a link to the **`SonyxDemos` class.**\nWhen browsing this class, you can select any of the demo messages, select the last line of the method comment (which is denoted by the quotes), and press \u003ckbd\u003eCmd\u003c/kbd\u003e + \u003ckbd\u003ed\u003c/kbd\u003e to run this demo.\nTo inspect how an example works, select a class or method in its example of interest (for instance, `SonyxDemoStream` or `sorted2`) and press \u003ckbd\u003eCmd\u003c/kbd\u003e + \u003ckbd\u003eb\u003c/kbd\u003e resp. \u003ckbd\u003eCmd\u003c/kbd\u003e + \u003ckbd\u003em\u003c/kbd\u003e to browse it resp. to look up its implementation.\n\nYou can **define your own sound probes** in any method you want.\nFor example, after opening the demo morph example, browse to `SonyxDemoMorph \u003e\u003e #mouseEnter:`, select an expression such as `self`, do a yellow-button click (usually, this corresponds to your right mouse button), and choose \"add sonyx probe\" from the menu.\nNow a probe annotation is displayed that you can use to configure the sound.\nUse the prehear icon to listen to it.\nWhenever you now invoke this method (in this example, move your mouse over the demo morph to do that), this sound is played.\nTo customize the sound, type into the probe annotation or click on any part of it to change it.\nFor instance, click on the default squeakSound (`#horn`) to replace it with another sound (maybe `#croak`).\nAfterward, press \u003ckbd\u003eCmd\u003c/kbd\u003e + \u003ckbd\u003es\u003c/kbd\u003e to save the sound probe and trigger it again.\n\nYou can **add further parameters to the sound** by selecting the entire `SonyxSound` expression with a click and pressing \u003ckbd\u003eCtrl\u003c/kbd\u003e + \u003ckbd\u003ei\u003c/kbd\u003e (insert).\nClick into the `msg` field to choose a parameter key (for instance, `balance:`).\nIn the next field, you can enter a value for this parameter (e.g., `0`).\nSave the probe again and trigger the sound to listen to the difference.\nYour sound can also dynamically reflect the context of the method invocation.\nTo do so, replace the parameter value with any Smalltalk expression that refers to the method context (e.g., `anEvent position x from: self left to: self right`).\nIf you now trigger the sound probe again, you can hear the difference based on the circumstances of the method invocation (e.g., at which position your mouse cursor has entered the demo morph).\n\nTo retrieve an overview of all currently playing sounds, open the **Sonyx Monitor** from the main docking bar on top of the screen in the Apps menu.\nIt displays a list of all sound probes that are defined in the system.\nWatch it while triggering some sounds.\nYou can select any sound probe to read the sound definition, browse the method where the probe is defined, disable/enable the probe, remove it, or prehear the sound.\n\n### Troubleshooting\n\nAt the current stage of this project, occasional hick-ups cannot be precluded.\nIn the event that you should stumble into any problems, next to contacting me an option is to check the following guide:\n\n- To stop all running sounds, press \u003ckbd\u003eCmd\u003c/kbd\u003e + \u003ckbd\u003edot\u003c/kbd\u003e.\n- If the sound monitor highlights no longer playing sounds, try the fixup options from the \"fixup (in case of trouble)\" button on the top.\n- If you cannot hear any sounds, try to stop all runnings sounds first.\n  If this does not help, open the sonyx monitor, press the fixup button, and choose \"Forget all errors\".\n- If any other terrible things have happened that stop you from using sonyx as expected, just close Squeak without saving the changes and try your luck again.\n\nIn any case, if you find any defect that you are able to reproduce, I will appreciate your bug report.\n\n## Architecture\n\nThe package structure of this project is organized as follows:\n\n\u003ctable\u003e\n\t\u003cthead\u003e\n\t\t\u003ctr\u003e\n\t\t\t\u003ctd\u003e\u003cstrong\u003ePackage\u003c/strong\u003e\u003c/td\u003e\n\t\t\t\u003ctd\u003e\u003cstrong\u003eDescription\u003c/strong\u003e\u003c/td\u003e\n\t\t\u003c/tr\u003e\n\t\u003c/thead\u003e\n\t\u003ctbody\u003e\n\t\t\u003ctr\u003e\n\t\t\t\u003ctd\u003e\u003ca href=\"packages/BaselineOfSonyx.package/\"\u003eBaselineOfSonyx\u003c/a\u003e\u003c/td\u003e\n\t\t\t\u003ctd\u003ePackage metadata.\n\t\t\tContains scripts to set up the deploy image and fill it with an initial welcome text.\u003c/td\u003e\n\t\t\u003c/tr\u003e\n\t\t\u003ctr\u003e\n\t\t\t\u003ctd\u003e\u003ca href=\"packages/Sonyx-Core.package/\"\u003eSonyx-Core\u003c/a\u003e\u003c/td\u003e\n\t\t\t\u003ctd\u003eContains core functionality of sonyx and integration into the Squeak system.\n\t\t\tProvides the sound API and a set of sound samples.\u003c/td\u003e\n\t\t\u003c/tr\u003e\n\t\t\u003ctr\u003e\n\t\t\t\u003ctd\u003e\u003ca href=\"packages/Sonyx-Demo.package/\"\u003eSonyx-Demo\u003c/a\u003e\u003c/td\u003e\n\t\t\t\u003ctd\u003eContains several examples that demonstrate the features of sonyx.\n\t\t\tBrowse \u003ccode\u003eSonyxDemos\u003c/code\u003e to see an overview of all of them.\u003c/td\u003e\n\t\t\u003c/tr\u003e\n\t\t\u003ctr\u003e\n\t\t\t\u003ctd\u003e\u003ca href=\"packages/Sonyx-Tests.package/\"\u003eSonyx-Tests\u003c/a\u003e\u003c/td\u003e\n\t\t\t\u003ctd\u003eEnd-to-end tests for sonyx.\u003c/td\u003e\n\t\t\u003c/tr\u003e\n\t\t\u003ctr\u003e\n\t\t\t\u003ctd\u003e\u003ca href=\"packages/Sonyx-UI.package/\"\u003eSonyx-UI\u003c/a\u003e\u003c/td\u003e\n\t\t\t\u003ctd\u003eView logic for the facilities implemented in the Core package. Includes sound annotations, a sound monitor, and integrates Sandblocks into the probe annotations.\u003c/td\u003e\n\t\t\u003c/tr\u003e\n\t\t\u003ctr\u003e\n\t\t\t\u003ctd\u003e\u003ca href=\"packages/Sonyx-Study-Base.package/\"\u003eSonyx-Study-Base\u003c/a\u003e\u003c/td\u003e\n\t\t\t\u003ctd\u003eFramework for running the \u003ca href=\"#user-study\"\u003euser study\u003c/a\u003e.\u003c/td\u003e\n\t\t\u003c/tr\u003e\n\t\t\u003ctr\u003e\n\t\t\t\u003ctd\u003e\u003ca href=\"packages/Sonyx-Study-RVV.package/\"\u003eSonyx-Study-RVV\u003c/a\u003e\u003c/td\u003e\n\t\t\t\u003ctd\u003eLED visualizations for a visual control condition in the study, based on \u003ca href=\"https://github.com/hpi-swa-teaching/live21-value-visualization\"\u003eRuntime Value Visualizations\u003c/a\u003e.\u003c/td\u003e\n\t\t\u003c/tr\u003e\n\t\t\u003ctr\u003e\n\t\t\t\u003ctd\u003e\u003ca href=\"packages/Sonyx-Study-Solutions.package/\"\u003eSonyx-Study-Solutions\u003c/a\u003e\u003c/td\u003e\n\t\t\t\u003ctd\u003eModel solutions to the programming tasks for the user study.\u003c/td\u003e\n\t\t\u003c/tr\u003e\n\t\t\u003ctr\u003e\n\t\t\t\u003ctd\u003e\u003ca href=\"packages/Sonyx-StudyTasks.package/\"\u003eSonyx-StudyTasks\u003c/a\u003e\u003c/td\u003e\n\t\t\t\u003ctd\u003eMaterials for the programming tasks for the user study.\u003c/td\u003e\n\t\t\u003c/tr\u003e\n\t\u003c/tbody\u003e\n\u003c/table\u003e\n\n## User Study\n\nWe have conducted a controlled experiment with 6 partipants to quantitatively evaluate the use of auditory display in exploratory programming.\nAll details on the design and the outcomes of the study are described [here](./docs/study).\n\n## Theoretical background, literature \u0026 acknowledgments\n\nThere is so much more to tell about this project!\nFor all the other details, including important related work and underlying goals of sonyx, please refer to the [wiki](https://github.com/LinqLover/sonyx/wiki/Material#project-slides-for-the-sonic-thinking-seminar).\nThere are also [acknowledgments](./ACKNOWLEDGMENTS.md) to many kind people.\nSpecial thanks go to Julia von Thienen for the statistical evaluation of the user study as well as Tom Beckmann ([@tom95](https://github.com/tom95)) and Patrick Rein ([@codeZeilen](https://github.com/codeZeilen)) for their unremitting support and close cooperation to get sonyx integrated into Babylonian and Sandblocks!\n\nTalk Slides: [Auditory Displays in Programming.]( https://linqlover.github.io/LinqLover/slides/ACUD%20Sonyx.pdf) On [*Making Things Audible*](https://acudmachtneu.de/events/1827/making-things-audible/), March 5–6, 2022. ACUD MACHT NEU, Berlin, Germany.\n\nPlease also refer to [UPSTREAM.md](./UPSTREAM.md) for a list of further implementational work that has been done in the context of this project.\n\n## Outlook \u0026 known limitations\n\nThis is only a prototype, and there are still plenty of other ideas that have not yet been tried out.\nYou can find a selection of small and big ideas in the [project board](https://github.com/LinqLover/sonyx/projects/1).\n\nSome known technical limitations to date are:\n\n- Limited mouse controls of probe expressions. While intended to be completely configurable as GUI, nested sounds and additional parameters still require keyboard control at the moment.\n- No compatibility with vanilla Babylonian - inserting examples in a method will play unexpected sounds.\n\nPlease do not hesitate to create an issue if you have further ideas or troubles!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flinqlover%2Fsonyx","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flinqlover%2Fsonyx","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flinqlover%2Fsonyx/lists"}