{"id":14979659,"url":"https://github.com/pageauc/sonic-track","last_synced_at":"2025-10-28T19:31:45.919Z","repository":{"id":74954791,"uuid":"85640823","full_name":"pageauc/sonic-track","owner":"pageauc","description":"Uses a raspberry pi camera or web cam and python opencv to track motion in camera view. Sends motion contour data to sonic-pi via osc interface to produce and control notes/sample. Includes ability to use onscreen menu areas to change synthesizer, octaves Etc.","archived":false,"fork":false,"pushed_at":"2021-07-27T20:08:33.000Z","size":67665,"stargazers_count":31,"open_issues_count":1,"forks_count":1,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-02-01T17:51:11.160Z","etag":null,"topics":["htmi","motion-data","motion-to-sound","motion-tracking","opencv","opencv3","pi-camera","psonic","raspberry-pi","rpi","sonic-pi","sound","speaker","youtube-sonic-track","youtube-video"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pageauc.png","metadata":{"files":{"readme":"Readme.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-03-21T00:24:39.000Z","updated_at":"2024-11-18T13:00:18.000Z","dependencies_parsed_at":"2023-07-25T20:31:21.247Z","dependency_job_id":null,"html_url":"https://github.com/pageauc/sonic-track","commit_stats":{"total_commits":87,"total_committers":1,"mean_commits":87.0,"dds":0.0,"last_synced_commit":"69dde77cafebceda13f1b4ba5e3f963b9b6e1d7f"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pageauc%2Fsonic-track","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pageauc%2Fsonic-track/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pageauc%2Fsonic-track/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pageauc%2Fsonic-track/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pageauc","download_url":"https://codeload.github.com/pageauc/sonic-track/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":238710126,"owners_count":19517651,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["htmi","motion-data","motion-to-sound","motion-tracking","opencv","opencv3","pi-camera","psonic","raspberry-pi","rpi","sonic-pi","sound","speaker","youtube-sonic-track","youtube-video"],"created_at":"2024-09-24T14:00:27.459Z","updated_at":"2025-10-28T19:31:37.619Z","avatar_url":"https://github.com/pageauc.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Raspberry Pi sonic-track.py a Sonic-pi Motion Track Demo\n#### Real Time Camera Motion Tracking of contour x,y,w,h values and generates sounds via sonic-pi.  \n#### Uses pi-camera or webcam, python3, Opencv3, python-sonic, pythonosc, sonic-pi, powered speaker or HDMI TV\n\n### NEW Release 0.91 Adds drums and restructures code and variables and adds option for zone grid lines on GUI\n### ver 0.95 of setup.sh purges previous sonic-pi and installs latest https://sonic-pi.net/files/releases/v3.3.1/sonic-pi_3.3.1_1_armhf.deb\n\n#### YouTube Air Drum Demo https://youtu.be/PSrzbeVX8DE\n#### YouTube Air Note Demo https://youtu.be/dC26PUYYR5E\n#### YouTube Motion Activated Menus https://youtu.be/PQjpskPAtR0\n#### YouTube sonic-track.py Intro https://youtu.be/RCIUlv431E0\n\n### Introduction\nThis demo app sends camera movement tracking data to sonic-pi music program.  sonic-track sends data to sonic-pi via psonic.py and pythonosc.\nYou will need a pi camera and a powered speaker connected to the Raspberry Pi audio/video plug via appropriate cables or you can attach RPI\nto an HDMI TV via HDMI cable and sound will redirect to TV speakers.\nI thought it would be interesting to point the camera at a fish tank or other source of\nvideo movement to see what sounds are generated. You can also stand in front of the camera and generate sounds via body movements.\n\nThis program demo is a very basic setup but more elaborate sonic-pi sound generation algorithms are possible.\n\n### How to Install\n\n#### Quick Install   \nEasy Install of sonic-track onto a Raspberry Pi Computer with latest Raspbian.\nThis is a whiptail menu system that allows install of opencv3 if required \n\n    curl -L https://raw.github.com/pageauc/sonic-track/master/setup.sh | bash\n\nFrom a computer logged into the RPI via ssh (Putty) session use mouse to highlight command above, right click, copy.  \nThen select ssh(Putty) window, mouse right click, paste.  The command should \ndownload and execute the sonic-track github setup.sh script and install sonic-track files and set permissions.  \nThis install can also be done directly on an Internet connected Raspberry Pi via a console or desktop terminal session and web browser.      \n***Note*** - A raspbian apt-get update and upgrade will be performed as part of install \nso it may take some time if these are not up-to-date.\n\n#### Manual Install   \nFrom logged in RPI SSH session or console terminal perform the following.\n\n    wget https://raw.github.com/pageauc/sonic-track/master/setup.sh\n    chmod +x setup.sh\n    ./setup.sh\n\n### Opencv3 Install (if required)\nsonic-track.py requires opencv3 to be installed on the latest RPI disto.\nI have written a menu driven install script on my GitHub repo here https://github.com/pageauc/opencv3-setup.  Use this if\nyou do not have opencv3 already installed. cv3-setup.sh is a whiptail menu that can walk you through installing \nopencv from source.  The menu system can select opencv version, installs dependencies, downloads source, perform cmake,\nrun compile perform make install of selected opencv version.  \nFor more details See github [Readme.md](https://github.com/pageauc/opencv3-setup/blob/master/Readme.md) \n\n### How to Run\nDefault is SSH or Terminal console only display. Make sure pixel desktop is\n enabled and logged in.  This can be done from raspi-config setting.\nThis is required for the sonic-track.sh to start sonic-pi on the desktop from\n an ssh only session.\n\nUse Nano to Edit config.py variable settings. Select PiCamera or USB web camera\n depending on what you have installed or want to use. \nIf images are oriented wrong then set camera hflip, vflip as appropriate.\n\nCustomize the synthPicks by entering comma delimited number entries\n from the synthList above.  The synth numbers can be arranged in any order. \nThis list is used to to switch synth when motion is detected in the hotspot area\n (usually top/left corner)  The noteOctavePicks can also be customized from the list\n of octaveList above. You may also want to look at notesSleepOnList Entries, notesSleepVarOn, Etc. \n See other variables and comments for additional variable customization settings. \n \nTo Run sonic-track from SSH session, console or GUI desktop terminal session execute the following commands.\nMake sure a speaker is connected to the pi before starting.\n\n    cd ~/sonic-track\n    ./sonic-track.sh   \n        \n#### GUI RPI Desktop Display Video\nWith a display, keyboard and mouse attached to the RPI, Login to the Pixel Desktop\n\nUse desktop menu to open a terminal session.  Use nano to set the following\nconfig.py variable then ctrl-x y to save change(s)\n\n    cd ~/sonic-track\n    nano config.py\n    \nedit the following setting the ctrl-x y to save\n    \n    windowOn = True\n\nYou can either start sonic-pi from the desktop then in the terminal window start sonic-track.py\n\nor from the desktop terminal session run\n\n    ./sonic-track.sh\n    \n### Change Settings\n\nEdit the config.py file and set variables as per comments\n\n    cd ~./sonic-track\n    nano config.py\n \nThe notePlayOn=True will turn on notes and drumPlayOn=True will turn on drum session or you can select both if\nyou are daring.  I recommend just one at a time to begin with.\nsonic-track.py uses motion contour center and width and height eg x, y, h, w motion data.\nThese values are used to generate notes and or drums and can also change sleep value if enambled.  Sound values are send\nto sonic-pi interface to create sounds.  Sounds can be modified using synth settings per config.py synthPicks.  The reference numbers\nare from the synthList and the synthPicks list can contain any number of values separated by commas and\n arranged in any order.  These can be changed when sonic-track.sh is running my creating movement in the hotspot area\n(default is top left corner 1/5 of the screen height and width.\nThere are several template config.py files to assist in setting up multiple configurations. These can be copied over\nthe original config.py.  You can also save custom configurations to any filename then copy over the config.py file\nto use it.  You may want to keep a backup copy of any config.py files that will be overwritten \n        \nFor more information about psonic see https://github.com/gkvoelkl/python-sonic    \nand midi values see  http://www.electronics.dit.ie/staff/tscarff/Music_technology/midi/midi_note_numbers_for_octaves.htm\n\nTo change how the notes algorithm works you would need to change sonic-track.py programing.  This will\nrequire reprogramming how notes are generated via x,y,w,h values.  \n\n    cd ~./sonic-track\n    nano sonic-track.py    \n\n### Prerequisites\n* Recommend a quad core Raspberry Pi computer running with an up-to-date Raspbian Jessie distro\n* RPI camera module or USB Web Cam installed/connected configured and tested to work. \n* Dependencies will be installed via setup.sh depending on your previous installs.  \n* opencv3 is required and can be installed via cv3-setup.sh on menubox.sh menu pick (if required)\n* sonic-pi is installed as part of the Jessie full install distro (not Lite)\n* An HDMI TV connected to HDMI port on RPI or a Powered speaker including cable between RPI 3.5 mm audio/video plug and speaker\n\nSpeaker 3.5mm audio/video jack similar to these cables https://www.adafruit.com/product/2881 \nor https://www.amazon.com/Parts-Express-3-5mm-Plug-Cable/dp/B0007V6JCK  \nthese are also available from other vendors.  Just google 3.5mm audio/video jack\n* You may also need a cable similar to this \nhttps://www.amazon.com/P316-06N-6-Inch-Stereo-Splitter-Adapter/dp/B00M5FKF9E/ref=sr_1_1?ie=UTF8\u0026qid=1490056641\u0026sr=8-1\u0026keywords=35+mm+rca+audio+cable\ndepending on the powered speaker audio IN connection requirements.\n\n### sonic-track.py - Basic concept of tracking moving objects\nThis Demo program detects motion in the field of view using opencv3 commands and returns movement \ncontours above a minimum size and returns the x,y,h,w of the movement contours. These values are then\nsent to sonic-pi via psonic.py and pythonosc.  sonic-track.sh can run in a SSH terminal \nsession only. The sonic-pi gui will be launched via xauth display commands.  Make sure the \nRaspberry Pi Jessie OS pixel GUI desktop is running.  This demo needs to run on\na quad core raspberry pi with the latest Jessie build installed.\n \n* Motion Track Demo YouTube Video http://youtu.be/09JS7twPBsQ   \n* YouTube sonic-track demo https://youtu.be/RCIUlv431E0\n* GitHub Repo https://github.com/pageauc/sonic-track\n\n#### Some ideas I would like to work on\n\n* Air drum eg bongo drum feature activated by body movement  (release 0.90 Added air drums)\n* Simon type sound motion game to try to repeat a series of notes using body movement\n ( see hotspot game https://github.com/pageauc/hotspot-game )\n* Implement bird call type game to use motion tracking to pick the bird associated with\n a sound from an on screen image of different birds. I have a poke-pi demo that allows displaying images on screen (not published)\n it allows for interactively taking an image of a toy or object then you can use the small icon in the motion tracking\n environment to do something.  I was visualizing something like pokemon\n* Same as birds idea above but for any animals Eg farmyard or wild animals.\n* Control a device and give audio feed back.  Eg higher sound for higher settings.\n* Save motion notes to a csv file or other data format\n* Read from a video file and generate notes based on motion tracking\n* Read csv file and replay previous notes or read from another data file and convert to\nnote ranges required for input\n* Create alternate configuration settings via separate config.py files.  This would allow\nchanging how notes are generated. (partly done for drums and notes)\n* Setup a method to easily change synth settings  (implemented pickList)\n\n### Credits  \nSome of this code is based on a YouTube tutorial by\nKyle Hounslow using C here https://www.youtube.com/watch?v=X6rPdRZzgjg\n\nThanks to Adrian Rosebrock jrosebr1 at http://www.pyimagesearch.com \nfor the PiVideoStream Class code available on github at\nhttps://github.com/jrosebr1/imutils/blob/master/imutils/video/pivideostream.py\n\n\n## ---------- Other Raspberry Pi Projects Based on Motion Tracking ------------\n\n### speed-camera.py - Object (vehicle) speed camera based on motion tracking\nTracks vehicle speeds or other moving objects in real time and records image \nand logs data. Now improved using threading for video stream and clipping of \narea of interest for greater performance.  \n* GitHub Repo https://github.com/pageauc/rpi-speed-camera\n* YouTube Speed Camera Video https://youtu.be/eRi50BbJUro  \n* RPI forum post https://www.raspberrypi.org/forums/viewtopic.php?p=1004150#p1004150  \n\n### cam-track.py - Tracks camera x y movements\nUses a clipped search image rectangle to search subsequent video stream images and returns\nthe location. Can be used for tracking camera x y movements for stabilization,\nrobotics, Etc.  \n* GitHub Repo https://github.com/pageauc/rpi-cam-track\n* YouTube Cam-Track Video https://www.youtube.com/edit?video_id=yjA3UtwbD80   \n* Code Walkthrough YouTube Video https://youtu.be/lkh3YbbNdYg        \n* RPI Forum Post https://www.raspberrypi.org/forums/viewtopic.php?p=1027463#p1027463   \n\n### hotspot-game.py - A simple motion tracking game\nThe game play involves using streaming video of body motion to get as many hits \nas possible inside shrinking boxes that randomly move around the screen. \nPosition the camera so you can see body motions either close or standing. \nPretty simple but I think kids would have fun with it and they just might \ntake a look at the code to see how it works, change variables or game logic.      \n* GitHub hotspot-game Repo https://github.com/pageauc/hotspot-game \n* YouTube Hotspot Gam Video https://youtu.be/xFl3lmbEO9Y       \n* RPI Forum Post https://www.raspberrypi.org/forums/viewtopic.php?p=1026124#p1026124   \n\n## ----------------------------------------------------------------------------\n  \nHave Fun   \nClaude Pageau    \nYouTube Channel https://www.youtube.com/user/pageaucp   \nGitHub Repo https://github.com/pageauc\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpageauc%2Fsonic-track","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpageauc%2Fsonic-track","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpageauc%2Fsonic-track/lists"}