Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/ejarzo/Shape-Your-Music
A web application for drawing music.
https://github.com/ejarzo/Shape-Your-Music
audio canvas draw-shapes geometry konva midi music polygon react shape sound tonejs webaudio
Last synced: about 1 month ago
JSON representation
A web application for drawing music.
- Host: GitHub
- URL: https://github.com/ejarzo/Shape-Your-Music
- Owner: ejarzo
- Created: 2018-02-18T04:59:34.000Z (almost 7 years ago)
- Default Branch: master
- Last Pushed: 2024-09-17T00:46:46.000Z (3 months ago)
- Last Synced: 2024-09-17T04:53:55.491Z (3 months ago)
- Topics: audio, canvas, draw-shapes, geometry, konva, midi, music, polygon, react, shape, sound, tonejs, webaudio
- Language: JavaScript
- Homepage: https://shapeyourmusic.dev
- Size: 14.2 MB
- Stars: 133
- Watchers: 3
- Forks: 22
- Open Issues: 13
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Shape Your Music
_A geometric sequencer that suggests one answer to the question "What does a shape sound like?"._
[https://shapeyourmusic.dev/](https://shapeyourmusic.dev)
## About
Shape Your Music is a musical [geoboard](https://en.wikipedia.org/wiki/Geoboard) that explores a new way of composing and performing music using geometry.
Using this app, one can:
- Draw multiple shapes to generate unique polyrhythms
- Manipulate shapes in real time to improvise and perform
- Change and adjust the sound that shapes of each color produces
- Experiment with different musical modes, keys, and tempos
- Use grid and syncing options to create defined rhythms and loops
- Move shapes up or down to transpose them within the scale, or side to side to move them in stereo space
- Record and export your project as either an audio file or as MIDI files
- Save projects and browse other people's creations
---
## đź“– User Manual
### Overview
Sound is created by drawing polygons on the canvas and clicking PLAY (or space bar). Each polygon represents a looping musical phrase.
There are two modes: Draw and Edit (toggle between them with Tab). While in DRAW mode, clicking on the canvas allows you to draw polygons by placing vertices. In EDIT mode, You can **add** vertices by clicking on an edge midpoint, or **delete** vertices by double-clicking them. You can also drag each polygon to adjust its position, or drag the entire shape to a new position. Shapes higher up on the canvas start at higher notes than shapes lower down. Moving shapes left or right places them in stereo space (left/right on your speakers or headphones). Also in EDIT mode, you can click on a shape to display a context menu with more options.
Each shape is a certain color. The current color with which you are drawing is controlled with the color palette in the toolbar. A shape's color determines which instrument it uses to produce sound. The sounds for each color can be controlled with the colored panels at the bottom of the screen; if the red panel is set to the "Cello" instrument, every red shape will make a cello sound.
When a shape plays, a node traverses the perimeter of the shape at a constant speed, sounding a note at each vertex. The first note of a shape is determined by the shape's `y` position on the plane. The note for each subsequent edge is determined by the angle between that edge and the previous edge. This angle determines the musical interval between the two notes(edges). For example: A sharp right turn means that the next note is much higher than the previous, while a shallow left turn means that the next note is a little lower. When the last point is reached, the loop starts again.
### Toolbar
The toolbar allows you to adjust various aspects of your project.
| Name | Description |
|----------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Play/Stop | Pressing play starts all shapes at their origin point. Shapes that are added during playback will start playing as soon as they are completed. |
| Record | Pressing record allows you to download your project as an audio file (.wav). If playback is stopped when you click record, recording will begin when you begin playback. If the project is playing when you click record, the recording will start instantly. Pressing stop or record again will end the recording and show a window where you can listen and download the file that was generated. |
| Color | Select the color of the shapes you are drawing. Different colored shapes produce different sounds. |
| Draw Tool | Draw mode allows you to create shapes. Click to place vertices. Click on the origin point to complete a shape. Right click to cancel. |
| Edit Tool | Edit mode allows you to adjust your shapes. Drag vertices to edit the perimeter of your shape. Drag the whole shape to move it. Click on a shape to show more detailed options (see shape controls). |
| Grid | When selected, the grid is shown and all points will snap to the grid when drawn, or when shapes are moved |
| Sync | When selected, shapes will snap and lock to the same length - so that they will loop at the same time. Shapes can be “halved’ or “doubled” so that they loop half or twice as often. This allows for a defined rhythm. |
| Proximity Mode | Turn on Proximity Mode to experience your project spatially, where the listening position is controlled with the mouse. Hover over the dropdown icon to adjust the listening radius. |
| Tempo | Change the speed of playback |
| Key | Select the root note |
| Scale | Select the musical scale (mode) |
| Clear | Delete all shapes |
### Exporting
There are two ways to export your project: as audio or as MIDI.
1. **Audio**: Recording audio works like many DAWs. Click the record button to begin recording. Recording will begin when playback starts and captures the audio output of the project. Clicking record again or stopping playback will stop the current recording, which can then be downloaded as a `.wav` file from the Downloads section in the sidebar.
2. **MIDI**: Click the `Export to MIDI` icon on the left to download the project as a folder of MIDI files, one for each shape. Since the tempo is not synced to any real BPM, the melodies may sound slower or faster when loaded into another program, but all shapes will maintain their relative lengths.
### Shape Menu
The Shape menu allows you to control a shape's properties. CLICK on a shape (in EDIT mode) to display this menu.
| Control | Description |
| -------------- | ------------------------------- |
| Volume Slider | The left side of the panel is a volume slider. Drag the white bar to adjust how loud the shape is |
| Mute (toggle) | When selected, the shape produces no sound |
| Solo (toggle) | When selected, the shape will be the only shape producing sound |
| Color | Change the shape's color. This will change its sound to the corresponding instrument for that color |
| \*2 | Doubles the perimeter of the shape, making the loop last for twice as long |
| \2 | Halves the perimeter of the shape, making the loop last for half as long |
| Reverse | Reverse the direction of playback. Reversing the direction will recalculate the notes based on the new angles - instead of simply playing the notes in opposite order |
| To Front | Moves the shape to the front of the canvas -- has no effect on the audio |
| To Back | Moves the shape to the back of the canvas -- has no effect on the audio |
| Duplicate | Creates a clone of the shape |
| Delete Shape | Deletes the shape from the canvas |
### Color Controller
The Color Controller allows you to change the sound generated by shapes of that color. Use the dropdown menu and arrow buttons to select the preset and the knobs to adjust the sound.
---
## đź’» Development
### Front End
The front end is a React app, created using [`create-react-app`](https://github.com/facebook/create-react-app).
It uses:
- [Tone.js](https://github.com/Tonejs/Tone.js) for music/audio generation
- [React Konva](https://github.com/konvajs/react-konva) for canvas manipulation
- [Teoria](https://github.com/saebekassebil/teoria) for music theory logic
- [Ant Design](https://ant.design/docs/react/introduce) for several UI components
To run: `yarn` and then `yarn start:app`.
### Backend
_NOTE: If you just want to play around with the client code, you can ignore this setup. Just use `yarn start:app`_
The backend consists of a [FaunaDB database](https://fauna.com/) and serverless [Netlify functions](https://www.netlify.com/docs/functions/) to provide an API for viewing and saving projects. It uses [Netlify Identity](https://www.netlify.com/docs/identity/) to handle users, account creation and login. All functions live in the [functions](https://github.com/ejarzo/Shape-Your-Music/tree/master/functions) folder.
If you want to be able to save data locally, you wil need a [FaunaDB account](https://dashboard.fauna.com/accounts/register) (it's free). Once you have an account, you will need a server key that you will set in your local environment.
If everything is already setup, run with `yarn start:server`.
#### FaunaDB Setup
1. Create a fauna account
2. On the [dashboard](https://dashboard.fauna.com/), click `New Database`
3. Go to the `Security` tab
4. Click `Create New Key` with role: `Server`
5. Click `Save`
6. Copy your key
7. In your local environment (NOTE: this will NOT be your react app .env file. This will be your `~/.bash_profile`, `~/.zhrc`, etc.), set a new variable FAUNADB_SERVER_SECRET to be your copied key
8. Reload your terminal session and run `yarn start:server`. The first time this runs, it will create and boostrap your Fauna DB. If you reload your Fauna dashboard, you should see a new `projects` collection and `all_projects` index.
For more, see the [Netlify example repo](https://github.com/netlify/netlify-faunadb-example) which I used as a reference.
#### Netlify Setup
Netlify is used to handle user accounts through their [Identity](https://www.netlify.com/docs/identity/) feature. To fully integrate with the backend and be able to save your projects locally, you will need a Netlify account. Once you have an account, use their [CLI](https://www.netlify.com/docs/cli/) to login and initialize a project for this repo. Try signing up through the UI and you should get a confirmation email that redirects to your netlify project.
### Testing
Tests are in [this repo](https://github.com/ejarzo/sym-tests). Because there is a lot going on with canvas and Web Audio, I found that E2E tests with [puppeteer](https://pptr.dev/) were the easiest way to confirm things are working.