Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/colinbdclark/Flocking
Flocking - Creative audio synthesis for the Web
https://github.com/colinbdclark/Flocking
Last synced: 25 days ago
JSON representation
Flocking - Creative audio synthesis for the Web
- Host: GitHub
- URL: https://github.com/colinbdclark/Flocking
- Owner: colinbdclark
- License: gpl-2.0
- Fork: true (lichen-community-systems/Flocking)
- Created: 2020-06-09T19:59:16.000Z (over 4 years ago)
- Default Branch: main
- Last Pushed: 2023-03-16T17:23:55.000Z (over 1 year ago)
- Last Synced: 2024-05-14T14:05:04.870Z (7 months ago)
- Language: JavaScript
- Homepage:
- Size: 53.3 MB
- Stars: 2
- Watchers: 2
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: MIT-LICENSE.txt
Awesome Lists containing this project
- awesome-sound - flocking.js
- awesome-javascript-audio - Flocking - declaratively create sounds, synths, effects, etc, as JSON objects, supports mouse/trackpad (Sound creation / Instrument recordings)
README
Flocking - Creative audio synthesis for the Web!
================================================
What is Flocking?
-----------------
Flocking is a JavaScript audio synthesis framework designed for artists and musicians
who are building creative and experimental Web-based sound projects.
It supports Firefox, Chrome, Safari on
Mac OS X, Windows, Linux, iOS, and Android.
Unlike comparable tools, Flocking is declarative. Its goal is to promote a uniquely
community-minded approach to instrument design and composition.
In Flocking, unit generators and synths are specified as JSON,
making it easy to save, share, and manipulate your synthesis algorithms.
Send your synths via Ajax, save them for later using HTML5 local data storage,
or algorithmically produce new instruments on the fly.
Because it's just JSON, every instrument you build using Flocking can be easily modified
and extended by others without forcing them to fork or cut and paste your code.
This declarative approach will also help make it easier to create new authoring,
performance, metaprogramming, and social tools on top of Flocking.
Flocking was inspired by the [SuperCollider](https://supercollider.github.io/)
desktop synthesis environment. If you're familiar with SuperCollider,
you'll feel at home with Flocking.
To learn more about Flocking's architecture and approach, please see:
Clark, C. and Tindale, Adam. "[Flocking: A Framework for Declarative Music-Making on the Web](https://github.com/colinbdclark/flocking-papers/blob/master/icmc-2014/flockingicmc2014.pdf?raw=true)"
in Georgaki, A. and Kouroupetroglou (eds.). _The Joint Proceedings of the ICMC and SMC_, (2014).
_[slides](colinclark.org/presentations/flocking-icmc-2014-slides.pdf)_
Flocking does not currently support the Web Audio API's new [AudioWorklet](https://www.w3.org/TR/webaudio/#audioworklet) specification, which allows for JavaScript-based signal processing code to run in a separate, real-time thread. This means that Flocking is not currently well-suited to applications that involve a lot of graphics rendering or user interaction. Work is underway on a new core for Flocking, called [Signaletic](https://github.com/colinbdclark/signaletic/tree/gh-1), which will provide full support for all Web Audio native nodes as well as the use of AudioWorklets.
Community
---------
Flocking has an inclusive and supportive community with several forums where you can ask for help
and share the projects you're working on.
### Mailing List
The [flocking mailing list](https://lists.idrc.ocadu.ca/mailman/listinfo/flocking)
is the place to ask questions, share code, and request new features.
### Chat
Flocking has an IRC channel. Join #flocking on irc.freenode.net.
Status and Roadmap
------------------
Flocking is in active development. It has bugs and it is growing quickly.
The project's [development roadmap](https://github.com/continuing-creativity/Flocking/wiki/Release-Plan)
is documented in the wiki. Plans include:
* Better support for interleaving Flocking unit generators with Web Audio API nodes
* A new format for specifying connections between unit generators
* A "live data merging" environment for live coding
* Graphical editing of Flocking synth defs
Unplanned features, bug fixes, and contributions are welcome and appreciated, of course. The Flocking project adheres to the [Contributor Covenant guidelines](https://contributor-covenant.org/version/1/2/0/), and is an open and welcoming community.
Documentation and Demos
-----------------------
* Flocking's [documentation](docs/) is in the repository
* Demos can be found in the [Flocking Playground](https://flockingjs.org/demos/playground)
* Other examples are located in the [examples repository](https://github.com/colinbdclark/flocking-examples)
Getting Started
---------------
The latest stable release of Flocking can be downloaded from the [Flocking releases](https://github.com/continuing-creativity/Flocking/releases) page.
Concatenated and minified Flocking builds, suitable for development and production respectively,
are included in the [dist directory](dist/). Flocking can also be built manually using Grunt.
Here's how to include Flocking's development file in your HTML page:
For more information on using Flocking in a browser,
read the [Getting Started](docs/getting-started.md) tutorial.
If module loaders are your thing, Flocking also supports the CommonJS and AMD styles.
Using Flocking
--------------
Flocking consists of a handful of primary components that are configured using JSON
specifications. These include: Unit Generators (ugens), Synths, Schedulers, and the Environment.
### Unit Generators ##
Unit generators, or _ugens_ for short, are the basic building blocks of synthesis;
they do the work of generating or processing audio signals in Flocking.
UGens have multiple inputs and a single output. Some unit generators support
multiple input or output multiple channels.
A unit generator can be wired to other unit generators,
supporting sophisticated signal processing graphs.
Unit generators implement one primary method, gen(numSamps),
which is responsible for processing a block of audio samples.
Typically, however, you never need to interact with unit generator instances directly.
Instead, you create declarative "unit generator definitions" (_ugenDefs_) objects,
letting Flocking take care of creating the actual unit generator instances.
UGenDefs are composed into trees called _synthDefs_.
Here's an example of a ugenDef:
{
id: "carrier", // A unique ID used when updating this ugen's input values.
ugen: "flock.ugen.sinOsc", // The fully-qualified name of the desired unit generator,
// specified as a "key path" (a dot-separated string).
rate: "audio", // The rate at which the unit generator should run.
inputs: { // The input values for this unit generator. Each UGen has different inputs.
freq: 440 // For convenience, these inputs don't need to be nested inside the "inputs"
}, // container, but you might want to for readability.
options: {
interpolation: "linear" // Other non-signal options such as interpolation rates, etc.
// Options are also specific to the type of unit generator.
}
}
### Synths ###
A Synth is a self-contained collection of unit generators that represents a
synthesizer, instrument, or signal processor of some kind.
Multiple synths can run at the same time, and they can be connected together
using shared interconnect buses.
For example, a mixing board Synth could be created to mix and process signals from
several tone-generating Synths and effect Synths.
To create a synth, you specify a synthDef option, which is a declarative
tree of unit generator definitions. Here's a simple example of a sine oscillator (named _carrier_)
whose amplitude is modulate by another sine oscillator, _mod_:
{
id: "carrier", // Name this unit generator "carrier," exposing it as an input to the synth.
ugen: "flock.ugen.sinOsc", // Sine oscillator ugen.
freq: 440, // Give it a frequency of 440 Hz, or the A above middle C.
mul: { // Modulate the amplitude of this ugen with another ugen.
id: "mod", // Name this one "mod"
ugen: "flock.ugen.sinOsc", // Also of type Sine Oscillator
freq: 1.0 // Give it a frequency of 1 Hz, or one wobble per second.
}
}
Synths can be updated in realtime by using the get() and set() methods.
Any unit generator with an id property in its ugenDef will automatically be
exposed as a named input to the synth. To update a unit generator, a _key path_ is used to specify
the desired input. Key paths are dot-delimited, path-like strings that allow you to address any part of the
unit generator tree. Here's an example of a key path:
"carrier.freq.mul" // Refers to the amplitude (mul) of the carrier's frequency.
#### Getting Synth Inputs ####
An input can be retrieved from a synth by invoking the get() method with a key path.
If the target of the path is a _value unit generator_, its value will be returned directly.
If it is any other kind of input, its ugen instance will be returned instead.
var freq = synth.get("carrier.freq");
#### Setting Synth Inputs ###
Synth inputs can be set by calling the aptly-named set() method.
Flocking's signal processing pipeline is dynamic, so unit generators can be added
or replaced at any time. Behind the scenes, everything is a unit generator,
even static values.
Updating a value:
synth.set("carrier.freq", 440);
Replacing the target unit generator with a new one:
synth.set("carrier.freq", {
ugen: "flock.ugen.sinOsc",
freq: 2
});
Updating multiple inputs at once:
synth.set({
"carrier.freq": 440,
"carrier.mul": 0.5,
"modulator.freq": 123
});
### Rates ###
To ensure reasonable performance on resource-constrained devices such as phones and low-power computers
(e.g. Chromebooks, Raspberry Pi), Flocking uses a block-based architecture. By default, ugens and synths will
produce 64 samples per block. This value is configurable by specifying the blockSize
option when initializing Flocking.
There are three primary types of signal rates in Flocking: audio, control,
and constant. Audio rate produces a full block of samples,
control rate produces one sample per block, and constant rate will alway produce the same value.
Synths also support two other rates, demand and scheduled. A demand rate synth
will only produce a value when its gen() method is invoked. Scheduled synths are under the
control of a scheduler instead of the sample output clock.
### The Environment ##
An Environment represents a whole "audio system" or "context" in Flocking.
It is responsible for evaluating all Synths instances and outputting their samples to the
current audio output strategy. An environment manages a list of Synth instances and evaluates them in order.
You should instantiate only one flock.enviro for your entire application.
The Flocking _shared environment_ is created by calling flock.init():
var environment = flock.init();
Before you'll hear any sound, the environment needs to be started. You only need to start the environment once.
This is done using the start() method:
environment.start();
To stop the environment from generating samples, use the stop() method:
environment.stop();
#### Synths and the Environment ####
By default, synths are automatically added to the end (or _tail_) of the environment's list of synths.
This means that synths will start playing immediately when you create them.
If you want to defer the playing of a Synth to a later time,
you can override the addToEnvironment option when you instantiate it:
var mySynth = flock.synth({
synthDef: {
ugen: "flock.ugen.sin",
freq: 440
},
addToEnvironment: false
});
If you need to manage the Environment's list of synths manually,
you can use the methods provided by the flock.nodeListComponent _grade_.
To add a synth to the head of the graph so that it will be evaluated first:
enviro.head(mySynth);
To add a synth to the tail of the graph so that it will be evaluated after all other synths):
enviro.tail(mySynth);
### Schedulers ###
A scheduler allows you to schedule changes to Synths at a particular time.
Currently, there is one type of Scheduler, the asynchronous scheduler.
Unfortunately, it is driven by the browser's notoriously inaccurate setTimeout() and setInterval() clocks,
which means that it will drift by up to 75 ms depending on the browser's load.
In practice, however, this drift is sufficient for scheduling many kinds of changes,
and if sample-level accuracy is needed, unit generators such as flock.ugen.sequence,
flock.ugen.change and flock.ugen.random can be used.
A block-accurate scheduler is planned for an upcoming release of Flocking.
In the meantime the asynchronous scheduler does a decent job of keeping "pleasantly inaccurate" time.
If you need to, you can always schedule arbitrary events using plain old functions:
// Fade out after 8 seconds.
band.scheduler.once(8, function () {
band.sinSynth.set({
"carrier.mul.start": 0.25,
"carrier.mul.end": 0.0,
"carrier.mul.duration": 1.0
});
});
The Flocking scheduler, in future releases, will be removed and replaced with [Bergson](https://github.com/colinbdclark/bergson), a highly accurate scheduler written specifically for audio and video applications.
Testing
-------
Flocking uses Testem and Vagrant to run its test suite across many of its supported environments, both on the host and in virtual machines running Windows and Linux.
### Prerequisites
1. [Vagrant](https://www.vagrantup.com/)
2. [VirtualBox](https://www.virtualbox.org/)
3. The GPII CI Plugin: 'vagrant plugin install vagrant-gpii-ci'
### Running Flocking's Test Suite
Run all tests on your host:
* `npm test`
Run only browser tests:
* `npm run browser-test`
Run all VM-based tests:
* `npm run all-vm-test`
Run all tests on a Vagrant-based Windows VM:
* `npm run windows-vm-test`
Run all tests on a Linux VM:
* `npm run linux-vm-test`
Stop all Vagrant VMs:
* `vagrant halt`
Destroy all Vagrant boxes, freeing up hard disk space:
* `npm run destroy-vms`
Compatibility
-------------
Flocking is currently tested on the latest versions of Firefox, Chrome, Safari, and Microsoft Edge
on Mac, Windows, Linux, iOS, and Android.
License
---------
Flocking is distributed under the terms of both the MIT or GPL 2 Licenses.
As a result, you can choose the license that best suits your
project. The full text of Flocking's [MIT](MIT_LICENSE.txt) and [GPL](GPL-LICENSE.txt) licenses are at the root of the repository.
Credits
-------
Flocking is developed by Colin Clark and the community.
It was named after a composition by [James Tenney](https://www.plainsound.org/JTwork.html),
a composer, thinker, and early pioneer of computer music who was my composition teacher and a
huge influence on my work. I hope you find this library useful enough to create projects as
beautiful and inspiring as Jim's _Flocking_.
### Thanks to:
* [Adam Tindale](http://adamtindale.com) for the tanh distortion unit generator, documentation improvements, and several of the Playground demos
* [Alfredo Matas](https://github.com/amatas) for test automation on Windows and Linux
* [Johnny Taylor](https://github.com/abledaccess) for styling improvements to the Playground
* [Dan Stowell](https://github.com/danstowell) for the Freeverb and Delay1 unit generators
* [Mayank Sanganeria](https://github.com/e7mac) for the flock.ugen.granulator unit generator
* [Vitus](https://github.com/derDoc) for his contributions to the original version of the interactive Playground
* [Myles Borins](https://github.com/thealphanerd) for pushing the limits of Flocking early in its development
* [Antranig Basman](https://github.com/amb26) for code review, architectural advice, and help with maths
* Alex Geddie for teaching me a ton about synthesis and computer music