...
- Support for modular synthesis-style signal processing graphs, which are composed of low-level signal units (Nodes, in Web Audio terminology)
- This should include signal processing primitives that are missing or implicit in the Web Audio API, such as envelopes, value nodes, common mathematical operators, etc.
- The ability to specify graphs of Web Audio API nodes declaratively (i.e. without requiring repetitive, imperative code to instantiate or connect nodes)
- Signal graphs should be authorable using visual and textual tools so that non-programmers can design sonification instruments
- Signal graph specifications should be easy to serialize, interpret, and modify as data structures using standard tools (i.e. JSON)
- Signals should provide viable defaults values such that users need only define their own overrides, which will be merged in by the framework
- It should be possible to export signal graphs created in a sonification authoring tool or editing environment into a format that can be easily instantiated in a "player" environment
- Can enable the creation of separate, higher level tooling that supports personalization and user authoring environments in which "end users" can customize or change a sonification's instrumental characteristics and mappings to data
- Minimal dependence on third party frameworks/libraries—where such minimalism doesn't end up leading towards the creation of ad hoc or implicit framework functionality—but which can be viably integrated into larger frameworks such as Flocking and Infusion
- The ability to change live signal graphs declaratively by address (i.e. by targeting signal specification data structures at named paths) such as:
- inserting new nodes into the graph
- removing nodes
- replacing one node with another
- updating audio parameter values
- The ability to define custom-typed nodes that are implemented using JavaScript-based signal processors (via the current ScriptProcessorNode API and eventually the AudioWorklet API when it is implemented)
- This should include, for example, the ability for Flocking to implement its own custom "flocking" type node
- Built-in, sensible defaults for all (or as many as is conceivably possible) node parameters and input values, such that nodes can be easily used in an "always live" authoring or playback environment
- The ability to define shared audio buffers that can be loaded from a URL or generated from a set of built-in function generators
What's Not in Scope (Yet)?
...
// and which can be routed to the AudioContext's destination node....
2-op FM Synthesis
This example is a one operator (i.e. one carrier, one modulator, and one amplitude envelope) FM brass instrument created by John Chowning, published in Computer Music by Dodge and Jerse. Notably, it illustrates how a node can be connected up as the input to several unit generators, and how multiple Nodes can be hooked up to a single input.
...
- We at least need a way of abstracting the inputs and outputs (i.e. routing to the AudioContext's destination) such that "graph fragments" could be constructed and wired up to each other. This is essential for any kind of higher-level composition elsewhere .(this will happen in layers 2/3)
- Elaborate on the the three-part architecture discussed with Antranig on Jan 20, 2017:
- A regularized, low-level "compile target" document format that does not include connection references, automatic expansion, anonymous nodes, or other conveniences intended for hand-coders but which may risk introducing irregularities or additional processing requirements that reduce compileability or interoperability
- An Infusion-supported component format that can be used to support authoring tools; this format can be compiled down to the low-level representation
- A high-level, textual coding representation roughly of the form described above, but with the introduction of a better means for naming and composing "graph fragments"
...