Ambisonic Plugin Generation

Background

Ambisonic encoding is the process of decomposing a sound field into spherical harmonics. The encoding matrix is the amount of spherical harmonics present at a specific device position. In mode-matching decoding, the decoding matrix is the pseudo-inverse of the encoding matrix. Ambisonic decoding is the process of reconstructing spherical harmonics into a sound field.

This example involves higher order ambisonics, which include traditional first-order ambisonics. In ambisonics, there is a relationship between the number of ambisonic channels and the ambisonic order:

       ambisonic_channels = (ambisonic_order + 1)^2 

For example: First-order ambisonics requires four audio channels while fourth-order ambisonics requires 25 audio channels.

Supported Conventions

The ambisonic design examples support up to seventh-order ambisonics with pseudo-inverse decoding.

Ambisonic Devices: Elements and Speakers

Ambisonic devices are divided into two groups: elements and speakers. Each device has an audio signal and metadata describing its position and operation. Elements correspond to multi-element microphone arrays, and speakers correspond to loudspeaker arrays for ambisonic playback.

The ambisonic encoder applies the ambisonic encoding matrix to raw audio from microphone elements. The position (azimuth, elevation) and deviceType of the microphone elements along with desired ambisonic order are needed to create the ambisonic encoding matrix.

The ambisonic decoder applies the ambisonic decoding matrix to ambisonic audio for playback on speakers. The position (azimuth, elevation) and deviceType of the speakers along with desired ambisonic order are needed to create the ambisonic decoding matrix.

Sound Field Representation

In order to capture, represent, or reproduce a sound field with ambisonics, the number of devices (elements or speakers) must be greater than or equal to the number of ambisonic channels.

For the encoding example, audio captured with a 32-channel spherical array microphone may be encoded up to fourth-order ambisonics (25 channels). For the decoding example, a loudspeaker array containing 64 speakers is configured for ambisonic playback up to seventh-order. If the playback content is fourth order ambisonics, then even though the array is set up for seventh- order, only fourth-order ambisonics is realized through the system.

       number_devices >= number_ambisonic_channels

For an encoder, if the number of devices (elements) is less than the number of ambisonic channels, then audio from the device (elements) positions may be represented in ambisonics, but a sound field is not represented. One or more audio channels may be encoded into ambisonics in an effort to position sources in an ambisonic field. Each encoder represents the intensity of the sound field to be encoded at a specified device (element) location.

For a decoder, if the number of devices (speakers) is less than the number of ambisonic channels, the devices (speakers) do not fully reproduce a sound field at the specified ambisonic order. A sound field may be reproduced at a lower ambisonic order. For example, third-order ambisonics played on a speaker array with 10 speakers can be realized as a second-order (9 channel) system with an additional speaker for playback. Each decoder represents an intensity of the ambisonic field at the specified device (speaker) position.

Pseudoinverse Decoding Method

There are many decoding options. This example uses pseudoinverse decoding, also known as mode matching. This decoding method favors regular-shaped device layouts. Other decoding methods may favor irregular-shaped device layouts.

Device Type

The deviceType for encoders turns the device (element) encoding on or off for a particular element. The deviceType for decoders turns the device (speaker) decoding on or off for a particular speaker. If the deviceType vector is omitted, then the deviceTypes are set to 1 (on). The intention behind deviceType is to provide flexibility of padding encoder inputs or decoder outputs with off channels to fit an ambisonic encoder or decoder plugin into an environment with fixed channel counts such as an 8-, 16- or 32-channel audio bus.

For example: A second-order ambisonic encoder with 14 elements has 14 inputs and 9 outputs. If you add two more devices (elements) with deviceType 0 (off) to the encoder, then the encoder has 16 inputs and 9 outputs. A fourth-order ambisonic decoder with 29 devices (speakers) has 25 inputs and 29 outputs. If you add three more devices (speakers) with deviceType 0 (off) to the decoder, then the channel count becomes 25 inputs and 32 outputs.

When the deviceType is set to 0 (off), the azimuth and elevation for that channel are ignored; however, a value is still needed. It is recommended to set the azimuth and elevation to 0 degrees when the device types are set to 0 (off).

Ambisonic Encoder Plugin

audiopluginexample.AmbiEncoderPlugin is built around the audioexample.ambisonics.ambiencodemtrx and audioexample.ambisonics.ambiencode functions. The number of devices (elements to be encoded) is the number of input channels of the encoder plugin. The ambisonic order determines the number of output channels of the encoder plugin.

audioexample.ambisonics.ambiencodemtrx generates the ambisonic encoder matrix from a given ambisonic order and a given device list. audioexample.ambisonics.ambiencode applies the ambisonic encoder matrix to raw audio resulting in ambisonic encoded audio. The formatting of the ambisonic audio may be specified with the audioexample.ambisonics.ambiencode function. The number of raw audio channels must equal the number of devices in the ambisonic encoder matrix.

The encoder plugin inherits directly from the audioPlugin base class. The plugin constructor calls audioexample.ambisonics.ambiencodemtrx to build the initial encoder matrix. The process function calls audioexample.ambisonics.ambiencode to apply the encoder matrix to the audio input. The output of the plugin is ambisonic encoded audio. The encoder matrix is recalculated only when a plugin property is modified which minimizes computations inside the process loop.

The plugin interface populates azimuth and elevation but not device type. The idea behind device type is to add off-channels to an encoder matrix to fit the matrix into a 8x-channel frame. For example: second-order has 9 channels, create a 16 channel encoder matrix, with the first 9 channels having device type of 1 (on) and the remaining 7 channels having device type of 0 (off).

audioTestBench(audiopluginexample.AmbiEncoderPlugin)

audioTestBench('-close')

Inspect Code | Run Plugin | Generate Plugin

Generate Custom Ambisonic Encoder Plugin

Generating ambisonic plugins can be an involved process. The ambiGenerateEncoderPlugin function streamlines the process of generating ambisonic encoder plugins. This function supports up to seventh-order ambisonics. Supported formats are 'acn-sn3d', 'acn-n3d', 'acn-fuma', 'acn-maxn', 'fuma-sn3d', 'fuma-n3d', 'fuma-fuma', 'fuma-maxn'. The function requires the following inputs:

% Provide a name for the audioPlugin class
name = 'myEncoderPlugin';

% Include a device list of element positions
device = [45 135 225 315 45 135 225 315; -45 -45 -45 -45 45 45 45 45];

% Specify the ambisonic order
order = 3;

% Specify the ambisonic format
format = 'acn-sn3d';

Run the function.

audioexample.ambisonics.ambiGenerateEncoderPlugin(name, device, order, format)

Once designed, the audio plugin can be validated, generated, and deployed to a third-party digital audio workstation (DAW).

Ambisonic Decoder Plugin

audiopluginexample.AmbiDecoderPlugin is built around the audioexample.ambisonics.ambidecodemtrx and audioexample.ambisonics.ambidecode functions. The ambisonic order determines the number of input channels of the decoder plugin. The number of devices (speakers locations) is the number of output channels of the decoder plugin.

audioexample.ambisonics.ambidecodemtrx generates the ambisonic decoder matrix from a given ambisonic order and a given device list. audioexample.ambisonics.ambidecode applies the ambisonic decoder matrix to ambisonic audio resulting in decoded audio. The formatting of the ambisonic audio may be specified with the audioexample.ambisonics.ambidecode function. audioexample.ambisonics.ambidecode determines the ambisonic order from the minimum of the ambisonic order of the input audio and the ambisonic order of the decoder matrix.

The decoder plugin inherits directly from the audioPlugin base class. The plugin constructor calls audioexample.ambisonics.ambidecodemtrx to build the initial decoder matrix. The process function calls audioexample.ambisonics.ambidecode to apply the decoder matrix to the audio input. The output of the plugin is decoded audio. The decoder matrix is recalculated only when a plugin property is modified which minimizes computations inside the process loop.

The plugin interface populates azimuth and elevation but not device type. The idea behind device type is to add off-channels to an encoder matrix to fit the matrix into a 8x-channel frame. For example: second-order has 9 channels, create a 16 channel encoder matrix, with the first 9 channels having device type of 1 (on) and the remaining 7 channels having device type of 0 (off).

audioTestBench(audiopluginexample.AmbiDecoderPlugin)

audioTestBench('-close')

Inspect Code | Run Plugin | Generate Plugin

Generate Custom Ambisonic Decoder Plugin

Generating ambisonic plugins can be an involved process. The ambiGenerateDecoderPlugin function streamlines the process of generating ambisonic decoder plugins. This function supports up to seventh-order ambisonics. Supported formats are 'acn-sn3d', 'acn-n3d', 'acn-fuma', 'acn-maxn', 'fuma-sn3d', 'fuma-n3d', 'fuma-fuma', 'fuma-maxn'. The function requires the following inputs:

% Provide a name for the audioPlugin class
name = 'myDecoderPlugin';

% Include a device list of speaker positions
device = [45 135 225 315 45 135 225 315; -45 -45 -45 -45 45 45 45 45];

% Specify the ambisonic order
order = 3;

% Specify the ambisonic format
format = 'acn-sn3d';

Run the function.

audioexample.ambisonics.ambiGenerateDecoderPlugin(name,device,order,format)

Once designed, the audio plugin can be validated, generated, and deployed to a third-party digital audio workstation (DAW).

See Also

Ambisonic Binaural Decoding

Related Topics

References

[1] Kronlachner, M. (2014). Spatial Transformations for the Alteration of Ambisonic Recordings (Master's thesis).

[2] https://en.wikipedia.org/wiki/Ambisonics

[3] https://en.wikipedia.org/wiki/Ambisonic_data_exchange_formats