We have just released Steam Audio 2.0 beta 13, which brings support for AMD TrueAudio Next technology. TrueAudio Next lets developers accelerate certain spatial audio processing tasks by reserving a portion of the GPU. Combined with Steam Audio's ability to model a wide range of acoustic phenomena, this enables increased acoustic complexity and an increased sense of presence in games and VR applications. Click here
to download the latest version of Steam Audio, or click here
to grab the source code for the Unity and FMOD Studio plugins.
What is TrueAudio Next?
AMD TrueAudio Next (TAN) is a software library that implements high-performance convolution – a filtering technique – for audio data on supported GPUs. TrueAudio Next also provides a feature called Resource Reservation, which allows developers to dedicate a portion of the GPU's computational resources to be spent on time-sensitive tasks such as audio processing. Here is a brief explanation of both of these high-level features of TAN.Convolution
In the context of audio, convolution is a mathematical operation that transforms one audio signal into another. It is essentially a specific kind of filtering. A very wide range of audio filters can be implemented using convolution, including echoes, frequency equalization, HRTF-based 3D audio, reverb, and more. When using convolution, an audio filter is expressed in terms of its impulse response (IR), which can be manually specified, recorded (as is often the case when using convolution for reverb), or calculated by a simulation algorithm (as is the case with Steam® Audio).
Audio engines used for games and VR usually support convolution reverb: various combinations of audio data can be filtered using convolution, to add reverb, after the user has specified an IR in some way. This is usually real-time convolution, which means that audio is filtered as it is being played. IRs used for reverb are often quite long: representing the IR for an airplane hangar with a 5-second reverb at a 48 kHz sampling rate requires (5 * 48000) = 240,000 audio samples = around 935 KB. Real-time convolution is a very compute-intensive operation, and audio engines usually allow only 2-4 convolution reverb filters running simultaneously, to keep CPU usage within acceptable limits. Convolution reverb can model a wide range of acoustic phenomena, and leads to an increased sense of presence, which is especially important in VR. However, because it is a compute-intensive algorithm, it is often avoided in favor of less compute-intensive, less detailed reverb algorithms.
TAN uses the GPU compute capabilities of supported AMD GPUs to accelerate convolution, allowing developers to increase the acoustic complexity and detail in their games and VR applications using convolution reverb.Resource Reservation
TAN also provides a feature called resource reservation, which allows developers to reserve a portion of the GPU exclusively for audio processing. GPUs are made up of several CUs; these are roughly the GPU equivalent of the multiple cores in a CPU. It is not possible to reserve groups of cores on a CPU for one process, but with AMD resource reservation, this becomes possible for the GPU's cores. A typical high-end GPU might have 36 CUs, for example. Usually, a game's renderer uses all available CUs.
With Resource Reservation, developers can reserve some small number, say 4, of the CUs for audio processing only. This dedicates 4 CUs for audio processing, while reducing the number of CUs available to the renderer by 4. Resource reservation does not allow more than 20 to 25% of the GPU CUs to be reserved.Figure: Reserving Compute Units (CUs) with TrueAudio Next.What TrueAudio Next Is NOT
TrueAudio Next currently does not perform ray tracing for sound. Ray tracing is one way of simulating acoustic phenomena and calculating an IR; TAN focuses on accelerating the task of filtering audio data using IR after the IR has been calculated (or manually specified).
In 2013, AMD introduced TrueAudio technology, which used dedicated DSP hardware available on certain AMD GPUs to accelerate some audio functions such as rendering 3D audio using Head-Related Transfer Functions (HRTFs), and convolution with static IRs. In contrast, TrueAudio Next uses standard, existing Radeon™ GPU resources to implement more general and highly scalable convolution algorithms, which can be used for a wider range of filters, including convolution reverb with time-varying IRs. As a software library built using OpenCL, TrueAudio Next can also be targeted to CPUs, but it is especially beneficial when it is used with GPU Resource Reservation.
How is TrueAudio Next useful to Steam® Audio?Indirect Sound
Steam® Audio uses physics-based simulation to model two ways in which sound travels from a source to a listener. Direct sound is sound that travels in a straight line from the source to the listener, possibly passing through solid objects. Indirect sound is sound that is emitted from the source, bounces off solid objects repeatedly, and then reaches the listener.
Steam® Audio models indirect sound by calculating an IR using ray tracing. This IR is then used in a convolution reverb effect to add indirect sound effects to either individual sound sources (in the case of source-centric convolution reverb) or a submix of sound reaching the listener (in the case of listener-centric convolution reverb).
Simulating and rendering indirect sound is the most computationally intensive part of what Steam® Audio does, and TrueAudio Next helps manage this workload.Ambisonics
Source-centric convolution reverb can be directional in nature: somebody else's distant footsteps heard through an open doorway sound more directional than one's own footsteps in a large room, for example. Steam® Audio uses Ambisonics to represent the directional variation of indirect sound. The higher the Ambisonics order, the more spatial detail that can be rendered in the indirect sound. On the other hand, the higher the Ambisonics order, the more IRs that are needed in the convolution reverb effect. For example, 2nd order Ambisonics requires 9 convolutions per source. This too, increases the computational cost of indirect sound.
Note that Steam® Audio does not use TrueAudio Next for applying HRTF-based 3D audio rendering. The computational cost of HRTF processing is significantly lower than that of convolution reverb: thousands of sources can be rendered with HRTF-based 3D audio using a single CPU core.
Why does Steam® Audio support TrueAudio Next?
Steam® Audio's existing CPU-based convolution has sufficient performance for many applications, particularly those that rely heavily on listener-centric reverb. However, with TrueAudio Next, Steam® Audio can offer more flexibility to developers.
There are two main reasons why we chose to support TrueAudio Next in Steam® Audio:
- If we can provide developers with more flexibility in choosing how their audio processing workloads are balanced on a user's PC, we want to do that. TrueAudio Next allows developers to choose how CPU and GPU resources work together to provide a compelling audio experience on the user's PC.
- If we can allow developers to offer an optional higher level of audio detail with their existing content on PCs that are powerful enough, we want to do that. With Steam® Audio, developers just specify higher settings to be used if a TAN-capable GPU is installed on the user's PC. Developers do not have to re-author any content to work with TAN. Steam® Audio can be used to seamlessly select either CPU- or GPU-based convolution, depending on the user's hardware configuration.
Steam® Audio's support for TrueAudio Next does not in any way restrict Steam® Audio to specific hardware or platforms. We hope that our support for TrueAudio Next encourages hardware and platform vendors to provide more options for developers to balance their audio workloads against graphics, physics, and other workloads, which in turn will help them create better audio experiences for their users.
What are the benefits of TrueAudio Next?Benefits of GPU convolution
TAN's OpenCL convolution algorithms move convolution from the CPU to the GPU, and help achieve significant audio processing speedups. This has the following advantages:
Figure: GPU convolution performance vs CPU convolution performance, for 1s IRs, 1st order Ambisonics, 1024-sample frames, and a 48 kHz sampling rate. The performance advantage of GPU convolution increases with number of sources. CPU convolution is usable up to a moderate number of sources; GPU convolution is necessary for large numbers of sources.Benefits of resource reservation
- Steam® Audio's CPU-based convolution is highly efficient, and is sufficient for a wide variety of games and VR applications, especially if only using listener-centric reverb. However, TAN allows us to give developers the option of reaching an even higher level of auditory complexity: increasing the number of sources, or the Ambisonics order for indirect sound, or the IR length, etc.
- By moving convolution from the CPU to the GPU, the fraction of time spent by the CPU on audio processing can be kept within the CPU budget chosen by the developer. This also frees up more CPU resources for use by other tasks such as physics simulation or AI.
Resource reservation causes TAN's convolution algorithms to run in isolation from any rendering, physics, or other tasks that the GPU is being used for. This has two broad advantages:
Figure: Performance improvements with increasing numbers of reserved CUs, for 1s IRs, 1st order Ambisonics, 1024-sample frames, and a 48 kHz sampling rate. The plot shows the maximum number of sources that can be processed within a 6ms budget. Increasing the number of reserved CUs allows more sources to be processed within the same amount of time.Figure: Impact on GPU rendering performance when reserving 4 CUs for TrueAudio Next processing. The plots show the average FPS observed on various GPU benchmarks when run at the same time as a TAN benchmark using Steam® Audio. Measurements obtained using a Radeon™ RX 480 GPU with 4 reserved CUs, 256 total convolution channels and 1.3s IRs. All values are shown as a percentage of a baseline that is obtained when the TAN benchmark is not running. For example, reserving 10% of the available CUs reduces the average FPS in the Timespy benchmark by about 10%.Figure: Impact on TAN processing time when heavy graphics workloads are running concurrently. CU reservation limits the impact of heavy graphics workloads on time-sensitive audio processing tasks on the GPU. Measurements obtained using a Radeon™ RX 480 GPU with 4 reserved CUs, 256 total convolution channels, 1.3s IRs, 1024-sample audio frames, and a sampling rate of 48 kHz.
- Increasing the number of reserved CUs predictably results in improved performance, even under the heaviest workloads.
- Because audio processing does not interfere with graphics, the audio processing does not result in sudden frame rate drops and stuttering.
- Because graphics does not interfere with audio processing, rendering a complex scene cannot cause audio processing to slow down, which in turn helps avoid unacceptable clicks, pops, or other audio glitches.
How do I enable TrueAudio Next in Steam® Audio?System Requirements
Using the Unity Plugin
Radeon™ RX 470 Series
Radeon™ RX 480 Series
Radeon™ RX 570 Series
Radeon™ RX 580 Series
Radeon™ R9 Fury
Radeon™ R9 Fury X
Radeon™ Pro Duo
Radeon™ RX Vega 56
Radeon™ RX Vega 64
Windows 10, 64-bit
To use TAN in a Unity project that uses the Steam® Audio Unity plugin, you must add a Steam® Audio Custom Settings component to the Steam® Audio Manager Settings object in every scene that you wish to use TAN in, and then set Convolution Option to TrueAudio Next. Refer to the documentation for more details on the other settings that are displayed when Convolution Option is set to TrueAudio Next.Figure: Enabling TrueAudio Next in the Steam® Audio Unity plugin.
Steam® Audio allows the developer to select the number of CUs to be reserved and to independently configure the higher-detail settings to be used when TrueAudio Next is available. When TrueAudio Next is not available, CPU convolution with a default configuration can be used seamlessly. TrueAudio Next, along with resource reservation, can also be enabled or disabled on a scene-by-scene basis.Using the FMOD Studio Plugin
To use TAN in an FMOD Studio project that is part of a Unity project that uses Steam® Audio FMOD Studio plugin, follow the instructions for enabling TAN in the Steam® Audio Unity plugin. No further action is required.Using the Unreal Engine 4 Plugin
To use TAN in an Unreal Engine 4 project, you can simply configure the Steam® Audio plugin settings to use convolution type "AMD TrueAudio Next". This will allow you to configure the number of compute units available for audio processing. No further action is required.
TAN for Steam® Audio is currently available to users who build from source via this pull request on GitHub
(requires membership in the EpicGames organization).Figure: Enabling TrueAudio Next in the Steam® Audio Unreal Engine plugin.Using the C API
To use TAN in an application that directly uses the Steam® Audio C API, you will need to perform two main steps:
- Initialize an OpenCL compute device that supports TAN. To do this, call iplCreateComputeDevice, typically at application startup. In the deviceFilter parameter, set type to IPL_COMPUTEDEVICE_GPU, and requiresTrueAudioNext to IPL_TRUE. This will cause Steam® Audio to try initializing OpenCL to use a GPU that supports TAN. If no such device is found on the user's PC, an error code is returned. The application should test this error code and take appropriate action in case of failure, such as falling back to Steam® Audio's built-in CPU-based convolution. This compute device should be passed to all Steam® Audio API functions that take a computeDevice parameter.
- Create an Environmental Renderer that uses TAN for convolution. When calling iplCreateEnvironmentalRenderer to create an environmental renderer, in the renderingSettings parameter, set convolutionType to IPL_CONVOLUTIONTYPE_TRUEAUDIONEXT. If the function returns a status code indicating success, then all subsequently created Convolution Effect objects will use TAN.
Note that when using TAN for convolution, iplGetMixedEnvironmentalAudio is much more efficient than calling iplGetWetAudioForConvolutionEffect for each Convolution Effect object separately.