Make sure you check for apps that can cause audio latency!

[HammockTime]

I was just about to get a refund for EXA because I wasn't satisfied with how accurately it played audio on hitting ringers. It turned out to be the fault of a utility I run in the background, called VoiceMeeter. This is a very nice and versatile little tool, but with software such as EXA, it really messes with latency. The worst part is that it didn't seem to add a constant latency which maybe you adapt to, but rather one that varied slightly and just threw things off while playing (also made it hard to follow the metronome).

People who have screen recording software running might also not be aware of the performance impact those kinds of apps can have in audio software such as EXA. Kill them all - well, unless you're doing a video, but then don't expect a very synchronized result.

[zk]

Thanks again for tracking down the source of this latency issue! I appreciate you taking the time to step through it with me via Reddit. And of course, your other feedback/ideas were great, too.

This discovery should help a lot of players who might run into similar issues. And, as the developer, it was a big relief to learn that this particular issue was external to EXA :)

If anyone else finds an audio app (or any other "running in the background" apps) that cause similar latency issues for EXA, please post them here.

[zk]

It's not exactly the same, but on my side, I've noticed that doing high-quality video capture of the EXA window (for example, using Open Broadcaster Software / OBS) can have a slight impact EXA's overall audio/visual performance. This impact may be more noticeable on machines that are closer to the minimum system requirements.

Hopefully this isn't much of an issue. So far, the players on Twitch streams and YouTube videos don't seem to have latency-related issues, even when recording in "mixed reality" view.

[Stamper]

@HammockTime, you are a star, I also use VoiceMeeter (who wouldn't, its awesome), but wow does it mess up the latency. Killed VoiceMeeter, reassigned sound output in control panel, job done, EXA now works like a dream. It was impossible to record anything before, and similar to you, I was about to give up, but hey, now its time to bash those ringers.

Big up, cheers ( and Big up to zachkinstner, I'm now following you on twitter, look forward to your next instalment(s), great app, thanks)

[HammockTime]

Glad it helped you too, @Stamper.

It also occurs to me that people who are into audio production/recording are pretty likely to have some kind of virtual audio routing/patching type of software that can be very useful when working with DAWs, sequencers, etc. If these people are also drawn to apps like EXA as possibly useful in their studio workflow, and they use the same system for VR as for desktop audio production, it could be that they wouldn't consider the latency implication of the utils mentioned above since, for desktop audio use, the impact could be within acceptable limits.

I guess what I'm saying is that this is something that should be quite prominently mentioned in the store description since these users might not be using Steam very much and so might be unfamiliar with this discussion forum.

[zk]

Killed VoiceMeeter, reassigned sound output in control panel, job done, EXA now works like a dream.

@HammockTime saves the day!

I have added this to the "System Requirements" section of the store page:

Additional Notes: Disable audio-routing apps (like VoiceMeeter) to avoid hit-to-sound latency issues (details: http://steamcommunity.com/app/606920/discussions/0/1327844097115617013 ).

[Stamper]

I use Cakewalk Sonar as my DAW and also use Audacity a fair bit, but of course you can explicitly specify which sound drivers to use within the apps themselves, so no latency issues. I use VoiceMeeter when recording play/game activity so that I can have separate audio streams between the game and 2 mics that I use, and then later mix them down. This is only when I use FRAPS. I guess I'm now going to have a problem if I ever want to record my EXA play time and have separate audio streams, may take a bit more manual jiggery pokery setting up each device rather than using VoiceMeeter. QED. Hmm.....watch this space (unless anyone else has any ideas?)

[HammockTime]

In the interest of documenting the apps that might affect EXA, I've looked around for Windows utils that can affect latency. It would be good to find out which do have a noticeable impact and which don't (if there are any), because these tools are useful when dealing with audio and it'd be nice to be able to have them running if at all possible.

Here are the ones I've found:

R3LAY VPB Virtual PatchBay (formerly JADE Engine)[www.lawo.com]
Virtual Audio Recorder[www.virtualaudiorecorder.com]
Synchronous Audio Router[sar.audio]
VB-Audio Virtual Cable and HiFi Cable[vb-audio.pagesperso-orange.fr]
Voicemeeter Virtual Audio Mixer[vb-audio.pagesperso-orange.fr]
Voicemeeter Banana Advanced Audio Mixer[vb-audio.pagesperso-orange.fr]
VBAN: The Audio Network[vb-audio.pagesperso-orange.fr]
JACK Audio Connection Kit[jackaudio.org]
Carla[kxstudio.linuxaudio.org]
Virtual Audio Streaming[www.virtualaudiostreaming.net]
Virtual Audio Stream[www.virtualaudiostream.com]
virtual audio capture grabber device[github.com]
CheVolume[www.chevolume.com]
Audio Router[github.com]
NTONYX Virtual Audio Cable[www.ntonyx.com]
Muzychenko Virtual Audio Cable[software.muzychenko.net]

Zach, I was also thinking that some users might not know these kinds of utils as audio "routing" or "patching" tools. They might simply be using them because they filled a need and a quick search led them there, or a friend might have recommended them. In other words, it might actually be useful to refer to the apps by name in order to alert users to possible latency issues.

[zk]

This list is fantastic -- thanks @HammockTime!

I think it would be useful to confirm whether these apps are specifically causing a problem for EXA, or if they would affect any app. I suspect it is the latter, where the latency just has a larger perceived impact with a music-performance app, since it is so sensitive to exact timing.

These background-audio apps would need to move data into an audio buffer (just like EXA does internally), and this is where latency occurs. Essentially, the app fills a buffer of size 256, 512, 1024, or X samples -- at 44100 samples/sec these are roughly 5ms, 10ms, and 20ms of added latency.

I found this Reddit post discussing audio lag when the user was streaming on Twitch. Here's a video that demonstrates the added latency at various buffer sizes, and shows how to reduce (but not eliminate) it:

https://www.youtube.com/watch?v=yFM80hjukO0

[Stamper]

I think the issue here is that you have 2 layers of latency. VoiceMeeter (VM) Input uses a VAIO driver (windows default), so EXA is using that to load its samples into and buffer. VoiceMeeter (VM) then takes that input and again buffers into either MME, WDM or KS depending on what your final output device is. Thus you have 2 layers of latency: EXA > VM > Output.

Better to just have the one layer and let EXA talk direct to the final output device and disable VM or similar.

Another final note. The VoiceMeeter (VM) VAIO Input, MUST to be set as your default sound device for VM to work. Any output device then used by VM in either A1 and/or A2 output will be locked exclusive by VM and cannot be used by any other application. I have had this cause problems if I do try to use one of the final output devices as a manually selected audio device in another app (some apps like Winamp, most DAWs, Audacity, for example let you choose the output device instead of using the windows default device. They dont like having VM running).

I dont use any of the other apps listed by HammockTime, so can't comment on those, sorry.

[zk]

I think the issue here is that you have 2 layers of latency.

Yes, although EXA does its best to avoid this buffer-based latency -- it predicts controller motion enough to schedule audio events in advance. If this prediction isn't far enough in advance, EXA will clip the beginning of a note's audio data rather than delay the start of the note.

[vb]

Voicemeeter Latency is also pending on configuration. the device selected as output A1 is the critical point. If using a good ASIO device (with 256 sample buffer) you may have less latency in your game than when using direct device in DirectX ... on the opposite If using MME device , you can get additional latency. Also the VAIO configuration can be optimized for a given application or given game, but we do not recommend to go below 4096 samples (without having read our tech note about this). At the end Voicemeeter can be optimized for minimal/optimal latency but this can required expertise in windows audio... feel free to contact us by e-mail if needing information about this...

[HammockTime]

@vb, when you say "a good ASIO device", do you mean the combination of hardware + ASIO driver or just the hardware? Can the integrated Realtek chipset on my mobo with ASIO4ALL driver can be considered "a good ASIO device" - I'm guessing not.

In any case, in scanning over the Unity docs on audio, it doesn't look like the lower level Windows audio features described here, https://docs.microsoft.com/en-us/windows-hardware/drivers/audio/low-latency-audio, are exposed. I wasn't aware that there'd been that much of an improvement in Windows 10, WRT audio latency.

[vb]

Originally posted by HammockTime:

@vb, when you say "a good ASIO device", do you mean the combination of hardware + ASIO driver or just the hardware? Can the integrated Realtek chipset on my mobo with ASIO4ALL driver can be considered "a good ASIO device" - I'm guessing not.

yes, I meant a real hardware device with a good ASIO driver. ASIO4ALL will add latency for nothing... It's better to connect Voicemeeter output A1 directly to WDM device (WASAPI).

Originally posted by HammockTime:

In any case, in scanning over the Unity docs on audio, it doesn't look like the lower level Windows audio features described here, https://docs.microsoft.com/en-us/windows-hardware/drivers/audio/low-latency-audio, are exposed. I wasn't aware that there'd been that much of an improvement in Windows 10, WRT audio latency.

Yes, there has been improvement, but in the reality, WASAPI is not as efficient as it should, and audio stability still depends on driver as well... So PC configuration, hardware audio and driver interface are 3 parameters to manage to get optimal configuration... not so easy for users... generally speaking audio ask users to experiment a bit to find the best working configuration...

[zk]

Hi @vb, from your comments, it sounds like there might be a particular set of options that would allow users to use VoiceMeeter (or others) without adding latency to EXA's audio. Is that correct? Or would this also require specialized hardware?

People seem to really like VoiceMeeter, so it would be great if there were a step-by-step guide to resolve this issue. I'd love to tell people "change these settings" rather than "disable that app"!

[vb]

yes, there is different things to do. I guess users want to connect EXA to Voicemeeter to be able to manage the sound to different audio device or application (VOIP).

In this case latency is basically given by 2 connections:
- The Voicemeeter Main Stream (given by output A1 device = Monitoring)
- The EXA to Voicemeeter connection (MME, DX, KS, WASAPI) with what buffer size...

For Voicemeeter Main Stream we recommend to first select ASIO device as output A1 (with 256 or 512 sample buffer)... otherwise WDM or KS with 512 sample buffers.

For the Virtual connection between EXA to Voicemeeter virtual input... The best latency would be to connect EXA to Voicemeeter Virtual ASIO driver (if EXA support ASIO interface)... otherwise by using WASAPI or KS interface... with buffer below 1024 samples (512 or 256 preferred)... if EXA is using Direct-X or MME to playback audio, latency will stay high.

...then the Voicemeeter VAIO internal latency could be also optimized to 3x1024 sample for example ... or much less... to be experimented...