SOURCE ENGINE AUDIO :: Head-Related Transfer Functions

[WildCat]

Edit (12 Jan 2017):

Valve has acquired Impulsonic. Impulsonic are the creators of Phonon 3D, which is the 3D audio system that Valve used to provide HRTF in CS:GO. I think this is great news because it means that Valve are taking 3D audio seriously and I think that we will see HRTF being introduced into other Valve games (especially any future VR titles). I also think that it could mean that the HRTF feature in CS:GO will continue to get improvements and updates.

- Screenshot[imgur.com] (announcement email I got from Impulsonic)
- Impulsonic's website[www.impulsonic.com]

Edit (8 Dec 2016):

We are truly blessed this day. Praise be unto Lord Gaben for He has lavished the faithful patrons of CS:GO with the everlasting joy of HRTFs. Praise be! Praise be.

CS:GO Blog: Holiday Season never sounded so good.

Original post (2 Oct 2013; last edited 9 Jan 2016):

Please consider adding head-related transfer functions[en.wikipedia.org] (HRTF) to the headphone mix in Source Engine games.

By implementing HRTFs, headphone users can enjoy a sound field that is much more natural, realistic and three dimensional. You'll be able to perceive the location of a sound coming from any arbitrary direction including behind, above and below.

With consumer-level head-mounted displays coming in the near future, the desire for true 3D audio with headphones will be even greater than it already is. The immersive, 3D visuals of VR deserve to be supplemented by similarly sophisticated audio.

I realize that implementing HRTFs may not be a trivial exercise. Therefore, as an alternative to doing this yourselves, I suggest integrating one of the existing solutions that supports HRTFs. I know of six options. The first two are both implementations of the OpenAL API:

• Rapture3D[blueripplesound.com]
  
• OpenAL Soft[kcat.strangesoft.net]

One of the advantages of OpenAL is that end-users can swap the implementation of OpenAL that a game uses. This means that merely implementing support for OpenAL will allow enthusiasts like myself to use Rapture3D or OpenAL Soft in Source games.

The other four options are:

• RealSpace 3D[realspace3daudio.com]
  
• Oculus Spatializer (Oculus Audio SDK)[developer.oculus.com]
  
• 3Dception[twobigears.com]
  
• Phonon 3D[www.impulsonic.com]

Video Demonstrations

For those not familiar with HRTFs and how they improve audio in games, have a listen to these demo videos. Note: You need to be using headphones.

Rapture3D:

http://www.youtube.com/watch?v=KKkI0W0_2Qg
http://www.youtube.com/watch?v=f4PJ68LHU-I
http://www.youtube.com/watch?v=zx4k8qewC_8
http://www.youtube.com/watch?v=iikd_hucZbY
http://www.youtube.com/watch?v=lqqiIR017SQ

RealSpace 3D:

http://www.youtube.com/watch?v=s3eOuqAmLAA

Interactive Demonstration

I created this interactive demo in Unity to show off RealSpace 3D.

• RealSpace 3D Audio v0.10.6 Demo[matthewkerswill.com] (148 MB)

Further reading

• I Want HRTFs In My Games![matthewkerswill.com]
  - A non-game-specific article I wrote on my personal website.
  
  
• OpenAL Audio for Source Engine
  - My suggestion thread on the old Steam forums that dates back to 2010.

[WildCat]

Messaggio originale di rim6o:

Do you think you could post instructions for linux as well. I wasn't able to enable HRTF for AAMFP.

After reinstalling Windows on my new SSD, I no longer have Ubuntu installed on my machine. I originally installed it specifically for testing if I could get HRTFs working in a Source engine game via OpenAL. Since I'm finished with that testing, I don't think I'll be installing Ubuntu again any time soon.

I wouldn't feel comfortable writing a guide with instructions for Linux without being able to double and triple-check my method while writing it.

I would expect the method for AAMFP to be the same as ATDD. I can't confirm that, though, because it never installed AAMFP on Linux. My testing was only in ATDD.

Messaggio originale di rim6o:

1. Could you elaborate on how Dolby Headphone or CMSS3D are inferior to OpenAL HRTF?

I actually wrote a few paragraphs on that in my guides. If you look in the 'Further Reading' section, you'll find it. I could potentially flesh out the explanation so let me know if there are any facets of the discussion that you think could be further expanded upon.

Messaggio originale di rim6o:

2. Is '3D surround sound' the correct term for what HRTF provides? It would be beneficial to diferentiate from gimmicks while discussing HRTF.

That's a good point. However, I think it's going to be difficult to differentiate good HRTF solutions from bad HRTF solutions merely with different terminology.

'3D surround sound' on its own is not sufficient because it doesn't mention headphones. I think that my use of '3D surround sound for stereo headphones' is acceptable. I think '3D headphone surround' is also pretty good and is, in fact, used by Blue Ripple Sound (makers of Rapture3D) on their website.

[rim6o]

@WildCat
Sorry for making an impression that I don't put enough effort before asking. I'm just short on time and often I merely skeem through articles. Will make sure to pay more attention in the future though. Off to do my homework!

Linux:
I was hoping you'd have an idea on what could go wrong in my case (libs replaced, hrtf = true and no actual HRTF in-game).

A tip:
You can install linux (unlike Windows) on ANY storage, including a USB dongle and on any partition on any of your drives (doesn't have to be the first partition).

Naming:
There are two reasons why I pay so much attention to naming:
a) half of 'technologies' out there are fake / don't reflect the promise of their names and fancy acronyms
b) the very reason why there is so little demand for proper 3D sound could be that most people are convinced they do get 3D sound because they've enabled the super-dooper surround sound option in their soundcard control panel.

Few days ago I watched an ArmA 3 gameplay video. Stereo panning as ♥♥♥♥.

PS. It was a guy at digitalcombatsimulator.com forums who brought my attention to 3D on stereo headphones but I would also like to thank you for the HRTF advocacy.

[Kazioo]

Messaggio originale di WildCat:

Messaggio originale di 2point4:

OpenAL is already in the Source engine. You just need to be using Linux or OSX.

I recently installed Ubuntu 13.04 64-bit in a new partition on my hard drive specifically for testing this. After a lot of fiddling around and tweaking, I'm starting to doubt that Source really uses OpenAL on Linux.

Maybe the problem is in outdated Steam for Linux libraries? I've found this on archlinux wiki:

https://wiki.archlinux.org/index.php/steam

Steam, by default, ships with a copy of every library it uses, packaged within itself, so that games can launch without issue. This can be a resource hog, and the slightly out-of-date libraries they package may be missing important features (Notably, the OpenAL version they ship lacks HRTF and surround71 support).

[WildCat]

Messaggio originale di Kazioo:

Maybe the problem is in outdated Steam for Linux libraries? I've found this on archlinux wiki:

https://wiki.archlinux.org/index.php/steam

Thanks for the tip but I actually already discovered that issue myself and updated the library. This allowed me to get HRTFs working in the Steam version of Amnesia: The Dark Descent but my Source engine test game, Portal, continued to sound the same.

[Monstieur]

You could just buy this: http://software.store.creative.com/software/sound-blaster-x-fi-mb3/106-21569.aspx

It's better than Rapture 3D because it works globally across the OS (it creates a new default audio playback device and pipes the processed audio out your existing device). Games that use OpenAL can access EAX 5.0 and will have Creative's superior surround positioning for 5.1 and HRTF for headphones. Games that don't use OpenAL but output pre-mixed surround will still benefit from Creative's superior HRTF for headphones but 5.1 will remain unaffected.

[WildCat]

Messaggio originale di Locutus:

You could just buy this: http://software.store.creative.com/software/sound-blaster-x-fi-mb3/106-21569.aspx

Thanks - I was not aware that Creative were selling their software separately from their audio cards. I decided to buy a copy to give it a test. Unfortunately, I don't think it's playing nicely with my ASUS Xonar Essence STX because the sound is extremely buzzy and distorted.

But, no matter... I can still provide comment on it because I have prior experience with Creative software. In the past, I have used an X-Fi XtremeGamer audio card and the features of this X-Fi MB3 software appear to be very similar to the software that came with the that card.

Messaggio originale di Locutus:

It's better than Rapture 3D because it works globally across the OS

You speak of this as an advantage but it's actually a huge disadvantage. I often listen to music when I'm not playing games and it would be completely unacceptable to have the surround effects being applied to the music. Any self-respecting music enthusiast or audiophile will tell you that music should be experienced as the artist intended it and not with any inappropriate post-processing applied to it.

Having to turn the Pro Studio system on and off before and after playing a game is an unacceptable user experience and not a solution to the issue.

Messaggio originale di Locutus:

Games that use OpenAL can access EAX 5.0 and will have Creative's superior surround positioning for 5.1 and HRTF for headphones.

Being able to access EAX 5.0 is only useful for old games that implemented it. EAX is now deprecated and you won't see any new games coming out with support for it. EAX has been replaced by EFX, which covers all the EAX functionality and is open. Rapture3D and OpenAL Soft both support EFX.

I can't comment on the quality of Creative's HRTFs because I don't know if they've upgraded them since I used 'CMSS-3D Headphone' on my X-Fi XtremeGamer. If they haven't upgraded them, then I'd describe them as decent but not great. I remember that sounds coming from above or below didn't sound particularly convincing.

Messaggio originale di Locutus:

Games that don't use OpenAL but output pre-mixed surround will still benefit from Creative's superior HRTF for headphones but 5.1 will remain unaffected.

For games that output pre-mixed surround, the software will be down-mixing to stereo and applying HRTFs in the process. This is the same thing that the likes of 'Dolby Headphone' and 'Razer Surround' do. I don't consider this to be an acceptable solution for providing HRTFs to headphone users in video games.

This sort of system is disadvantaged in the following ways:

• The input for the system only provides 5 or 7 positions from which sound may emanate (depending on whether you're running the game in 5.1 or 7.1 mode). This is not good enough positional accuracy.
  
• 5.1 and 7.1 systems don't provide any elevation information. Therefore, you will not be able to hear any difference between sounds coming from above and below.

[Monstieur]

Messaggio originale di WildCat:

Unfortunately, I don't think it's playing nicely with my ASUS Xonar Essence STX because the sound is extremely buzzy and distorted.

Try setting all the playback devices to 24-bit 48000Hz in Windows.

Messaggio originale di WildCat:

You speak of this as an advantage but it's actually a huge disadvantage. I often listen to music when I'm not playing games and it would be completely unacceptable to have the surround effects being applied to the music. Any self-respecting music enthusiast or audiophile will tell you that music should be experienced as the artist intended it and not with any inappropriate post-processing applied to it.

Having to turn the Pro Studio system on and off before and after playing a game is an unacceptable user experience and not a solution to the issue.

I mean OpenAL will always be available to ALL applications including those that don't use Rapture3D.

I always keep everything switched off for my 5.1 speakers. I simply install it so that OpenAL & EAX are available and surround positioning is accurate compared to onboard playback device in Windows.

Messaggio originale di WildCat:

For games that output pre-mixed surround, the software will be down-mixing to stereo and applying HRTFs in the process. This is the same thing that the likes of 'Dolby Headphone' and 'Razer Surround' do. I don't consider this to be an acceptable solution for providing HRTFs to headphone users in video games.

This sort of system is disadvantaged in the following ways:

• The input for the system only provides 5 or 7 positions from which sound may emanate (depending on whether you're running the game in 5.1 or 7.1 mode). This is not good enough positional accuracy.
  
• 5.1 and 7.1 systems don't provide any elevation information. Therefore, you will not be able to hear any difference between sounds coming from above and below.

These kinds of games usually have a built in Headphones setting which applies HRTFs to the individual effects.

[WildCat]

Messaggio originale di Locutus:

Try setting all the playback devices to 24-bit 48000Hz in Windows.

Unfortunately, this didn't work for me. However, it does seem like it might be related to sample rate because the sound of the distortion and buzzing does change slightly when altering these settings. I couldn't find any setting that came anywhere close to eliminating it, though.

Messaggio originale di Locutus:

These kinds of games usually have a built in Headphones setting which applies HRTFs to the individual effects.

If you know of any specific games that provide a Headphone mode with built in HRTFs, I would be very interested to know so that I can test them out.

[Monstieur]

Messaggio originale di WildCat:

If you know of any specific games that provide a Headphone mode with built in HRTFs, I would be very interested to know so that I can test them out.

All Valve's games...

[WildCat]

Messaggio originale di Locutus:

All Valve's games...

I am very familiar with Valve's games and I know for a fact that the Source engine's Headphone mode doesn't implement HRTFs. If it did, I would not have made this thread.

When the 'Speaker Configuration' is set to 'Headphone', the Source engine only does basic one-dimensional left-to-right panning for localization of sound sources.

[Kazioo]

I think Battlefield 3 had a simple HRTF implementation in the Headphone mode (definitely on PS3, I didn't check Headphone options in the PC version).

[WildCat]

Messaggio originale di Kazioo:

I think Battlefield 3 had a simple HRTF implementation in the Headphone mode (definitely on PS3, I didn't check Headphone options in the PC version).

I don't have Battlefield 3 installed anymore so I had a quick listen to Battlefield 4 instead. On PC, the audio options menu provides a setting for 'Speaker Configuration', which can be set to either 'Stereo' or 'Surround'. There's also a 'Speaker Type' setting, which can be set to 'Headphones', 'War Tapes', 'TV', 'Hi-fi', or 'Home Cinema'.

When the 'Speaker Configuration' is set to 'Stereo', it appears that they're doing a little more than most games do in the sound localization department. Essentially, sounds coming from behind are made quieter than sounds coming from in-front. This effect is present for all the 'Speaker Type' settings and not just 'Headphones'.

Whilst I can sort-of appreciate that little touch, I don't really consider it to be a proper HRTF unless direction-specific frequency response alterations are being made. A sound that is merely lower in volume could be mistaken as coming from further away rather than coming from behind. I guess you could call it a 'simple HRTF' but it is certainly very, very simple.

[Kazioo]

I think it was called "enhanced stereo mode":

http://annoyedadmin.files.wordpress.com/2011/11/bf3.png

The difference was very noticeable, but it didn't work well for me. My own footspets sounded like the legs weren't under the camera.

EA DICE, Audio director Bence Pajor:

We currently have a really rough version of HRTF in the headphones setting. There are currently no plans for a proper HRTF setting, but we are investigating better spatial awareness.

http://blogs.battlefield.com/2014/02/ask-dice-sound-design/

[rim6o]

If even DICE doesn't give a rat's buttock over true 3D sound then I think it's all a downward spiral. Correct me if I'm wrong but OpenAL doesn't seem to flourish neither.

BTW, Wildcat, are you fine with general 3D audio related discussions in this topic?

[WildCat]

Messaggio originale di rim6o:

If even DICE doesn't give a rat's buttock over true 3D sound then I think it's all a downward spiral. Correct me if I'm wrong but OpenAL doesn't seem to flourish neither.

I wouldn't say it's a downward spiral. It's true that OpenAL is rarely used by game developers these days but OpenAL doesn't have to be the medium through which HRTFs are implemented. It think it's only a matter of time before this sort of technology catches on. I just hope it will be sooner rather than later.

GenAudio is making its entry into this market with it's AstoundSound plugins for Wwise and FMOD and we will most likely see games starting to use them soon. It remains to be seen, however, whether AstoundSound turns out to be a decent 3D audio solution or whether GenAudio is just selling snake oil.

I have to admit that I have not been particularly impressed with their demo videos thus far. That said, I would need to test-drive some kind of interactive demo before passing judgement and I have not been able to find any such demos available for download.

Here's their latest demo video. What do you think?
https://www.youtube.com/watch?v=GeokLJGaco4

Messaggio originale di rim6o:

BTW, Wildcat, are you fine with general 3D audio related discussions in this topic?

Yeah, I'm fine with that. Please be my guest. :)