Interleaved Reprojection now enabled for all applications by default

[aaron.leiby]

Alex Vlachos talked a bit about this at GDC 2016 two weeks ago during his presentation, “Advanced VR Rendering Performance”. We are treating this as a last resort safety net against poorly performing applications. If you get too close to using up the available frame time (either due to cpu or gpu work) the compositor will drop into half-time mode where we reproject every other frame -- thus "interleaved reprojection".

The important thing for developers to keep in mind is that when this happens, your game will be updating at 45hz instead of the normal 90hz, and you need to be robust to this dynamically changing while your app is running. So anywhere you have hardcoded assumptions about running at 90fps should check to see if we are currently presenting frames multiple times or not. This is done by checking Compositor_FrameTiming's m_nNumFramePresents, which will normally be 1, but will jump to 2 when interleaved reprojection is active.

The latest Unity SteamVR plugin does this when setting fixedDeltaTime for physics updates, and it is a good example of how to check for this situation.

if (lockPhysicsUpdateRateToRenderFrequency)
{
       var vr = SteamVR.instance;
       if (vr != null)
       {
              var timing = new Compositor_FrameTiming();
              timing.m_nSize = (uint)System.Runtime.InteropServices.Marshal.SizeOf(typeof(Compositor_FrameTiming));
              vr.compositor.GetFrameTiming(ref timing, 0);

              Time.fixedDeltaTime = Time.timeScale / vr.hmd_DisplayFrequency;
              Time.maximumDeltaTime = Time.fixedDeltaTime * timing.m_nNumFramePresents;
       }
}

Automatic interleaved reprojection can be disabled for development purposes by unchecking the "Allow reprojection" checkbox under the Performance settings in SteamVR.

The frame timing graph draws a red line on the stacked gpu view when interleaved reprojection is active.

Slides for Alex's talk can be found here:
http://alex.vlachos.com/graphics/Alex_Vlachos_Advanced_VR_Rendering_Performance_GDC2016.pdf

To force reprojection on for testing, use shift+r in the mirror window. Then you can toggle between presenting each frame once or twice using the r key. Use shift+r again to go back to automatic mode.

[cc_sander]

Reprojection sounds great. Does it automatically jump back to 90 if it can? I think it will be an issue if it doesn't because glitches in performance are pretty common even if you app hits 90 most of the time.

Also - a side note to your example: Adjusting fixedDeltaTime in Unity at runtime is generally a bad idea. I have done a lot of experiments with this in Unity, and the Physics system and interactions will be different if you don't have a consistent fixedDeltaTime. For example, a flying ball that hits another ball will act differently between to otherwise identical simulations if the fixedDeltaTime is different. This could lead to game physics being different for different players or even changing within the same play session if it is being dynamically adjusted. Both forces and collisions will be different enough to be noticeable. Better is to pick a fixed frame rate based on level of physics accuracy your game requires (could even be higher than 90fps), and not have it locked to your render framerate at all. In my case I have picked 90fps physics rate, and if game drops to 45fps, I will let physics still run at 90fps.

[aaron.leiby]

Originally posted by cc_sander:

Reprojection sounds great. Does it automatically jump back to 90 if it can? I think it will be an issue if it doesn't because glitches in performance are pretty common even if you app hits 90 most of the time.

Yes - if you bring up SteamVR's frame timing graph, you will see a red line on the stacked gpu view when interleaved reprojection is active and can get a feel for how it gets turned on/off. There is also a checkbox to display the frame timing in-headset (attached to the back of your right controller). Displaying the frame timing in headset incurs a little bit of overhead (you'll see the green gpu band get a bit thicker) so keep that in mind when using it to track down issues.

Also, good catch on the Unity physics update behavior. Thanks!

[KillahInstinct]

I might have missed the announcement but does this also mean the Steam VR performance demo is adapted to see if you meet the new bare minimum specs Vlachos was talking about?

IE: I'm hoping I can by with the bare minimum specs till the next-gen cards swings around but not sure where I am without actualling having a Vive - I think he said he was aiming for the 670.

[[RUST]Grumplestiltskin]

I'm having an unusual issue with our game (Hot dogs, Horseshoes and Hand Grenades). Basically on low-to-mid spec cpus, when 'Allow reprojection' is active, physics objects are exploding (by orders-of-magnitude amounts of velocity). A friend of mine (w 970, older AMD cpu) recorded this video:

https://www.youtube.com/watch?v=E0cGe9xtwVk

Ignore the red lines for missed frames, as they were caused by him recording video. No frames are dropped when this user runs the game without reprojection. However when reprojection is enabled, every single physics object, upon being 'grabbed' explodes off in a random direction.

I'm not doing anything particularly unusual. My core physics code functions very similarly to NewtonVRs, and yet I seem to be the only one experiencing this. Any thoughts from anyone?

[Strabby | Fresh W]

@[RUST]Grumplestiltskin - Yeah, we had a similar issue in Babel : Tower to the gods. When dropping frames (which is quite common when we're running in editor), the tower likes to bounce up and down and occasionally collapse! Or just do a nice little dance. Either way, not the nicest thing to be happening!

It seems to be caused by the fixedDeltaTime changing when the reprojection kicks in. You can fix this by turning off the "Lock Physics Update Rate To Render Frequency" on the Steam VR_Render component of the [SteamVR] object.


You might also want to set your fixed timestep (in the time settings) to 0.01111111 if it's not already, to make sure it updates 90 times a second.

[aaron.leiby]

I updated that bit of code in the example above given cc_sander's advice. Have you tried updating SteamVR_Render locally to match?

[[RUST]Grumplestiltskin]

aaron: I noticed there's a line there that's not in SteamVR. This one:
Time.maximumDeltaTime = Time.fixedDeltaTime * timing.m_nNumFramePresents;

I would suggest heavily against changing maximumDeltaTime to be so low, as it can actually just... straight up prevent fixedupdate callbacks from firing, which can prevent things like... stabilizing physics code from firing at all.

[[RUST]Grumplestiltskin]

Update: Removed the line changing fixeddeltatime. All bizarre physics behavior went away when projection kicks in.

[ayf690]

Just to clarify, where and what did you change to fix it?

[[RUST]Grumplestiltskin]

Setting lockPhysicsUpdateRateToRenderFrequency to false in SteamVR_Render, or simply blowing out the entire code-block listed at the origination of this post.

My tester with a low-spec machine (2012 amd cpu) that drops into reprojection mode constantly with my game found all errant physics behavior went away after this change.

Given this, I would suggest that anyone with a physics-heavy game (where physics drives logic) should _not_ dynamically change fixedDeltaTime based on current framerate.

[aaron.leiby]

FYI the edited version above does not dynamically modify fixed delta time, but it does set it to match the rendered frequency (so things work properly on headsets that run at 90hz vs 75hz).

Instead, it is modifying maximumDeltaTimemail to allow physics to update twice when reprojection kicks in.

[[RUST]Grumplestiltskin]

Gotcha. I will plug that code in manually and test it in my case to see how everything responds to it.

[cc_sander]

Glad you updated the code. Another thing of note is if you want Unity physics consistent between different headsets that have different refresh rates, you will still want fixeddeltatime to be consistent, even if this means you get multiple physics updates per rendered frame with one of the headsets. I found this to be the case when developing with Oculus DK1 and DK2 at same time. Also, if you want to run a physics framerate lower than headset render framerate (let's say 60hz), you might want to turn on interpolation on the rigidbodies. By doing that, the rigidbodies positions will be updated during update based on interpolation between the last 2 physics frames and you will get buttery smooth motion that adapts perfectly to the render framerate this way.

[AK0]

Reprojection seems to be disagreeing with our game, whenever we get a possible network spike the reprojection kicks in (seems like a couple times per second we get near the threshold) and its constantly forcing our game to 45hz. It is currently making Hover Junkers unplayable, but when we turn off the steamvr options the game runs buttery smooth. Is there anything we can do to force it off or turn up the threshhold or anything?

[zAERAN]

I'm having the same issue. Reprojection broke a LOT of things in MinigolfVR. I've had to move literally everything to a FixedUpdate loop and tell Unity not to sync the physics to the refresh rate.

It's pretty annoying having to alter large segments of code this soon before launch.