Stage Mode for Room Scale Games

[Ian (Sealost)]

I am the developer of The Thrill of the Fight, which is a room scale only game that keeps its content within the user's play area. For the game to function properly on Quest, the Quest must be using a stage reference space. Otherwise the user can do a system-level recenter, which shifts the play area outside of the user's boundaries and breaks the game.

When I try TotF through Steam Link, the Steam Link app seems to be using a local floor reference space, which allows system-level recentering and makes the game unplayable.

Is there any way or any plans for a game to trigger Steam Link to use a stage reference space on the Quest side? It would probably be ideal if Steam Link handled this transparently, with OpenVR games automatically forcing a stage reference space and OpenXR games automatically using whichever reference space type the game is using on the PC side. (And as a side note, the Quest seems to provide an incorrect play area size if it's not using a stage reference space, so you'll likely need to retrieve and sync the play area size after the stage reference space is created.)

If automatic syncing is not an option, you might want to do what Virtual Desktop does and provide an explicit user-controlled setting that forces a stage area reference space on the Quest side. It sounds like this may be a desirable option anyway for hybrid tracking setups.

[danwillm]

Hi,
Thanks for the report.

Could you confirm whether this issue is also present when using other headsets with SteamVR+OpenXR (index, vive,etc.)?

[Ian (Sealost)]

Could you confirm whether this issue is also present when using other headsets with SteamVR+OpenXR (index, vive,etc.)?

At first I thought this question didn't make sense in context since I was talking about the Quest system recenter functionality you get from holding down the recenter button, but I jumped in with a Rift S in SteamVR and found similar issues I was not previously aware of. Now I'm just all kinds of confused about how the Steam Link app is running on the Quest side.

So yes, a very similar issue is also present when using a Rift S through SteamVR by hitting the recenter button on the SteamVR dashboard. I had not noticed this option before, and I was not aware that this functionality existed. Previously, I thought no recenter option was shown to users that had a room scale boundary configured, and this has not previously been an issue for TotF going all the way back to 2016. Is this something new that was added in SteamVR 2.0?

On a fresh start of SteamVR, the tracking origin seems to be properly centered where the play area should be, and TotF works fine, but using that recenter option will shift the play area and break the game. Recentering *before* starting TotF can cause some especially horrible behavior with the size of the play area.

My original post was based on assumptions from how other Quest-to-PC streaming apps have had a hard time getting play area support working properly. But because of the PC recentering problems, I now don't know what's a problem with my game in PC SteamVR and what's a problem with Steam Link, if any.

So I guess here's the first question I need to find an answer to: What do I need to do to ensure TotF remains centered to the user's real-world play area in normal PC SteamVR and to stop the dashboard recenter from breaking the game? For context, TotF uses the legacy OpenVR API. Is there anything I can do under that API, or am I going to need to update the game to use OpenXR with a stage reference space (which according to the spec should already behave how I'm describing and be unable to be recentered)?

If this isn't the right place to get the answer to that question, can you direct me the right place to get help with that?

Once I have that answer, then I can see what problems remain with Steam Link, if any.

[danw]

Hi,
Thanks for the reply.

This sounds like a problem with your app and OpenVR - not something Steam Link specific.

There shouldn't have been any new behavior introduced in SteamVR 2.0 - I just gave your app at test on pre-2.0 and noticed a recenter still repositions you in the world.

It sounds like you're using vr::TrackingUniverseStanding in your app? If so, that universe is affected by recenters and there's nothing apps can do to turn that off.

Although it's not recommended, you could try using vr::TrackingUniverseRawAndUncalibrated, which isn't affected by recentering, but you'll need to do the mapping between that and the play area rectangle yourself to position the play area correctly - You can call IVRChaperone::GetPlayAreaRect to get the play area rectangle (0,0,0 will be the current standing universe center), and can get the current transform from raw to standing space with IVRSystem::GetRawZeroPoseToStandingAbsoluteTrackingPose.

While I would be recommending to use OpenXR here, as of right now, the implementation of the OpenXR stage space in SteamVR will actually be affected by recenters, so that it places 0,0,0 of the stage space to the position of the recenter. This should still be compliant with the spec, but I understand that it's not ideal (this behavior was added before extensions like local floor). We'll put this down as something to look into change, and I'll try to keep you updated if we do change the behavior.

Hope this helps, feel free to let me know if you have any other questions or clarifications.

Thanks!

[Ian (Sealost)]

(My two previous replies were a little wordy because I found out more information while I was writing them. I've edited them down into one relatively shorter reply and deleted the two that were here before.)

For context, TotF's main development period was from 2016 to 2020. The dashboard recentering seems to have been added in mid 2020, and I think this was a big mistake. Valve basically pioneered the idea of the play area, and the whole point of the standing universe and room setup process was to provide a mapping between the virtual world and the user's real-world space. Adding a transparent system-level recenter allows the user to break that mapping and invalidate the play area. This change introduced the potential to break already-released games, and no clear solution was provided for games that needed to remain mapped to the user's real-world space.

Now, more than 3 years later, I'm not sure what solution makes sense going forward. In hindsight, a new universe should have been added to allow for system-level recentering without changing the behavior of already-released games. However, fixing the problem that way now would introduce behavior changes for games released in the meantime. If you even plan to fix this in OpenVR, I think it would make sense to just add a new "stage" universe that isn't affected by recentering.

I think your OpenXR STAGE reference space implementation absolutely needs to change, though. It seems very clear in the specs that STAGE spaces should not be affected by recentering. The spec explicitly states that LOCAL spaces can have a system-level recenter, but it does not provide that allowance for STAGE spaces. The STAGE space itself is described in terms of a play area rectangle, which would be pointless if a recenter was possible. Even the name "stage" implies that it's a specific fixed area of real-world space for the player to walk around within.

The LOCAL_FLOOR extension adds new functionality and does not change anything about how the core spec treats STAGE spaces. The LOCAL_FLOOR spec makes it clear the extension is needed specifically because STAGE spaces cannot be recentered under the core spec:

The core OpenXR specification describes that standing-scale experiences are meant to use the STAGE reference space. However, using the STAGE forces the user to move to the stage space in order to operate their experience, rather than just standing locally where they are.

As for your workaround, in addition to you not recommending using the uncalibrated universe, I don't think it's reasonable to create in-game space calibration when SteamVR users have already done that mapping on a system level. Hopefully we can find a solution that utilizes the functionality that has already been created in SteamVR specifically to handle this exact scenario.

I notice that the SteamVR loading screen shows a gray outline of my play area located exactly where it should be regardless of where I recenter. This is true in Steam Link or when using a PC headset. What is the SteamVR loading screen doing to keep the play area in the correct location?

And as for Steam Link, traditionally it has been hard for other Quest streaming apps to get things right for room scale games, so I'm not ruling out that there may be additional problems with Steam Link even if we get the above issues fixed. As it stands right now, though, you can run TotF on a PC headset and it works fine, but you cannot run TotF through Steam Link due to the automatic recentering.

[Ian (Sealost)]

Happy new year! I'm hoping to revisit this issue.

I think the problem I originally raised is specific to Steam Link, and I hope we can get a solution in place. Originally, I was asked this:

Originally posted by danwillm:

Could you confirm whether this issue is also present when using other headsets with SteamVR+OpenXR (index, vive,etc.)?

In trying to answer that question, I learned that SteamVR allows a forced recenter that breaks room scale games with PC headsets. However, this doesn't seem to apply to Steam Link. When connected through Steam Link, the option to do that forced recenter is missing from the SteamVR dashboard. In a different thread where someone else asked about the missing icon, you said:

Originally posted by danwillm:

Recentering is handled by the headset, not SteamVR. You can hold the meta button on the right controller to recenter.

My original post was about how on Quest, room scale games need to use stage tracking to prevent the headset from allowing the user recenter. If recentering is handled by the headset like you said (which makes sense and was my assumption in the original post), then Steam Link will not work with room scale games unless the app or the user can make Steam Link use stage tracking on the Quest side of things.

So I'm back to my original question - can you either make it that stage tracking PCVR games automatically trigger the Steam Link app to use stage tracking on the Quest or can you add a feature to manually allow users to force stage tracking on the Quest side like Virtual Desktop does?

I notice that the SteamVR loading screen shows a gray outline of my play area located exactly where it should be regardless of where I recenter. This is true in Steam Link or when using a PC headset. What is the SteamVR loading screen doing to keep the play area in the correct location?

I'm curious about the answer to this question, too, as it might provide a possible workaround.

[Reonu]

I'm not a dev, but this confuses me. "Recentering" doesn't change where the play area is: the play area covers the same exact physical space. It just changes what the "origin" of the area is. So I can make it so that the "origin" is a corner of the play area, but the play area didn't suddenly move. I'm still in the corner of the play area. Sounds like games need to be updated to account for this concept, but the play area hasn't moved anywhere.

You can check this by enabling the option in SteamVR to always display the play area rectangle on the ground, overlayed on top of the game. The play area never moves, no matter how much you recenter. If you also use OVR Advanced Settings to display a center marker, you will see that when you "recenter" only the center marker moves, inside the play area. So, as weird as it sounds, you can make it so that the "center marker" is in the corner of the play area. Again, the play area NEVER moves.

And this feature is very useful for users. For example, when playing Beat Saber, it wants me to stand in a certain spot (what steamVR considers to be the "center"), but I completely disagree with where it wants me to stand. It makes the most sense to stand facing my bed since that area has the most space to move sideways, and the most space in front of me to extend my arms. So I just place myself there and then "recenter". Bam, done. My life would be way harder if this feature didn't exist.

[Ian (Sealost)]

Originally posted by Reonu:

There's a lot of history and complexity here regarding this topic, so I think the best and most simple way I can reply is just to explain how the OpenXR spec currently handles all of this.

The OpenXR spec provides three different reference spaces, to accommodate three different styles of games:

• The LOCAL reference space, which can have its tracking origin be recentered relative to your head. It is intended for "seated scale" games, which are mainly racing and cockpit games.
  
• The LOCAL_FLOOR reference space, which can have its tracking origin be recentered relative to the floor under where you are standing. It is intended for "standing scale" games, which are games that either always keep their interactable content within reach or use artificial movement to put the interactable content within reach. Most VR games are standing scale, including Beat Saber.
  
• The STAGE reference space, which permanently keeps its tracking origin centered to your physical play area. It is intended for "room scale" games, which keep their interactable content within your play area and generally require physical movement to put it within reach.

The only information OpenXR provides about the play area is its width and height. For a game to utilize the play area, it must center that width and height around the STAGE reference space's tracking origin.

In short, games just use the reference space they need, and it all works great. TotF can use a STAGE reference space and keep the fights within your play area. Beat Saber can use a LOCAL_FLOOR reference space so you can recenter wherever you want. You don't need to give up one to get the other. On Quest, it all works exactly like this. On PC, it all still works well enough even though SteamVR isn't handling STAGE spaces correctly.

However, using Steam Link and other apps like it cause an additional complication that needs to be resolved. If the game that's running on your PC needs to remain centered to your play area, but the Steam Link app that's running on the Quest is using a LOCAL_FLOOR reference space isn't centered to your play area, then the PC game will be off-center. That PC game using a STAGE reference space needs the Quest to also use a STAGE reference space in order to work properly. Virtual Desktop handles this by allowing users to manually choose to use a STAGE reference space. What I'm asking for here is either a manual option like VD provides or an automatic method that uses a STAGE reference space on Quest if one is being used on PC.

With all that said, I also noticed that SteamVR is simultaneously showing both the recentered position AND the correct physical location of the play area. I asked how they were doing that in my post. The thing to keep in mind, though, is that SteamVR can "cheat" and access more information than what is available through the APIs. SteamVR still knows the calibrated play area's original location even if games can't access that information after a user recenters. Steam Link can also "cheat" and access information that isn't provided to the game running on PC. You could easily check a reference point in the STAGE and LOCAL_FLOOR spaces to relate them to each other and render both the center point and the play area in the correct location, but games running on PC don't have the ability to do that through Steam Link, even though the Steam Link app itself has access to that information.

[danw]

Hi,
Thanks for following up.

The Steam Link client will always uses the Stage space to retrieve poses. You can prove this for yourself if you try get poses in the RawAndUncalibrated universe and notice that it is not affected by recentering on the headset. It does, however, use the Local space to get the seated and standing universe's origin relative to the Stage space. When you query poses in the Standing or Seated universe, you're getting this transform applied on top of the RawandUncalibrated universe.
Where I think you're noticing a difference in behavior to the Index (or any lighthouse hmd), is that the Index will always initially set the standing universe's origin to the center of the play space. Steam Link, on the other hand, will forward whatever the Local space's origin is relative to the Stage space for the standing universe's origin, which almost certainly won't be the center of the play space. The behavior of actually recentering the standing/seated universes is exactly the same in Steam Link as it is of the Index.

As for working out where the "stage" origin is in OpenVR, we don't intend to change the behavior of OpenVR, but like I mentioned before, you can actually work out an origin of the "stage" space with what's already provided:
When you call GetPlayAreaRect, you're going to get a quad that stays in some place in the real world. This quad is positioned relative to the standing universe origin, so its coordinates will change every time a user recenters. You can use this to position things within the room, and this is in fact what SteamVR is using internally to position the floor box in the construct like you mentioned, it's not using any internal methods.
I'll include a code snippet at the bottom, it doesn't include the actual functions to do the vector and matrix operations, but the approach is what's important. Is this what you're looking for?

I hope this clears some things up for you, let me know if you have any other questions.

vr::HmdQuad_t rectPlayArea;
vr::VRChaperone()->GetPlayAreaRect( &rectPlayArea );

vr::HmdVector3_t DsubA = HmdVector_Subtract(rectPlayArea.vCorners[3], rectPlayArea.vCorners[2]);
vr::HmdVector3_t BsubA = HmdVector_Subtract(rectPlayArea.vCorners[1], rectPlayArea.vCorners[2]);

vr::HmdMatrix34_t mayPlayAreaRotation = s_mat34Identity;
{
vr::HmdVector3_t X = HmdVector_Normalized(DsubA);
vr::HmdVector3_t Z = HmdVector_Normalized(BsubA);

vr::HmdVector3_t Y = HmdVector_Cross(Z, X);

mayPlayAreaRotation.m[0][0] = X.v[0];
mayPlayAreaRotation.m[1][0] = X.v[1];
mayPlayAreaRotation.m[2][0] = X.v[2];
mayPlayAreaRotation.m[0][1] = Y.v[0];
mayPlayAreaRotation.m[1][1] = Y.v[1];
mayPlayAreaRotation.m[2][1] = Y.v[2];
mayPlayAreaRotation.m[0][2] = Z.v[0];
mayPlayAreaRotation.m[1][2] = Z.v[1];
mayPlayAreaRotation.m[2][2] = Z.v[2];
}

vr::HmdMatrix34_t matPlayAreaTranslation = s_mat34Identity;
{
vr::HmdVector3_t Translation;
Translation.v[0] = rectPlayArea.vCorners[0].v[0] + rectPlayArea.vCorners[1].v[0] + rectPlayArea.vCorners[2].v[0] + rectPlayArea.vCorners[3].v[0];
Translation.v[1] = rectPlayArea.vCorners[0].v[1] + rectPlayArea.vCorners[1].v[1] + rectPlayArea.vCorners[2].v[1] + rectPlayArea.vCorners[3].v[1];
Translation.v[2] = rectPlayArea.vCorners[0].v[2] + rectPlayArea.vCorners[1].v[2] + rectPlayArea.vCorners[2].v[2] + rectPlayArea.vCorners[3].v[2];
Translation = HmdVector_ScalarMultiply(Translation, 1.f / 4.f);

matPlayAreaTranslation.m[0][3] = Translation.v[0];
matPlayAreaTranslation.m[1][3] = Translation.v[1];
matPlayAreaTranslation.m[2][3] = Translation.v[2];
}

vr::HmdMatrix34_t matTransformStageCenterFromStanding = HmdMatrix_Multiply(matPlayAreaTranslation, mayPlayAreaRotation);

[Ian (Sealost)]

That explanation of how Steam Link is keeping these spaces in sync was exactly what I needed. Makes perfect sense. Thanks for providing that.

It's good to know that GetPlayAreaRect works the way you described. The documentation[github.com] says "It's always centered around standing tracking space origin" which made me think it wouldn't be useful to solve this problem (and I misunderstood your 0,0,0 comment to mean the same thing).

It's understandable that you don't want to change the way OpenVR works, and this appears to be a perfectly fine solution, so no worries there.

However, I would like to move over to OpenXR if possible, and it sounds like there's not a solution currently available under that API. I do hope you consider disallowing the system-level recenter for the stage reference space and adding the "local floor" reference space for that use case instead. For now, though, I'll just continue using OpenVR.

[danw]

We've just released a beta that implements EXT_local_floor and also stops the stage space from being recentered.
Let me know if this works for you. Thanks.

[Ian (Sealost)]

Originally posted by danwillm:

We've just released a beta that implements EXT_local_floor and also stops the stage space from being recentered.
Let me know if this works for you. Thanks.

Holy cow! I did not expect this so fast. Thanks for hearing me out and implementing this. I'll let you know if I run into any issues with it.

[Reonu]

Originally posted by danwillm:

We've just released a beta that implements EXT_local_floor and also stops the stage space from being recentered.
Let me know if this works for you. Thanks.

Hello, this new beta version prevents me from recentering in Pistol Whip which is super annoying. It's a smilar game to Beat Saber in that it expects me to mostly stay in what it considers to be the "center" and move mostly sideways but always around that center area. Being able to define a new center by holding a new button is super convenient. I can't do this anymore which is very frustrating. If I'm understanding the exchange here correctly, the dev was already able to do what they wanted to do with a different approach, so I don't think releasing an update that breaks existing behaviour is the move here. Please reconsider this change before taking it out of beta. Thanks.

[danw]

After some feedback from folks about the recent changes to the stage space about it breaking a few apps, we're going to roll it back to being recentered for this release cycle.

XR_EXT_local floor will remain in the beta to allow apps (and engines) time to switch over to using it, and we're thinking about ways to satisfy the needs of games that want to be fixed in real space, while maintaining compatibility with older titles that take advantage of the recentering features.

[Ian (Sealost)]

I think it makes sense not to rush this, so that affected apps get fair warning.

However, I don't think this change *breaks* any existing games, does it? The games still work correctly, and the user still stands somewhere that they explicitly communicated to the runtime that they were willing to stand. Those users even still have the option to change that standing point by going through room setup. They just lose out on the convenience of doing that in one click until the app gets updated.

Meanwhile, *not* having this change *does* break games, at least under OpenXR. Recentering a room scale game breaks it, and that seems to be unavoidable with Steam Link. I have a note on my Steam store page warning Quest users not to buy my game because it doesn't work with streaming apps. which I would love to be able to remove.

Would it be possible to add a setting to SteamVR to allow users to force recentering behavior for stage spaces (with it defaulting off)? That would allow room scale games to work by default and encourage games that want it to switch to local_floor while still giving users a way to access that recentering convenience in the meantime.