OpenXR bugs and spec violations

[siddok]

I'm developing a DX12 OpenXR app. I spent 6 days adapting workarounds for SteamVR's OpenXR spec violations, bugs and limitations and this is a comprehensive lists of problems I had to deal with.

Posting this with good vibes only with the hope that somebody at Valve can potentially address (some of) this issues.

1. XR_SWAPCHAIN_USAGE_UNORDERED_ACCESS_BIT is completely ignored by the runtime and crashes the app when you try to create UAVs out of swapchain's texture, this is also a spec violation:

This is bad because it prevents you from writing directly to the swapchain, resulting in an extra copy pass of ~50mb and two wasteful full-screen draw-calls. Which is significant at least on lower end hardware.

2. sRGB pixel formats are not supported by the runtime.

Not sure what led to this decision and why this artificial limitations is created, given that there is no auto-gamma-correction like for example on Apple platforms.

Previously I relied on hardware accelerated sampling for gamma-correction but now I had to move

pow(color, 2.2)

to my shaders, which is bad and also slower.

3. Swapchains are created in a very large size. The one that is returned for SteamLink is 2508x2508 which is tremendeously bigger than Quest Pro resolution (which I run SteamLink on) and much bigger than the swapchain created by Oculus OpenXR implementation which is 2x2224x2160.

It is okay to allocate larger swapchains to account for reprojection warping, but it also explains why so many games face performance issues when using SteamVR: the runtime forces them to use excessively high resolution. 30% more pixels in my case.

4. Swapchain textures are not transitioned to the expected state according to the spec. This is also a spec violation.

And you can't transition it yourself because you don't have an access to BeforeState.

This makes debug layer absolutely unusable as the output gets polutted by endless error messages every frame.

Debug layer also indicates few more problem inside runtime, some of them seems to be a bug and another one looks like a DirectX 11 limitation.


Overall, supporting SteamVR made my codebase worse, testing of Gracia is now significantly harder and performance is worse because I had to remove some optimizations that were possible on other runtimes for the sake of maintainability.

[LordDaniel]

Not the first time I am hearing this on SteamVR. Recently Virtual Desktop got their own OpenXR implementation, and they showed better performance and full compatibility with games and apps. Like, it is clearly SteamVR has some issues, and I do hope Valve sees it and actually reworks the OpenXR implementation.

Thanks for OP for researching the bugs, and for making it more detailed than “I feel like it could be better”.

[aaron.leiby]

2. sRGB pixel formats are not supported by the runtime.

Could you elaborate on this? SteamVR's DX12 OpenXR binding supports the following sRGB formats:
DXGI_FORMAT_R8G8B8A8_UNORM_SRGB
DXGI_FORMAT_B8G8R8A8_UNORM_SRGB

Are you seeing specific situations where this is not behaving as expected? If so, please provide details such as which headset you are testing with, etc. Thanks.

Note: As per the spec - DXGI resources will be created with their associated TYPELESS format, but the runtime will use the application-specified format for reading the data.

3. Swapchains are created in a very large size.

Please elaborate. Swapchains are allocated using the width and height specified by the application via XrSwapchainCreateInfo.

Presumably you are referring to recommendedImageRectWidth/Height values returned by XrViewConfigurationView?

SteamVR performs a mini-benchmark on startup and adjusts recommended resolution based on your video card's performance.

If you look in C:\Program Files (x86)\Steam\logs\vrcompositor.txt you will find output like the following:

Fri Dec 15 2023 12:28:09.652 [Info] - ****************************************** Begin GPU speed ******************************************
Fri Dec 15 2023 12:28:09.714 [Info] - GPU Vendor: "NVIDIA GeForce RTX 4070" GPU Driver: "31.0.15.4617"
Fri Dec 15 2023 12:28:09.790 [Info] - MeasureGpuMegaPixelsPerSecond(0): Returning 2308 MP/sec. Total CPU time 0.08 seconds.
Fri Dec 15 2023 12:28:09.790 [Info] - GPU speed from average of 6 median samples: 2307
Fri Dec 15 2023 12:28:09.791 [Info] - ******************************************* End GPU speed *******************************************
Fri Dec 15 2023 12:28:09.791 [Info] - HMD driver recommended: 2016x2240 120.0Hz HiddenArea(24.88%) = 814 MP/sec
Fri Dec 15 2023 12:28:09.791 [Info] - Raw ideal render target scale = 2.83

Users have the ability to override this in SteamVR's Settings (globally) or on a per-application basis. Applications are free to ignore this, but users typically expect it to work in apps. Some applications uses it as a basis for more complicated techniques like dynamically resizing the viewport based on real-time performance feedback.

[aaron.leiby]

1. XR_SWAPCHAIN_USAGE_UNORDERED_ACCESS_BIT is completely ignored by the runtime

SteamVR v2.2.3 now handles swapchain usage flags correctly for D3D12 (as well as mipmap and sample count). Thanks for bringing this to our attention!
https://steamcommunity.com/app/250820/eventcomments/4036978698783065341/

4. Swapchain textures are not transitioned to the expected state according to the spec.

This is correct, but a bit more complex. SteamVR's OpenXR runtime currently transitions D3D12 swapchain images to D3D12_RESOURCE_STATE_RENDER_TARGET (or D3D12_RESOURCE_STATE_DEPTH_WRITE) inside of xrWaitSwapchainImage rather than xrAcquireSwapchainImage. This change was made to address some issues in both Unreal and Unity some time ago. In theory, this should not matter to your app since Acquire should be treated as "peek index" to allow building command lists for the next frame. However, applications must call Wait on the image before they are allowed to submit those command lists to the queue.

Applications are allowed to Acquire all the images in a swapchain. If the transition is performed during that operation, then the compositor/runtime would not be able continue using them (e.g. for reprojection). The entire point of Wait is to handle this. I suspect that distinction was perhaps made after these extensions were originally drafted.

There are ongoing discussions within the OpenXR working group about updating the spec. The Vulkan extension has the same issue. In that particular case, SteamVR is still performing the transition in Acquire, while the Oculus PC runtime transitions in Wait fwiw.

[siddok]

@aaron.leiby thank you for responding

I'm trying a version 2.2.3 and unfortunately now the app stopped working at all. Same code, no changes. This is the crash from console:

```
D3D12 ERROR: ID3D12Device::CreateCommittedResource: D3D12_RESOURCE_DESC::Flags cannot have D3D12_RESOURCE_FLAG_DENY_SHADER_RESOURCE set without D3D12_RESOURCE_FLAG_ALLOW_DEPTH_STENCIL, D3D12_RESOURCE_FLAG_VIDEO_DECODE_REFERENCE_ONLY or D3D12_RESOURCE_FLAG_VIDEO_ENCODE_REFERENCE_ONLY. [ STATE_CREATION ERROR #599: CREATERESOURCE_INVALIDMISCFLAGS]
Exception thrown at 0x00007FFAC41C567C in D3D12Sample.exe: Microsoft C++ exception: _com_error at memory location 0x000000D5BE2FDDD0.
Fri Dec 22 2023 16:22:28.755 [Info] - Failed to create D3D12 swapchain texture
xrCreateSwapchain(context->xr_session, &swapchain_info, &handle) at src\XRSwapchain.h:48 failed with -2
```

We can not create a swapchain anymore with new SteamVR OpenXR implementation.



However, I was able to examine one of the problem: with SRGB formats. And they are supported now! In fact _SRGB format is the first in returned SupportedSwapchainFormats. It wasn't the case with previous versions (I manually examined every DXGI_FORMAT value and none of SRGB formats were in that returned array)

And regarding swapchain size, yeah, this mini benchmark can explain things because I have a high-end 4090 so it is likely crunching numbers at faster pacing, but anyway I still believe that SteamVR should introduce a reasonable cap over native resolution of the headset because currently SteamVR works way slower than Oculus OpenXR with no visible benefits. Also, aspect ratio is different (it is 1:1 on Steam VR and 1:1.03 on Oculus for the same headset). Another thing is SteamVR's swapchain is not divisible by 16, which is a bit painful at least in our usecase.

[aaron.leiby]

D3D12 ERROR: ID3D12Device::CreateCommittedResource: D3D12_RESOURCE_DESC::Flags cannot have D3D12_RESOURCE_FLAG_DENY_SHADER_RESOURCE set without

Try setting XR_SWAPCHAIN_USAGE_SAMPLED_BIT when creating your swapchains. I'm not sure what situation this should not be set, but the OpenXR spec says to set D3D12_RESOURCE_FLAG_DENY_SHADER_RESOURCE if XR_SWAPCHAIN_USAGE_SAMPLED_BIT is not set. Any swapchain image submitted to the runtime will need to be sampled, so it seems like an error to not just always set this.

Also, as mentioned over here[github.com], we've switched away from using ID3D12Device::CreateCommittedResource for D3D12 OpenXR swapchain images to using ID3D12CompatibilityDevice::CreateSharedResource instead, but that change hasn't shipped yet.

I still believe that SteamVR should introduce a reasonable cap over native resolution of the headset

By default, it caps the resolution at 1.5x the driver recommended resolution.

aspect ratio is different (it is 1:1 on Steam VR and 1:1.03 on Oculus for the same headset)

Which headset? There was a bug in Steam Link that was causing the driver reported recommended resolution to be 2048x2048, but that has since been fixed to report the native panel resolution for each device now.

Another thing is SteamVR's swapchain is not divisible by 16, which is a bit painful at least in our usecase.

SteamVR rounds to the nearest multiple of 4. OpenXR applications specify swapchain image size, so there is nothing stopping you from further rounding to the nearest 16 instead.

[siddok]

Try setting XR_SWAPCHAIN_USAGE_SAMPLED_BIT

It worked! Thank you!

However the log is still polluted by this now, looks like the state of the swapchains is still wrong, even after Acquire/Wait call (which I always used):

D3D12 ERROR: ID3D12CompatibilityDevice::ReflectSharedProperties: Resource provided was not shared by D3D11, or with a D3D11 desc. [ MISCELLANEOUS ERROR #916: REFLECTSHAREDPROPERTIES_INVALIDOBJECT]
D3D12 ERROR: ID3D12CommandQueue::ExecuteCommandLists: Using ResourceBarrier on Command List (0x0000022B2DBDA3B0:'Unnamed ID3D12GraphicsCommandList Object'): Before state (0x80: D3D12_RESOURCE_STATE_PIXEL_SHADER_RESOURCE) of resource (0x0000022B30D42AC0:'CSxrSwapchainD3D12OpenVR 10 i1') (subresource: 0) specified by transition barrier does not match with the state (0x4: D3D12_RESOURCE_STATE_RENDER_TARGET) specified in the previous call to ResourceBarrier [ RESOURCE_MANIPULATION ERROR #527: RESOURCE_BARRIER_BEFORE_AFTER_MISMATCH]
D3D12 ERROR: ID3D12CompatibilityDevice::ReflectSharedProperties: Resource provided was not shared by D3D11, or with a D3D11 desc. [ MISCELLANEOUS ERROR #916: REFLECTSHAREDPROPERTIES_INVALIDOBJECT]
D3D12 ERROR: ID3D12CommandQueue::ExecuteCommandLists: Using ResourceBarrier on Command List (0x0000022B2DBDA3B0:'Unnamed ID3D12GraphicsCommandList Object'): Before state (0x80: D3D12_RESOURCE_STATE_PIXEL_SHADER_RESOURCE) of resource (0x0000022B30D42AC0:'CSxrSwapchainD3D12OpenVR 10 i1') (subresource: 1) specified by transition barrier does not match with the state (0x4: D3D12_RESOURCE_STATE_RENDER_TARGET) specified in the previous call to ResourceBarrier [ RESOURCE_MANIPULATION ERROR #527: RESOURCE_BARRIER_BEFORE_AFTER_MISMATCH]
D3D12 ERROR: ID3D12CommandQueue::ExecuteCommandLists: Using Draw on Command List (0x0000022B37D88040:'Unnamed ID3D12GraphicsCommandList Object'): Resource state (0xF9EE1A0: D3D12_RESOURCE_STATE_PIXEL_SHADER_RESOURCE) of resource (0x0000022B38099770:'CSxrSwapchainD3D12OpenVR 10 i2') (subresource: 0) is invalid for use as a render target. Expected State Bits (all): 0xF9EE180: D3D12_RESOURCE_STATE_RENDER_TARGET, Actual State: 0xF9EE160: D3D12_RESOURCE_STATE_PIXEL_SHADER_RESOURCE, Missing State: 0x4: D3D12_RESOURCE_STATE_RENDER_TARGET. [ EXECUTION ERROR #538: INVALID_SUBRESOURCE_STATE]
D3D12 ERROR: ID3D12CommandQueue::ExecuteCommandLists: Using Draw on Command List (0x0000022B37D88040:'Unnamed ID3D12GraphicsCommandList Object'): Resource state (0xF9EE1A0: D3D12_RESOURCE_STATE_PIXEL_SHADER_RESOURCE) of resource (0x0000022B38099770:'CSxrSwapchainD3D12OpenVR 10 i2') (subresource: 1) is invalid for use as a render target. Expected State Bits (all): 0xF9EE180: D3D12_RESOURCE_STATE_RENDER_TARGET, Actual State: 0xF9EE160: D3D12_RESOURCE_STATE_PIXEL_SHADER_RESOURCE, Missing State: 0x4: D3D12_RESOURCE_STATE_RENDER_TARGET. [ EXECUTION ERROR #538: INVALID_SUBRESOURCE_STATE]
D3D12 ERROR: GPU-BASED VALIDATION: DrawInstanced, RenderTarget state invalid, Slot [0], Incompatible resource state: Resource: 0x0000022B38099770:'CSxrSwapchainD3D12OpenVR 10 i2', Subresource Index: [0], Resource State: D3D12_RESOURCE_STATE_PIXEL_SHADER_RESOURCE(0x80), Required State Bits: D3D12_RESOURCE_STATE_RENDER_TARGET(0x4), Draw Count [0], Dispatch Count [0], Command List: 0x0000022B37D88040:'Unnamed ID3D12GraphicsCommandList Object', [ EXECUTION ERROR #942: GPU_BASED_VALIDATION_INCOMPATIBLE_RESOURCE_STATE]
D3D12 ERROR: GPU-BASED VALIDATION: DrawInstanced, RenderTarget state invalid, Slot [0], Incompatible resource state: Resource: 0x0000022B38099770:'CSxrSwapchainD3D12OpenVR 10 i2', Subresource Index: [1], Resource State: D3D12_RESOURCE_STATE_PIXEL_SHADER_RESOURCE(0x80), Required State Bits: D3D12_RESOURCE_STATE_RENDER_TARGET(0x4), Draw Count [0], Dispatch Count [0], Command List: 0x0000022B37D88040:'Unnamed ID3D12GraphicsCommandList Object', [ EXECUTION ERROR #942: GPU_BASED_VALIDATION_INCOMPATIBLE_RESOURCE_STATE]
D3D12 ERROR: ID3D12CompatibilityDevice::ReflectSharedProperties: Resource provided was not shared by D3D11, or with a D3D11 desc. [ MISCELLANEOUS ERROR #916: REFLECTSHAREDPROPERTIES_INVALIDOBJECT]
D3D12 ERROR: ID3D12CommandQueue::ExecuteCommandLists: Using ResourceBarrier on Command List (0x0000022B2DBDA3B0:'Unnamed ID3D12GraphicsCommandList Object'): Before state (0x80: D3D12_RESOURCE_STATE_PIXEL_SHADER_RESOURCE) of resource (0x0000022B38099770:'CSxrSwapchainD3D12OpenVR 10 i2') (subresource: 0) specified by transition barrier does not match with the state (0x4: D3D12_RESOURCE_STATE_RENDER_TARGET) specified in the previous call to ResourceBarrier [ RESOURCE_MANIPULATION ERROR #527: RESOURCE_BARRIER_BEFORE_AFTER_MISMATCH]
D3D12 ERROR: ID3D12CompatibilityDevice::ReflectSharedProperties: Resource provided was not shared by D3D11, or with a D3D11 desc. [ MISCELLANEOUS ERROR #916: REFLECTSHAREDPROPERTIES_INVALIDOBJECT]
D3D12 ERROR: ID3D12CommandQueue::ExecuteCommandLists: Using ResourceBarrier on Command List (0x0000022B2DBDA3B0:'Unnamed ID3D12GraphicsCommandList Object'): Before state (0x80: D3D12_RESOURCE_STATE_PIXEL_SHADER_RESOURCE) of resource (0x0000022B38099770:'CSxrSwapchainD3D12OpenVR 10 i2') (subresource: 1) specified by transition barrier does not match with the state (0x4: D3D12_RESOURCE_STATE_RENDER_TARGET) specified in the previous call to ResourceBarrier [ RESOURCE_MANIPULATION ERROR #527: RESOURCE_BARRIER_BEFORE_AFTER_MISMATCH]
D3D12 ERROR: ID3D12CommandQueue::ExecuteCommandLists: Using Draw on Command List (0x0000022B37CCEA70:'Unnamed ID3D12GraphicsCommandList Object'): Resource state (0xF9EE1A0: D3D12_RESOURCE_STATE_PIXEL_SHADER_RESOURCE) of resource (0x0000022B38088A00:'CSxrSwapchainD3D12OpenVR 10 i0') (subresource: 0) is invalid for use as a render target. Expected State Bits (all): 0xF9EE180: D3D12_RESOURCE_STATE_RENDER_TARGET, Actual State: 0xF9EE160: D3D12_RESOURCE_STATE_PIXEL_SHADER_RESOURCE, Missing State: 0x4: D3D12_RESOURCE_STATE_RENDER_TARGET. [ EXECUTION ERROR #538: INVALID_SUBRESOURCE_STATE]
D3D12 ERROR: ID3D12CommandQueue::ExecuteCommandLists: Using Draw on Command List (0x0000022B37CCEA70:'Unnamed ID3D12GraphicsCommandList Object'): Resource state (0xF9EE1A0: D3D12_RESOURCE_STATE_PIXEL_SHADER_RESOURCE) of resource (0x0000022B38088A00:'CSxrSwapchainD3D12OpenVR 10 i0') (subresource: 1) is invalid for use as a render target. Expected State Bits (all): 0xF9EE180: D3D12_RESOURCE_STATE_RENDER_TARGET, Actual State: 0xF9EE160: D3D12_RESOURCE_STATE_PIXEL_SHADER_RESOURCE, Missing State: 0x4: D3D12_RESOURCE_STATE_RENDER_TARGET. [ EXECUTION ERROR #538: INVALID_SUBRESOURCE_STATE]
D3D12 ERROR: GPU-BASED VALIDATION: DrawInstanced, RenderTarget state invalid, Slot [0], Incompatible resource state: Resource: 0x0000022B38088A00:'CSxrSwapchainD3D12OpenVR 10 i0', Subresource Index: [0], Resource State: D3D12_RESOURCE_STATE_PIXEL_SHADER_RESOURCE(0x80), Required State Bits: D3D12_RESOURCE_STATE_RENDER_TARGET(0x4), Draw Count [0], Dispatch Count [0], Command List: 0x0000022B37CCEA70:'Unnamed ID3D12GraphicsCommandList Object', [ EXECUTION ERROR #942: GPU_BASED_VALIDATION_INCOMPATIBLE_RESOURCE_STATE]
D3D12 ERROR: GPU-BASED VALIDATION: DrawInstanced, RenderTarget state invalid, Slot [0], Incompatible resource state: Resource: 0x0000022B38088A00:'CSxrSwapchainD3D12OpenVR 10 i0', Subresource Index: [1], Resource State: D3D12_RESOURCE_STATE_PIXEL_SHADER_RESOURCE(0x80), Required State Bits: D3D12_RESOURCE_STATE_RENDER_TARGET(0x4), Draw Count [0], Dispatch Count [0], Command List: 0x0000022B37CCEA70:'Unnamed ID3D12GraphicsCommandList Object', [ EXECUTION ERROR #942: GPU_BASED_VALIDATION_INCOMPATIBLE_RESOURCE_STATE]
D3D12 ERROR: ID3D12CompatibilityDevice::ReflectSharedProperties: Resource provided was not shared by D3D11, or with a D3D11 desc. [ MISCELLANEOUS ERROR #916: REFLECTSHAREDPROPERTIES_INVALIDOBJECT]
D3D12 ERROR: ID3D12CommandQueue::ExecuteCommandLists: Using ResourceBarrier on Command List (0x0000022B2DBDA3B0:'Unnamed ID3D12GraphicsCommandList Object'): Before state (0x80: D3D12_RESOURCE_STATE_PIXEL_SHADER_RESOURCE) of resource (0x0000022B38088A00:'CSxrSwapchainD3D12OpenVR 10 i0') (subresource: 0) specified by transition barrier does not match with the state (0x4: D3D12_RESOURCE_STATE_RENDER_TARGET) specified in the previous call to ResourceBarrier [ RESOURCE_MANIPULATION ERROR #527: RESOURCE_BARRIER_BEFORE_AFTER_MISMATCH]
D3D12 ERROR: ID3D12CompatibilityDevice::ReflectSharedProperties: Resource provided was not shared by D3D11, or with a D3D11 desc. [ MISCELLANEOUS ERROR #916: REFLECTSHAREDPROPERTIES_INVALIDOBJECT]
D3D12 ERROR: ID3D12CommandQueue::ExecuteCommandLists: Using ResourceBarrier on Command List (0x0000022B2DBDA3B0:'Unnamed ID3D12GraphicsCommandList Object'): Before state (0x80: D3D12_RESOURCE_STATE_PIXEL_SHADER_RESOURCE) of resource (0x0000022B38088A00:'CSxrSwapchainD3D12OpenVR 10 i0') (subresource: 1) specified by transition barrier does not match with the state (0x4: D3D12_RESOURCE_STATE_RENDER_TARGET) specified in the previous call to ResourceBarrier [ RESOURCE_MANIPULATION ERROR #527: RESOURCE_BARRIER_BEFORE_AFTER_MISMATCH]

Which headset?

I tested with Quest Pro around 1.5 week ago, I'll see if newer version returns swapchains of different format

[aaron.leiby]

D3D12 ERROR: ID3D12CommandQueue::ExecuteCommandLists: Using ResourceBarrier on Command List (0x0000022B2DBDA3B0:'Unnamed ID3D12GraphicsCommandList Object'): Before state (0x80: D3D12_RESOURCE_STATE_PIXEL_SHADER_RESOURCE) of resource (0x0000022B30D42AC0:'CSxrSwapchainD3D12OpenVR 10 i1') (subresource: 0) specified by transition barrier does not match with the state (0x4: D3D12_RESOURCE_STATE_RENDER_TARGET) specified in the previous call to ResourceBarrier []

SteamVR names its command lists, so presumably this is from your app? This shows an attempted transition from D3D12_RESOURCE_STATE_PIXEL_SHADER_RESOURCE when the resource is in D3D12_RESOURCE_STATE_RENDER_TARGET. SteamVR's xrWaitSwapchainImage performs this transition (pixel shader resource to render target), so your app should not need to do this.

I do not see these errors when testing with hello_xr fwiw.
https://github.com/KhronosGroup/OpenXR-SDK-Source/tree/main/src/tests/hello_xr

[siddok]

I double checked to be sure, but we don't do transition on swapchain resources anywhere. In fact our code as simple as this:

swapchain->Acquire();

D3D12_CPU_DESCRIPTOR_HANDLE uiRTV = swapchain->rtv.GetCPU(swapchain->img_id);
auto g_CommandList = renderer->commandListRing->GetCurrentCommandList()->Get();

const float clearColor[] = { 0.0f, 0.2f, 0.4f, 1.0f };
g_CommandList->OMSetRenderTargets(1, &uiRTV, FALSE, nullptr);
g_CommandList->ClearRenderTargetView(uiRTV, clearColor, 0, nullptr);

ImGui::Render();
ImGui_ImplDX12_RenderDrawData(ImGui::GetDrawData(), g_CommandList.Get());

swapchain->Release();

And it emits the error. We get an error for every active swapchain in the rendering frame. Acquire does call "Wait" under the hood.

Also, no such errors on other runtimes.

[aaron.leiby]

Do you have any XR layers installed like OpenXR Toolkit?