Blackscreen with Read Dead Redemption 2

[Mr.FreshDachs]

Hello community,

I have a problem with playing Red Dead Redemption 2 via Steam Remote Play in my local network.

The problem is, that the encoder won't switch to "Game polled D3D11 NV12" when launching the game. Instead it stays at "Desktop DWM NV12", which results in that I only see a black screen instead of the game.

The graphic API ingame is DirectX12. When I launch another DX12-game (Hitman 2 in my case) the encoder switches to "Game polled D3D11 NV12" and Remote Play works flawlessly.

I already tinkered around with every en- and decoder options steam is offering and tried fullscreen or windowed modes. It stays at "Desktop DWM NV12" and the screen of the game is black.

If I use the Vulkan API in Red Dead Redemption 2, the encoder will actually recognise the game and will switch to "RGB Vulkan" and the stream is working (kind of), but I get HORRIBLE performance issues. FPS will drop to 23 as soon as the stream starts. A quick online search told me that Steam Remote Play has a lot of issues with the vulkan API. So this is not an option either. 23 fps just hurts too much.

All drivers and versions are up to date. Here are my specs:
CPU: AMD Ryzen 5 5600X
GPU: Radeon RX 580, Software-Version 21.9.1
RAM: 16 GB DDR4

For reference I'll include a snippet of the streaming_log.txt:

[2021-09-20 18:51:58] CLIENT: Marking frame 2097 complete with status k_EStreamFrameResultDroppedLate
[2021-09-20 18:52:00] Setting target framerate: 59.75 [timing: game 10.00, capture 0.14, convert 0.00, encode 1.07, network 2.22, decode 4.46, display 2.35]
[2021-09-20 18:52:00] CLIENT: Targeting 59.75 FPS
[2021-09-20 18:52:00] CLIENT: Got control packet k_EStreamControlSetTargetFramerate
[2021-09-20 18:52:01] CLIENT: Marking frame 2284 complete with status k_EStreamFrameResultDroppedLate
[2021-09-20 18:52:02] CLIENT: SteamNetworkingSockets connection: Connected UDP@[2a02:2454:9976:4100:803c:240a:3786:8a69]:53928 Ping: 2ms IN: 444.2kbit 432.9 pkt/s qual 99.9% OUT: 71.8kbit 67.6 pkt/s qual 100.0%
[2021-09-20 18:52:05] CLIENT: Marking frame 2518 complete with status k_EStreamFrameResultDroppedLate
[2021-09-20 18:52:05] CLIENT: Marking frame 2528 complete with status k_EStreamFrameResultDroppedLate
[2021-09-20 18:52:05] CLIENT: Marking frame 2530 complete with status k_EStreamFrameResultDroppedLate
[2021-09-20 18:52:05] CLIENT: Marking frame 2531 complete with status k_EStreamFrameResultDroppedLate
[2021-09-20 18:52:05] CLIENT: Marking frame 2534 complete with status k_EStreamFrameResultDroppedLate
[2021-09-20 18:52:05] CLIENT: Flushing audio, 130 ms queued, 126 ms maximum, delivery delay = 13
[2021-09-20 18:52:07] CLIENT: Marking frame 2627 complete with status k_EStreamFrameResultDroppedLate
[2021-09-20 18:52:07] "SessionStats"
{
"GameNameID" "Red Dead Redemption 2"
"appid" "1174180"
"TimeSubmitted" "1632156727"
"ResolutionX" "1406"
"ResolutionY" "790"
"CaptureDescriptionID" "Desktop DWM NV12 + libx264 main (8 threads)"
"DecoderDescriptionID" "VideoToolbox hardware decoding"
"BandwidthLimit" "30000"
"FramerateLimit" "0"
"Transport" "4"
"SlowGamePercent" "0"
"SlowCapturePercent" "0"
"SlowConvertPercent" "0"
"SlowEncodePercent" "0"
"SlowNetworkPercent" "4.45292139053344727"
"SlowDecodePercent" "0"
"SlowDisplayPercent" "0"
"AvgClientBitrate" "88.7859268188476562"
"StdDevClientBitrate" "91.1075210571289062"
"AvgServerBitrate" "306.321197509765625"
"StdDevServerBitrate" "0"
"AvgLinkBandwidth" "100000.0078125"
"AvgPingMS" "20.5756301879882812"
"StdDevPingMS" "25.5810165405273438"
"AvgCaptureMS" "0.151865988969802856"
"StdDevCaptureMS" "0.0936429351568222046"
"AvgConvertMS" "0.0101772453635931015"
"StdDevConvertMS" "0.272330909967422485"
"AvgEncodeMS" "1.18425488471984863"
"StdDevEncodeMS" "0.373619049787521362"
"AvgNetworkMS" "9.22682571411132812"
"StdDevNetworkMS" "25.8984813690185547"
"AvgDecodeMS" "4.39264869689941406"
"StdDevDecodeMS" "0.748838067054748535"
"AvgDisplayMS" "12.4463930130004883"
"StdDevDisplayMS" "5.27780866622924805"
"AvgFrameMS" "51.5953559875488281"
"StdDevFrameMS" "24.0828933715820312"
"AvgFPS" "60.1777610778808594"
"StdDevFPS" "9.96183013916015625"
"BigPicture" "1"
"KeyboardMouseInput" "1"
"SteamControllerInput" "0"
"TouchControllerInput" "0"
"GameControllerInput" "0"
"XBox360ControllerInput" "0"
"XBoxOneControllerInput" "0"
"PS3ControllerInput" "0"
"PS4ControllerInput" "0"
"OtherControllerInput" "0"
"WasSpectating" "0"
"RemotePlayTogether" "0"
}

Nevermind the bad resolution and network connection, I testet this with an old laptop on wifi, but its the same issue for stable wired connections on different devices (TV, PC).

My guess is, that because RDR2 is launched out of Rockstars Game Launcher, steam somehow doesn't detect it as a "game" to switch it's encoder but somehow thinks it's another desktop application.
On the other hand the decoder switching IS working for the Vulkan API, which is just performing horribly....
It's the steam version of the game by the way (which still opens the rockstar launcher, but I bought it on steam).

Any solutions to this problem that I found on other threads (fullscreen instead of windowed, dis/enabling vsync, only starting the stream after the game has finished loading) didn't help so far.

What may help if I somehow could force steam to use different encoders. Because the only problem as it seems is that steam just chooses the wrong one. Does anyone know a possibility to do so? I didn't find anything so far....

I hope that someone could maybe help me.
Thanks in advance.

[_I_]

steam has some issues capturing dx12 and vulkan
set it to dx11 if you can

[Mr.FreshDachs]

Unfortunately RDR2 only supports Vulkan and DX12 :/
But Hitman 2 and Civilization 6 worked with DX12 (even tho the encoder says "game polled D3D11" which sounds more like DX11 to me, but it works.

[_I_]

htiman2 is dx11
https://store.steampowered.com/app/863550/HITMAN_2/

rdr2 min gpu is a gtx 770 which is dx11 also
https://store.steampowered.com/app/1174180/Red_Dead_Redemption_2/

[Mr.FreshDachs]

Hmmm but in the hitman 2 launcher I specifically choose "DirectX 12" (same in Civ 6).
Is there a way to find out which API is really used at the moment (while the game is running)?
In RDR2 I only have Vulkan and DX12 as options, but I may take a look, if I somehow can get DX11 to run.

[_I_]

try asking in the games forum

[Mr.FreshDachs]

I actually already did. Thanks so far for your time.

[alenjenny]

Did you manage to find a fix?