Steam Link NVENC Fails

[76561199733777532]

Steam Link refuses to use hardware encoding. Log and key OS details below. The only log entry that indicates a problem says “NVENC - Submitting frame for encoding failed: 20”. Nothing else relevant is system logs. Tested Steam stable and beta clients.


OS: Ubuntu 24.04
Kernel: 6.8.8-2-pve
nVidia Driver: 550.90.07 (also tested 555 series)
GPU: RTX 4070 Ti Super


[2024-07-15 15:28:45][95889.269672] Video Encoder Changed
[2024-07-15 15:28:45][95889.270758] >>> Capture method set to Desktop OpenGL NV12 + NVENC HEVC
[2024-07-15 15:28:45][95889.271701] >>> Capture resolution set to 1920x1080
[2024-07-15 15:28:45][95889.271711] >>> Capture colorspace set to BT.601 limited range
[2024-07-15 15:28:45][95889.272305] NVENC - Submitting frame for encoding failed: 20
[2024-07-15 15:28:45][95889.323678] >>> Client video decoder set to iOS Metal hardware decoding
[2024-07-15 15:28:45][95889.374419] CLIENT: iOS Metal hardware decoding
[2024-07-15 15:28:45][95889.374432] CLIENT: Got control packet k_EStreamControlStartVideoData
[2024-07-15 15:28:45][95889.374439] CLIENT: Got control packet k_EStreamControlVideoEncoderInfo
[2024-07-15 15:28:45][95889.374447] CLIENT: Got control packet k_EStreamControlSetCursorScale
[2024-07-15 15:28:45][95889.542810] Detected 40 logical processors, using 4 threads
[2024-07-15 15:28:45][95889.542832] Configuring encoder: [threads=4][width=1920][height=1080][preset=superfast][tune=zerolatency]
[2024-07-15 15:28:45][95889.546420] Video Encoder Changed
[2024-07-15 15:28:45][95889.547519] >>> Capture method set to Desktop OpenGL NV12 + libx264 main (4 threads)
[2024-07-15 15:28:45][95889.559543] Detected 40 logical processors, using 4 threads
[2024-07-15 15:28:45][95889.559562] Configuring encoder: [threads=4][width=1920][height=1080][preset=superfast][tune=zerolatency]
[2024-07-15 15:28:45][95889.562539] Video Encoder Changed
[2024-07-15 15:28:45][95889.629763] CLIENT: Got control packet k_EStreamControlStopVideoData
[2024-07-15 15:28:45][95889.629804] CLIENT: iOS Metal hardware decoding

[Mahler]

I am also hitting the same problem. My symptom is that the steam link can see my machine, connects to it, then I get a black screen with a mouse on it which I can control for about 10 seconds when the steam link disconnects.

Here's an excerpt from my logs:

OS: Arch Linux
Kernel: 6.10.2-arch1-1
NVIDIA driver: 555.58.02
GPU: GTX 1080

[2024-10-05 15:36:54][11883.315972] Video Encoder Changed
[2024-10-05 15:36:54][11883.317027] >>> Capture method set to Desktop OpenGL NV12 + NVENC H264
[2024-10-05 15:36:54][11883.317843] >>> Capture resolution set to 1280x800
[2024-10-05 15:36:54][11883.317856] >>> Capture colorspace set to BT.601 limited range
[2024-10-05 15:36:54][11883.318237] NVENC - Submitting frame for encoding failed: 20
[2024-10-05 15:36:54][11883.418215] >>> Client video decoder set to Marvell hardware decoding
[2024-10-05 15:36:54][11883.418248] CLIENT: Marvell hardware decoding
[2024-10-05 15:36:54][11883.418259] CLIENT: Got control packet k_EStreamControlStartVideoData
[2024-10-05 15:36:54][11883.418269] CLIENT: Got control packet k_EStreamControlVideoEncoderInfo
[2024-10-05 15:36:54][11883.418277] CLIENT: Got control packet k_EStreamControlSetCursorScale
[2024-10-05 15:36:54][11883.574661] CGameStreamVideoStageVAAPI: Reinitializing 1280x800 at 30000 Kbit/s and 59.94 FPS with quality 1
[2024-10-05 15:36:54][11883.583569] ffmpeg verbose: libva: VA-API version 1.21.0
[2024-10-05 15:36:54][11883.583618] ffmpeg verbose: libva: User environment variable requested driver 'nvidia'
[2024-10-05 15:36:54][11883.583803] ffmpeg verbose: libva: Trying to open /usr/lib32/dri/nvidia_drv_video.so
[2024-10-05 15:36:54][11883.584017] ffmpeg verbose: libva: Found init function __vaDriverInit_1_16

[Mahler]

Turning off hardware encoding for remote play 'fixes' the issue at the cost of additional latency.

[That Cat from Stranger Things]

For me it sees NVENC H264 but not NVENC HEVC. This is on the latest Steam Link app with AV1 support and the most recent public version of the Steam client for Linux.

However, I am using the 535 drivers instead of 550 or 555, proprietary modules on a GTX 1660 Ti.

Before updating the Steam Link app, it was properly defaulting to HEVC.

[That Cat from Stranger Things]

Originally posted by Mahler:

Turning off hardware encoding for remote play 'fixes' the issue at the cost of additional latency.

Are you using the open kernel modules? Nvidia recently made the switch so all future drivers use the open kernel modules which do NOT include NVENC. You must explicitly install with the proprietary modules to use NVENC.

Edit: Oh wait, I see it's a GTX 1080. This piece of advice still applies for those with Turing and newer though.

[That Cat from Stranger Things]

As of the latest Steam Linux stable update, NVENC HEVC works again for Steam Play/Steam Link. It was a update desync between Windows and Linux Steam clients.

[Gord0n페카]

hola soy nuevo por aqui, podrain ayudarme. Gracias