Pi 4B HEVC stutter and graphics driver

[Dr. Balanced]

Running HEVC decoding will cause severe stuttering about every 3 seconds, with the stuttery period lasting for about 1 second. Quality settings do not matter, this happens just the same at 4k60 25mbps and 480p30 3mbps. H264 will also regularly stutter but not as badly, with stutters being less severe and few more seconds apart, but that is sadly limited to 1080p on the Pi 4. Performance graph is perfectly smooth regardless, which has me scratching my head. Has anyone with a Pi 4B been able to get smooth playback? It should be possible since VLC can play 4k60 footage no problem.

Why does steam link not support the KMS driver and V4L2 HEVC decode? Enabling the KMS driver (dtoverlay=vc4-kms-v3d-pi4) causes the steamlink app not to run (specifically with an error saying the kms driver is not supported) and enabling V4L2 HEVC decoding (dtoverlay=rpivid-v4l2) causes streaming to bug out and I believe software decoding is used instead. The stream is zoomed in massively and only appears on the bottom third of the screen, so it's a massively zoomed in view of the top left of the steam big picture UI on the bottom third of the screen with the normal raspberry pi desktop on the top two thirds. This happens to me at 4k normally unless I add gpu_mem=256 to config.txt but with this it happens no matter what the quality setting is.

Both of these features are (I think?) supposed to improve HEVC decode performance so it would be nice to see them working.

@Slouken

[boltronics]

Stable 4k60 on a Pi 4B has proven to be very difficult. I purchased the Pi hardware because I recently got myself a new 4K TV (with @120Hz, HDM2.1 support) and this article suggested it was very easy to achieve 4K@60Hz with the latest Pi model:

https://medium.com/@monofuel34089/running-your-raspberry-pi-4-at-4k60hz-78010a26e98d

It turns out that it's completely misleading - he may be outputting at a 4K resolution, but the video stream he's decoding is only 1920x1080. You just can't do 4K@60 as easily as the author says - but it is possible... sort of! Here I present my findings in trying to get it working.

My primary game machine is a Debian Buster GNU/Linux desktop with a Radeon VII card, custom built Mesa 20.2.0 stack and a 2560x1440 144Hz Freesync monitor.

It turns out that you can't select 4K in-game over Steam Link on a 4K TV if the desktop monitor can't also show 4K, so I purchased a HDMI plug from Amazon that makes the graphics card think there is a second 4K@60 monitor connected. I then wrote a little script called `toggledisplay` and made a shortcut in my Xfce window manager (Win+Home) to call it. I then made a second shortcut (Win+End) to call `toggledisplay fake4k` to switch output to the second fake 4K monitor exclusively.

$ cat toggledisplay 
#!/bin/bash

if [ "${1}" != "fake4k" ]
then
    xrandr --output DisplayPort-0 --mode 2560x1440 --rate 144 --primary
    xrandr --output HDMI-A-0 --off
else
    xrandr --output DisplayPort-0 --off
    xrandr --output HDMI-A-0 --primary
    xrandr --output HDMI-A-0 --mode 3840x2160
    xrandr --output HDMI-A-0 --rate 60
    xrandr > ~/OUTPUT
fi
$

After switching between modes, I looked at the OUTPUT file:

$ cat ~/OUTPUT 
Screen 0: minimum 320 x 200, current 3840 x 2160, maximum 16384 x 16384
DisplayPort-0 connected (normal left inverted right x axis y axis)
   2560x1440     59.95 + 143.86   119.88    99.90  
   1920x1200     59.95  
   1920x1080    119.96   143.85    60.00    60.00    50.00    59.94  
   1600x1200     59.95  
   1680x1050     59.95  
   1600x900      60.00  
   1280x1024     75.02    60.02  
   1440x900      59.95  
   1280x800      59.81  
   1152x864      75.00  
   1280x720      60.00    50.00    59.94  
   1024x768      75.03    60.00  
   832x624       74.55  
   800x600       75.00    60.32  
   720x576       50.00  
   720x480       60.00    59.94  
   640x480       75.00    60.00    59.94  
   720x400       70.08  
DisplayPort-1 disconnected (normal left inverted right x axis y axis)
DisplayPort-2 disconnected (normal left inverted right x axis y axis)
HDMI-A-0 connected primary 3840x2160+0+0 (normal left inverted right x axis y axis) 480mm x 270mm
   1920x1080     60.00 + 120.00   119.88    50.00    24.00    23.98  
   4096x2160     24.00    23.98  
   3840x2160     30.00*   30.00    25.00    24.00    29.97    23.98  
   2880x1800     60.00  
   2560x1600     59.97  
   2560x1440     59.95  
   1920x1200     59.95  
   1680x1050     59.88  
   1600x900      60.00  
   1280x1024     75.02  
   1440x900      59.90  
   1280x800      59.91  
   1280x720      60.00    50.00    59.94  
   1024x768      75.03    70.07    60.00  
   800x600       72.19    75.00    60.32    56.25  
   720x576       50.00  
   720x480       60.00    59.94  
   640x480       75.00    72.81    60.00    59.94  
$

So for some reason, the amdgpu driver isn't seeing the 60Hz option of the fake HDMI monitor (I saw there is an open bug report about this somewhere on bugs.freedesktop.org which has been around for years...), making this endeavour pointless. Unfortunately I couldn't find any fake DisplayPort 4k@60 plugs anywhere. Maybe I could manually specify a fake resolution for the plug in my xorg.conf file... I haven't investigated that avenue at this time. Even just trying to use Steam Link to output 2560x1440 to my TV was proving to be problematic (the Pi would show a scrambled image on the TV) so I decided to give up on this approach.

Next, I tried Windows 10. Annoying because I have around 3.7TB of games installed on my GNU/Linux partition. I didn't need to use the 4K@60 dongle in Windows because the AMD drivers have a super-sampling mode feature built in which, when enabled, tricks Windows into thinking it's running on a 3840x2160@60Hz monitor, so I can select a 3840x2160 resolution in game without issue.

Now back to Steam Link - I can only output in 4K on the Pi if I select Fast encoding - otherwise it seems to drop back to 1920x1080 automatically. I have a wired Gigabit Ethernet LAN (both machines directly connected via Gigabit Ethernet switches) so heaps of networking bandwidth. So long as HEVC was selected, I was able to stream at 4K@60 - but I was getting huge slowdowns every ~3 seconds, as you also indicated.

According to the stats overlay (if you enable it on the Pi Steam-Link software), the issue is with the Pi's ability to decode. It's just not quick enough to handle it. My desktop is barely breaking a sweat (I was using the old Sniper Elite V2 as a test since it loads quickly, supports 4K, also runs under Proton if needed, and isn't too demanding for modern hardware, even on maximum detail settings).

Using the AMD driver software, I tried capping the frame rate to 30 FPS (it's a power saving option). As expected, the Pi could stream that fine. I tried bumping the cap incrementally. It wasn't until I set it to 35 FPS where I started seeing some stuttering. Not too frequently, but occasionally it was there. 36 FPS... stuttering increased, etc. Any more than that and the stuttering was very frequent.

I tried messing around with bandwidth caps, toggling the dynamic resolution option, even toggling the hardware encoding option for AMD, etc. All options either had no effect, or made things worse.

Something I noticed in the Steam Link options was the extra encoder/decoder options for streaming from Nvidia hardware. My wife has a GTX 1080 in her computer running Windows 10, so I figured I'd try streaming from her machine for a bit. She also uses a 2K monitor, and I couldn't find an equivalent super-sampling feature from Nvidia so I used my fake 4K@60 HDMI plug, and toggled to that "screen" using Win+P before connecting to it with Steam Link from the Pi.

Things were working much better, but I still couldn't quite hit that magical 4K@60 FPS without stutter - but I was close! I guess the Pi hardware can decode the compression Nvidia drivers use better.

One last thing to try - overclocking the Pi and upgrading the firmware. I got myself an Argon ONE Raspberry Pi 4 Case, which is basically one big heat sink with a fan in it (and it's cool here in Melbourne at this time anyway - the Pi temps never reached 50 degrees), and read over both of these articles:

https://magpi.raspberrypi.org/articles/how-to-overclock-raspberry-pi-4

https://www.tomshardware.com/reviews/raspberry-pi-4-b-overclocking,6188.html

My config.txt currently looks like this:

disable_overscan=1
hdmi_drive=2
arm_freq=2147
over_voltage=6
gpu_freq=750
dtparam=i2c_arm=on
dtparam=audio=on
gpu_mem=512

[pi4]
dtoverlay=vc4-fkms-v3d
max_framebuffers=2

[all]
hdmi_enable_4kp60=1
enable_uart=1

and now, when connected to my wife's computer, I can stream games at 4K@60Hz with a mostly stable frame rate. Not 100% stable... eventually the Steam Link stats graph will report a stutter - but I'm not really noticing it so maybe it's a very brief stutter compared to what it was before.

The only problem is that my wife's upset at me because it wasn't so long ago that I got my Radeon VII, and she's always using her computer and doesn't like me touching it anyway. I can't bring myself to game at 1920x1080@60 when I could just use my desktop and get 2K 144Hz so I've never been able to actually use Steam Link on the Pi in the few weeks that I've owned it. Totally sucks!

Hopefully Valve can find a way to make 4K@60Hz work on the Pi without relying on a Windows+Nvidia host hardware combination and heavy overclocking. In the meantime, when it comes to gaming on my TV, I'm sadly heading back to my Xbox.

[spadz93]

I'm having this problem just trying to get 1080p60hz. I'll try some of the above, hopefully it helps a little bit

EDIT: yeah, tried OC'ing the RPI4, and i'm using a 1080ti. I'm not having a good time here. I have a gigabit router, cat5e running both my PC and RPI4 back to the router, i don't really know what else to do here

[boltronics]

I don't think overclocking should be required for FHD@60 as it worked pretty much out of the box for me. Run a bandwidth test and also check the stats overlay to see the bottleneck. Try different games - a couple I tried wouldn't run over Steam Link. Check your GPU power saving settings and limit to 60FPS if possible. Disable vsync, etc.