[solved] Re-enabling hardware en- and decoding in SteamOS 3.4

[[KAMI] RedWyvern]

edit: As per the so far unannounced SteamOS 3.4.2 update, hardware encoding has been enabled again!
If for some reason you want to you can still use the updated script to downgrade, though I recommend against this.
WARNING: THE SCRIPT BREAKS STEAMOS 3.5 AND UP

edit: In the update of 21 December;
https://steamcommunity.com/app/1675200/eventcomments/3727323242807343433/
All SteamOS update channels are now affected.
As expected, the temporary solution as posted no longer, but can be made to work again, for now.
More detailed update on this:
https://steamcommunity.com/app/1675200/discussions/0/3541546590709961979/?ctp=2#c3727323242807512617

edit: Temporary solution found: https://steamcommunity.com/app/1675200/discussions/0/3541546590709961979/#c3541546590717333369

As reported in:
https://github.com/ValveSoftware/SteamOS/issues/903
https://steamcommunity.com/app/1675200/eventcomments/5568165891217922394/?ctp=5#c5568165891223664535
https://steamcommunity.com/app/1675200/discussions/1/3541546590703047609/
https://steamcommunity.com/app/1675200/discussions/1/3541546590703291288/
the new 3.4 Preview update lacks hardware video en- and decoding.
Because US software patents seem to be involved, it's uncertain whether Valve can fix this.
As it messes with how I use my Steam Deck, I am experimenting with patching the feature back in, to not be stuck on version 3.3 in the future if it doesn't get solved on standard SteamOS.

[FireGate_13]

hardware de-encoding in windows is lost from the begining.I cannot understand.. how all AMD gpus have hardware acceleration and steam's apu is not supported?

[[KAMI] RedWyvern]

Continuing from https://steamcommunity.com/app/1675200/discussions/1/3541546590703291288/#c3541546590707204758

I've attempted to replace the Mesa driver with the standard Arch Linux one, which does support VAProfileH264Main according to a vainfo command.
When attempting to use Steam Link with this, I am currently getting the following errors:

[2022-11-17 20:39:20] ffmpeg verbose: libva: VA-API version 1.15.0
[2022-11-17 20:39:20] ffmpeg verbose: libva: Trying to open /usr/lib32/dri/radeonsi_drv_video.so
[2022-11-17 20:39:20] GameScope focus changed to appID 587620
[2022-11-17 20:39:20] ffmpeg error: libva: /usr/lib32/dri/radeonsi_drv_video.so has no function __vaDriverInit_1_0
[2022-11-17 20:39:20] ffmpeg verbose: libva: va_openDriver() returns -1
[2022-11-17 20:39:20] ffmpeg error: Failed to initialise VAAPI connection: -1 (unknown libva error).
[2022-11-17 20:39:20] CGameStreamVideoStageVAAPI: Failed to create device context: Input/output error

I will continue experimenting with a hacky fix, by either solving this library issue or downgrading to the old Mesa package instead.

For those wanting to continue where I left off for now, I am currently using this script to install the Arch Linux package on SteamOS:

#!/bin/bash

mkdir mesa_packages
cd mesa_packages

echo -e "Downloading the Arch Linux Mesa packages...\n\n\n"
wget -nc \
    "https://europe.mirror.pkgbuild.com/extra/os/x86_64/glu-9.0.2-3-x86_64.pkg.tar.zst" \
    "https://europe.mirror.pkgbuild.com/multilib/os/x86_64/lib32-libva-mesa-driver-22.2.3-1-x86_64.pkg.tar.zst" \
    "https://europe.mirror.pkgbuild.com/multilib/os/x86_64/lib32-mesa-22.2.3-1-x86_64.pkg.tar.zst" \
    "https://europe.mirror.pkgbuild.com/multilib/os/x86_64/lib32-mesa-vdpau-22.2.3-1-x86_64.pkg.tar.zst" \
    "https://europe.mirror.pkgbuild.com/multilib/os/x86_64/lib32-vulkan-radeon-22.2.3-1-x86_64.pkg.tar.zst" \
    "https://europe.mirror.pkgbuild.com/extra/os/x86_64/libva-mesa-driver-22.2.3-1-x86_64.pkg.tar.zst" \
    "https://europe.mirror.pkgbuild.com/extra/os/x86_64/mesa-22.2.3-1-x86_64.pkg.tar.zst" \
    "https://europe.mirror.pkgbuild.com/extra/os/x86_64/mesa-utils-8.5.0-2-x86_64.pkg.tar.zst" \
    "https://europe.mirror.pkgbuild.com/extra/os/x86_64/mesa-vdpau-22.2.3-1-x86_64.pkg.tar.zst" \
#    "https://europe.mirror.pkgbuild.com/extra/os/x86_64/opencl-mesa-22.2.3-1-x86_64.pkg.tar.zst" \
# Disabling this causes an obvious partial upgrade, but also prevents a pacman error.
    "https://europe.mirror.pkgbuild.com/extra/os/x86_64/vulkan-radeon-22.2.3-1-x86_64.pkg.tar.zst" \
|| echo -e "\n\n\nWARNING: Some downloads failed; this might not work."

echo -e "\n\n\nReplacing the SteamOS Mesa install with the Arch Linux one..."
sudo steamos-readonly disable
sudo pacman -U *
sudo steamos-readonly enable

echo -e "\n\n\nYour Deck will restart now, hopefully with reinstated hardware en- and decoding."
sleep 8
sudo reboot

[[KAMI] RedWyvern]

I highly recommend not doing any of this, unless you know exactly what you are doing and are able to restore a broken SteamOS install.



I have moved to mixing the old versions of libva-mesa-driver, lib32-libva-mesa-driver, mesa-vdpau and lib32-mesa-vdpau with the current version of the rest of the package.
This risky mix of versions results in the following streaming log:

[2022-11-18 21:04:00] ffmpeg verbose: libva: VA-API version 1.15.0
[2022-11-18 21:04:00] ffmpeg verbose: libva: Trying to open /usr/lib32/dri/radeonsi_drv_video.so
[2022-11-18 21:04:00] ffmpeg error: libva: dlopen of /usr/lib32/dri/radeonsi_drv_video.so failed: libLLVM-13.so: cannot open shared object file: No such file or directory
[2022-11-18 21:04:00] ffmpeg verbose: libva: va_openDriver() returns -1
[2022-11-18 21:04:00] ffmpeg error: Failed to initialise VAAPI connection: -1 (unknown libva error).
[2022-11-18 21:04:00] CGameStreamVideoStageVAAPI: Failed to create device context: Input/output error

This indicates that the older hardware acceleration driver is working with the new version, but lacking some dependencies of the old SteamOS version.

Which might be solvable, I will continue on this path to see if it can be made to work, knowing that it might break in an update (after re-installing).
It would be a better approach to compile a new package of one version, as attempted in https://github.com/ValveSoftware/SteamOS/issues/903#issuecomment-1312727044 but for now I am sticking to the Valve-compiled ones.

To install these older packages, I am using the following script:

#!/bin/bash

mkdir 3.3.2_mesa_hw_accel_pkgs
cd 3.3.2_mesa_hw_accel_pkgs

echo -e "Downloading some SteamOS 3.3.2 (20221005.1) Mesa packages...\n\n\n"
wget -nc \
    "https://steamdeck-packages.steamos.cloud/archlinux-mirror/jupiter/os/x86_64/lib32-libva-mesa-driver-22.0.2.150118.radeonsi_20220427-1-x86_64.pkg.tar.zst" \
    "https://steamdeck-packages.steamos.cloud/archlinux-mirror/jupiter/os/x86_64/lib32-mesa-vdpau-22.0.2.150118.radeonsi_20220427-1-x86_64.pkg.tar.zst" \
    "https://steamdeck-packages.steamos.cloud/archlinux-mirror/jupiter/os/x86_64/libva-mesa-driver-22.0.2.150118.radeonsi_20220427-1-x86_64.pkg.tar.zst" \
    "https://steamdeck-packages.steamos.cloud/archlinux-mirror/jupiter/os/x86_64/mesa-vdpau-22.0.2.150118.radeonsi_20220427-1-x86_64.pkg.tar.zst" \
|| echo -e "\n\n\nWARNING: Some downloads failed; this might not work.\n         It is safer to close Konsole now, no SteamOS modifications have been made yet."

echo -e "\n\n\nDowngrading the SteamOS Mesa hardware acceleration packages (to 22.0.2.150118)..."
sudo steamos-readonly disable
sudo pacman -U *
sudo steamos-readonly enable

echo -e "\n\n\nYour Deck will restart now, hopefully with reinstated hardware acceleration."
sleep 8
sudo reboot

[[KAMI] RedWyvern]

I highly recommend firstly using the output of

pacman -Qs mesa

to update the package download URL list in the script, obviously before doing any of these crazy hacks.
But here is another script to restore the Mesa packages to the at the time of writing current version:

#!/bin/bash

mkdir 3.4-P_mesa_packages
cd 3.4-P_mesa_packages

echo -e "Downloading the SteamOS 3.4 Preview (20221116.100) Mesa packages...\n\n\n"
wget -nc \
    "https://steamdeck-packages.steamos.cloud/archlinux-mirror/jupiter-beta/os/x86_64/mesa-22.2.0.157819.radeonsi_3.4.0-3-x86_64.pkg.tar.zst" \
    "https://steamdeck-packages.steamos.cloud/archlinux-mirror/extra-beta/os/x86_64/glu-9.0.2-3-x86_64.pkg.tar.zst" \
    "https://steamdeck-packages.steamos.cloud/archlinux-mirror/jupiter-beta/os/x86_64/lib32-libva-mesa-driver-22.2.0.157819.radeonsi_3.4.0-3-x86_64.pkg.tar.zst" \
    "https://steamdeck-packages.steamos.cloud/archlinux-mirror/jupiter-beta/os/x86_64/lib32-mesa-22.2.0.157819.radeonsi_3.4.0-3-x86_64.pkg.tar.zst" \
    "https://steamdeck-packages.steamos.cloud/archlinux-mirror/jupiter-beta/os/x86_64/lib32-mesa-vdpau-22.2.0.157819.radeonsi_3.4.0-3-x86_64.pkg.tar.zst" \
    "https://steamdeck-packages.steamos.cloud/archlinux-mirror/jupiter-beta/os/x86_64/vulkan-radeon-22.2.0_devel.156801.jupiter_22.3.2-2-x86_64.pkg.tar.zst" \
    "https://steamdeck-packages.steamos.cloud/archlinux-mirror/jupiter-beta/os/x86_64/lib32-vulkan-radeon-22.2.0_devel.156801.jupiter_22.3.2-2-x86_64.pkg.tar.zst" \
    "https://steamdeck-packages.steamos.cloud/archlinux-mirror/jupiter-beta/os/x86_64/libva-mesa-driver-22.2.0.157819.radeonsi_3.4.0-3-x86_64.pkg.tar.zst" \
    "https://steamdeck-packages.steamos.cloud/archlinux-mirror/extra-beta/os/x86_64/mesa-utils-8.5.0-2-x86_64.pkg.tar.zst" \
    "https://steamdeck-packages.steamos.cloud/archlinux-mirror/jupiter-beta/os/x86_64/mesa-vdpau-22.2.0.157819.radeonsi_3.4.0-3-x86_64.pkg.tar.zst" \
    "https://steamdeck-packages.steamos.cloud/archlinux-mirror/jupiter-beta/os/x86_64/opencl-mesa-22.2.0.157819.radeonsi_3.4.0-3-x86_64.pkg.tar.zst" \
|| echo -e "\n\n\nWARNING: Some downloads failed; this might not work.\n         It is safer to close Konsole now, no SteamOS modifications have been made yet."

echo -e "\n\n\nOverwriting the SteamOS Mesa driver (with 22.2.0.157819)..."
sudo steamos-readonly disable
sudo pacman -U *
sudo steamos-readonly enable

echo -e "\n\n\nYour Deck will restart now, back to the 3.4 Preview Mesa version."
sleep 8
sudo reboot

[[KAMI] RedWyvern]

Success!

Downgrading the Mesa driver to the version used in the beta restored hardware accelerated video encoding.
Here is a screenshot of Okami HD being streamed to my desktop, with low CPU usage:
https://steamcommunity.com/sharedfiles/filedetails/?id=2890726528
streaming_log.txt confirms that it is using hardware acceleration:

[2022-11-19 20:40:00] ffmpeg verbose: libva: VA-API version 1.15.0
[2022-11-19 20:40:00] ffmpeg verbose: libva: Trying to open /usr/lib32/dri/radeonsi_drv_video.so
[2022-11-19 20:40:00] ffmpeg verbose: libva: Found init function __vaDriverInit_1_13
[2022-11-19 20:40:00] ffmpeg verbose: libva: va_openDriver() returns 0
[2022-11-19 20:40:00] ffmpeg verbose: Initialised VAAPI connection: version 1.15
[2022-11-19 20:40:00] ffmpeg verbose: VAAPI driver: Mesa Gallium driver 22.0.2 for AMD Custom GPU 0405 (vangogh, LLVM 13.0.0, DRM 3.45, 5.13.0-valve31-1-neptune).
[2022-11-19 20:40:00] ffmpeg verbose: Driver not found in known nonstandard list, using standard behaviour.
[2022-11-19 20:40:00] ffmpeg verbose: Input surface format is nv12.
[2022-11-19 20:40:00] ffmpeg verbose: Using VAAPI profile VAProfileH264Main (6).
[2022-11-19 20:40:00] ffmpeg verbose: Using VAAPI entrypoint VAEntrypointEncSlice (6).
[2022-11-19 20:40:00] ffmpeg verbose: Using VAAPI render target format YUV420 (0x1).

	"CaptureDescriptionID"		"Desktop PipeWire DMABUF + VAAPI H264"
	"DecoderDescriptionID"		"VAAPI DRM hardware decoding"

Out of curiosity to the reports of it not launching, I tested Linux native Portal, which started fine on SteamOS 3.4 Preview with the downgraded Mesa driver:
https://steamcommunity.com/sharedfiles/filedetails/?id=2890726569

The technical details
By installing the missing dependencies I have gotten Valve's 22.0.2.150118 Mesa version to work on the current SteamOS 3.4 Preview build 20221116.100.
The older driver requires version 13.[something] of llvm-libs and lib32-llvm-libs, but the rest of SteamOS needs version 14.[something].
While Arch Linux's llvm13-libs package can be successfully installed alongside the normal version, the 32-bit version and Valve's packages downgrade the main package instead, which breaks SteamOS.
Installing the versions alongside each other is possible, either by making a package for this or by installing it manually, without using pacman.

Tomorrow I will follow up with a script for those daring enough to replicate this driver downgrade but unable to follow along.
I do not know how reliable this is, I only know that it works for how I use my Deck, FOR NOW.

[[KAMI] RedWyvern]

Mesa downgrade script

For those wanting to experiment with the current SteamOS 3.4 Preview build 20221116.100 but are running into driver related regressions, here is a script to downgrade the Mesa video driver to version 20.0.2, the one used in SteamOS 3.3.2.
Run this script is at your own risk.
For me the downgraded driver appears to work correctly, but this might not be the case for you or in later SteamOS 3.4 builds.

Instructions (Desktop Mode):

1. Paste all the contents of the code block found below in Kate > Save As somewhere you can find (with a sensible filename).
   
2. Right-click the saved file > Properties > Permissions > check Is executable > OK
   
3. Right-click the file again > Run In Konsole
   
4. When it asks, enter your sudo password to apply the downgrade (set one with passwd before if needed).
   
5. After the Deck restarts, it will be running Mesa 20.0.2 (instead of at the time of writing 20.2.0).

The script for downgrading to Mesa 20.0.2:

#!/bin/bash

echo -e "Saving previous package version URLs..."
mkdir mesa_downgrade_data; cd mesa_downgrade_data
date >> revert_downloads.txt; pacman -Sp \
    glu \
    lib32-libva-mesa-driver \
    lib32-mesa \
    lib32-mesa-vdpau \
    lib32-vulkan-radeon \
    libva-mesa-driver \
    mesa \
    mesa-utils \
    mesa-vdpau \
    opencl-mesa \
    vulkan-radeon \
>> revert_downloads.txt

echo -e "\n\n\nDownloading and extracting SteamOS LLVM 13.0.0 libraries...\n\n\n"
wget -nc \
    "https://steamdeck-packages.steamos.cloud/archlinux-mirror/multilib/os/x86_64/lib32-llvm-libs-13.0.0-1-x86_64.pkg.tar.zst" \
    "https://steamdeck-packages.steamos.cloud/archlinux-mirror/extra/os/x86_64/llvm-libs-13.0.0-6-x86_64.pkg.tar.zst" \
|| { echo -e "\n\n\nERROR: Failed to download required LLVM packages."; read; exit; }
tar -xaf lib32-llvm-libs-13.0.0-1-x86_64.pkg.tar.zst usr/lib32/libLLVM-13.so usr/lib32/libLLVM-13.0.0.so
tar -xaf llvm-libs-13.0.0-6-x86_64.pkg.tar.zst usr/lib/libLLVM-13.so usr/lib/libLLVM-13.0.0.so

echo -e "\n\n\nDownloading SteamOS Mesa 20.0.2 packages...\n\n\n"
mkdir mesa_packages; cd mesa_packages
wget -nc \
    "https://steamdeck-packages.steamos.cloud/archlinux-mirror/extra/os/x86_64/glu-9.0.2-2-x86_64.pkg.tar.zst" \
    "https://steamdeck-packages.steamos.cloud/archlinux-mirror/jupiter/os/x86_64/lib32-libva-mesa-driver-22.0.2.150118.radeonsi_20220427-1-x86_64.pkg.tar.zst" \
    "https://steamdeck-packages.steamos.cloud/archlinux-mirror/jupiter/os/x86_64/lib32-mesa-22.0.2.150118.radeonsi_20220427-1-x86_64.pkg.tar.zst" \
    "https://steamdeck-packages.steamos.cloud/archlinux-mirror/jupiter/os/x86_64/lib32-mesa-vdpau-22.0.2.150118.radeonsi_20220427-1-x86_64.pkg.tar.zst" \
    "https://steamdeck-packages.steamos.cloud/archlinux-mirror/jupiter/os/x86_64/lib32-vulkan-radeon-22.2.0_devel.152974.jupiter_22.2.1.3-1-x86_64.pkg.tar.zst" \
    "https://steamdeck-packages.steamos.cloud/archlinux-mirror/jupiter/os/x86_64/libva-mesa-driver-22.0.2.150118.radeonsi_20220427-1-x86_64.pkg.tar.zst" \
    "https://steamdeck-packages.steamos.cloud/archlinux-mirror/jupiter/os/x86_64/mesa-22.0.2.150118.radeonsi_20220427-1-x86_64.pkg.tar.zst" \
    "https://steamdeck-packages.steamos.cloud/archlinux-mirror/extra/os/x86_64/mesa-utils-8.4.0-6-x86_64.pkg.tar.zst" \
    "https://steamdeck-packages.steamos.cloud/archlinux-mirror/jupiter/os/x86_64/mesa-vdpau-22.0.2.150118.radeonsi_20220427-1-x86_64.pkg.tar.zst" \
    "https://steamdeck-packages.steamos.cloud/archlinux-mirror/jupiter/os/x86_64/opencl-mesa-22.0.2.150118.radeonsi_20220427-1-x86_64.pkg.tar.zst" \
    "https://steamdeck-packages.steamos.cloud/archlinux-mirror/jupiter/os/x86_64/vulkan-radeon-22.2.0_devel.152974.jupiter_22.2.1.3-1-x86_64.pkg.tar.zst" \
|| { echo -e "\n\n\nERROR: Failed to download required Mesa packages."; read; exit; }

echo -e "\n\n\nInstalling LLVM 13.0.0 libraries and downgrading Mesa to 20.0.2..."
sudo steamos-readonly disable
sudo cp -r ../usr /
sudo pacman -U *
sudo steamos-readonly enable

echo -e "\n\n\nYour Deck will restart now, applying the Mesa downgrade..."
sleep 8
sudo reboot

[inFantry-32]

Originally posted by KAMI RedWyvern:

edit: temporary solution found: https://steamcommunity.com/app/1675200/discussions/0/3541546590709961979/#c3541546590717333369

As reported in:
https://github.com/ValveSoftware/SteamOS/issues/903
https://steamcommunity.com/app/1675200/eventcomments/5568165891217922394/?ctp=5#c5568165891223664535
https://steamcommunity.com/app/1675200/discussions/1/3541546590703047609/
https://steamcommunity.com/app/1675200/discussions/1/3541546590703291288/
the new 3.4 Preview update lacks hardware video en- and decoding.
Because US software patents seem to be involved, it's uncertain whether Valve can fix this.
As it messes with how I use my Steam Deck, I am experimenting with patching the feature back in, to not be stuck on version 3.3 in the future if it doesn't get solved on standard SteamOS.

+1

[[KAMI] RedWyvern]

So far, after the driver downgrade SteamOS 3.4 has been a solid experience for me, no other major issues while playing a few older games over Steam Link and using Desktop Mode as a Linux tablet.
SteamOS even appears to detect the downgrade correctly and re-downloads the shader cashes for the older version.
The new KDE Plasma version and inclusion of KDE Connect are nice upgrades and work well on Mesa 20.0.2.
Downgrading even appears to solve the onscreen keyboard related crashes in Desktop Mode:
https://steamcommunity.com/app/1675200/discussions/1/5568165891228744102/#c3541546590719815755

If you do run into problems after downgrading the driver, post about it here, it might be solvable.

PS. The recent 20221121.100 Slightly More Evil update is practically identical to 20221116.100, as with any OS update it does require running the script again.

[[KAMI] RedWyvern]

Originally posted by bean:

100% off-topic

Do you mind if I add you, bro/bro-ette?
you are super knowledgeable

my attempts to diy turn into a penguin via Steam Deck were thwarted when I realized I could use it to play video games ootb. :CyberVision:

Oh sure, go ahead... and thanks for the compliment I guess.

[[KAMI] RedWyvern]

Update: SteamOS build 20221202.101 still exhibits at least the desktop crashing on keyboard closing issue.
Downgrading the driver works correctly in this build as well, besides LLVM 13 all other dependencies are present.
Build 20221121.100 had one similar desktop crash while typing, but this was the only crash I had since downgrading and is probably unrelated.

[SolSoCoG]

I did an update via Discover of some mesa/desktop packages and am now greeted with missing h264 en/decoding, the script above unfortunately didn't fix it, probably because it doesn't take flatpak into account?

[[KAMI] RedWyvern]

Originally posted by Nettelbeck:

I did an update via Discover of some mesa/desktop packages and am now greeted with missing h264 en/decoding, the script above unfortunately didn't fix it, probably because it doesn't take flatpak into account?

Would you mind sharing some more details, like what exact app you are using?
Make sure this app has the permission to use hardware acceleration, install and use the Flatseal app to make sure this is the case.
I did a quick test in OBS and the hardware encoding is working there for me, both appearing in the settings and using less CPU than x264 when recording.
https://steamcommunity.com/sharedfiles/filedetails/?id=2898711220
You are correct in that the script does not take Flatpak into account, which also uses their own runtime so it might be isolated, but as I have working h264, the SteamOS driver could have something to do with it.
As the latter only gets downgraded if the script fully completes, did this happen?
It should only be ran on SteamOS 3.4, ask for a password that should have been set before with a passwd command, then typing Y to confirm that you want to install the old driver and finish with rebooting the Deck.
If it didn't do this you can make sure all the prerequisites are met and run it again.

[semoraz]

Originally posted by Nettelbeck:

I did an update via Discover of some mesa/desktop packages and am now greeted with missing h264 en/decoding, the script above unfortunately didn't fix it, probably because it doesn't take flatpak into account?

Another KDE Platform update just arrived via Discover, which fixed yesterday's issues (when using Moonlight) on my side.

Edit: Talking about the issues in the stable branch here.

[LordDunlow]

Originally posted by semoraz:

Originally posted by Nettelbeck:

I did an update via Discover of some mesa/desktop packages and am now greeted with missing h264 en/decoding, the script above unfortunately didn't fix it, probably because it doesn't take flatpak into account?

Another KDE Platform update just arrived via Discover, which fixed yesterday's issues (when using Moonlight) on my side.

Edit: Talking about the issues in the stable branch here.

This fixed my problem as well. It was really causing problems with Moonlight streaming from my desktop PC. Any idea how I prevent this being disabled again? I'm on stable.

[Chris]

Moonlight is also detecting decoders for me again after those updates on the Discover store. I'm on the beta branch (SteamOS 3.3).