What are the constant shader pre-cache downloads for?

[Moe Lester]

I understand that unlike windows which can compile shaders during gameplay without any stuttering, SteamOS will experience constant hitches/freezes while they compile so it needs to download them before you open the game. I verified this by trying some GOG games and the performance was bad.

My question is why does Steam Deck download shader pre-cache for my existing games every day? Shouldn’t they all be downloaded at the same time as the game as being installed? How is it possible I’ve already played half a game and Steam still downloads shader pre-cache for it?

[drsxzgsa]

You can disable pre-cache downloads so they only update when you start a game.

[Moe Lester]

Originally posted by drsxzgsa:

You can disable pre-cache downloads so they only update when you start a game.

Why are they updating in the first place? Are they missing from random sections of the game?

[drsxzgsa]

Not sure. If the game(s) gets an update then the shader will too I guess??

Pretty annoying I know. That’s why I’ve disabled updates until I start a game.

[Moe Lester]

Originally posted by drsxzgsa:

Not sure. If the game(s) gets an update then the shader will too I guess??

Pretty annoying I know. That’s why I’ve disable updates until I start a game.

The games I play haven’t gotten an update in ten years.

[Moe Lester]

My issue isn’t with constant downloads, it’s that I want to know if steam leaves random sections of a game without pre-compiled shaders.

[Mahjik]

We on the consumer side don't have the details. What we do know is that shaders need to be recompiled if something changes in the game or OS (including GPU drivers). So it's not just the game hasn't been updated in while.

What I have always suspected is that Valve's shader compilation is also covering their non-Stable pipelines (Beta & Preview). So for those on Stable, they get shader updates more frequently when it may not be required. I have no idea if that is true as we cannot see into Valve's processes.

[thetargos]

Shader cache change whenever there is a new driver update or system update. Since there have been several updates in the last few weeks to multiple levels of Steam infrastructure (Steam client, SteamOS 3.6, 3.7 and 3.8, even minor changes can trigger this, etc), I would not be surprised if there were even more in the pipe.

Shaders downloaded are built versions for the SD hardware "ready to consume and use". Yes, the hardware CAN and DOES build shaders on the go, alas, it is one of the major issues with stuttering in all platforms. Valve has mitigated this by means of 1) having games run through Proton to process the bulk (not all) shaders before launching games (hence, the processing Vulkan shaders screen), and 2) have a cache of shaders assigned for every installed games that requires them to skip or shorten this and to allow a better overall experience.

This is especially true for games made for Windows and DirectX, since Proton and Wine would have to translate shader data on-the-fly (a process which consumes resources, both CPU and RAM), and can degrade performance. Remember DirectX is only available on Windows and on Linux we only have Vulkan and OpenGL ("slower") as available translate targets, having shaders ready for use while not automatically improves performance, allows better utilization of available resources for maximum.performance, instead of allocating them for the translation and build tasks. Even regular Linux native applications can make use of shader cache (driver-sided) to improve performance upon re-runs of the same code.

This is partly why with every part of the graphics stack that varies, the cache has to be regenerated. Yes, induces a point of pain, but it also alleviates many more. I for one do not mind downloading the cache, BUT I do reckon that it IS a thorn on the side of a "Quick game session" in time constraint scenarios. You can skip 'em, and in many cases the game plays normally (maybe even using the older cache), in some other cases performance tanks.

[Moe Lester]

Originally posted by thetargos:

Yes, the hardware CAN and DOES build shaders on the go, alas, it is one of the major issues with stuttering in all platforms.

It doesn’t cause stuttering on windows in any game from any decade.

Also there is clearly some shader cache downloads when nothing in the system has updated or changed. The issue has nothing to do with wanting to open games as fast as possible, it’s why aren’t these shaders all downloaded at the beginning before opening the game the first time, to ensure no stutters happen anywhere so that we can get a smooth windows-like experience.

[Mahjik]

Originally posted by Moe Lester:

It doesn’t cause stuttering on windows in any game from any decade.

It depends on the game. I played a game on my Deck that didn't have a shader cache, so it was building them on the fly. It was not the best experience. Some new games, like Elden Ring, can build them inside them game itself (i.e. the devs realized the shader building was a performance hit while playing so they offered a mechanism to pre-build them as well). Other newish games do it to..

Originally posted by Moe Lester:

why aren’t these shaders all downloaded at the beginning before opening the game the first time, to ensure no stutters happen anywhere so that we can get a smooth windows-like experience.

You can change when downloads are done in Steam settings. You can say only update when I launch a game. If you have games installed that you play infrequently, that could save you a download or two.

[Deport Them All]

Its steam showing you that you don't own anything.

[Deport Them All]

Originally posted by Mahjik:

Originally posted by Moe Lester:

It doesn’t cause stuttering on windows in any game from any decade.

It depends on the game. I played a game on my Deck that didn't have a shader cache, so it was building them on the fly. It was not the best experience. Some new games, like Elden Ring, can build them inside them game itself (i.e. the devs realized the shader building was a performance hit while playing so they offered a mechanism to pre-build them as well). Other newish games do it to..

Originally posted by Moe Lester:

why aren’t these shaders all downloaded at the beginning before opening the game the first time, to ensure no stutters happen anywhere so that we can get a smooth windows-like experience.

You can change when downloads are done in Steam settings. You can say only update when I launch a game. If you have games installed that you play infrequently, that could save you a download or two.

Nice non solutions. Why should we have to download updates for 10 year old games that are content complete? Shill.

[Moe Lester]

Originally posted by Mahjik:

Originally posted by Moe Lester:

It doesn’t cause stuttering on windows in any game from any decade.

It depends on the game. I played a game on my Deck that didn't have a shader cache, so it was building them on the fly. It was not the best experience. Some new games, like Elden Ring, can build them inside them game itself (i.e. the devs realized the shader building was a performance hit while playing so they offered a mechanism to pre-build them as well). Other newish games do it to..

Yes the simple evidence is that if you play any non-Steam game on SteamOS there will be stutters every 2 seconds. If you play the exact same game on windows it will be a smooth experience.
So shader compilation during gameplay is a non-issue on Windows to begin with. SteamOS’s need to download them isn’t some kind of advantage, it’s a limitation of the system that can’t compile them properly during gameplay like windows can. I’m fine with this, but what I’m not fine with is that only SOME of the shader pre-cache for each game is downloaded, leaving the game open to stutters even when it’s a Steam game, while we get shader pre-cache updates randomly downloaded once a day or once a week with seemingly nothing triggering them besides that they’re missing from the game that I’m already playing.
So my question is still, what is triggering these downloads every day? It’s clearly neither a system update nor a game update.

[Coffee]

For pre-cache!

Hard to believe but it is.

[thetargos]

Originally posted by Moe Lester:

Originally posted by thetargos:

Yes, the hardware CAN and DOES build shaders on the go, alas, it is one of the major issues with stuttering in all platforms.

It doesn’t cause stuttering on windows in any game from any decade.

Also there is clearly some shader cache downloads when nothing in the system has updated or changed. The issue has nothing to do with wanting to open games as fast as possible, it’s why aren’t these shaders all downloaded at the beginning before opening the game the first time, to ensure no stutters happen anywhere so that we can get a smooth windows-like experience.

Many old games do not have issues building their shaders on the fly, especially those from the SM 2.0 and 3.x era, some even newer.

More recent games, regardless of the engine, but one in particular can elicit the issue more than others, exhibit this, especially when transitioning areas and panning view in new areas. There is a slight delay, stemming from shader building. Engine tricks such as motion blur usually hide this. Note that this is also regardless of the hardware used. Granted, some hardware configurations may be "worse" than others. This issue is even worse in a NON NATIVE environment such as Linux, and the need to translate from HLSL to GLSL and build. Nvidia compiler (cg) usually is faily efficient and fast. AMD compiler (used to be LLVM) was not as fast, until the introduction of ACO a few years back. And even these may incur in that delay.

Granted, it is true at times the cache can corrupt itself resulting in constant download, needless to say it is annoying as hell. One game in particular in my library was especially susceptible to this and downloading 4 GiB of shaders for Spyro every time I launched Steam was not fun.

If you detect such behavior you can 1) disable the cache for that particular game or 2) for all games. I usually micromanage this out of noticing in which games the cache was more useful. When a game starts constantly downloading the cache, it is best practice to delete the one present and test, but to each, their own.

[Moe Lester]

Originally posted by thetargos:

Originally posted by Moe Lester:

It doesn’t cause stuttering on windows in any game from any decade.

Also there is clearly some shader cache downloads when nothing in the system has updated or changed. The issue has nothing to do with wanting to open games as fast as possible, it’s why aren’t these shaders all downloaded at the beginning before opening the game the first time, to ensure no stutters happen anywhere so that we can get a smooth windows-like experience.

Many old games do not have issues building their shaders on the fly, especially those from the SM 2.0 and 3.x era, some even newer.

More recent games, regardless of the engine, but one in particular can elicit the issue more than others, exhibit this, especially when transitioning areas and panning view in new areas. There is a slight delay, stemming from shader building. Engine tricks such as motion blur usually hide this. Note that this is also regardless of the hardware used. Granted, some hardware configurations may be "worse" than others. This issue is even worse in a NON NATIVE environment such as Linux, and the need to translate from HLSL to GLSL and build. Nvidia compiler (cg) usually is faily efficient and fast. AMD compiler (used to be LLVM) was not as fast, until the introduction of ACO a few years back. And even these may incur in that delay.

Granted, it is true at times the cache can corrupt itself resulting in constant download, needless to say it is annoying as hell. One game in particular in my library was especially susceptible to this and downloading 4 GiB of shaders for Spyro every time I launched Steam was not fun.

If you detect such behavior you can 1) disable the cache for that particular game or 2) for all games. I usually micromanage this out of noticing in which games the cache was more useful. When a game starts constantly downloading the cache, it is best practice to delete the one present and test, but to each, their own.

Thank you for the explanation. Are all shader caches downloaded before the game is first played? And are shader cache downloads after this point the result of just the cache corrupting itself? Meaning there’s no cache missing from the game?