Shader Precaching Single Thread Performance

[Mike]

The fossilize process seems to run only as a single thread process and takes a lot longer than I would presume it would as a multi threaded process. I have latest Nvidia drivers (465.31). Any idea why it is still single threaded?

[class101 [OLED deck]]

Here on Steam client beta when the shaders are compiling, I see 8x fossilize_replay processes, I guess it is corresponding to the number or my cpu cores and it takes 100% of the cpu when working. So no I don't see any problem here.

Do you have only one fozzilize process working on a multi core cpu ? It must take a long time !

[Mike]

Yes it only uses one, I am also using Steam Beta. I wonder if it is a nvidia issue, intel issue, not sure. It is annoying, takes about 5 minutes or so, but it is not a problem beyond that.

[thetargos]

I run on nVidia HW and that driver version (GTX1080, tho), Beta client and I also see fossilize_replay take up as many threads as 6-8 (on my 12-core Ryzen), and when "processing shaders" it can take up all available threads (and still take 'a while' to be done).

Not sure what the issue might be, anything else updated beside the drivers?

[GuRu Asaki]

2 Questions?

1.) Are you using just 1 or both Options under Shader Pre-Caching?
--->> I ask, cause i'm only using the top Option called Pre-Caching.

2.) In the Shader Tab, it states this...

- May slightly Increase Disk & Bandwith Usage when Enabled.
---->> Could this be why your experiencing issues?

[class101 [OLED deck]]

I think you shouldn't use the term threads but instead fossilize_replay parallel forks, they are both completely different things

It does not really matter to know if fossilize_replay is a true multithread process, what matters is how much fossilize_replay forks are running on your system and the CPU usage.

If you see 8 forks, all taking the whole 100% of your CPU, it works as intended, it does not really matter to know if one of these process is multithread or singlethreaded, most are probably single threaded because I don't see a reason to multithread a command line tool supposed to be forked, this would be overkill.

Nor multiple single threaded forks, nor one multithreaded process but not both... It would be counter efficient.

Originally posted by Mike:

Yes it only uses one, I am also using Steam Beta. I wonder if it is a nvidia issue, intel issue, not sure. It is annoying, takes about 5 minutes or so, but it is not a problem beyond that.

I think you should report this on Steam for Linux github, 5min is a lot indeed, maybe something wrong at determining your cpu cores

https://github.com/ValveSoftware/steam-for-linux

I took a screenshot of the shaders compiling on my system for RDR2, it took around 3min but for a game of this size and if it used one process, it would surely have taken 8x more time so around 25min. (largest game yet I have caught to compile shaders that long in 3min, it is quicker for most games because they are smaller)

https://imgur.com/a/b6qWbq6

[GuRu Asaki]

If you see 8 forks, all taking the whole 100% of your CPU, it works as intended?


Why would you want Games to take up 100% of your CPU, though?
----->> Wouldn't that cause your CPU to struggle, especially at long periods of time?

Also, I understand how fossilize & fossilize_replay is different from Threads,
but maybe you could explain how it's different as maybe not everyone knows
exactly how they are different?

[Zyro]

Originally posted by GuRu Asaki:

Why would you want Games to take up 100% of your CPU, though?

Why would you want parts of the CPU you bought to idle around?
It's like hiring 8 workers and complaining that all of them are actually working... ;)

But I guess this precalculation should run before the actual game play, anyways.

Originally posted by GuRu Asaki:

Also, I understand how fossilize & fossilize_replay is different from Threads,
but maybe you could explain how it's different as maybe not everyone knows
exactly how they are different?

The difference between spawning threads or processes is not to important in this context. A thread is "lighter", threads are more connected to each other and can communicate easier. But in the end, if it's supposed to calculate as quick as it can, all that matters is that it makes full sue of the CPU.

[Aoi Blue]

Originally posted by catbox:

thats a good issue to bring up

i dont use steam shader precaching. preferring to let my GPU handle it.

I rely on Mesa's internal precache instead, myself.