fsync testing instructions

[Pierre-Loup]

If you run games with Proton, you might want to test fsync! For more details, refer to this announcement.

Source code:

If you know how to build the kernel, the testing branch is over there:

https://gitlab.collabora.com/krisman/linux/commits/futex-wait-multiple-master

Distribution packages:

Arch Linux

A linux-fsync package is available on the AUR at:

https://aur.archlinux.org/pkgbase/linux-fsync/

An Arch repository with pre-built packages is also available, see pinned comments on the AUR page for more information.

Ubuntu

We're maintaining PPAs for Ubuntu 18.04[launchpad.net] and Ubuntu 19.04[launchpad.net]. It's recommended to ppa-purge any other PPAs that include the kernel, like our SteamVR PPA.

After adding the PPA, installing `linux-mfutex-valve` and rebooting into our kernel should be the only step needed.

For example, to start testing on Ubuntu 18.04:

sudo add-apt-repository ppa:valve-experimental/kernel-bionic
sudo apt install linux-mfutex-valve
sudo reboot

Testing instructions:

Mostly we are trying to make sure all games work fine with fsync; a slight CPU use reduction might be observed during testing.

If using Proton 4.11, fsync is enabled by default if the right kernel support is detected. You should see console messages in the form of:

fsync: up and running.

To narrow down any new issues, the PROTON_NO_FSYNC=1 variable can be set to revert to esync even when an fsync-capable kernel is installed.

If you can reproduce a crash or performance regression that consistently goes away with PROTON_NO_FSYNC=1, please report it in the dedicated fsync issue[github.com].

Post any performance results with 4.11 and fsync toggled on/off to the performance results issue[github.com].

[donmartio]

Thanks for the clarifications. I fixed the typo and missing sudo commands (there was one more).

[kwahoo]

Generic or Ubuntu compilation using patches from AUR:

wget https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.2.8.tar.xz
git clone https://aur.archlinux.org/linux-fsync.git
tar xvf linux-5.2.8.tar.xz
cd linux-5.2.8/
cp /boot/config-$(uname -r) .config
make menuconfig
patch -p1 < ../linux-fsync/futex-wait-multiple-5.2.1.patch 
patch -p1 < ../linux-fsync/futex-Consolidate-duplicated-timer-setup-code.patch 
make -j16
sudo make modules_install
sudo make install
cd /lib/modules/5.2.8/
sudo find . -name *.ko -exec strip --strip-unneeded {} +
sudo update-initramfs -c -k 5.2.8
sudo update-grub

Początkowo opublikowane przez engie❤cat:

make -j12 is an overkill if you have less than 12 cpu cores. oversaturating the job scheduler results in slower compile times, typically recommended number is number of cores+1.

This may be true for Intel chips, but not for Ryzen where SMT scales much better. Numbers for my Ryzen 1700 (8c/16t):

time make -j16
real    16m6,696s
user    189m55,285s
sys     23m16,853s

time make -j9
real    20m43,030s
user    148m56,788s
sys     18m39,866s

[lucifer]

make -j* refers to virtual cpu cores (i.e. threads) not physical cores I'm fairly certain. So your 1700 would be used the best with make -j16. I'm sure that's what engie❤cat was referring to.

Just like how my 4-core HT has better compilation times with make -j8/ rather than -j4

[kwahoo]

Początkowo opublikowane przez lucifer:

make -j* refers to virtual cpu cores (i.e. threads) not physical cores I'm fairly certain. So your 1700 would be used the best with make -j16. I'm sure that's what engie❤cat was referring to.

I don't think so. donmartio uses 6 core Ryzen and all 6-core Ryzen have 12 threads.

[engie❤cat]

Początkowo opublikowane przez kwahoo:

This may be true for Intel chips, but not for Ryzen where SMT scales much better. Numbers for my Ryzen 1700 (8c/16t):

time make -j16
real    16m6,696s
user    189m55,285s
sys     23m16,853s

time make -j9
real    20m43,030s
user    148m56,788s
sys     18m39,866s

this is true regardless of cpu manufacturer: if there's more jobs than threads that your cpu can efficiently handle - there's going to be a slowdown in the scheduler. that number, which i have (not stellarly accurately) referred to as the number of cores, can be found with

grep processor /proc/cpuinfo | wc -l

here's my numbers for a 4-thread processor ran on a nox machine, with mrproper cleanup and reboots after each compilation, with nothing else running but compiler:

# time make -j4
real    11m42.784s
user    38m9.725s
sys     4m29.266s

# time make -j5
real    11m41.300s
user    38m32.541s
sys     4m27.329s

# time make -j12
real    12m7.158s
user    40m3.313s
sys     4m39.405s

as evident, the results are slightly (but not really noticeably) better with -j5, but -j12 increases compile time by almost half a minute, and that's with my optimized config. try your ryzen with -j32 and see what happens.

[®omano]

Isn't the 'rule' to max performance to do number of cores or threads - 1 (this way you also have a 'responsive' system not frozen)? I remember using all of my cores/threads to compile made my system unresponsive and eventually crashed.

[x_wing]

Początkowo opublikowane przez ®omano:

Isn't the 'rule' to max performance to do number of cores or threads - 1 (this way you also have a 'responsive' system not frozen)? I remember using all of my cores/threads to compile made my system unresponsive and eventually crashed.

Do you mean that you had a kernel panic or you just decided to reboot?

[kwahoo]

Początkowo opublikowane przez engie❤cat:

this is true regardless of cpu manufacturer: if there's more jobs than threads that your cpu can efficiently handle - there's going to be a slowdown in the scheduler. that number, which i have (not stellarly accurately) referred to as the number of cores, can be found with

You criticized donmartio for using -j12 on 6c/12t CPU.

[engie❤cat]

Początkowo opublikowane przez ®omano:

Isn't the 'rule' to max performance to do number of cores or threads - 1 (this way you also have a 'responsive' system not frozen)? I remember using all of my cores/threads to compile made my system unresponsive and eventually crashed.

no. performance and interactive responsiveness are two different beasts, don't mix them up. more importantly, if your system's gone unresponsive and then crashed - that's not a scheduler overhead issue, it's an out of memory (OOM) issue. you've likely tried to run a memory-heavy compilation process with eleventy billion tabs open in the background, leading to a system freeze and eventual termination (or crash) of the fattest process, causing the dependencies of that process to misbehave.

Początkowo opublikowane przez kwahoo:

You criticized donmartio for using -j12 on 6c/12t CPU.

no. I have given a general advice not to oversaturate the job server, given that the majority of users use quad core processors, some without even hyperthreading. many people don't know what -j# flag even means, and -j12 is not a good default. you could use -j$(grep proc /proc/cpuinfo|wc -l) instead, and bash would unpack it to set the right amount of jobs regardless of the processor used. this is for everyone's information, not to be a jerk, you've got it wrong.

[L Y N N]

OWW THANKS

[Aoi Blue]

Początkowo opublikowane przez kwahoo:

Początkowo opublikowane przez engie❤cat:

this is true regardless of cpu manufacturer: if there's more jobs than threads that your cpu can efficiently handle - there's going to be a slowdown in the scheduler. that number, which i have (not stellarly accurately) referred to as the number of cores, can be found with

You criticized donmartio for using -j12 on 6c/12t CPU.

However, since none of the jobs are going to use 100% of a thread's capability 100% of the time, it's often better to set it to slightly higher than your thread count.

The exact count depends on your CPU's hyper-threading efficiency and your particular compile job's efficiency.

Typically on many modern hyperthreaded CPUs only FPU and well packed SSE instructions will ever run aground in core sharing.

1 job per a core/thread would work fine. However 1 job more than that is usually the standard in order to fully utilize resources.

[Marlock]

Początkowo opublikowane przez TOVOT:

ok that the discussions of Steam are not really the most serious place in the world by definition, but your comment, ina discussion where those who register want real updates, what should it be used for?

don't reply to these empty posts, just report them as likely spam or bot...

helps Steam algorithms figure out which profiles are created only for that kind of useless purpose (as they get flagged often and generate no real interaction with normal users)

edit: also replying add to the amount of noise for other subscribers... as does my response now too... and so on if anybody else chimes in over this... (sorry everyone!)

[TOVOT]

Początkowo opublikowane przez Marlock:

Początkowo opublikowane przez TOVOT:

ok that the discussions of Steam are not really the most serious place in the world by definition, but your comment, ina discussion where those who register want real updates, what should it be used for?

don't reply to these empty posts, just report them as likely spam or bot...

helps Steam algorithms figure out which profiles are created only for that kind of useless purpose (as they get flagged often and generate no real interaction with normal users)

edit: also replying add to the amount of noise for other subscribers... as does my response now too... and so on if anybody else chimes in over this... (sorry everyone!)

You are right, if any spam is answered, it creates double the noise, I am good-bye
but I have grown tired of making the Internet a better place, in my opinion (and maybe it will seem excessive), you should be able to access the Internet with your identity card (electronic of course), and only an account (via ID card) in each site, no more than one

[Aoi Blue]

Początkowo opublikowane przez TOVOT:

Początkowo opublikowane przez Marlock:

don't reply to these empty posts, just report them as likely spam or bot...

helps Steam algorithms figure out which profiles are created only for that kind of useless purpose (as they get flagged often and generate no real interaction with normal users)

edit: also replying add to the amount of noise for other subscribers... as does my response now too... and so on if anybody else chimes in over this... (sorry everyone!)

You are right, if any spam is answered, it creates double the noise, I am good-bye
but I have grown tired of making the Internet a better place, in my opinion (and maybe it will seem excessive), you should be able to access the Internet with your identity card (electronic of course), and only an account (via ID card) in each site, no more than one

Yep, just flag the spam and move on.

[Bago Team]

i cant get any Kernel above 5.0 to work with Ubuntu 18 MESA Driver Performance almost drop by 50% or even more in every game from i change from 4.18 to any kernel 5.0+