A Brief History of Swapchain Time (Why High Mouse Polling Rates Will Kill Framerate)

[Kaldaien]

I have reached the conclusion that the Raw Input mouse polling rate problems I have been struggling with for the better part of half a month are unsolvable without divine intervention, so consider this little rant some kind of pre-cursor to a technical blog post / call to action to get Microsoft involved or to get game developers to use Raw Input the correct way :)


Direct Input is layered on top of Raw Input on Windows 10 (and presumably 8 and 7 as well). It spawns a dedicated thread and creates a child thread with a message pump and uses the message-based WM_INPUT rather than buffered Raw Input read API.

In Fullscreen Exclusive / "Game" mode, Microsoft introduces a revised scheduling strategy that unfairly biases CPU time to the thread that owns the swapchain.

As I understand it, the way fullscreen mode optimizes this stuff, the swapchain thread gets increased scheduler timeslices. This is in contrast to normal UI optimization strategy, where the foreground window thread gets a priority boost but is subject to the same amount of CPU time per-scheduler interval as its siblings.

Here in lies what I think is the crux of the problem. Modern game engines do not do anything particularly important on the swapchain thread; many are doing multi-threaded rendering.

The swapchain thread arguably does not need longer timeslices in a state-of-the-art D3D11 / D3D12 engine that is using multiple threads working in tandem to stage the drawing command streams. What the swapchain thread actually needs is ...... a higher priority and an event or signal for the thread to wait-on. The swapchain thread should find other work to do or simply sleep untill signaled.

So, the whole D3D12 = Flip Model + Windowed Mode design that seemed weird at first actually makes a ton of sense and fullscreen exclusive mode is arguably doing things that games don't need in a modern engine.

Anyway, the Fullscreen Exclusive swap chain thread is hindering the ability of Direct Input to actually get the Raw Input events it reads off of its queue. Things will be going fine for a while, with maybe only 15 to 20 events piling up over the course of a single 60 FPS (16.6667 ms) frame. But if the queue accumulates more than 20 events, things pile up at an exponential rate and I've observed many frames where there were 300 or more mouse events to drain from Direct Input's sink before input latency goes back to normal.

Microsoft could possibly implement a dampening system to smooth this all out. It's critically important to get the cluster@#$% of backlogged mouse move events cleared, but ... the more time Direct Input spends clearning these events during a frame, the more likely it is going to disrrupt other game-related threads and now you just lost framerate because (of all the silly things) you moved your mouse too fast !!

Something ha gotta be done about this, we are trading input latency for framerate, when the simpler solution would be to introduce mouse polling rate as an official configurable part of Windows.

-------------

Here is the bottom line -- there is no user or developer-facing mechanism to control the sample rate of an input device in Windows.

Your gaming mouse may have a (probably buggy) and (probably clunky) utility bundled with it that can change the mouse's sampling rate, but this needs to be an official part of Windows and not a driver hack.

-------------

I can detect when the sample rate is too high and issue a warning to the user, but that is really the best I can do for the PC gaming community. You either run your games in windowed mode such that mouse polling is more consistently performed or you turn your mouse sample rate down.

Under no circumstances should you crank your mouse's sample rate up to its maximum and then run a game in fullscreen exclusive. That is the anti-pattern that demonstrates why certain behavior that produces high peak framerates also produces catastrophic stutter. Windowed mode wins this race with nice steady input vs. render timing even if your peak FPS is never as high.

Now for the really ironic twist

IF YOU HAVE A G-SYNC MONITOR. Fullscreen G-Sync + >= 1000 Hz Mouse Sample Rate = EVIL. I will fight anyone to the death who tries to claim otherwise :)

[Kaldaien]

Round 2: What happens if you let keyboard input bungle up the event loop?

From my casual analysis of Monster Hunter: World, I present the keyboard analog of a mouse that's polled too quickly

Idle (if I engage my framerate limiter, the graph is flat as Kansas and green as grass)

https://steamcommunity.com/sharedfiles/filedetails/?id=1583467662

Holding any key down (believe me, THIS is noticeable, holy crap!)

https://steamcommunity.com/sharedfiles/filedetails/?id=1583467913

Non-defeatable keyboard repeats produce a similar problem to a mouse that won't stop force-feeding the game with small movement events. Just like with the mouse, Fullscreen Exclusive mode is problematic.

UNLIKE the mouse problem, keyboard repeat rate has been configurable in Windows for its entire existence. You could theoretically solve this problem by reducing key repeat rate, but that's a system-wide setting so also not the ideal solution we need.

Here is the final solution to the problem I have been working on

https://steamcommunity.com/sharedfiles/filedetails/?id=1583471990

That's with me holding the same keyboard buttons down and it's no longer causing the game to lose its freaking mind.

• Flip Model[blogs.msdn.microsoft.com] is a Windows 8 / 10 feature, it is pure magic for solving this problem and SK can retrofit it into just about any D3D11 game running on Windows 8.1 or higher.