NVENC works beautifully - When it works

[Greg]

After seeing the beta client update notes I was pretty excited to try out the new NVENC encoder path. Unfortunately I tried just about every D3D9 I could and none of them would use the NVENC encoder except the source engine mod "Lambda Wars".

In lambda wars my display latencies were around 13-14ms which is phenomenal for not using IFR or FBC.

Is there a reason NVENC wouldn't work with other D3D9 games?

I'll post some streaming_log's shortly.

[salxicha]

Hey Henryg

I believe its time for a better GUI on both client and host in-home streaming tabs.

If could choose the encoding method easily it would be a nice way to perform tests.
I suggest using something like OBS (Open Broadcaster Software) where we can easly select the parameters, FPS and even the " intensity " of the encoding process.


For reference:
https://jp9000.github.io/OBS/settings/encodingsettings.html



As for OBS, in order to have NVENC providing similar image results to x264 we need to configure NVENC recording as 30 to 40 mbs (30000 / 40000).

How this works for Steam? NVENC is more bandwidth consuming?

[Greg]

This is great news. Every game that previously defaulted to software rendering now properly uses NVENC. I haven't noticed the aforementioned quality problems reappear even with a rather long play session either. Thanks for the late night build guys

So, the only real way to compare performance across different paths is to use a high-speed camera with a side-by-side monitor setup. The 240 FPS "slo-mo" mode on recent iPhones is actually quite good for this :)

I think you just sold me on a new iPhone better than Apple ever has.

If could choose the encoding method easily it would be a nice way to perform tests.
I suggest using something like OBS (Open Broadcaster Software) where we can easly select the parameters, FPS and even the " intensity " of the encoding process.

I honestly think this is a bit much and would honestly probably cause more problems then they would help. It would also take much longer to implement these things as custom parameters and create a nice GUI around the whole thing.

A simple reorderable list where we could set the encoder priority would be a HUGE leap in the right direction. To be completely honest as much as I like NVENC the idea of it taking precedence over Async + NVIFR scares the living crap out of me. Something like this would definitely ease the transition.

[werner]

NVEnc so far is the best encoding method. The graphics quality is awesome just the same as local and almost no frame loss compared to non streaming. Good stuff.
Btw. I second that we need a better gui to enforce the encoding method per game if not for anything else than for testing!

[Magbed@Linux! 🐧]

For me its always using NVFBC did you enable anything or install anything extra from the nvidia driver , im running a 680GTX. Tried with old and new games and they all use the same. What game are you trying NVENC with

[Greg]

For me its always using NVFBC

You would have to disable NVFBC first. There are some scripts posted a few weeks back on the in-home streaming board to disable it easily.

[Kaldaien]

I am finding that NVENC does not want to work at all anymore on my system. The data path is now DWM or "d3d11 polled" and encoding is done by lib h264 on the CPU with 4 threads. Image quality is better, but lord oh mighty is framerate sloppy.

I don't know if it's related, but I wrote a project that converts DX10/11 games to use the DXGI Flip Presentation Model for improved DWM performance. I've seen other software on my system properly use NVENC for frames captured even when the games are using my DLL to retrofit flip model, so I'm at a loss.

I do know that Steam's overlay doesn't like the flip presentation model at all and I had to add a second SwapChain::Present (...) call to get its hooks to work right. I don't think the overlay hooks Present1 (...).

Maybe something could be done about that problem while you're at it? :P It's awkward doing a no-wait Present with a zero-area region just to get Steam to draw its overlay at the end of a frame. This problem is only going to get worse with DX12 and Windows Store apps, which are required to use the Flip Presentation Model.

[killwhitey]

I don't know why you don't just give people the options to choose their encoding path at the backend of Steam?

[Kaldaien]

killwhitey の投稿を引用：

I don't know why you don't just give people the options to choose their encoding path at the backend of Steam?

Very good question. I've been tempted at times to use the NDA version of NVAPI and implement this stuff myself since for months at a time in-home Streaming works great and then it all goes to hell. I'd like something that doesn't change constantly and I'd GET that if I said the hell with it all and did this myself.

[Greg]

Very good question. I've been tempted at times to use the NDA version of NVAPI and implement this stuff myself since for months at a time in-home Streaming works great and then it all goes to hell.

You could always become a contributor to the moonlight project. I'm pretty sure it started out the same exact way.

[Kaldaien]

< Greg の投稿を引用：

Very good question. I've been tempted at times to use the NDA version of NVAPI and implement this stuff myself since for months at a time in-home Streaming works great and then it all goes to hell.

You could always become a contributor to the moonlight project. I'm pretty sure it started out the same exact way.

I looked at that project in the past. There's very little I could do because it's mostly Java and I don't know the first thing about Java. I barely know C++'s standard library, my work basically covers C and the C standard library. Sort of backed myself into a corner with my career path, I think -- with all these popular and unfamiliar languages like Java, Ruby :P

[Greg]

I looked at that project in the past. There's very little I could do because it's mostly Java and I don't know the first thing about Java. I barely know C++'s standard library, my work basically covers C and the C standard library.

Java is C++ without pointers, it's pretty simple if you can read C/C++. I actually thought the same thing as you though (The project is in java) but it's actually just the client frontend that is written in java. The actual capture and decode logic is in C and bound to java via JNI.

Here's the directory for their decoder files for example -
https://github.com/moonlight-stream/moonlight-pc/tree/master/jni/nv_avc_dec

As much as I hate managed languages I've seen a steady decrease in C/C++ developer positions and a massive increase in Java positions. As a C/C++ programmer this makes me sad, as a guy who knows 8 different programming languages, this really just makes me a lot of money. It'd probably be more than worth it to try to pick Java up.


To be completely honest I've thought about this, A LOT. (writing my own streamer) Getting the capture "server" to work doesn't seem all that hard. Surprisingly it's the client that sounds a lot harder to write. As well as all of the issues related to window focus, mirroring input, network prediction, error handling, dealing with ♥♥♥♥♥♥♥♥ quirks (read: bugs) with the Nvidia drivers.

[Kaldaien]

Oh, that's pretty cool. But after poking around, I don't actually see anything that takes advantage of NVENC. This is a pretty common problem in open source projects because most useful features NVIDIA GPUs provide are walled off by an NDA. This makes them extraordinarily unfriendly to the open source community and is a constant cause of frustration. You sign your soul away to NVIDIA to use anything special on their hardware :-\

This actually just wraps itself around the protocol NVIDIA already devised for Shield streaming. I was thinking a bit more grandiose in scale, wanting to handle the encoding myself with every conceivable option tunable :) I'm completely turned off by the 3 radio buttons and 2 check boxes Valve passes for in-home streaming configuration, especially when from version to version those settings have different meanings :P

[Kaldaien]

Yeah, I probably really do need to pick-up Java. I'm slow to adopt new language features though and some of the crazy operators and keywords in Objective C will forever confuse the hell out of me. Lots of things in Java and Obj-C code and even new stuff added to c++11 look like parse errors to my eyes :P Just reading the code becomes an awkward affair full of "what the hell is a @ doing here?"

[Greg]

Kaldaien の投稿を引用：

Yeah, I probably really do need to pick-up Java. I'm slow to adopt new language features though and some of the crazy operators and keywords in Objective C will forever confuse the hell out of me. Lots of things in Java and Obj-C code and even new stuff added to c++11 look like parse errors to my eyes :P Just reading the code becomes an awkward affair full of "what the hell is a @ doing here?"

I'll admit Objective-C was hard to pick up but after 3-4 months it became second nature. There's actually a lot of nice things about objective-C that I just don't find in other languages. The bracket syntax is scary at first but it makes complete sense in the end. I actually think it looks cleaner when you start having a lot of nested calls.

@"" signifies that it's an objective-C string class and not a primitive C string. Objective-C is just a layer on top of standard C so anything you can do in C still applies.

This allows you to treat the string as a fully qualified class and do things like this:

NSString *helloWorld = [@"Hello " stringByAppendingString:@"World"];

[werner]

coming back to NVENC Rocket League crashes in NVENC encoding and takes Steam with it if you hit the goal with the ball (first tutorial lesson). Steam Link Client, latest beta of Steam Server. NVidia 970 card.