Streaming from a virtual machine

[Z3R0]

So inspired by Linus Sebastian's 10 gamers-one-pc video, I set about to create a gaming virtual machine. I bought a Dell T5500 workstation with a Xeon E5620 (roughly equivalent to an i7-920), installed ESXi 6.0 and dropped in a GTX 670. The gaming VM (4 vCPU, 8GB RAM) works well when gaming in front of the workstation, but streaming from it stinks. There are frequent stutters, occasional "Slow network" warnings, and its just generally unplayable.

A different PC, with an i5-3570K and a Quadro FX 3800 connected to the same WiFi router (Cisco E4200) via Gigabit ethernet streams just fine, though I can't play DX11 titles on the old GPU. I'm streaming to an HP 6460b laptop about 20' and one wall away. Can anyone tell me if there's anything I can do to improve performance?

[ReBoot]

Well, does your virtualizer and your hardware support GPU pass through? You need that for performance.

[Z3R0]

Yes, as I said games play fine on a monitor attached directly to the VM host. Its when trying to stream that performance is awful.

[ReBoot]

Well, how are you encoding? Does your virtualizer pass the encoding unit through?

[kreiselhoschi]

It doesn´t and what Bananaman tries to do absolutely makes no sense. Streaming w/o virtualisation using the original setup and the GTX670 would work well with IHS....

[Z3R0]

Why doesn't this make sense? I want to use the PC for things other than gaming, like running multiple Linux and Windows servers simultaneously along with a gaming VM. This particular workstation can have up to 12 cores and 24 threads so I'm trying to prove that it can work first, then I'll upgrade my CPUs. It also works nicely with ESXi to have a console on the workstation itself, instead of having to connect remotely all the time.

I'm half way there because I got GPU passthrough to work and can now play games on ESXi. But I'd like to be able to play games from my laptop or tablet with integrated graphics anywhere in the house. I'd also like to retire/sell the 3570K so I don't have so much heat in my office.

My Advanced Hosting settings are as follows:
Change desktop resolution to match streaming client - Enabled
Dynamicall adjust capture resolution to improve performance - Enabled
Use NVFBC capture on Nvidia GPU - Enabled
Hardware encoding - Enabled
Hardware encoding on Nvidia GPU - Enabled
Hardware encoding on AMD GPU - Enabled
Hardware encoding on Intel GPU - Enabled
Number of software encoding threads - Automatic
Prioritize network traffic - Enabled

I realize that the AMD and Intel encoding are unneccessary, but I didn't figure there was any harm enabling them.

[ReBoot]

Does the stream actually use the HW encoder? I honestly don't think so. Have you checked?

[Z3R0]

The log is a bit hard to understand, but I got this from the log on the VM trying to stream the game:

[2017-05-20 02:39:21] >>> Capture method set to Desktop BitBlt RGB + libyuv + scale + NVENC H264
[2017-05-20 02:39:21] >>> Capture resolution set to 1366x768
[2017-05-20 02:39:21] >>> Capture method set to Desktop BitBlt RGB + libyuv + NVENC H264
[2017-05-20 02:39:21] >>> Capture resolution set to 1084x642
[2017-05-20 02:39:21] >>> Capture method set to Desktop DWM NV12 + NVENC H264

The NVENC H264 indicates that the Nvidia encoder is working, right?

[kreiselhoschi]

I apologise, with the additional infos you provide it now makes sense (although I still don´t understand why you are trying to use a Quadro GPU for gaming purposes...).^^

NVENC indeed indicates that HW acceleration now is working.

[Z3R0]

After some more research and testing, I determined that it is actually a network problem. I'll either have to upgrade my router or make sure there is absolutely no other activity than game streaming on wifi.

[Grimm]

if you are using KVM this is a good, guide https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF

Basicly cpu pinning, hugepages.

oh and use the steam beta client on client machines (not the host), there is a stutering bug on linux so it seems.

[Z3R0]

Nope, using ESXi. I did some research, and it's a good bit simpler to set up with VMWare. I may try it at some point with KVM for the learning experience, but for now I just wanted to get it running. Plus I'm running a non-UEFI video card, which complicates matters with KVM.