Streaming Big Picture to Ubuntu

[stackman]

While we're waiting for the official Steam streaming to appear, I managed to hack together a version that works well enough for those interested.

I used Splashtop to stream the video, but when going to Linux there was an unnaceptable audio lag. Going to Windows on the same machine had worse video and audio performance. To get around that I used Netcat to send an audio capture from LiveInCode to PulseAudio. That brought the delay down to under a second - noticable in videos but good for gameplay. I captured the controller inputs with a program called VirtualHere and sent it to from Ubuntu to the Windows box. There's no noticable lag there.

My gaming PC is running SLI 460's and an i7 930 with 12 GB RAM and a 240 GB SSD. The Ubuntu PC was given to me from a friend who thought it was broken. I traded a pint of ice cream to a dude in a computer store for parts to get it working again - not exactly top of the line. It's a Celeron 900 @ 2.20 GHz with Intel integrated GM45 graphics and 2 GB RAM.

I guess the point of this is that a spare laptop is enough to stream. I was even able to get things working by streaming the video and audio to a Motorola Photon 4G, although I still had to route the controller through the laptop. Anyone with a Raspberry Pi should be able to recreate something near what I have with few problems.

Also, all of the software used to do this, with the exception of the games themselves, was either open source or offered unlimited trials. So if you have a spare laptop and want to try this out yourself, all you need is time and patience.

http://www.youtube.com/watch?v=JLPPX7dSuhg

[XÆЯO_Vince]

Nice streaming set-up. My current streaming set-up on Linux is with a modified version of Teamviewer 9 and Synergy (to properly lock mouse & keyboard context).

I originally used Splashtop for Linux but noticed that its in beta and for some reason had not only bad audio latency but the streamed video was only coming in at about 15 FPS. What tweaks did you use to get higher video FPS in Linux Splashtop?

Using Teamviewer 9, however, I get about a half-second audio delay and 25 to 30 FPS for the most part--although streamed video quality does suffer with noticable compression artifacts; No input latency either over my home Wifi network. The only problem with Teamviewer is audio doesn't work out of the box and the Linux verson happens to use Wine as its foundation and so that verson of Wine it uses has to be replaced with another one with working audio--I've managed to do that pretty easy enough.

I will have to check out VirtualHere to get a gamepad to work as I'm only using mouse and keyboard at the moment.

[stackman]

Thanks. I tried just about everything until I hit the sweet spot with this, so I figured I'd share it.

To fix the audio problem I just disabled the audio in splashtop and set up a separate stream from my gaming PC for audio. The audio latency in splashtop was SO bad that I figured any other solution would be better. There are a number of proprietary solutions, but for me just using Netcat to make a direct connection from Windows to Linux and then sending audio data over was the fastest. The great thing about Netcat is that it just doesn't care what data you're sending - it just goes. There's still some latency, but it's not enough to be an issue.

Right now I'm getting a good 30 FPS, but I'm trying to find some tweaks to improve that. Grab Uncompyle2 to look at what's inside the splasthop .pyc files if you're so inclined, that might help. Also, VirtualHere is great - it creates a virtual USB connection for anything - I could just as easily use a mouse and keyboard as a gamepad. Latency is a complete non-issue with it.

[XÆЯO_Vince]

What Netcat command strings did you use for the Windows and Ubuntu Netcat to capture and send audio? Not really familiar with Netcat myself but I'd like to try it and compare to what I have now.

[stackman]

Check out ubuntuforums.org/showthread.php?t=1121603 comment #7. That's pretty much what I did. I made a .zip with linco and netcat and a little instruction file. I'll upload that sometime tomorrow, but it's just basically what they're talking about in that thread.

[Balderick]

Thx for sharing and discussing this.

I had been living in hope Valve were going to make a late announcement at CES regarding In Home Streaming but alas not to be. http://www.bbc.co.uk/news/technology-25665902
There seems to be more and more reports of Valve's In Home Streaming will be using the same technology as nVidia Shield/ nVidia Experience technology which is not a streaming solution for everybody or most machines so am waiting with bated breath on what Valve's solution really is for In Home Streaming.

[myro]

I used briefly splashtop from windows to windows and noticed the following problems(tested with Guild Wars 2):
- Artifacts on quick turns.
- FPS drops below 20 FPS on quickly changing scenes on the client (host running at 140 FPS).
I used a i7/660gtx as host which supports Intel QuickSync and NVidia's NVENC as hardware transcoder. Unfortunately, I have no idea which hardware transcoder splashtop prefers.

Did you encounter similar issues?

@[T40-1]PvtBalderick: Valve might start with NVidia's NVENC for beta tests, but I would guess that they will support AMD's VCE and Intel QuickSync in the long run.

[stackman]

I have Win7 and Ubuntu on my laptop and running the same hardware I saw a substantial framerate improvement on the video of Splashtop for Ubuntu vs Windows. I can't give you any hard numbers because I didn't spend much time testing it in Windows since the difference was so immediately noticable. Anyway, I needed Linux in order to run VirtualHere to get my gamepad working the way I wanted it.

I did notice, though, that there were different audio issues in Windows. In Ubuntu the audio is just out of sync by a substantial amount - unusably so - from the moment you load it. In Windows the audio starts out synced but gradually gets worse as you go.

Honestly, if you've got the time and inclination it might be worth your while to throw Ubuntu onto some flash memory and boot from that to give it a try.

[stackman]

Just a quick update. I was able to tweak a registry setting for the Splashtop streamer that seems to dramatically improve the FPS of the stream. I don't know what the computer equivilant of the butt dyno is, but that's all this is. From a few tests this just seems substantially improved, I don't have actual numbers. The tradeoff seems to be a slightly increased audio lag - everything else is identical.

So to summarize: last night I got 720p and 1080p streaming working consistently with very minor artifacts and barely noticable audio lag at 30 FPS. As of now I've got 720p with very minor artifacts running at what feels like (and should be, according to the edits I made) 60 FPS and slightly more noticable audio lag. A note on my network topography: my gaming computer is connected to my router through gigabit ethernet and my laptop is connecting wirelessly with 802.11n on the 2.4GHz band. The laptop is approximately twenty five feet from the router with one wall in between.

[myro]

İlk olarak stackman tarafından gönderildi:

I have Win7 and Ubuntu on my laptop and running the same hardware I saw a substantial framerate improvement on the video of Splashtop for Ubuntu vs Windows. I can't give you any hard numbers because I didn't spend much time testing it in Windows since the difference was so immediately noticable. Anyway, I needed Linux in order to run VirtualHere to get my gamepad working the way I wanted it.

I did notice, though, that there were different audio issues in Windows. In Ubuntu the audio is just out of sync by a substantial amount - unusably so - from the moment you load it. In Windows the audio starts out synced but gradually gets worse as you go.

Honestly, if you've got the time and inclination it might be worth your while to throw Ubuntu onto some flash memory and boot from that to give it a try.

I already have ubuntu (well, actually linux mint) on my laptop. I just assumed that win -> win would give better results, without considering to try linux.

I'll do that thanks.

PS.: Could you link/post your registry tweak? Thanks.

[stackman]

I'll post an archive with the reg tweaks and hard to find programs along with step by step instructions a little later - can't get to it just now.

[XÆЯO_Vince]

I recorded a video of my streaming set-up in Archlinux and will be posting it on Youtube shortly. It should work equally well on SteamOS and any other distribution.

I look forward to seeing stackman's how-to though. I'd like to try Splashtop again if I can get decent FPS from it.

[stackman]

My wife's flight is delayed so I have some time to write up a quick how-to.

Who This is For

This is for someone running a gaming PC with Steam on Windows who wants to stream HD content over a local network to a low-power Ubuntu device. I'm going to go ahead and assume that you have enough knowledge of Ubuntu to modify protected files without breaking anything and enough knowledge of Windows to modify the registry without breaking anything. I'm also assuming that you know your IP addresses.

Here's the software you'll need:

Steam PC

• Splashtop Streamer You'll use this to stream video from your Steam PC to Ubuntu. If the audio works well enough for you then this will be all you need for streaming.
  
• VirtualHere Download the Windows Client. This will read the inputs from your gamepad on Ubuntu
  
• LiveInCode This is the first piece of the simple audio stream that you'll set up to your Ubuntu computer.
  
• Netcat This will open a connection to your Ubuntu computer that you'll send the audio from LiveInCode through.

Linux PC

• Splashtop Client This will display your video stream from your Steam PC. It's in Beta and it's still a little quirky. Oftentimes the stream won't want to start, but eventually it will connect.
  
• VirtualHere You'll download the server software. This is going to send the inputs from your gamepad (or any attached usb device) to Steam.
  
• wmctrl If you have problems setting Splashtop Client to fullscreen this will fix that for you.
  
• PulseAudio This is built into Ubuntu and it will receive the audio stream from Netcat and LiveInCode.

Step 1

Install Splashtop on both machines and verify that it works. Check the audio quality and lag to determine if you'll need to use Netcat and LiveInCode.

Step 2

Install VirtualHere on both machines and verify that it works. If you are unable to connect to the VirtualHere server from your Windows machine, manually input your Ubuntu machine's IP address in the VirtualHere configuration. Also, check the ports.

Step 3

If you are not happy with the audio from Splashtop ensure that Splashtop Streamer is set to send audio to the local computer only extract both LiveInCode and Netcat into the same directory on your Steam PC. I just use c:/LiveInCode. In that directory, create a .bat file with the following:

linco -B 16 -C 2 -R 44100 | nc <Your Ubuntu PC IP address> 5000

I named the file Win_to_Lin_audio.bat, but you can name it whatever you like.

Step 3a

Set your audio Recording device to "System Audio Driver" or "What U Hear" or whatever your system has that routes all audio back to the system. If you don't have or can't find anything, check Airfoil. It ships with a System Audio Driver that can be installed.

Step3b

On your Ubuntu computer open the /etc/pulse/default.pa file and add the following to the bottom of the file:

load-module module-simple-protocol-tcp port=5000

Restart your Ubuntu computer.

Step3c

Run the .bat file you created at the beginning of Step 3 and verify that it is routing audio to your Ubuntu PC. This will route ALL AUDIO from your Win PC to your Ubuntu PC. I'm not saying you should use the microphone on your PC to convince your significant other / roommates that your Ubuntu PC is haunted, but you could.

*Most of Step 3 can be done from your Ubuntu machine through Splashtop to save you having to run back and forth.

Step 4

On your Ubuntu PC open the following file: opt/splashtop-client/SRSrvdlg.ui. At around line 50 you will see two resolution settings. Modifying these two settings will allow you to choose those two resolutions from the Splashtop Client Launcher. IMPORTANT NOTE: Splashtop Client will only accept 2 options. You can modify the options present, but cannot have more than 2 options. OTHER IMPORTANT NOTE: Ensure that your Steam PC is able to output the resolution you want Splashtop Client to run at. If you're using a 4:3 monitor and want to output to a 16:9 HD TV make sure that you set up a custom resolution for your video card or Splashtop Client will not function properly.

Step 5

On your Steam PC download the fps.txt file from the bottom of this[support-splashtoppersonal.splashtop.com] page. Rename the file to fps.reg and run it. It will add a registry entry to your computer at

HKEY_CURRENT_USER\Software\Splashtop Inc.\Splashtop Remote Server 

Run regedit and navigate to that location. There will an entry called SharpFPS. Modify the value from 5 to 60 in decimal.

Step 6

If you are unable to get Splashtop Client to run in fullscreen, install wmctrl and run the following from the terminal:

wmctrl -r <the name of your Splashtop window> -b toggle,fullscreen

That should do it. I've found, so far, that 720p is the best combination of quality and performance. Your mileage will, of course, vary. I hope this helps and I'll be making a YouTube video going through all of this sometime this week.

[XÆЯO_Vince]

Wow...as soon I applied that SharpFPS registry key and changed it to 60, Splashtop client streams like a hot knife thru butter--super smooth FPS upto 60 FPS even. I get a solid 30-40 FPS now in Bioshock and it feels slightly smoother than even with TeamViewer. This rocks! I also changed the default resolutions and will now try wmctrl to enable full-screen.

The audio latency for some reason seems to be acceptable about roughly the same as with TeamViewer. Maybe its because the distro I'm using defaults to Alsa rather than PulseAudio, which has been known to increase latency in certain cases. I don't think the NetCat method will be universally needed, perhaps only on distros that have PulseAudio.

Do you think VirtualHere would work with a bluetooth PS3 DualShock 3 controller or does it have to be wired USB gamepad? What if the DualShock 3 was plugged in with its USB cable?

[Balderick]

ref ps3 bluetooth controller http://gaming.stackexchange.com/a/118018 the cable is for charging only. Switching bluetooth off in OS and using controller with cable will verify.

[stackman]

You'll be able to get a PS3 controller working if you have a bluetooth dongle connected to your VirtualHere server's USB port. VirtualHere will route the data from the USB to the client so it will appear on your client computer as a local bluetooth connection - just as if you had plugged it into your client computer directly. If bluetooth is built in to your server computer then you would need a USB dongle.

I'm glad that you got Splashtop running better! It's great to see that Steam Streaming will only need a thin client to work well.