Unity question: how to stream a VR Unity scene over WiFi?

[piwakawaka]

The following used to work, but in the last few days something has changed, and now it doesn't.

I've been experimenting with various developer workflows with Unity and the Quest, including the Link cable and Virtual Desktop (sideloaded via SideQuest) to stream SteamVR content to the headset over WiFi.

I have the Oculus Integration assets added to my Unity project, with a OVRPlayerController active in the scene. I do not have SteamVR assets added.

If I have the Link cable connected, I can hit the "Play" button in Unity and a few seconds later I'll be able to view my VR scene in the headset. This works pretty flawlessly, and I'm using is as my main workflow.

Last week I somehow managed to get this to work over WiFi (without the Link cable), but I cannot remember exactly how I did it. I'm reasonably sure, but not completely certain, that I was using Virtual Desktop and clicked "Play" in Unity with the Virtual Desktop Streamer app open, but no matter what I do today I'm not able to get this to stream, it simply shows the Unity IDE and the "Game" window, as it would on a monitor. I've tried with/without SteamVR loaded, but it doesn't seem to matter.

I'm 100% sure this worked last week.

Now when I hit Play in Unity, with SteamVR running (from VD), and then I right-click the SteamVR tray icon and choose "Display VR View", what I see is the SteamVR Home scene, unless I quit SteamVR Home first, in which case I just see the plains/mountains "lobby" instead.

I have my Unity screen divided into both graphical/3D views. I see a square briefly flash up on the Editor side (the viewport perhaps?), and the Game side shows the view from the camera rig initial position. But it does not move/respond to the HMD movement at all. It's not tracking the HMD. If I open "Display VR View" in SteamVR (right click the tray icon) then it shows a view of SteamVR Home (which is what I'd see if I put my HMD on), and that view is tracking the HMD.

I'm using Virtual Desktop 1.10.2 (sideloaded). Quest software 15.0.0.200.something (but without the new UI for some reason).

I'm a little lost for ideas at this stage.

[ggodin]

You probably have to inject Unity by right-clicking the Streamer icon in the system tray and clicking “Inject Game” then locating the Unity executable. I haven’t tested this btw, just my guess to make it work

[piwakawaka]

@ggodin - thank you for replying. I have actually tried this a few times, both injecting Unity and Unity Hub. Neither worked unfortunately. When I had this working last week, I'm almost certain I didn't do that either - it just seemed to work. I showed a colleague who agrees I wasn't dreaming - it definitely worked a few days ago. But no longer. I notice my Quest did upgrade itself to 15.0.0 recently, quite possibly in-between it working and now, which makes me suspicious of that.

[ggodin]

I'll put this on my list to test and I'll get back to you

[piwakawaka]

Thank you, I look forward to hearing from you in due course.

FWIW, here are some of the steps I've tried (amongst others) in the order executed:

* Powered down PC, powered down the Quest
* Booted up PC, open my Unity project
* Start Virtual Desktop streamer application
* Power up the Quest
* Open VD and connect to my PC - desktop is automatically displayed
* Press Oculus button on Left controller, to bring back the VD menu
* Point and click at "Launch SteamVR"
* After 10-20 seconds, this puts me into SteamVR Home
* Long-press the left controller Oculus button, select "Switch to Desktop"
* Click the Play button in Unity (can't see my mouse cursor so use one of the Touch controllers)
* Quick press the Oculus button on the left controller, select "Switch to VR"
* I'm back in SteamVR Home, not my scene.

I've tried lots of other combinations also, but this is the sequence I *think* I was doing last week when it was working for me.

[piwakawaka]

For the record - the one and only case I've managed to get working is to do a Windows build in Unity, then "Inject Game" the resultant build's .exe. So far this is the only way I've been able to get anything I create in Unity to show up in VR on the Quest via WiFi.

Not sure if it's relevant, but when the build's .exe starts up, it "fights" with SteamVR for a few seconds, rapidly flicking between the SteamVR content and my build's content in my HMD, then finally settles on my build's content. I guess there's some brief fighting going on between the two processes.

Of course the Link cable works fine, so I'll continue to use this.

[piwakawaka]

Hi @ggodin, just wondering if you've had a chance to look into this since your last reply? I'd really like to know whether this *should* work, because from what I've read elsewhere there are people using this all the time, and I'm 99% sure it did work for me the first time I tried it too. But since then I've probably tried 10-20 different things and none have worked. Does it work for you?

[piwakawaka]

Solved: apparently it's related to the new XR framework in Unity. For Unity streaming to work, it must use the older OpenVR framework instead. Hopefully support for the XR framework happens soon. The author has said that there are no games that support the new XR plugin yet so it may take some time - the irony being that I'm one of hopefully many developers planning to do this, but decided not to (for now) because there's no developer support in VD for it yet... chicken-and-egg :)

[ggodin]

Sorry for not updating the post. Yeah like you said, OpenXR not supported by virtual desktop yet (no games use it since it just came out so it’s a low priority). Using an older version will be the workaround for now

[piwakawaka]

@ggodin - I'm hoping to start a new VR project with Unity soon and would like to use OpenXR (with Unity streaming of course, for dev workflow). I was just wondering where you're at with this feature now in 2021?

[ggodin]

Set SteamVR as your OpenXR runtime and it will work with Virtual Desktop