Run SteamVR Unity Project as Background application type

[Jaymis]

I'm a filmmaker and have been working on different approaches for mixed-reality videos.
I just released the first iteration of a camera gimbal which can track the player's physical location and keep them in frame:

https://www.youtube.com/watch?v=hyQ75urqXtk

https://twitter.com/jaymis/status/783520528661417984

I'd love for one of my next steps to be running this alongside existing VR software, with the camera gimbal in 3rd-controller mixed-reality mode, so it could automatically follow player movement in the room/game.

I've looked at the OpenVR API docs, and found some information about vr::VR_Init() and using the "background" application type (this thread is current, and covers much of the same ground), but it seems that the Unity SteamVR scripts are designed to run as "scene" applications.

Is it possible to have a Unity/SteamVR project init to run as background?

I'm not a great programmer, so would love to stay inside Unity/Playmaker until I've prototyped the hardware/interface questions and can collaborate on Good Code with someone who has actual skills.

Please and thankyou.

Jaymis

/edit: Youtube instead of Vimeo link, for easier video-clicking.

[BOLL]

I wrote my reply on Vimeo before ;) No idea if any game engine can do it, but other applications sure can be background ones. This is the current interface of my software: http://i.imgur.com/BnwJafL.png SteamVR not running right now as I'm watching the Oculus keynote.

[BOLL]

And to actually try to be helpful, you can check out this[github.com] project on GitHub, it's a Unity app that runs in the background to insert an overlay into games. Maybe you can find what you are looking for in there :)

[BOLL]

So I looked through the source, on line 78 in \Assets\SteamVR\Scripts\SteamVR.cs the application type is set:

OpenVR.Init(ref error, EVRApplicationType.VRApplication_Overlay);

I'm not sure if this is a file that one should edit, but when I search I don't find this being set anywhere else. You could try swapping the type out to EVRApplicationType.VRApplication_Background and see if it does what you want in your project.

[Jaymis]

Haha BOLL, we're seeing each other everywhere on the internet these days :) I guess the robot-camera-tracker world is even smaller than the VR one :)

That project looks very interesting. I'll give it a try today! After posting last night, I found HeadlessOverlayToolkit[github.com], from the same user, but I had some troubles getting it going, so I'll have a try with both of these today.

[Jaymis]

After a couple of hours hacking (I'm a filmmaker first, coder abooout, fifty-seventh), I got in touch with the creator of HeadlessOverlayToolkit to tell him what I was working on. We emailed a bit and he gave me a couple of pointers, so I kept working.

A couple of hours later he's added support for 3rd and 4th controllers to the Beta branch of HeadlessOverlayToolkit[github.com].

Thanks Hotrian :)

[BOLL]

Aha, nice! I'll jump to email :) And yeah, I subscribe to the VR forums so I read most new threads... the Internet feels pretty small sometimes ;)

[BOLL]

Yo @Jaymis, just as a random followup as I just today picked up Unity again for the first time in years, wow, so happy I finally got around to it. In any case, I'm making a companion app and instead of importing the SteamVR plugin I used the openvr.dll and C# header directly and successfully registered my Unity application as a background app :D

Something I had to do to get the transformation matrix from OpenVR to work properly in Unity was to reverse rotation on the X-axis and position on the Z-axis. I'm pretty sure it could be done differently with the result of a different forward direction, but this is what I ended up with for it to look correct.

Anyway, sorry for resurrecting the thread, I just found it funny that when I searched before I started working on this today, this thread which I had participated in was the top Google result, so I figured I could just as well reply after my happily successful experimentation!

[Laiden]

Originally posted by BOLL:

instead of importing the SteamVR plugin I used the openvr.dll and C# header directly and successfully registered my Unity application as a background app :D

Hey BOLL, I'm trying to run a unity desktop app capturing the tracking data, together with a usual VR application, thats how google brought me here. Can you give me hints how you registered your unity as background? In this case, I do not use Unitys procedure to load a VR device and the integrated VR support stays disabled?

My current unity project uses the integrated XR API with OpenVR to access my VIVE.
In the past, I created a c++/qt app + OpenVR API to capture and render to the VIVE.
However, I do not know how to do the OpenVR init myself within Unity, because it does everything under the hood automatically.

If I cant make this work, the alternative were to capture the tracking in a standalone app and design a interface/plugin for the unity project maybe?

[Laiden]

I know this is a old thread, so its okay not having a response immediately. Meanwhile I was able to archieve what I wanted. In case other people interested, here's what I've done:

# First I unload the Unity OpenVR package, because I'm not certain what revision of OpenVR it uses and there's only the .dll, I couldn't find a readable header file to it. Did a dump of the lib - it seems as if no modification to the available methods were made though.
Means Unity's own XR implementation is disabled and its methods can not be used.

# Downloaded the built OpenVR API with supplied C# header from the official git repository. The binary itself is compiled in c++ and it exports some interface functions for Win, Linux, Mac. The repo also has some demo samples to look at.

# Took a short crashcourse about .NET CLI and marshalling "C" code from c#

# Put the binary into Assets/Plugins/x86_64 and the .cs header in Assets/Scripts of my Unity project. That was everything required to start using it in scripts. It just worked. No assembly referencing, static libs or including sorts of required. Magic O.O

# I did some minor convinience modifications to the .cs header (Usually thats a bad idea for future updates, I know). Basically I split the file and namespace plus moved some platform specific method arguments out of the wrapper class methods. Also the shorthand 'IVRSystem System' was renamed to resolve namespace conflict with .Net System

# The final script then includes the static wrapper class 'OpenVR' by 'using Valve.VR;' and accesses the VR device by the property OpenVR.Sys

# Ofc the initialization has to be done as mentioned already, have SteamVR running, then call OpenVR.Init with the Application Background argument. The wrapper class manages the fetching of the interface and instantiation as a singleton.

[Laiden]

Originally posted by BOLL:

reverse rotation on the X-axis and position on the Z-axis

Thats because transformation and Quaternions defined in a mathematical "right-hand" system. Maybe caused by OpenGL, render environments calculate vertices in a left-hand system. Means rotation and the direction of the 3rd axis is reversed