Reality Blender 3D overlays in action with Valve Index cameras

[PyrofrogStudios]

https://youtu.be/Xq4eLUaqUTA

[jakelogg]

Can you share your Reality Blender settings for that video? I'm trying to replicate it with a cockpit but the instruments float all over the screen as I turn my head. Since I now have a valve index and we have the same camera, the settings should transfer over, except for the left and right image alignment.

[PyrofrogStudios]

There is a guide showing usage:
https://steamcommunity.com/sharedfiles/filedetails/?id=1508492092

[jakelogg]

I read the guide. What I'm struggling with is the interplay between Z and Width on the overlay tab. As well as the distortion on the image tab. Since you have the exact same camera I have, I was hoping to just quickly map your settings to mine. The video you made looks fantastic, but I'm having trouble replicating it on my own.

[PyrofrogStudios]

Oh right I would ask in the comments of the video as unfortunately it’s not mine. I wish I had that setup but I can’t even buy an index in my country at the moment. It’s a shame as I can’t really develop things much further without one.

[jakelogg]

For anyone reading this thread in the future, here's what worked for me for the valve index camera:

- exit steam home so you're in the blank world
- hold one controller at arms length

- on the overlay tab, leave x and y values at 0, set width to 1
- adjust the Z value until the camera feed controller is about the same size as the virtual controller (for me, the sweet spot was 0.55)

- on the image tab, close your right eye and adjust the x and y values until the camera feed's controller matches the virtual controller. repeat for the right image side.

- decide whether you want to be glancing right or left
- move your controller to the edge of your peripheral vision on the preferred glance side
- check apply correction and adjust K1 until the controllers match up

- move your controller to the other side of your peripheral vision, crop out the image at the point it looks disjointed, repeat for the other image side
- note: the left and right crop fields are reversed for image sides. if you want to be glancing right, for the left side you'll need crop out the left, and for the right side you'll need to crop out the right. this was was counter intuitive for me.

This method allows me to see things near me very well, but if I look at the base station further away, I see 2 base stations equidistant from the virtual base station.

Here are the settings I ended up with.

Overlay tab:
X: 0
Y: 0
Z: 0.55
Width: 1

Image tab:
Image Left:
X: -1.226
Y: -0.424

Image Right:
X: 1.0879
Y: -0.089

K1 distortion about 50%, K2 zero. Cropped left side out because I'm mostly glancing to the right.

[PyrofrogStudios]

Thanks for sharing your findings. How have you found it? Is it useable? Image feed clear or blurry by the time you size it?

It would benefit from camera calibration but haven’t had time to develop it further.

[jakelogg]

It's pretty awesome! Far more immersive than any flight simulation setup I've had thus far.

The latency is kind of annoying at first (obviously not your fault), and it's very hard to exactly map the physical knobs and buttons to the virtual world. But for now I just turned the opacity down a little bit and it just sort of looks like I'm a ghost in the virtual world turning knobs and pressing buttons.

Thanks for developing it!

[jakelogg]

As for image quality, clarity has never been an issue for me. The hard part is aligning the images but at worst you just see two clear images far apart. If you find time to develop this further, I'm happy to help you test.

[PyrofrogStudios]

Thanks for the info.
I believe this has the potential to create the most immersive cockpit experience possible. But there are some hard problems to solve for it to be perfected. I get the feeling that the big players are obviously trying to get VR in the mainstream to pay off as even though VR is a perfect match for simrigs it is a tiny market. So it’s just me a single man indie dev trying to solve it. Add to that I have other software that sells and pays the bills I have to prioritise that over Reality Blender. Hopefully I can get some spare time to work on it again in the future. I will definitely contact you to test any new versions.
Thanks.

[KydDynoMyte]

I assume this is even better now with the new stereo corrected passthrough on the Index?