Avatar overlay while INSIDE a VR game.

[CyberObsequiousメタル猫又V4]

Hello. I have been testing out some different programs for placing an avatar over a twitch stream. So far there are issues with all of them which I cannot seem to circumvent.

TL;DR - is it simply not possible to run a VR program that tracks an avatar simultaneously while playing a VR game? Thanks for any help in advance.

Here's a rundown of issues I have ran into this week:
-Vmagicmirror: this is great for keyboard/mouse play but in VR it does not track head/hand movement.

-VirtualCast: Steam VR game, can't load two steam VR apps at one time. Most of the games I'm streaming are SteamVR besides VRchat. Also, OBS refuses to record virtualCast for some reason.

-Tracking World: This one seems like the most promising of the bunch, but recent steam updates stop the program from disabling HMD rendering, and it also launches steamVR even though it's a non steam game.
There are some other programs like flipside Studio but I've been struggling with finding support as there doesn't seem like much help in troubleshooting loading .FBX avatars.

Summary: Flipside studio on the quest store runs fine with Steam VR games at the same time, but my ineptitude to import models properly into it is leaving me stranded.

Tracking world handles the avatars the best and seems like the easiest to use, but steamVR doesn't want two programs running.

[BOLL]

Just as a side note, for something to use the tracking of SteamVR devices, it needs to launch SteamVR, so hard to get around that. At least as far as I know.

It is a possible to connect to SteamVR and grab poses and stuff in the background, I've actually made a group of utilities that does just that. If these applications renders to the headset they're scene applications and will indeed not run at the same time as other VR titles. They would need to change the application type to background.

If you find a nice avatar display system that works in a browser, it's possible to add as a browser source if you use OBS, then I could help piping tracking data to it through one of my applications[github.com] 😅 You can see what I made for my streaming overlay in this video, using the app mentioned.

[CyberObsequiousメタル猫又V4]

That's quite interesting BOLL. So the second last block on the bottom is a live avatar of you? Is it possible to load VRM files as the displayed character? This seems a little over my head as I have only played around with code a little bit.

[BOLL]

Yeah it's just my headset and controllers displayed as balls like 3D was done 30 years ago 😅 Any more realistic avatar would require a proper 3D engine, like three.js, and probably some inverse kinematics solution and then .fbx loader. Absolutely no idea how much work that would be but probably a lot 🤣 I only provide the pose data for the devices in my application.

[Burningspaceman]

You want LIV

[CyberObsequiousメタル猫又V4]

Originally posted by Burningspaceman:

You want LIV

I forgot I posted this, but someone reached out to me shortly after and YES! LIV is what's up. Thanks for responding!