Desktop webcam face tracking ?

[Quetz]

Hey, is there an option to use a webcam to track the head or at least the eyes on an avatar ?
I know the mouth is animated using sound.
Maybe even have hands tracking in a seating position ?
I've only found an old reddit thread talking about this in 2018, but i can't seem to find anything else from full body webcam tracking.
Thanks!

[Fawks]

Nope, this is not an option.
The closest you can get to tracking stuff outside of VR is a Kinect, but that is mainly used as a cheap mediocre alternative for Full Body Tracking.

[なの〈Nano〉]

I don't think there's a option or easy way to do it


----TLDR----
The closest thing I think of that you can say it's 'in-game option' is the Avatar 3.0 puppet menus control, which you can control hands, tails, facial expressions, hand gestures and many stuffs(maybe eyes as well?), but you will have to customizes your avatars with unity editor and uploads it, thankfully there are many tutorial videos though
---------

There are some stuffs which you can emulate the 'fake VR headset and controllers' and let the VR device handler program SteamVR to recognition it as legit stuffs, it's mostly tons of less effectiveness and many limits to control smoothly&good since the devices are not designed for VR tracking(can be kinda okay judges by how or how you setting it up). This answers the webcam, head, hands, you can track the fingers by the way, it's super annoying by the way.(lol)

And for the eyes and facial, VRChat don't track eyes even you have the supported VR headset like VIVE Pro Eye, the newer stuff VIVE facial tracker also not supported I assume, there do have some VR programs supported, just not right now in VRChat I guess. Also, the eye movements you seeing in the VRChat are automated, also most people doing facial expressions with control button&gestures manually by the way.

Well, but I can't say it's not possible since I saw people made a new stuff to control the Avatar 3.0 parameters externally, but it's not like 'adding fake devices,' since such parameters can't normally controlled externally in VRChat. I just can't really recommends it(maybe I should not even mention it?), but since it also brings the possibility that eye parameters are not locked(maybe?), I also wonder if you can manual the eyes with the Avatar 3.0 puppet menu... who will manual their eyes though, but guess options are always better? This brings the conclusion, ....wait, it's in the TLDR and I moved it to the top

[Quetz]

I understand!
Thanks for the detailed response :).

[goyohann.]

Quetz a écrit :

Hey, is there an option to use a webcam to track the head or at least the eyes on an avatar ?
I know the mouth is animated using sound.
Maybe even have hands tracking in a seating position ?
I've only found an old reddit thread talking about this in 2018, but i can't seem to find anything else from full body webcam tracking.
Thanks!

Actually, with OSC you can use a leap motion controller and a normal webcam for face and hand tracking. You would just need an external application that takes the feed from your devices and converts it into blendshape floats for the new VRChat OSC feature.

[Frank]

This thread was quite old before the recent post, so we're locking it to prevent confusion.