intel realsense based green screen?

[Mechcondrid]

so are there plans to include using an intel realsense camera in lieu of a physical green screen?
for instance the razer stargazer can do this with obs (and other software) via the intel realsense Sdk library

[𝚛𝚞𝚞]

We're exploring options other than the Kinect, but haven't found a good solution yet!

[Mechcondrid]

well the realsense SDK does make it somewhat easier since it is a built in function of the SDK and the driver to take the depth map and use that to exclude the background from the resulting image
the greenscreen function is actually part of some of the code samples i believe

[564.3]

Was looking for that as well, but LIV Cix and Mystery explained on Discord that RealSense use a different technology (structured light) than Kinect (time of flight). The resulting depth map have a lower density of measured points. This approximation lead to ghosting and noise, and they are not happy with the result.

[564.3]

Researchers using the raw data are also complaining on Intel forums. After checking if there is not a problem in the environment, the response is to use their post-processing libs. I was hoping for something better...

[𝚛𝚞𝚞]

The main issue here is that each "dot" in the structured light projection from the Realsense camera doesn't contribute to a single pixel in the RGB output - in fact, my understanding of it is each "depth pixel" is sampled from a few of the structured light dots, so the effective resolution of the depth map is even lower.

This makes it really hard to get a good background removal with it, as you end up with depth "blobs" that are much larger than a single pixel in the RGB output. Compared to a traditional camera and a chroma key background, you'll see very "blobby" results.

ToF cameras don't have this same limitation, but they do face an issue with certain colours affecting the perceived depth - but for background removal, that's not a problem!

[Mechcondrid]

well when using my stargazer with obs for background removal the only issue i've had with blobs is on objects sharing a plane (hand and something right behind my hand) but other than that it has been pretty rock solid; theres an occasional border on things like baggy or loose clothes but havent noticed much otherwise.
now whether or not thats due to the stargazer being a better built device that uses the sr300 driver and libraries i dont know but i have had very little issues with it

[𝚛𝚞𝚞]

Could you share some screenshots of the background removal quality you get? We're always open to new solutions!

[Mechcondrid]

ya sure will have to set it back up; i use it with my laptop and its on my desktop as my windows login device